CN114080795A

CN114080795A - Image sensor and electronic device

Info

Publication number: CN114080795A
Application number: CN202080034789.6A
Authority: CN
Inventors: 姚国峰; 沈健
Original assignee: Shenzhen Goodix Technology Co Ltd
Current assignee: Shenzhen Goodix Technology Co Ltd
Priority date: 2020-06-10
Filing date: 2020-06-10
Publication date: 2022-02-22
Also published as: WO2021248379A1

Abstract

An image sensor (200) and an electronic device are provided, which can improve the signal-to-noise ratio and the output speed of the image sensor (200), and simultaneously can also take into account the resolution and the aspect ratio of an image, thereby improving the overall performance and the user experience of the image sensor (200). An image sensor (200) includes: the pixel array (210) comprises a plurality of rhombic pixel units arranged in a rhombic shape, wherein half of the rhombic pixel units are positioned in a first target row in the pixel array (210), the other half of the rhombic pixel units are positioned in a first target column in the pixel array (210), the first target row is an odd-numbered row or an even-numbered row in the pixel array (210), and the first target column is an odd-numbered column or an even-numbered column in the pixel array (210); half of the pixel values of the pixel cells are used for 2 in 1 pixel synthesis in the horizontal direction, and the other half of the pixel values of the pixel cells are used for 2 in 1 pixel synthesis in the vertical direction to form a plurality of target pixel values of the image sensor (200).

Description

Image sensor and electronic device

Technical Field

The present application relates to the field of sensors, and more particularly, to an image sensor and an electronic device.

Background

An image sensor is an electronic device that converts an optical image into a digital signal, and generally includes a pixel array composed of a plurality of pixel cells, each pixel cell in the pixel array being used to form a pixel value in the image. In order to improve the signal-to-noise ratio and the output speed of the image sensor, a pixel synthesis (binning) method may be used to generate an image, i.e., to combine the pixel values of a plurality of pixel units and output the combined pixel value as a new pixel value. However, when the image is generated by adopting the pixel synthesis method, the disadvantages of the aspect ratio change, the resolution reduction and the like of the image are caused, and the user experience is influenced.

Therefore, how to improve the signal-to-noise ratio and the output speed of the image sensor, and simultaneously, consider the aspect ratio and the resolution of the image, and improve the overall performance and the user experience of the image sensor is a technical problem to be solved urgently.

Disclosure of Invention

The embodiment of the application provides an image sensor and an electronic device, which can improve the signal-to-noise ratio and the output speed of the image sensor, and simultaneously can give consideration to the resolution ratio and the length-width ratio of an image, thereby improving the overall performance and the user experience of the image sensor.

In a first aspect, an image sensor is provided, including: the pixel array comprises a plurality of rhombic pixel units which are arranged in a rhombic shape, wherein half of the rhombic pixel units are positioned in a first target row in the pixel array, the other half of the rhombic pixel units are positioned in a first target column in the pixel array, the first target row is an odd-numbered row or an even-numbered row in the pixel array, and the first target column is an odd-numbered row or an even-numbered row in the pixel array; the pixel values of the half of the pixel units are used for carrying out 2 in 1 pixel synthesis in the horizontal direction, and the pixel values of the other half of the pixel units are used for carrying out 2 in 1 pixel synthesis in the vertical direction to form a plurality of target pixel values of the image sensor.

According to the scheme, the pixel units in the image sensor 200 are arranged in a diamond shape and are arranged in the diamond shape, 2-to-1 pixel synthesis is carried out on the pixel values of the pixel units in the arrangement mode, the size proportion of an image obtained after the pixel synthesis is unchanged, the calculation load of the image sensor can be reduced, the signal-to-noise ratio and the output speed of the image sensor are improved, the resolution ratio and the size proportion of the image can be considered, and therefore the overall performance and the user experience of the image sensor are improved.

In one possible embodiment, the image sensor further includes: the filter array comprises a plurality of rhombic filter units which are arranged in a rhombic shape, and the rhombic filter units are correspondingly arranged above the rhombic pixel units one by one; half of the diamond-shaped filter units are located in a second target row of the filter array, the color of the filter units in each row of the second target row is the same, the other half of the diamond-shaped filter units are located in a second target column of the filter array, the color of the filter units in each column of the second target column is the same, the second target row is an odd-numbered row or an even-numbered row of the filter array, and the second target column is an odd-numbered row or an even-numbered row of the filter array.

According to the scheme of the embodiment of the application, the filter array is arranged in the image sensor, the filter array comprises filter units with multiple colors, the collection of color images can be realized, meanwhile, the filter units are arranged into a diamond shape and are arranged in the diamond shape, the specific arrangement mode of the filter units with different colors in the filter array is designed, the requirement of 2-in-1 pixel synthesis of the corresponding pixel array is met, the calculation load of the image sensor is reduced, the signal-to-noise ratio and the output speed of the image sensor are improved, the resolution ratio and the size ratio of the image are considered, the color images are output, and the overall performance and the user experience of the image sensor are further improved.

In a possible embodiment, the filter array includes a plurality of filter unit groups arranged in a diamond shape, and the filter unit group includes four adjacent filter units, two adjacent filter units at a common vertex angle in the four adjacent filter units have the same color, and two adjacent filter units at a common side have different colors.

In one possible embodiment, the filter array includes filter units of three colors, which are a first filter unit, a second filter unit, and a third filter unit, respectively.

In a possible embodiment, each of the second target rows is a first filter unit, and the second target column is a second filter unit or a third filter unit; or each of the second target rows is a first filter unit, and the second target row is a second filter unit or a third filter unit.

In a possible embodiment, half of the second target row is the second filter unit, and the other half of the second target row is the third filter unit; alternatively, half of the second target row is the second filter unit and the other half of the second target row is the third filter unit.

In one possible implementation, half of the columns in the second target column and the other half of the columns are odd columns and even columns in the second target column, respectively; or, half rows and the other half rows in the second target row are respectively odd rows and even rows in the second target row.

In a possible embodiment, the number of the second filter units is equal to the number of the third filter units, and the sum of the number of the second filter units and the number of the third filter units is equal to the number of the first filter units.

In one possible embodiment, the filter array includes four color filter units, which are a first filter unit, a second filter unit, a third filter unit and a fourth filter unit, respectively.

According to the scheme of the embodiment of the application, the filter array comprises the filter units with 4 colors, so that the quality of a color image can be improved, and the overall performance and the user experience of the image sensor are further improved.

In one possible embodiment, the target row of the filter array is the first filter unit or the fourth filter unit, and the target row of the filter array is the second filter unit or the third filter unit; or, the target row of the filter array is the first filter unit or the fourth filter unit, and the target row of the filter array is the second filter unit or the third filter unit.

In a possible embodiment, half of the rows in the second target row are the first filter units, the other half of the rows in the second target row are the fourth filter units, half of the columns in the second target column are the second filter units, and the other half of the columns in the second target column are the third filter units; or, half of the second target column is the first filter unit, the other half of the second target column is the fourth filter unit, half of the second target row is the second filter unit, and the other half of the second target row is the third filter unit.

In one possible implementation, half of the columns in the second target column and the other half of the columns are odd columns and even columns in the second target column, respectively; half of the second target rows and the other half of the second target rows are odd rows and even rows of the second target rows, respectively.

In a possible embodiment, the number of the first filter units is equal to the number of the fourth filter units, and the number of the second filter units is equal to the number of the third filter units; the sum of the number of the second filter units and the number of the third filter units is equal to the sum of the number of the first filter units and the number of the fourth filter units.

In a possible embodiment, the first filter unit, the second filter unit and the third filter unit are configured to pass light signals of three colors respectively, and the wavelength bands of the light signals of the three colors cover the visible wavelength band.

In one possible embodiment, the first filter unit, the second filter unit and the third filter unit are respectively three colors of red, green, blue, cyan, magenta and yellow.

In a possible embodiment, the first filter unit is a green filter unit, the second filter unit and the third filter unit are a red filter unit and a blue filter unit, respectively.

In one possible embodiment, the fourth filter unit is a visible light or non-visible light filter unit.

In one possible embodiment, the fourth filter unit is a white filter unit, a gray filter unit or a near-infrared filter unit.

In one possible implementation, the number of the target pixel values is half of the number of pixel units in the pixel array, and the target pixel values are used for forming a target pixel value array arranged in a square shape; in the target pixel value array, one of two adjacent target pixel values in the horizontal direction or the vertical direction is a target pixel value obtained through 2-in-1 pixel synthesis in the horizontal direction, and the other one is a target pixel value obtained through 2-in-1 pixel synthesis in the vertical direction.

In a possible embodiment, in the target pixel value array, one of two target pixel values adjacent in the horizontal direction or the vertical direction is a target pixel value obtained by summing or averaging pixel values of two pixel units adjacent to a common vertex on the first target row in the pixel array, and the other is a target pixel value obtained by summing or averaging pixel values of two pixel units adjacent to a common vertex on the first target column in the pixel array.

In one possible embodiment, the image sensor further includes: a pixel synthesis circuit; the pixel synthesis circuit is connected to the pixel array and used for carrying out 2-in-1 pixel synthesis on the half of pixel units in the horizontal direction and carrying out 2-in-1 pixel synthesis on the other half of pixel units in the vertical direction so as to form a plurality of target pixel values.

In one possible embodiment, the image sensor further includes: a row control circuit and a column control circuit; the row control circuit is connected with a plurality of rows of pixel units in the pixel array through a plurality of row control lines, and each row of pixel units in the plurality of rows of pixel units are arranged at the same vertex angle; the column control circuit is connected with a plurality of columns of pixel units in the pixel array through a plurality of column control lines, and each column of pixel units in the plurality of columns of pixel units are arranged at the same vertex angle; the pixel synthesis circuit is connected to the pixel array through the column control circuit.

In one possible embodiment, the image sensor further includes: the analog-to-digital conversion circuit and the signal processing circuit; the analog-to-digital conversion circuit is connected with the pixel synthesis circuit and is used for converting the target pixel values output by the pixel synthesis circuit into digital pixel values; the signal processing circuit is used for processing the plurality of digital pixel values to obtain a color image.

In one possible implementation, the image sensor is a Complementary Metal Oxide Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

In a second aspect, an electronic device is provided, which includes the image sensor of the first aspect or any one of the possible implementations of the first aspect.

The image sensor is arranged in the electronic equipment, and the image sensor is high in corresponding speed, so that the output speed of images in the electronic equipment can be increased, the overall performance of the electronic equipment can be improved, and the user experience is improved.

Drawings

Fig. 1 is a schematic structural diagram of an image sensor.

FIG. 2 is a schematic diagram of a horizontal 2-in-1 pixel synthesis.

Fig. 3 is a schematic diagram of an image before and after 2 in 1 pixel synthesis.

Fig. 4 is a schematic top view of an image sensor provided according to an embodiment of the present application.

Fig. 5 is a schematic top view of another image sensor provided in accordance with an embodiment of the present application.

Fig. 6 is a schematic sectional view of the image sensor of fig. 5 taken along a-a'.

Fig. 7 is a schematic top view of another image sensor provided in accordance with an embodiment of the present application.

Fig. 8 is a schematic top view of another image sensor provided in accordance with an embodiment of the present application.

Fig. 9 is a schematic diagram of 2-in-1 pixel synthesis for the image sensor of fig. 5.

FIG. 10 is a schematic top view of another image sensor according to an embodiment of the present application.

FIG. 11 is a schematic top view of another image sensor according to an embodiment of the present application.

Fig. 12 is a schematic diagram of 2-in-1 pixel synthesis for the image sensor of fig. 10.

Fig. 13 is a schematic structural diagram of an image sensor according to an embodiment of the present application.

Detailed Description

The technical solutions in the embodiments of the present application will be described below with reference to the accompanying drawings.

It should be understood that the specific examples are provided herein only to assist those skilled in the art in better understanding the embodiments of the present application and are not intended to limit the scope of the embodiments of the present application.

It should also be understood that the various embodiments described in this specification can be implemented individually or in combination, and the examples in this application are not limited thereto.

The technical solution of the embodiment of the present application may be applied to various image sensors, such as a Complementary Metal Oxide Semiconductor (CMOS) image sensor (CIS) or a Charge Coupled Device (CCD) image sensor, but the embodiment of the present application is not limited thereto.

As a common application scenario, the image sensor provided in the embodiment of the present application may be applied to a smart phone, a camera, a tablet computer, and other mobile terminals or other terminal devices having an imaging function.

Fig. 1 shows a schematic structural diagram of an image sensor. As shown in fig. 1, the image sensor 100 includes: a pixel array (pixel array)110, a row selection circuit 120, a column selection circuit 130, a control circuit 140, a pixel synthesis circuit 150, an analog to digital converter (ADC) circuit 160, and a signal processing circuit 170.

Specifically, as shown in fig. 1, a plurality of square pixel cells in the pixel array 110 are arranged in M rows × N columns, where M, N is a positive integer. Generally, the row direction of the M rows and the column direction of the N columns are perpendicular to each other on the plane of the pixel array 110. In some cases, for convenience of description, in one plane, two directions perpendicular to each other, such as a row direction and a column direction in the present application, may be referred to as a horizontal direction and a vertical direction.

In the pixel array 110 shown in fig. 1, any side of each square pixel cell is parallel or perpendicular to the row direction or the column direction.

Optionally, the pixel unit may include a photodiode, a field effect switching transistor, and other devices for receiving an optical signal and converting the optical signal into a corresponding electrical signal.

Alternatively, if the image sensor needs to acquire a color image, a Color Filter Array (CFA) may be disposed above the pixel array 110, wherein one color filter may be disposed above each pixel unit, and for the purpose of description, the pixel unit over which the color filter is disposed is also referred to as a color pixel unit, for example, the pixel unit over which the red filter is disposed is referred to as a red pixel unit, the pixel unit over which the green filter is disposed is referred to as a green pixel unit, and so on.

Currently, most CFAs of image sensors use a Bayer (Bayer) format based on three primary colors of RGB, for example, as shown in fig. 1, a CFA in Bayer format is disposed above the pixel array 110, and the pixel array 110 uses 2 × 2 pixel units as basic units, each of which includes 1 red pixel unit, 1 blue pixel unit, and 2 green pixel units, wherein two green pixel units are disposed adjacent to each other at a common top corner.

Row select circuit 120 is coupled to each row of pixel cells in pixel array 110 via M row control lines, and may be used to turn on and off each pixel cell in each row of pixel cells. For example, the row selection circuit 120 is connected to the gate of the fet of each pixel unit in the first row of the pixel array 110 via a row control line, and controls the operating state of the photodiode by turning on or off the fet. Wherein, M row control lines are all parallel to the horizontal direction.

The column selection circuit 130 is connected to each column of pixel units in the pixel array 110 through N column control lines, and may be configured to select a signal value output of each pixel unit in each column. For example, the column selection circuit 130 is connected to the source of the fet of each pixel unit in the first column in the pixel array 110 through a column control line, and controls the output of the electrical signal converted by the photodiode. Wherein, N column control lines are all parallel to the vertical direction.

The control circuit 140 is connected to the row selection circuit 120 and the column selection circuit 130, and is configured to provide timing for the row selection circuit 120 and the column selection circuit 130, control the row selection circuit 120 and the column selection circuit 130 to select a pixel unit in the pixel array 110, and output a pixel value of the pixel unit.

Alternatively, after the row selecting circuit 120, the column selecting circuit 130 and the control circuit 140 cooperate with the pixel value generated by the pixel array 110 to output, the pixel value of the pixel array 110 is transmitted to a pixel combining circuit 150, and the pixel values of a plurality of pixel units are combined and output as a new value.

In some embodiments, the binning circuit 150 may be a summing circuit for summing the pixel values of a plurality of pixel units. Further, the binning circuit 150 may also be a summing and averaging circuit for summing pixel values of a plurality of pixel units and calculating an average value.

It can be understood that the pixel values output by the pixel array 110 are analog signal quantities, and the binning circuit 150 is configured to sum or average the analog pixel values to obtain new analog pixel values, and send the binned new analog pixel values to the ADC circuit 160 for analog-to-digital conversion, and convert the analog pixel values into digital pixel values to form a digital image, so as to facilitate subsequent image processing by the signal processing circuit 170 to output an optimized color image.

Alternatively, the signal processing circuit 170 may include, but is not limited to, an Image Signal Processor (ISP) for performing linearization processing, dead pixel removal, noise removal, color correction, demosaic (demosaic), Automatic Exposure Control (AEC), Automatic Gain Control (AGC), Automatic White Balance (AWB), and the like on the digital image.

FIG. 2 shows a schematic diagram of a horizontal 2 × 1binning (2 in 1 pixel synthesis).

As shown in fig. 2, wherein (a) illustrates a plurality of pixel values arranged in a Bayer pattern, each pixel value of the plurality of pixel values corresponding to each pixel cell of the image sensor; (b) the graph shows (a) new pixel values after the plurality of pixel values in the graph have undergone horizontal 2 × 1 binning.

Specifically, when 2 × 1binning in the horizontal direction is performed, two adjacent pixel values of the same color are combined in a row of pixel units. For example, (a) the red pixel values R11 and R13 in the graph are combined to form (b) a new red pixel value R '11 in the graph, which new red pixel value R' 11 may be the sum or average of R11 and R13; similarly, the green pixel values G12 and G14 in (a) are combined to form a new green pixel value G '12 in (b), which new green pixel value G' 12 may be the sum or average of G12 and G14.

After 2 × 1binning in the horizontal direction, the number of new pixel values is half of the number of pixel values before binning, and the image size formed by the new pixel values is also compressed to half of the previous one in the horizontal direction. For example, in the drawing, 32 pixel values are shared in total, and the size of the formed image is H (horizontal direction) × V (vertical direction), and after 2 × 1binning in the horizontal direction, in the drawing, 16 new pixel values are shared in total, and the size of the formed image is H/2 (horizontal direction) × V (vertical direction).

Fig. 2 exemplarily shows a schematic diagram of 2 × 1binning in the horizontal direction, and it can be understood that if 2 × 1binning is performed on the pixel values in the vertical direction, the number of new pixel values is also half of the number of pixel values before binning, and the image size formed by the new pixel values is compressed to be half of the previous one in the vertical direction. For example, if (a) a plurality of pixel values in the figure undergo 2 × 1binning in the vertical direction, the size of an image formed is H (horizontal direction) × V/2 (vertical direction).

It can also be understood that the process of vertical 2 × 1binning is similar to the process of horizontal 2 × 1binning in fig. 2, and specifically, the pixel values of two same colors that are adjacent to each other in a column of pixel units are combined. The specific scheme may refer to the related description in fig. 2 above, and is not described herein again.

FIG. 3 shows a schematic image after 2 × 1 binning.

Fig. 3 (a) is an image acquired by an original pixel array, fig. 3 (b) is an image formed after 2 × 1 binding in the horizontal direction, and fig. 3 (c) is an image formed after 2 × 1 binding in the vertical direction.

As can be seen from fig. 2 and 3 and the description thereof, when the pixel values are combined in the 2 × 1binning scheme, although the number of new pixel values is reduced and the calculation load of the subsequent signal processing circuit 170 is reduced, the size ratio of the processed image is changed, and the image is compressed in the horizontal or vertical direction, which may affect the user experience, thereby limiting the application scenarios of the 2 × 1 binning.

Although the pixel values can be processed by 2 × 2binning, 3 × 3binning, etc. without affecting the image size ratio, 4 pixel values are combined into one pixel value in the 2 × 2binning mode, and 9 pixel values are combined into one pixel value in the 3 × 3binning mode, and the resolution of the image is reduced with the increase of the binning degree, which also affects the user experience.

Based on the above problems, the present application provides an image sensor, which adjusts the structure and arrangement form of a pixel array, and processes the pixel value of the pixel array in a 2 × 1binning manner, where the size ratio of an image after 2 × 1binning is the same as the size ratio of an original image collected by the pixel array, so that the calculation load of the image sensor can be reduced, the signal-to-noise ratio and the output speed of the image sensor can be improved, and the resolution and the size ratio of the image can be considered, thereby improving the overall performance and the user experience of the image sensor.

Hereinafter, the image sensor according to the embodiment of the present application will be described in detail with reference to fig. 4 to 13.

It should be noted that, for the sake of understanding, the same structures are denoted by the same reference numerals in the embodiments shown below, and detailed descriptions of the same structures are omitted for the sake of brevity.

Fig. 4 is a schematic top view of an image sensor 200 according to an embodiment of the present disclosure.

As shown in fig. 4, the image sensor 200 includes:

the pixel array 210 comprises a plurality of rhombic pixel units arranged in a rhombic shape, wherein half of the rhombic pixel units are positioned in a first target row in the pixel array 210, the other half of the rhombic pixel units are positioned in a first target column in the pixel array 210, the first target row is an odd-numbered row or an even-numbered row in the pixel array 210, and the first target column is an odd-numbered column or an even-numbered column in the pixel array 210;

half of the pixel units of the diamond-shaped pixel units are used for carrying out 2 in 1 pixel synthesis in the horizontal direction, and the other half of the pixel units are used for carrying out 2 in 1 pixel synthesis in the vertical direction to form a plurality of target pixel values of the image sensor.

As shown in fig. 4, the pixel array 210 includes 9 rows and 9 columns of diamond-shaped pixel units, the row direction is a horizontal direction, the column direction is a vertical direction, and every two adjacent pixel units in the plurality of pixel units are connected at a common vertex angle in each row or each column. And in the plurality of pixel units, any side of each pixel unit forms an included angle of 45 degrees with the horizontal direction.

Alternatively, in the pixel array 210 shown in fig. 4, the first target row is an even-numbered row, and the first target column is an even-numbered column, wherein half of the pixel units in the pixel array 210 are located in the even-numbered row and the other half of the pixel units are located in the even-numbered column.

The pixel values of half of the pixel units on the even-numbered row are used for performing 2 × 1binning in the horizontal direction, that is, in the horizontal direction, the pixel values of two pixel units in the even-numbered row are used for synthesizing a new target pixel value, and the pixel values of the other half of the pixel units on the even-numbered column are used for performing 2 × 1binning in the horizontal direction, that is, in the vertical direction, the pixel values of two pixel units in the even-numbered column are used for synthesizing a new target pixel value, after pixel synthesis, the obtained plurality of target pixel values are half of the number of pixel units before 2 × 1binning, and the image size formed by the plurality of target pixel values is compressed by the same amount in both the horizontal direction and the vertical direction, for example, if the image size formed by the pixel array 210 in fig. 4 is H (horizontal direction) × V (vertical direction), then the image size formed after 2 × 1binning is H '(horizontal direction) × V' (vertical direction), wherein H'/V ═ H/V.

Therefore, according to the scheme of the application, the plurality of pixel units in the image sensor 200 are arranged in a diamond shape and in a diamond shape, in the arrangement mode, 2-to-1 pixel synthesis is performed on the pixel values of the plurality of pixel units, the size proportion of an image obtained after the pixel synthesis is unchanged, the calculation load of the image sensor can be reduced, the signal-to-noise ratio and the output speed of the image sensor are improved, the resolution ratio and the size proportion of the image can be considered, and therefore the overall performance and the user experience of the image sensor are improved.

Fig. 5 is a schematic top view of another image sensor 200 provided in an embodiment of the present application, and fig. 6 is a schematic cross-sectional view of the image sensor 200 along the direction a-a'.

As shown in fig. 5 and 6, the image sensor 200 further includes:

the filter array 220 comprises a plurality of rhombic filter units which are arranged in a rhombic shape, and the rhombic filter units are correspondingly arranged above the rhombic pixel units one by one;

half of the diamond-shaped filter units are located in a second target row in the filter array 220, the color of the filter units in each row in the second target row is the same, the other half of the diamond-shaped filter units are located in a second target column in the filter array 220, the color of the filter units in each column in the second target column is the same, the second target row is an odd-numbered row or an even-numbered row in the filter array 220, and the second target column is an odd-numbered column or an even-numbered column in the filter array 220;

specifically, the top view shown in fig. 5 is also a schematic diagram of an arrangement of the filter array 220 according to the embodiment of the present application.

In one possible embodiment, as shown in fig. 6, a plurality of filter units in the filter array 220 may be disposed on an upper surface of a plurality of pixel units of the pixel array 210; in another possible implementation, the plurality of filter units in the filter array 220 may be suspended above the plurality of pixel units in the pixel array 210.

Further, as shown in fig. 6, as an example, each filter unit in the filter array 220 is correspondingly disposed right above each pixel unit in the pixel array 210, in other words, the center of each filter unit coincides with the center of its corresponding pixel unit in the vertical direction. Except for this way, each filter unit in the filter array 220 is correspondingly disposed above and obliquely above each pixel unit in the pixel array 210, at this time, each pixel unit in the pixel array 210 can receive the optical signal in the oblique direction, and the embodiment of the present application does not limit the specific position of the filter array 220.

Specifically, in the embodiment of the present application, the color of one half of the filter cells located on the second target row is different from the color of the other half of the filter cells located on the second target column.

In some embodiments, the filter array 210 includes a plurality of filter unit groups arranged in a diamond shape, and the filter unit group includes four adjacent filter units, two filter units adjacent to a common vertex in the four adjacent filter units have the same color, and two filter units adjacent to a common edge have different colors. In the target rhombus, two filter units which are adjacent to each other at a common vertex angle in the horizontal direction are positioned in a second target row, and two filter units which are adjacent to each other at a common vertex angle in the vertical direction are positioned in a second target column.

Alternatively, filter elements of three colors may be included in the filter array 220.

For example, as shown in fig. 5, the filter unit 220 includes three color filter units of a first filter unit 221, a second filter unit 222, and a third filter unit 223.

The first filter unit 221, the second filter unit 222, and the third filter unit 223 may be filter units of three primary colors, red, green, and blue (RGB), may also be filter units of three complementary colors, cyan, magenta, and yellow (CMY), and may also be filter units of two complementary colors of one primary color, or two complementary colors of one complementary color.

In some embodiments, the first filter unit 221 is a green filter unit, the second filter unit 222 is a red filter unit, and the third filter unit 223 is a blue filter unit, and at this time, the filter array 220 in fig. 5 is a Bayer-format filter array. The number of green filter cells is twice the number of red or blue filter cells.

In other embodiments, the first filter unit 221 is a yellow filter unit, the second filter unit 222 is a red filter unit, and the third filter unit 223 is a blue filter unit, wherein the light signal transmitted by the yellow filter unit includes a green light component and a red light component.

It can be understood that the light signals of three colors after the first filter unit 221, the second filter unit 222 and the third filter unit 223 are superposed to form white light, in other words, the light signal bands of the three colors can cover the visible light band.

In other words, the color of the first filter unit 221, the color of the second filter unit 222, and the color of the third filter unit 223 are not specifically limited in this embodiment.

In the image sensor 200 shown in fig. 5, the filter array 220 includes 16

first filter units

221, 8

second filter units

222, and 8 third filter units 223, and the 32 filter units are arranged in 9 rows and 9 columns in a diamond shape.

Specifically, in the filter array 220 shown in fig. 5, the second target row is an even-numbered row and the second target column is an even-numbered column, and in the horizontal direction, the filter units in each row on the even-numbered row are all the first filter units 221 in the same color, while the filter units in each row on the non-target row, that is, the filter units in each row on the odd-numbered row are not in the same color, and specifically, the filter units in each row on the odd-numbered row include filter units of two colors, namely, the second filter unit 222 and the third filter unit 223.

In the vertical direction, the filter units in each column on the even-numbered columns are of the same color, wherein the 2 nd column and the 6 th column are both the second filter unit 222, and the 4 th column and the 8 th column are both the third filter unit 223, and meanwhile, in the embodiment of the present application, the non-target column, that is, the filter units in each column on the odd-numbered columns are also of the same color, and specifically, the filter units in each column on the odd-numbered columns are the first filter units 221.

Further, as shown in fig. 5, the second filter cells 222 are located at odd-numbered columns in the second target column, and the third filter cells 223 are located at even-numbered columns in the second target column. Specifically, in the 4-column second target column, the 1 st and 3 rd columns are the second filter unit 222, and the 2 nd and 4 th columns are the third filter unit 223.

It is understood that the filter array 220 shown in fig. 5 can be formed by arranging a plurality of array units 202, wherein each array unit 202 is square, and any side of the square array unit forms an angle of 45 ° with the horizontal direction or the vertical direction. And each array unit 202 includes 2 filter element groups, for a total of 8 filter elements, where the 8 filter elements include three colors. Specifically, one array sheetIn the cell 202, one filter unit group includes a first filter unit 221 and a second filter unit 222, and the other filter unit group includes a first filter unit 221 and a third filter unit 223. If each filter unit in the array unit 202 is a square with a side length of a, the distances between the centers of two adjacent array units in the horizontal direction and the vertical direction are both

Wherein a is a positive number.

Fig. 7 shows a schematic top view of another image sensor 200.

As shown in fig. 7, in the embodiment of the present application, the second target row is an even-numbered row and the second target column is an even-numbered column, and in the horizontal direction, the filter units in each row in the even-numbered row are all the first filter units 221 in the same color, while the filter units in each row in the non-target row, that is, the filter units in each row in the odd-numbered row are not in the same color, and specifically, the filter units in each row in the odd-numbered row include filter units of two colors, namely, the second filter unit 222 and the third filter unit 223.

In the vertical direction, the filter units in each column on even columns are of the same color, wherein the 2 nd column and the 4 th column are both the second filter unit 222, and the 6 th column and the 8 th column are both the third filter unit 223, and meanwhile, in the embodiment of the present application, the filter units in each column on non-target columns, that is, on odd columns, are all the first filter units 221.

Further, as shown in fig. 7, the second filter units 222 are located in the first two columns of the second target column, and the third filter units 223 are located in the second two columns of the second target column. Specifically, among the 4 columns of the second target column, the 1 st and 2 nd columns are the second filter unit 222, and the 3 rd and 4 th columns are the third filter unit 223.

In the embodiment shown in fig. 5 and 7, the number of filter units in each row and each column in the filter array 220 is equal, wherein half of the second target columns are the second filter units 222 and the other half of the second target columns are the third filter units 223, so that in the filter array 220 of the embodiment of the present application, the number of the second filter units 222 and the number of the third filter units 223 are equal, and the sum of the number of the second filter units 222 and the number of the third filter units 223 is equal to the number of the first filter units 221.

It is understood that, in the case that the filter array 220 includes filter units of three different colors, fig. 5 and 7 show two schematic diagrams in this case, wherein the number of rows and columns of the filter array 220 is not particularly limited in the embodiment of the present application, and the filter array 220 in the embodiment of the present application includes, but is not limited to, the two arrangements shown in fig. 5 and 7.

Alternatively, the second filter unit 222 may also be located at the 1 st and 4 th columns in the second target column in fig. 5 and 7, and the third filter unit 223 may also be located at the 2 nd and 3 rd columns in the second target column in fig. 5 and 7.

Alternatively, the second filter unit 222 may also be located at the 1 st column among the second target columns in fig. 5 and 7, and the third filter unit 223 may also be located at the 2 nd to 4 th columns among the second target columns in fig. 5 and 7.

In other words, in the embodiment of the present application, it is only necessary to satisfy the condition that the color of the filter unit in each column of the second target column is the same, and the second filter unit 222 and the third filter unit 223 are specifically located in which columns of the second target column, and the number relationship between the second filter unit 222 and the third filter unit 223 is not specifically limited in this embodiment of the present application.

In the embodiment shown in fig. 5 and 7, each row of the second target row is a first filter unit 221, a part of the second target column is a second filter unit 222, and another part of the second target column is a third filter unit 223.

Fig. 8 shows a schematic top view of another image sensor 200. In the filter array 220 shown in fig. 8, each column of the second target row is a first filter unit 221, and a part of the second target row is a second filter unit 222 and a part of the second target row is a third filter unit 223.

The filter array 220 shown in fig. 8 can be understood as a filter array 220 obtained by rotating the filter array 220 in fig. 5 by 90 ° clockwise with its center as a rotation center.

In the image sensor 200 shown in fig. 8, the second target rows are even rows and the second target columns are even columns, and in the vertical direction, the filter units in each column on the even columns are the same color and are all the first filter units 221, while the filter units in the non-target columns, i.e., the filter units in each column on the odd columns are not the same color, and specifically, the filter units in each column on the odd columns each include filter units of two colors, i.e., the second filter unit 222 and the third filter unit 223.

In the horizontal direction, the filter units in each row on the even rows are of the same color, wherein the 2 nd column and the 6 th row are the second filter unit 222, and the 4 th row and the 8 th row are the third filter unit 223, and meanwhile, in the embodiment of the present application, the filter units in each row on the non-target row, that is, the odd row, are also of the same color, and specifically, the filter units in each row on the odd row are the first filter units 221.

Further, as shown in fig. 8, the second filter unit 222 is located at an odd-numbered line in the second target line, and the third filter unit 223 is located at an even-numbered line in the second target line. Specifically, in the 4 rows of the second target column, the 1 st row and the 3 rd row are the second filter unit 222, and the 2 nd row and the 4 th row are the third filter unit 223.

It is understood that, in the embodiment of the present application, the filter array 220 may also be the filter array 220 that is obtained by rotating the filter array 220 in fig. 7 by 90 ° clockwise with its center as a rotation center.

It can be further understood that, in the embodiment of the present application, only the condition that the color of the filter unit in each row of the second target row is the same needs to be satisfied, where the second filter unit 222 and the third filter unit 223 are specifically located in which rows of the second target row, and the number relationship between the second filter unit 222 and the third filter unit 223 is also not specifically limited in the embodiment of the present application.

For convenience of description, hereinafter, a pixel unit under the first filter unit is referred to as a first pixel unit, and similarly, a pixel unit under the second filter unit is referred to as a second pixel unit, and a pixel unit under the third filter unit is referred to as a third pixel unit.

Taking the image sensor 200 in fig. 5 as an example, a process of pixel synthesis by a pixel unit in the pixel array 210 in a case where the filter array 220 includes filter units of three colors will be described.

Alternatively, for the image sensor 200 in fig. 5, pixel cells of even rows may be subjected to 2 × 1binning in the horizontal direction, and pixel cells of even columns may be subjected to 2 × 1binning in the vertical direction.

Taking the first filter unit 221 as a green filter unit, the second filter unit 222 as a red filter unit, and the third filter unit 223 as a blue filter unit in fig. 5 as an example, fig. 9 shows a schematic diagram of 2 × 1binning for the pixel array 210 in the image sensor 200 in fig. 5, where the first pixel unit corresponding to the first filter unit 221 is a green pixel unit (denoted by letter "G" in the figure), the second pixel unit corresponding to the second filter unit 222 is a red pixel unit (denoted by letter "R" in the figure), and the third pixel unit corresponding to the third filter unit 223 is a blue pixel unit (denoted by letter "B" in the figure).

As shown in (a) of fig. 9, in the horizontal direction, 2 × 1binning is performed on the green pixel cells of each row in the even-numbered rows, for example, in the 2 nd row, the green pixel values G23 and G25 of the two green pixel cells of the 3 rd column and the 5 th column are combined to form a new target green pixel value G '12, and the green pixel values G27 and G29 of the two green pixel cells of the 7 th column and the 9 th column are combined to form a new target green pixel value G' 14.

In the vertical direction, 2 × 1binning in the vertical direction is performed on the pixel unit of each of the even columns, specifically, 2 × 1binning in the vertical direction is performed on the red pixel units of the 2 nd and 6 th columns, and the blue filter units of the 4 th and 8 th columns.

For example, in the red pixel cell of column 2, the red pixel values R12 and R32 of the two red pixel cells of row 1 and row 3 are merged to form a new target red pixel value R '11, and the red pixel values R52 and R72 of the two red pixel cells of row 5 and row 7 are merged to form a new target red pixel value R' 31. In addition, in the blue pixel cell of the 4 th column, the blue pixel values B34 and B54 of the two blue pixel cells of the 3 rd row and the 5 th row are merged to form a new target blue pixel value B '22, and the blue pixel values B74 and B94 of the two blue pixel cells of the 7 th row and the 9 th row are merged to form a new target blue pixel value B' 42.

As shown in fig. 9 (b), the new plurality of target pixel values form a square array, and in the horizontal direction or the vertical direction, of two adjacent target pixel values, one is obtained by two adjacent pixel units located on the first target column through 2 × 1 binding in the vertical direction, and the other is obtained by two adjacent pixel units located on the first target row through 2 × 1 binding in the horizontal direction.

After the 2 × 1binning, the number of the new target pixel values is half of the number of the pixel values before the binning, that is, half of the number of the pixel units in the pixel array 210, and the image size formed by the new target pixel values is compressed by the same amount in both the horizontal direction and the vertical direction, for example, if the image size formed by the graph (a) in fig. 9 is H (horizontal direction) × V (vertical direction), the image size formed after the 2 × 1binning is H ' (horizontal direction) × V ' (vertical direction), where H '/V ═ H/V.

Therefore, according to the scheme of the embodiment of the application, the image sensor is provided with the filter array, the filter array comprises filter units with multiple colors, the collection of color images can be realized, meanwhile, the filter units are arranged in a diamond shape and in a diamond shape, the specific arrangement modes of the filter units with different colors in the filter array are designed, the requirements of 2 × 1 binding in the horizontal direction or 2 × 1 binding in the vertical direction of the corresponding pixel units are met, the calculation load of the image sensor is reduced, the signal-to-noise ratio and the output speed of the image sensor are improved, the resolution ratio and the size ratio of the image are considered, the color images are output, and the overall performance and the user experience of the image sensor are further improved.

Fig. 5 to 9 exemplarily show a related art of the image sensor 200 in a case where the filter array 220 includes filter units of three colors.

Optionally, the filter array 220 may further include filter units of 4 colors.

Fig. 10 shows a top view of another image sensor 200.

As shown in fig. 10, in the image sensor 200, the filter array 220 includes four color filter cells of a first filter cell 221, a second filter cell 222, a third filter cell 223, and a fourth filter cell 224.

The fourth filter unit 224 may be a visible light filter unit having a color different from that of the first filter unit 221, the second filter unit 222, and the third filter unit 223, for example, it may be a white filter unit or a gray filter unit, or a non-visible light filter unit, for example, a near-infrared filter unit. The color of the fourth filter unit 224 is not particularly limited in the embodiment of the present application.

In the embodiment shown in fig. 10, the filter array 220 includes 8

first filter units

221, 8

second filter units

222, 8

third filter units

223, and 8 fourth filter units 224, and the 32 filter units are arranged in 9 rows and 9 columns in a diamond shape.

Specifically, in the filter array 220 shown in fig. 10, the second target row is an even-numbered row and the second target column is an even-numbered column, and in the horizontal direction, a plurality of filter units in each row on the even-numbered row are the same color, wherein the 2 nd row and the 6 th row are the fourth filter unit 224, and the 4 th row and the 8 th row are the first filter unit 221. And the filter cells of each row on the non-target row, i.e., the odd-numbered row, are not of the same color, and specifically, the filter cells of each row on the odd-numbered row each include filter cells of two colors, i.e., the second filter cell 222 and the third filter cell 223.

In the vertical direction, the filter units in each column on the even-numbered columns are of the same color, wherein the 2 nd column and the 6 th column are both the second filter unit 222, and the 4 th column and the 8 th column are both the third filter unit 223, meanwhile, in the embodiment of the present application, the filter units in each column on the non-target column, that is, the odd-numbered columns are not of the same color, specifically, the filter units in each column on the odd-numbered columns include filter units of two colors, namely, the first filter unit 221 and the fourth filter unit 224.

Further, as shown in fig. 10, the fourth filter unit 224 is located at an odd-numbered line among the second target lines, and the first filter unit 221 is located at an even-numbered line among the second target lines. Specifically, in the 4 columns of the second target row, the 1 st row and the 3 rd row are the fourth filter unit 224, and the 2 nd row and the 4 th row are the first filter unit 221. In addition, the second filter cells 222 are located at odd-numbered columns in the second target column, and the third filter cells 223 are located at even-numbered columns in the second target column. Specifically, in the 4-column second target column, the 1 st and 3 rd columns are the second filter unit 222, and the 2 nd and 4 th columns are the third filter unit 223.

It is understood that the filter array 220 shown in fig. 10 can be formed by arranging a plurality of array units 202, wherein each array unit 202 is square, and any side of the square array unit forms an angle of 45 ° with the horizontal direction or the vertical direction. And each array unit 202 includes 2 filter unit groups, for a total of 8 filter units, where the 8 filter units include four colors. Specifically, one filter unit group includes a first filter unit 221 and a second filter unit 222, and the other filter unit group includes a third filter unit 223 and a fourth filter unit 224. If each filter unit in the array unit 202 is a square with a side length of a, the distances between the centers of two adjacent array units in the horizontal direction and the vertical direction are both

Wherein a is a positive number.

Fig. 11 shows a schematic top view of another image sensor 200.

As shown in fig. 11, in the embodiment of the present application, the second target row is an even-numbered row and the second target column is an even-numbered column, and in the horizontal direction, the filter units in each row on the even-numbered row are of the same color, wherein the rows 2 and 4 are the fourth filter unit 224, and the rows 6 and 8 are the first filter unit 221. And the filter cells of each row on the non-target row, i.e., the odd-numbered row, are not of the same color, and specifically, the filter cells of each row on the odd-numbered row each include filter cells of two colors, i.e., the second filter cell 222 and the third filter cell 223.

In the vertical direction, the filter units in each column on the even-numbered columns are of the same color, wherein the 2 nd column and the 4 th column are both the second filter unit 222, and the 6 th column and the 8 th column are both the third filter unit 223, and meanwhile, in the embodiment of the present application, the filter units in each column on the non-target column, that is, the odd-numbered columns, each include filter units of two colors, namely, the first filter unit 221 and the fourth filter unit 224.

Further, as shown in fig. 11, the fourth filter units 224 are positioned in the first two rows of the second target rows, the first filter units 221 are positioned in the second two rows of the second target rows, the second filter units 222 are positioned in the first two columns of the second target columns, and the third filter units 223 are positioned in the second two columns of the second target columns. Specifically, in the 4 rows of the second target row, the 1 st row and the 2 nd row are the fourth filter cells 224, the 3 rd row and the 4 th row are the first filter cells 223, and in the 4 columns of the second target column, the 1 st column and the 2 nd column are the second filter cells 222, and the 3 rd column and the 4 th column are the third filter cells 223.

In the embodiment shown in fig. 10 and 11, the number of filter units in each row and each column in the filter array 220 is equal, wherein half of the objects in the second object row are the first filter units 221, the other half of the objects are the fourth filter units 224, half of the objects in the second object column are the second filter units 222, and the other half of the objects are the third filter units 223, so that in the filter array 220 of the embodiment of the present application, the number of the first filter units 221 and the fourth filter units 224 is equal, the number of the second filter units 222 and the third filter units 223 is equal, and the sum of the number of the second filter units 222 and the third filter units 223 is equal to the sum of the number of the first filter units 221 and the fourth filter units 224.

It is understood that, in the case that the filter array 220 includes filter units of four different colors, fig. 10 and 11 show two schematic diagrams in this case, wherein the number of rows and columns of the filter array 220 is not particularly limited in the embodiment of the present application, and the filter array 220 in the embodiment of the present application includes, but is not limited to, the two arrangements shown in fig. 10 and 11.

Alternatively, the first filter cell 222 may be further located at the 1 st row and the 4 th row in the second target row in fig. 10 and 11, the fourth filter cell 224 may be further located at the 2 nd row and the 3 rd row in the second target row in fig. 10 and 11, the second filter cell 222 may be further located at the 1 st column and the 4 th column in the second target column in fig. 10 and 11, and the third filter cell 223 may be further located at the 2 nd column and the 3 rd column in the second target column in fig. 10 and 11.

Alternatively, the first filter unit 222 may be further located at row 1 in the second target row in fig. 10 and 11, the fourth filter unit 224 may be further located at rows 2 to 4 in the second target row in fig. 10 and 11, the second filter unit 222 may be further located at column 1 in the second target column in fig. 10 and 11, and the third filter unit 223 may be further located at columns 2 to 4 in the second target column in fig. 10 and 11.

In other words, in the embodiment of the present application, it is only necessary to satisfy the condition that the color of the filter unit in each row of the second target row is the same and the color of the filter unit in each column of the second target column is the same, where the first filter unit 221 and the fourth filter unit 224 are specifically located in which rows of the second target row, where the second filter unit 222 and the third filter unit 223 are specifically located in which columns of the second target column, and the number relationship among the first filter unit 221, the second filter unit 222, the third filter unit 223, and the fourth filter unit 224 is not specifically limited in this embodiment.

In the above-described embodiment shown in fig. 10 and 11, in the filter array 220, a part of the second target row is the first filter cell 221, another part of the second target row is the fourth filter cell 224, a part of the second target column is the second filter cell 222, and another part of the second target column is the third filter cell 223.

It is to be understood that, in some embodiments, in the filter array 220, a part of the second target row is the first filter cell 221, another part of the second target row is the fourth filter cell 224, a part of the second target row is the second filter cell 222, and another part of the second target row is the third filter cell 223.

That is, the filter array 220 in fig. 10 or 11 can be understood as a rotated filter array 220 rotated 90 ° clockwise with the center thereof as the rotation center.

The technical solution after rotation can be referred to the related description of fig. 10 and fig. 11, and is not described herein again.

Taking the image sensor 200 in fig. 10 as an example, a process of pixel synthesis by the pixel unit in the pixel array 210 in a case where the filter array 220 includes filter units of four colors will be described.

Alternatively, with the image sensor 200 in fig. 10, pixel cells of even rows may be subjected to 2 × 1binning in the horizontal direction, and pixel cells of even columns may be subjected to 2 × 1binning in the vertical direction.

Taking the first filter unit 221 as a green filter unit, the second filter unit 222 as a red filter unit, the third filter unit 223 as a blue filter unit, and the fourth filter unit 224 as a white filter unit in fig. 10 as an example, fig. 12 shows a schematic diagram of 2 × 1binning for the image sensor 200 in fig. 10, where the first pixel unit corresponding to the first filter unit 221 is a green pixel unit (denoted by letter "G" in the figure), the second pixel unit corresponding to the second filter unit 222 is a red pixel unit (denoted by letter "R" in the figure), the third pixel unit corresponding to the third filter unit 223 is a blue pixel unit (denoted by letter "B" in the figure), and the fourth pixel unit corresponding to the fourth filter unit 224 is a white pixel unit (denoted by letter "W" in the figure).

As shown in fig. 12 (a), horizontally, 2 × 1binning in the horizontal direction is performed on each row of pixel cells in the even-numbered rows, specifically, 2 × 1binning in the horizontal direction is performed on the white filter cells in the 2 nd and 6 th rows, and the green filter cells in the 4 th and 8 th rows.

For example, in row 2, the white pixel values W23 and W25 of the two white pixel cells of columns 3 and 5 are combined to form a new target white pixel value W '12, and the white pixel values W27 and W29 of the two white pixel cells of

columns

7 and 9 are combined to form a new target white pixel value W' 14. In row 4, the green pixel values G41 and G43 of the two green pixel cells of columns 1 and 3 are combined to form a new target green pixel value G '21, and the green pixel values G45 and G47 of the two green pixel cells of columns 5 and 7 are combined to form a new target green pixel value G' 23.

As shown in fig. 12 (b), the new target pixel values form a square array, and in the horizontal direction or the vertical direction, of two adjacent new target pixel values, one is obtained by vertically 2 × 1binning of two adjacent pixel units located on the first target column, and the other is obtained by horizontally 2 × 1binning of two adjacent pixel units located on the first target row.

After the 2 × 1binning, the number of the new target pixel values is half of the number of the pixel values before the binning, and the image size formed by the new target pixel values is compressed equally in both the horizontal direction and the vertical direction, for example, if the image size formed in (a) in fig. 12 is H (horizontal direction) × V (vertical direction), the image size formed after the 2 × 1binning is H '(horizontal direction) × V' (vertical direction), where H '/V' is H/V.

In the image sensor 200 shown in fig. 5 to 12, the filter array 220 includes filter units of three or four colors, and the filter array 220 forms a diamond array in which the number of rows and columns is equal, so that a plurality of target pixel values obtained after 2 × 1binning are arranged in a square array, and thus the processed image is a square image.

It is understood that in the embodiment of the present application, the filter array 220 may also include only two color filter units, for example, the second target column in fig. 5 or 7, i.e., the even columns include only one color filter unit, and the odd columns also include only one color filter unit; likewise, the second target row in fig. 5 or 7, i.e., the even rows, includes only one color filter cell, and the odd rows also include only one color filter cell. The light signals of the two colors passing through the filter units of the two colors are superposed to form white light.

It is also understood that the filter array 220 may also include only filter units of more than four colors, for example, in fig. 10 or fig. 11, the second target column may include filter units of more than three colors, or the second target row may also include filter units of more than three colors, and the color types of the filter units in the filter array 220 are not specifically limited in the embodiment of the present application.

In addition, in the embodiment of the present application, the filter array 220 may form a diamond array with unequal numbers of rows and columns, and the rectangular image is obtained after 2 × 1 binning. The size of the filter array 220 is not particularly limited in the embodiments of the present application.

In addition, in the embodiments of the application, the first target row, the first target column, the second target row, and the second target column are illustrated as even rows and even columns, in this application, the first target row, the first target column, the second target row, and the second target column may also be odd rows or odd columns, and specific related schemes thereof may be referred to the above description, which is not repeated herein.

Based on the pixel array 210 and the filter array 220, fig. 13 shows a schematic structural diagram of another image sensor 200.

Optionally, as shown in fig. 13, in the embodiment of the present application, the image sensor 200 further includes: a pixel synthesis circuit 230, wherein the pixel synthesis circuit 230 is connected to the pixel array 210, and is used for performing 2-in-1 pixel synthesis on half of the pixel units in the pixel array 210 in the horizontal direction, and performing 2-in-1 pixel synthesis on the other half of the pixel units in the vertical direction.

Specifically, the half of the pixel units are pixel units located in a first target row in the pixel array 210, and the other half of the pixel units are pixel units located in a first target column in the pixel array 210, where the first target row is an odd-numbered row or an even-numbered row, and the first target column is an odd-numbered column or an even-numbered column.

Alternatively, the pixel synthesis circuit 230 may have the same function as the pixel synthesis circuit 150 in fig. 1, and is specifically a summing circuit or an averaging circuit.

In some embodiments, the pixel synthesis circuit 230 is configured to sum or average pixel values of two pixel units adjacent to a common top corner on a first target column in the pixel array 210 for 2-in-1 pixel synthesis in the horizontal direction, and the pixel synthesis circuit 230 is further configured to sum or average pixel values of two pixel units adjacent to a common top corner on the first target column in the pixel array 210 for 2-in-1 pixel synthesis in the vertical direction.

Specifically, the process of the pixel synthesis performed by the pixel synthesis circuit 230 on the pixel units in the pixel array 210 can refer to the related description in fig. 9 and fig. 12, and is not repeated herein.

Alternatively, the pixel synthesis circuit 230 may be an analog circuit that receives pixel values of pixel units as analog signal values, for example, the pixel values are voltage signal values output by each pixel unit.

As shown in fig. 13, the image sensor 200 further includes: a row selection circuit 240, a column selection circuit 250, and a control circuit 260, wherein the control circuit 260 is connected to the row selection circuit 240 and the column selection circuit 250, and the pixel synthesis circuit 230 is connected to the pixel array 210 through the column selection circuit 250.

Specifically, the row selection circuit 240 is connected to each row of pixel units in the pixel array 210 through a plurality of row control lines, and two adjacent pixel units in each row of pixel units are connected at a common top corner. Optionally, the plurality of row control lines are all parallel to one of the edges of the square image sensor chip and arranged in a horizontal direction.

The column selection circuit 250 is connected to each column of pixel units in the pixel array 210 through a plurality of column control lines, and in each column of pixel units, two adjacent pixel units are also connected at the same common top corner. Optionally, the plurality of row control lines are all parallel to another side of the square image sensor chip and arranged in a vertical direction.

In some embodiments, the row selection circuit 240 and the column selection circuit 250 may have the same structure and function as the row selection circuit 120 and the column selection circuit 130 in fig. 1, the row selection circuit 240 is used for turning on and off each pixel unit in each row of pixel units, and the column selection circuit 250 is used for selecting a signal value output of each pixel unit in each column and transmitting the signal value of each pixel unit to the pixel synthesis circuit 230.

Alternatively, the control circuit 260 may be a control unit in the image sensor 200, and for example, it may be specifically a processor, configured to output a control signal to control operations of the row selection circuit 240 and the column selection circuit 250, so as to control a pixel value output of a specific pixel unit in the pixel array 210, and therefore, through the control circuit 260, the row selection circuit 240 and the column selection circuit 250, pixel values of adjacent pixel units in the first target row and the first target column may be output to the pixel synthesis circuit 230, so as to implement synthesis of the pixel values.

Optionally, as shown in fig. 13, the image sensor 200 may further include an analog-to-digital conversion circuit 270 and a signal processing circuit 280.

The specific functions of the analog-to-digital conversion circuit 270 and the signal processing circuit 280 can be referred to the description of the analog-to-digital conversion circuit 160 and the signal processing circuit 170 in fig. 1, and are not described herein again.

In addition to the image sensor 200 provided in the embodiments of the application, the present application also provides an electronic device, which may include the image sensor 200 in any of the embodiments described above.

The electronic device may be any electronic device having an image capturing function, for example, the electronic device may specifically be a mobile terminal such as a mobile phone and a computer, a shooting device such as a camera and a video camera, an Automatic Teller Machine (ATM), and the like.

In some embodiments, the signal processing circuit 280 is not located in the image sensor 200, but located in a processing unit of an electronic device in which the image sensor 200 is located, for example, if the electronic device is a mobile phone, the signal processing circuit 280 may be an image signal processing unit in a processor of the mobile phone, or the signal processing circuit 280 may also be a separate image signal processing chip in the mobile phone, and the embodiment of the present application does not limit the specific hardware form of the signal processing circuit 280.

It should be understood that the processor of the embodiments of the present application may be an integrated circuit chip having signal processing capabilities. In implementation, the steps of the above method embodiments may be performed by integrated logic circuits of hardware in a processor or instructions in the form of software. The Processor may be a general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic device, or discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like. The steps of the method disclosed in connection with the embodiments of the present application may be directly implemented by a hardware decoding processor, or implemented by a combination of hardware and software modules in the decoding processor. The software module may be located in ram, flash memory, rom, prom, or eprom, registers, etc. storage media as is well known in the art. The storage medium is located in a memory, and a processor reads information in the memory and completes the steps of the method in combination with hardware of the processor.

It will be appreciated that the image sensor of embodiments of the application may also include memory, which may be volatile memory or non-volatile memory, or may include both volatile and non-volatile memory. The non-volatile Memory may be a Read-Only Memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash Memory. Volatile Memory can be Random Access Memory (RAM), which acts as external cache Memory. By way of example, but not limitation, many forms of RAM are available, such as Static random access memory (Static RAM, SRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic random access memory (Synchronous DRAM, SDRAM), Double Data Rate Synchronous Dynamic random access memory (DDR SDRAM), Enhanced Synchronous SDRAM (ESDRAM), Synchronous link SDRAM (SLDRAM), and Direct Rambus RAM (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to comprise, without being limited to, these and any other suitable types of memory.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other ways. For example, the above-described apparatus embodiments are merely illustrative, and for example, the division of the units is only one logical division, and other divisions may be realized in practice, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the methods described in the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.

The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any person skilled in the art can easily conceive of the changes or substitutions within the technical scope of the present application, and shall be covered by the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

An image sensor, comprising:

the pixel array comprises a plurality of rhombic pixel units arranged in a rhombic shape, wherein half of the rhombic pixel units are positioned in a first target row in the pixel array, the other half of the rhombic pixel units are positioned in a first target column in the pixel array, the first target row is an odd-numbered row or an even-numbered row in the pixel array, and the first target column is an odd-numbered column or an even-numbered column in the pixel array;

the pixel values of the half of the pixel units are used for carrying out 2 in 1 pixel synthesis in the horizontal direction, and the pixel values of the other half of the pixel units are used for carrying out 2 in 1 pixel synthesis in the vertical direction so as to form a plurality of target pixel values of the image sensor.
The image sensor of claim 1, further comprising:

the filter array comprises a plurality of rhombic filter units which are arranged in a rhombic shape, and the rhombic filter units are correspondingly arranged above the rhombic pixel units one by one;

half of the diamond-shaped filter units are located in a second target row in the filter array, the color of the filter units in each row in the second target row is the same, the other half of the diamond-shaped filter units are located in a second target column in the filter array, the color of the filter units in each column in the second target column is the same, the second target row is an odd-numbered row or an even-numbered row in the filter array, and the second target column is an odd-numbered column or an even-numbered column in the filter array.
The image sensor of claim 2, wherein the filter array comprises a plurality of filter unit groups arranged in a diamond shape, and the filter unit groups comprise four adjacent filter units, two filter units adjacent to a common vertex in the four adjacent filter units have the same color, and two filter units adjacent to a common edge have different colors.
The image sensor according to claim 2 or 3, wherein the filter array includes filter cells of three colors, which are a first filter cell, a second filter cell, and a third filter cell, respectively.
The image sensor of claim 4, wherein each of the second target rows is a first filter unit, and the second target column is a second filter unit or a third filter unit; or,

each of the second target rows is a first filter unit, and the second target row is a second filter unit or a third filter unit.
The image sensor of claim 5, wherein half of the second target column is the second filter unit and the other half of the second target column is the third filter unit; or,

half of the lines in the second target row are the second filter unit, and the other half of the lines in the second target row are the third filter unit.
The image sensor of claim 6, wherein half of the second target columns and the other half of the second target columns are odd columns and even columns, respectively, of the second target columns; or,

half rows and the other half rows of the second target rows are odd rows and even rows of the second target rows, respectively.
The image sensor according to claim 6 or 7, wherein the number of the second filter units is equal to the number of the third filter units, and a sum of the number of the second filter units and the number of the third filter units is equal to the number of the first filter units.
The image sensor according to claim 2 or 3, wherein the filter array includes four color filter units, which are a first filter unit, a second filter unit, a third filter unit, and a fourth filter unit, respectively.
The image sensor of claim 9, wherein the target row of the filter array is a first filter unit or a fourth filter unit, and the target row of the filter array is a second filter unit or a third filter unit; or,

the target column of the filter array is a first filter unit or a fourth filter unit, and the target column of the filter array is a second filter unit or a third filter unit.
The image sensor of claim 10, wherein half of the rows in the second target row are the first filter units, the other half of the rows in the second target row are the fourth filter units, half of the columns in the second target column are the second filter units, and the other half of the columns in the second target column are the third filter units; or,

half row in the second target row is the first filter unit, the other half row in the second target row is the fourth filter unit, half row in the second target row is the second filter unit, the other half row in the second target row is the third filter unit.
The image sensor of claim 11, wherein half of the second target columns and the other half of the second target columns are odd columns and even columns, respectively, of the second target columns;

half of the second target rows and the other half of the second target rows are odd rows and even rows, respectively, of the second target rows.
The image sensor according to claim 11 or 12, wherein the number of the first filter units is equal to the number of the fourth filter units, and the number of the second filter units is equal to the number of the third filter units;

the sum of the number of the second filter units and the number of the third filter units is equal to the sum of the number of the first filter units and the number of the fourth filter units.
The image sensor according to any one of claims 4 to 13, wherein the first filter unit, the second filter unit and the third filter unit are configured to pass light signals of three colors respectively, and the wavelength bands of the light signals of the three colors cover a visible light wavelength band.
The image sensor of claim 14, wherein the first, second, and third filter cells are of three colors, red, green, blue, cyan, magenta, and yellow, respectively.
The image sensor of claim 15, wherein the first filter unit is a green filter unit, the second filter unit and the third filter unit are a red filter unit and a blue filter unit, respectively.
The image sensor of any of claims 12 to 16, wherein the fourth filter cell is a visible or non-visible filter cell.
The image sensor of claim 17, wherein the fourth filter unit is a white filter unit, a gray filter unit, or a near-infrared filter unit.
The image sensor of any one of claims 1 to 18, wherein the plurality of target pixel values is half the number of pixel cells in the pixel array and is used to form a square array of target pixel values;

in the target pixel value array, one of two adjacent target pixel values in the horizontal direction or the vertical direction is a target pixel value obtained through 2-in-1 pixel synthesis in the horizontal direction, and the other one is a target pixel value obtained through 2-in-1 pixel synthesis in the vertical direction.
The image sensor of claim 19, wherein one of two adjacent target pixel values in the target pixel value array in a horizontal direction or a vertical direction is a target pixel value obtained by summing or averaging pixel values of two pixel units adjacent to a common vertex on the first target row in the pixel array, and the other is a target pixel value obtained by summing or averaging pixel values of two pixel units adjacent to a common vertex on the first target column in the pixel array.
The image sensor of any one of claims 1 to 20, further comprising: a pixel synthesis circuit;

the pixel synthesis circuit is connected to the pixel array and used for performing 2-in-1 pixel synthesis on one half of the pixel units in the horizontal direction and performing 2-in-1 pixel synthesis on the other half of the pixel units in the vertical direction to form the plurality of target pixel values.
The image sensor of claim 21, further comprising: a row control circuit and a column control circuit;

the row control circuit is connected with a plurality of rows of pixel units in the pixel array through a plurality of row control lines, and each row of pixel units in the plurality of rows of pixel units are arranged at the same top angle;

the column control circuit is connected to a plurality of columns of pixel units in the pixel array through a plurality of column control lines, and each column of pixel units in the plurality of columns of pixel units are arranged at a common vertex angle;

the pixel synthesis circuit is connected to the pixel array through the column control circuit.
The image sensor of claim 21 or 22, further comprising:

the analog-to-digital conversion circuit and the signal processing circuit;

the analog-to-digital conversion circuit is connected to the pixel synthesis circuit and is used for converting the target pixel values output by the pixel synthesis circuit into digital pixel values;

the signal processing circuit is used for processing the plurality of digital pixel values to obtain a color image.
The image sensor of any one of claims 1 to 23, wherein the image sensor is a Complementary Metal Oxide Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.
An electronic device, comprising:

the image sensor of any one of claims 1 to 24.