WO2023098552A1

WO2023098552A1 - Image sensor, signal processing method and apparatus, camera module, and electronic device

Info

Publication number: WO2023098552A1
Application number: PCT/CN2022/133951
Authority: WO
Inventors: 林稔杰; 李沛德
Original assignee: 维沃移动通信有限公司
Priority date: 2021-11-30
Filing date: 2022-11-24
Publication date: 2023-06-08
Also published as: CN114071035A

Abstract

The present application relates to the technical field of image processing. Disclosed are an image sensor, a signal processing method and apparatus, a camera module and an electronic device. The image sensor comprises a pixel array, wherein the pixel array comprises a plurality of pixel groups, each of the pixel groups consists of nine pixel units, the nine pixel units comprise three first pixel units and six second pixel units, the first pixel units comprise at least one of the following: a white pixel unit and an infrared pixel unit, and the photosensitive wavelength band of the second pixel units is 400-700 nm.

Description

Image sensor, signal processing method, device, camera module and electronic equipment

Cross References to Related Applications

This application claims the priority of the Chinese patent application with the application number 202111454305.6 filed on November 30, 2021, and the title of the invention is "image sensor, signal processing method, device, camera module and electronic equipment", which is incorporated by reference in its entirety incorporated into this application.

technical field

The present application belongs to the technical field of image processing, and specifically relates to an image sensor, a signal processing method, a device, a camera module and electronic equipment.

Background technique

At present, the market competition of mobile terminal products is becoming more and more fierce, especially in the aspect of cameras of smart electronic devices, the competition is extremely fierce. In order to increase the sensitivity of the image sensor to achieve better night scene details and color effects, many technologies are adopted, including larger pixel size to increase the sensitivity of the image sensor itself, and larger lens aperture to increase the amount of light entering. High dynamic range chip reading technology is used for multi-frame synthesis to expand the dynamic range, dual lenses are used to enhance the details of objects and take into account the color of objects, and various new color filter arrangements to improve color reproduction accuracy and low light at the same time environmental sensitivity.

Although the corresponding products on the market have been improved in many aspects, the image sensor still cannot effectively improve the color reproduction, which eventually leads to deviations between the imaged color and the real scene, resulting in a decline in user experience.

Contents of the invention

The present application aims to provide an image sensor, a signal processing method, a device, a camera module, and an electronic device, which can increase the degree of color reproduction of the image sensor.

In order to solve the above-mentioned technical problems, the application is implemented as follows:

In the first aspect, the embodiment of the present application proposes an image sensor, the image sensor includes:

A pixel array, the pixel array includes a plurality of pixel groups, each of the pixel groups is composed of nine pixel units, and the nine pixel units include three first pixel units and six second pixel units, the The first pixel unit includes at least one of the following: a white pixel unit and an infrared pixel unit, and the photosensitive wavelength band of the second pixel unit is 400-700 nm.

In the second aspect, the embodiment of the present application provides an image signal processing method, which is applied to the above-mentioned image sensor, and the method includes:

Acquiring a first image signal of an Nth row in the pixel array, where the first image signal is a signal corresponding to a first pixel unit in the Nth row;

Acquire a second image signal of the N+M row in the pixel array, the second image signal includes a signal corresponding to the second pixel unit in the N+M row, where N is a positive integer, and M is taken as Value is 1 or 2;

Image processing is performed based on the first image signal and the second image signal.

In a third aspect, an embodiment of the present application provides an image signal processing device, which is applied to the above-mentioned image sensor, and the device includes:

A first image signal acquisition module, configured to acquire a first image signal of an Nth row in the pixel array, where the first image signal is a signal corresponding to a first pixel unit in the Nth row;

The second image signal acquisition module is configured to acquire a second image signal of the N+Mth row in the pixel array, the second image signal includes a signal corresponding to the second pixel unit in the N+Mth row, wherein , N is a positive integer, and the value of M is 1 or 2;

An image processing module, configured to perform image processing based on the first image signal and the second image signal.

In a fourth aspect, the embodiment of the present application provides a camera module, which includes the image sensor described in the first aspect.

In a fifth aspect, the embodiment of the present application provides an electronic device, which includes the camera module described in the second aspect.

In the embodiment of the present application, by arranging a plurality of pixel groups in the pixel array, and the pixel groups include pixel units of various photosensitive wavelength bands, the color reproduction degree of the image sensor can be effectively improved.

Additional aspects and advantages of the application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application.

Description of drawings

The above and/or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, wherein:

FIG. 1 is a partial schematic diagram of a pixel array in an embodiment of the present application;

Fig. 2 is one of the structural schematic diagrams of the pixel group in the embodiment of the present application;

Fig. 3 is the second schematic diagram of the structure of the pixel group in the embodiment of the present application;

Fig. 4 is a schematic diagram of reading the data of row N of the pixel array in Fig. 1;

Fig. 5 is a schematic diagram of reading the N+1 row data of the pixel array in Fig. 1;

Fig. 6 is a schematic diagram of reading the data of row N+2 of the pixel array in Fig. 1;

Fig. 7 is one of the structural schematic diagrams of two photosensitive elements arranged in the red pixel unit in the embodiment of the present application;

Fig. 8 is the second structural schematic diagram of setting two photosensitive elements in the red pixel unit in the embodiment of the present application;

Fig. 9 is the third schematic diagram of the structure of two photosensitive elements arranged in the red pixel unit in the embodiment of the present application;

Fig. 10 is the fourth schematic diagram of the structure of two photosensitive elements arranged in the red pixel unit in the embodiment of the present application;

Fig. 11 is a schematic structural diagram of four photosensitive elements arranged in a red pixel unit in the embodiment of the present application;

Fig. 12 is a schematic structural diagram of six photosensitive elements arranged in the red pixel unit in the embodiment of the present application;

13 is a schematic structural diagram of a camera module according to an embodiment of the present application;

FIG. 14 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application.

Detailed ways

Embodiments of the present application will be described in detail below, examples of which are shown in the drawings, in which the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below by referring to the figures are exemplary, are only for explaining the present application, and should not be construed as limiting the present application. Based on the embodiments in this application, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of this application.

The features of the terms "first" and "second" in the description and claims of the present application may explicitly or implicitly include one or more of these features. In the description of the present application, unless otherwise specified, "plurality" means two or more. In addition, "and/or" in the specification and claims means at least one of the connected objects, and the character "/" generally means that the related objects are an "or" relationship.

In the description of the present application, it should be understood that the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", " Back", "Left", "Right", "Vertical", "Horizontal", "Top", "Bottom", "Inner", "Outer", "Clockwise", "Counterclockwise", "Axial", The orientation or positional relationship indicated by "radial", "circumferential", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the application and simplifying the description, rather than indicating or implying the referred device or element Must be in a particular orientation, constructed, and operate in a particular orientation, and thus should not be construed as limiting of the application.

In the description of this application, it should be noted that unless otherwise specified and limited, the terms "installation", "connection", and "connection" should be understood in a broad sense, for example, it can be a fixed connection or a detachable connection. Connected, or integrally connected; it can be mechanically connected or electrically connected; it can be directly connected or indirectly connected through an intermediary, and it can be the internal communication of two components. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood in specific situations.

An image sensor, a signal processing method, a device, a camera module, and an electronic device according to an embodiment of the present application will be described below with reference to FIGS. 1-11 .

In the first aspect, as shown in FIGS. 1 to 3 , an embodiment of the present application provides an image sensor, including:

Pixel array 61, described pixel array 61 comprises a plurality of pixel groups, as shown in Figure 2 and Figure 3, each described pixel group is all made up of nine pixel units, and described nine pixel units comprise three first pixels A unit 611 and six second pixel units, the first pixel unit 611 includes at least one of the following: a white pixel unit, an infrared light pixel unit, and the photosensitive wavelength band of the second pixel unit is 400-700nm. Nine pixel units are arranged in each pixel group, and the nine pixel units are respectively composed of a first pixel unit 611 and a second pixel unit. Wherein the first pixel unit 611 is composed of a white pixel unit and/or a red pixel unit, and the second pixel unit can be any pixel unit with a photosensitive wavelength band between 400-700nm, for example, the second pixel unit can be composed of a cyan pixel unit (Cyan, C), magenta pixel unit (Magenta, M), yellow pixel unit (Yellow, Y), red pixel unit (Red, R), green pixel unit (Green, G) or blue pixel unit (Blue, B), etc. The pixel unit is composed. According to the different requirements of different manufacturers for image sensors, the type of pixel units included in the second pixel unit can be adjusted, and the pixel group composed of second pixel units with more colors can greatly improve the color reproduction of the image sensor to improve user experience.

Specifically, the first pixel unit 611 is a white pixel unit or an infrared pixel unit. The ultra-wide spectrum of the white pixel unit is suitable for enhancing the details of objects in low light. The infrared light pixels can detect the infrared light of the environment, or further enhance the edge details of objects in low light with the help of externally emitted infrared light sources. Provide a better night scene experience.

Optionally, six second pixel units are arranged around one first pixel unit 611 in the pixel array 61 . Six second pixel units are arranged around each first pixel unit 611 , and by adjusting the wavelength band of each pixel unit in the six first pixel units 611 , it is possible to further increase the color reproduction of the sensor.

Optionally, the photosensitive wavelength bands corresponding to the six second pixel units are all different. When the photosensitive wavelength bands of the six second pixel units surrounding the first pixel unit 611 are all different, in the data of the first pixel unit 611, the six second pixel units can complement the color accordingly, so as to ensure the Further improve the color reproduction.

Optionally, the shapes of the first pixel unit 611 and the second pixel unit are regular hexagons. The regular hexagonal pixel unit can ensure that the six second pixel units around the first pixel unit 611 can be in close contact with the first pixel unit 611 to avoid light leakage and also reduce the volume of the pixel array 61 .

Optionally, the second pixel unit includes at least one of the following: a cyan pixel unit, a magenta pixel unit, a yellow pixel unit, a red pixel unit, a green pixel unit, and a blue pixel unit. According to different usage scenarios of the image sensor, when making a pixel group, the six second pixel units around the first pixel unit 611 can be of the same photosensitive wavelength band, or the six second pixel units around the first pixel unit 611 can be The photosensitive wavelength bands are all different, and the photosensitive wavelength bands of the six second pixel units around the first pixel unit 611 can also be adjusted to be partly the same. Depending on the application scenarios of the image sensor, corresponding adjustments can be made during the manufacturing process. Wherein, if the six second pixel units around the first pixel unit 611 are of the same light-sensing wavelength band, then the color of the light-sensing wavelength band will have a very high degree of reproduction. If the photosensitive wavelength bands of the six second pixel units around the first pixel unit 611 are all different, then the pixel group is suitable for an environment with complex colors, and can have a better reproduction degree for various colors.

Optionally, as shown in FIG. 2 and FIG. 3, each pixel group includes one cyan pixel unit, one magenta pixel unit, one yellow pixel unit, one red pixel unit, One said green pixel unit and one said blue pixel unit. Cyan, magenta, and yellow are complementary colors of red, green, and blue. By setting a cyan pixel unit, a magenta pixel unit, a yellow pixel unit, and a red pixel unit on the peripheral side of the first pixel unit 611, respectively, A green pixel unit and a blue pixel unit can greatly increase the reproduction degree of the pixel group for different colors. When the pixel array 61 is composed of the pixel groups arranged in the above-mentioned arrangement, no matter what scene is taken, the image sensor 6 can guarantee a very high degree of color reproduction.

Optionally, the second pixel unit includes a first sub-pixel unit 612 and a second sub-pixel unit 613, and the first sub-pixel unit 612 includes at least one of the following: cyan pixel unit, magenta pixel unit, yellow pixel unit unit, the second sub-pixel unit 613 includes at least one of the following: a red pixel unit, a green pixel unit, and a blue pixel unit;

Three of the first pixel units 611 and three of the second sub-pixel units 613 are arranged around each of the first sub-pixel units 612, and three of the second sub-pixel units 613 are arranged around each of the second sub-pixel units 613. One pixel unit 611 and three first sub-pixel units 612 .

By distinguishing the specific pixel units contained in the first sub-pixel unit 612 and the second sub-pixel unit 613, the structure of each pixel group is further defined, and in the process of reading data, it can be ensured that the first When the sub-pixel unit 612 is used, the second sub-pixel unit 613 on the peripheral side can improve the degree of color reproduction. It is guaranteed that when shooting different scenes, it can have a high degree of color reproduction and can cope with environments with more complex colors.

Specifically, the first sub-pixel unit 612 includes a cyan pixel unit, a magenta pixel unit and a yellow pixel unit, and the second sub-pixel unit 613 includes a red pixel unit, a green pixel unit and a blue pixel unit. Taking the first pixel unit 611 as the center, the periphery of the first pixel unit 611 includes a red pixel unit, a green pixel unit and a blue pixel unit as well as a cyan pixel unit, a magenta pixel unit and a yellow pixel unit. As shown in FIG. 2 , the pixel units surrounding the first pixel unit 611 in the counterclockwise direction can be: red pixel unit, cyan pixel unit, green pixel unit, magenta pixel unit, blue pixel unit and yellow pixel unit. This sorting manner is only a partial arrangement among the numerous pixel units in the pixel array 61 . When reading data from the above-mentioned first pixel unit 611, the color information of the first sub-pixel unit 612 and the second sub-pixel unit 613 around the first pixel unit 611 can be collected, which is beneficial to color accuracy at the pixel level reduction while avoiding color decoding across large ranges.

Optionally, the pixel group further includes a microlens, and one microlens is correspondingly provided for each pixel unit.

Specifically, the image sensor also includes a microlens layer, and the microlens layer and the pixel array 61 are correspondingly arranged;

A plurality of microlenses are disposed on the microlens layer, and the plurality of microlenses correspond to the first pixel unit 611 , the first sub-pixel unit 612 and the second sub-pixel unit 613 one-to-one. A plurality of microlenses on the microlens layer protrude in the opposite direction away from the pixel array 61, and are used to gather the incident light to ensure that the light in the corresponding area can accurately pass through the corresponding pixel unit, and finally be captured by the pixel array 61. Received by the corresponding photosensitive element.

Optionally, each pixel unit includes one, two, four or six photosensitive elements. As shown in Figures 7 to 10, each pixel unit includes two photosensitive elements, as shown in Figure 11, each pixel unit includes four photosensitive elements, and as shown in Figure 12, each pixel unit includes six photosensitive elements . By adjusting the number of photosensitive elements set in each pixel unit, the phase detection structure corresponding to each pixel unit can be changed, thereby changing the multi-directional autofocus speed and focus sensitivity under different light intensities, and at the same time improving the focus accuracy .

Specifically, when two photosensitive elements are arranged in one pixel unit, the photosensitive elements can be arranged on both sides of the separation line in a partition manner as shown in FIGS. 7 to 10 , so that phase focusing can be performed from different directions. When there are four photosensitive elements in one pixel unit, the photosensitive elements can be arranged on both sides of the separation line in a partitioned manner as shown in Figure 11, so that phase focusing can be performed from more directions. When six photosensitive elements are set in one pixel unit, the photosensitive elements can be arranged on both sides of the dividing line in a partitioned manner as shown in Figure 12, which can realize phase focusing from more directions and further improve the accuracy of phase focusing.

In the second aspect, the embodiment of the present application discloses an image signal processing method, which is applied to the above-mentioned image sensor, and the method includes:

Acquiring a first image signal of the Nth row in the pixel array 61, where the first image signal is a signal corresponding to the first pixel unit 611 in the Nth row;

Acquiring the second image signal of the N+M row in the pixel array 61, the second image signal includes the signal corresponding to the second pixel unit in the N+M row, where N is a positive integer, and M The value is 1 or 2;

Wherein, the Nth row is the first image signal corresponding to the first pixel unit 611, the N+Mth row is the second image signal corresponding to the second pixel unit, and the image processing is performed based on the first image signal and the second image signal, which can Effectively analyze the color, greatly reducing the complexity of luminous flux correction and color correction processing. It can restore the color more easily and improve the image quality.

The second image signal includes an image signal corresponding to the second sub-pixel unit 613 in row N+1, and an image signal corresponding to the second sub-pixel unit 612 in row N+2.

Wherein, the pixel array 61 includes a first pixel unit 611, a first sub-pixel unit 612 and a second sub-pixel unit 613, and the first pixel unit 611, the first sub-pixel unit 612 and the second sub-pixel unit 613 are all of the same size. Regular hexagon, the first pixel unit 611, the first sub-pixel unit 612 and the second sub-pixel unit 613 are arranged in an array and each side is adjacent to each other; with the first pixel unit 611 as the center, the six first pixel units 611 Each side is connected with different pixel units, wherein the first sub-pixel unit 612 and the second sub-pixel unit 613 are arranged at intervals. One of the first sub-pixel unit 612 and the second sub-pixel unit 613 is a three primary color pixel unit, and the other is a complementary color pixel unit.

The Nth row of the pixel array 61 is the first pixel unit 611, the N+1th row of the pixel array 61 is the second sub-pixel unit 613, and the N+2th row of the pixel array 61 is the first sub-pixel unit 612 ; When the image sensor is used to take pictures, the exposure and reading of related data starts from the first row of the Nth row, and the data exposed and read out in the first row are all the first pixel unit 611 . The solid line part in Figure 4 to Figure 6 corresponds to the number of rows of read data, Figure 4 is a schematic diagram of reading the Nth row of data, Figure 5 is a schematic diagram of reading the N+1th data, and Figure 6 is the schematic diagram of reading the N+th row Schematic representation of 2 rows of data.

The plurality of second sub-pixel units 613 in row N+1 can pass through three different light wavelength bands, and the three adjacent second sub-pixel units 613 in row N+1 can pass through different light wave bands; wherein, The second sub-pixel unit 613 includes pixels with three color attributes, such as red pixel unit, green pixel unit and blue pixel unit respectively. The read out data are related data of red pixel unit, green pixel unit and blue pixel unit. At the same time, the colors of the adjacent second sub-pixel units 613 in row N+1 are different, for example, the adjacent second sub-pixel units 613 in row N+1 are red pixel units, green pixel units and blue pixel units in turn, The arrangement order of these three types of pixel units can be exchanged, but adjacent pixel units cannot have the same color.

The plurality of first sub-pixel units 612 in row N+2 can pass through three different light wavelength bands, and the light wave bands that can pass through the first sub-pixel unit 612 and the light wave band that can pass through the second sub-pixel unit 613 are the same. different, and the wavelength bands of light that the three adjacent first sub-pixel units 612 in the N+2 row can pass through are different. Wherein, the first sub-pixel unit 612 includes pixels of three colors, for example, a cyan pixel unit, a magenta pixel unit and a yellow pixel unit respectively. At the same time, the color attributes contained in the first sub-pixel unit 612 and the second sub-pixel unit 613 are all different, that is to say, the first sub-pixel unit 612 may include a cyan pixel unit, a magenta pixel unit and a yellow pixel unit, and the second The sub-pixel unit 613 may include a red pixel unit, a green pixel unit, and a blue pixel unit. When performing exposure and reading related data in row N+2, the exposed and read data are all cyan pixel units, magenta pixel units and yellow pixel units. At the same time, the colors of the adjacent first sub-pixel units 612 in row N+2 are different, for example, the adjacent first sub-pixel units 612 in row N+2 are cyan pixel units, magenta pixel units and yellow pixel units in sequence, The arrangement order of these three types of pixel units can be exchanged, but adjacent pixel units cannot have the same color.

Through the above arrangement, the color properties of adjacent pixel units of any pixel unit are different, and the wavelength bands of light that can pass through are also different. Specifically, in the area where the image sensor reads data, the numbers of the first pixel unit 611 , the first sub-pixel unit 612 and the second sub-pixel unit 613 respectively account for one-third of the total. The first sub-pixel unit 612 includes three kinds of pixel units that can pass through different light wavelength bands, so the pixel units in the first sub-pixel unit 612 that can pass through three different light wave bands account for one-third respectively. The second sub-pixel unit 613 includes three kinds of pixel units capable of passing through different light wavelength bands, so the pixel units in the second sub-pixel unit 613 that can pass through three different light wave bands account for one-third respectively. With the first pixel unit 611 as the center, different pixel units are connected to the six sides of the first pixel unit 611. For example, the outer peripheral side of the first pixel unit 611 is sequentially pasted with the first sub-pixel unit 612 and the second sub-pixel unit. The sub-pixel units 613 , the first sub-pixel units 612 and the second sub-pixel units 613 are three in number respectively. That is to say, the first sub-pixel unit 612, the second sub-pixel unit 613, the first sub-pixel unit 612, the second sub-pixel unit 613, and the first sub-pixel unit 612 are attached in sequence on the outer peripheral side of the first pixel unit 611. , the second sub-pixel unit 613 . Wherein, the first sub-pixel unit 612 and the second sub-pixel unit 613 are respectively pixel units capable of passing through different wavelength bands of light, for example, the first sub-pixel units 612 arranged at intervals around the first pixel unit 611 in the counterclockwise direction are sequentially The cyan pixel unit, the magenta pixel unit and the yellow pixel unit, and the second sub-pixel unit 613 arranged counterclockwise at intervals around the first pixel unit 611 are a red pixel unit, a green pixel unit and a blue pixel unit in sequence.

Taking the red pixel unit in the second sub-pixel unit 613 as the center, the six sides of the red pixel unit are connected with different pixel units, specifically including three first pixel units 611 and three first sub-pixel units 612, three One first pixel unit 611 and three first sub-pixel units 612 are alternately pasted on the peripheral side of the second sub-pixel unit 613 in sequence. The colors of the three first sub-pixel units 612 are different, and the pixel units pasted on the six sides of the second sub-pixel unit 613 can be cyan pixel units, white pixel units, yellow pixel units, white pixel units, and magenta pixel units. and white pixel cells.

With the yellow pixel unit in the first sub-pixel unit 612 as the center, different pixel units are connected to the six sides of the yellow pixel unit, specifically including three first pixel units 611 and three second sub-pixel units 613, three One first pixel unit 611 and three second sub-pixel units 613 are pasted alternately on the peripheral side of the first sub-pixel unit 612 in sequence. The colors of the three second sub-pixel units 613 are different, and the pixel units pasted on the six sides of the first sub-pixel unit 612 are red pixel units, white pixel units, blue pixel units, white pixel units, green pixel units and White pixel cells.

The above arrangement of the first pixel unit 611, the first sub-pixel unit 612 and the second sub-pixel unit 613 in the pixel array 61 is only a partial example, and the above-mentioned staggered arrangement can be realized and the colors read out in each row are the same The effect of the pixel unit is acceptable.

The first sub-pixel unit 612 is the complementary color of the second sub-pixel unit 613, and can refer to the color information of the first pixel unit 611 to improve the color reproduction accuracy. Perceived sensitivity to color in brightness. The spectrum of the first pixel unit 611 is more suitable for enhancing the details of the ground light, so that the color can be reproduced more accurately. Through the above-mentioned arrangement, the color distribution of adjacent pixels seen by the pixels with the same color attribute can be consistent, that is, the pixels with the same color attribute have the same optical crosstalk, which is beneficial to simplify subsequent color correction. At the same time, it is effectively avoided that the color output of the same line contains signals of different color attributes, which increases the difficulty of reading the timing of the circuit. Since the pixel units in the three adjacent rows are different, the corresponding color attributes are also different. Therefore, different exposure controls can be adopted for different color attributes to avoid overexposure and underexposure, and at the same time improve the distance between pixels of the same color attribute. object motion blur correction accuracy.

In the third aspect, the embodiment of the present application discloses an image signal processing device, which is applied to the above-mentioned image sensor, and the device includes:

A first image signal acquisition module, configured to acquire a first image signal of an Nth row in the pixel array 61, where the first image signal is a signal corresponding to the first pixel unit 611 in the Nth row;

The second image signal acquisition module is configured to acquire a second image signal of the N+M row in the pixel array 61, the second image signal includes a signal corresponding to the second pixel unit in the N+M row, Wherein, N is a positive integer, and the value of M is 1 or 2;

By processing the first image signal and the second image signal through the above signal processing device, the restoration degree of imaging colors can be greatly improved.

Optionally, the centerlines of the plurality of first pixel units 611 in the Nth row are located on the same straight line, and the centerlines of the first pixel units 611 in the Nth row and the second subpixel unit in the N+1th row The top edge of 613 is overlapped; the center lines of the multiple second sub-pixel units 613 in the N+1th row are located on the same straight line, and the center lines of the second sub-pixel units 613 in the N+1th row are aligned with the Nth The bottom edge of the first pixel unit 611 in the row is set to coincide, and the center line of the second sub-pixel unit 613 in the N+1th row is set to coincide with the top edge of the first sub-pixel unit 612 in the N+2th row; The centerlines of a plurality of first sub-pixel units 612 in row N+2 are located on the same straight line, and the centerlines of the first sub-pixel units 612 in row N+2 are aligned with the second sub-pixel units in row N+1. The bottom edges of the pixel units 613 are arranged overlapping. Taking the first pixel unit 611 as the color processing center as an example, after N lines are exposed, the distance between N+1 lines and N lines is only half the width of a pixel unit. At the same time, the color information of the first sub-pixel unit 612 and the second sub-pixel unit 613 around each first pixel unit 611 in N rows is also collected, which is beneficial to the restoration of color accuracy at the pixel level while avoiding spanning a large range color solution mosaic.

Optionally, the L-th column of the pixel array 61 is alternately formed by the first pixel unit 611, the first sub-pixel unit 612 and the second sub-pixel unit 613 in sequence, and the first sub-pixel unit 612 in the L-th column can pass through the light wavelength band Similarly, the second sub-pixel units 613 in the L-th column can pass through the same wavelength band of light. The L column contains the first pixel unit 611, the first sub-pixel unit 612 and the second sub-pixel unit 613. At the same time, the first sub-pixel unit 612 in the L column can only pass through the same light wavelength band, and the second sub-pixel unit in the L column The sub-pixel unit 613 can only pass through the same light wavelength band. At the same time, the pixel units in the L+1 column and the pixel units in the L column are dislocated.

In the fourth aspect, as shown in FIG. 13 , the embodiment of the present application provides a camera module, including the image sensor 6 described in any one of the above embodiments.

The camera module adopts the image sensor 6 in the present application so that the camera module can have the technical effect brought by the above-mentioned image sensor 6 (sensor).

Optionally, the camera module also includes:

A circuit board 7, the image sensor 6 is electrically connected to the circuit board 7;

The lens 2 is arranged on the side of the image sensor 6 away from the circuit board.

Wherein, the camera module can also include a protective film 1 arranged on the lens 2;

Cooperate with the lens 2 to drive the voice coil motor 3 of the lens 2;

The bracket 4 used for the camera module itself, the voice coil motor 3 is connected to the bracket 4;

The infrared filter 5 is arranged in the bracket 4 and coincides with the optical path of the lens 2;

Image sensor 6 is arranged in support 4, and is positioned at the optical path downstream of infrared filter 5;

The circuit board 7 is used for circuit connection of the camera module, and a connector 8 is provided at the end of the circuit board 7 to connect the circuit board 7 to the device where the camera module is located.

The camera module works as follows:

Step 101: adjust the lens 2.

In the structure of the camera module, the lens 2 (Lens) is used for focusing and focusing, and the lens 2 is wrapped and fixed by the voice coil motor 3, and the upper and lower ends of the voice coil motor 3 are connected to the shrapnel. When focusing, electrify the motor to generate electromagnetic force, which is finally balanced with the elastic force of the shrapnel. The position of the motor can be controlled by the magnitude of the electrification, and then the motor and lens 2 can be pushed to the focus position.

The voice coil motor 3 includes an upper cover, an upper spring piece, a lower spring piece, a casing, a motor, a magnet, a coil, a base, a mirror holder and terminals. The upper cover protects the motor. When the upper spring piece is deformed, a force is exerted on the motor, and the sum of the lower spring piece balances the electromagnetic force. The shell is the main frame of the fixed part of the motor, which has the function of magnetic conduction and improves the effective utilization rate of the magnet. When the coil passes current, an upward thrust is generated under the action of the magnetic field of the magnet, which drives other parts of the moving part to move together. The magnet generates a magnetic field, so that the energized coil generates electromagnetic force under the action of its magnetic field, so that the moving part carrier moves with the Lens. When the lower spring is deformed, it will exert force on the motor, and the sum of the upper spring will balance the electromagnetic force. The base is integrated with the motor, and the integrated structure is directly assembled with the flexible circuit board, or the base and the motor are separated, and the motor is matched with the mirror base. Terminal, the mobile phone supplies power to the motor through the terminal.

Step 102: The image sensor 6 receives light.

The scene light is projected to an infrared filter (IR filter), and the function of the infrared filter is to filter out unnecessary light projected to the image sensor 6, so as to prevent the image sensor 6 from generating false colors/ripples, so as to improve its effective resolution and color rendition . The light after passing through the infrared filter can be sensed by the image sensor 6 .

Step 103: image processing.

To the image that 102 steps obtain, by the processing of ISP, compression generates JPG image, can then

Used for saving or previewing.

In a fifth aspect, the embodiment of the present application provides an electronic device, including the camera module described in the above embodiment. Shooting through the electronic device can capture images with more accurate color reproduction effects through the image sensor 6 in the above-mentioned camera module. The electronic device has the technical effect achieved by the above-mentioned camera module.

FIG. 14 is a schematic diagram of a hardware structure of an electronic device implementing an embodiment of the present application.

The electronic device 1000 includes but not limited to: a radio frequency unit 1001, a network module 1002, an audio output unit 1003, an input unit 1004, a sensor 1005, a display unit 1006, a user input unit 1007, an interface unit 1008, a memory 1009, a processor 1010 and Camera modules and other components.

The camera module includes an image sensor, and the image sensor includes a pixel array, and the pixel array includes a plurality of pixel groups, and each of the pixel groups is composed of nine pixel units, and the nine pixel units include three A first pixel unit and six second pixel units, the first pixel unit includes at least one of the following: a white pixel unit and an infrared light pixel unit, and the photosensitive wavelength band of the second pixel unit is 400-700nm.

Optionally, six second pixel units are arranged around one first pixel unit in the pixel array.

Optionally, the photosensitive wavelength bands corresponding to the six second pixel units are all different.

Optionally, the shapes of the first pixel unit and the second pixel unit are regular hexagons.

Optionally, the second pixel unit includes at least one of the following: a cyan pixel unit, a magenta pixel unit, a yellow pixel unit, a red pixel unit, a green pixel unit, and a blue pixel unit.

Optionally, each pixel group includes one cyan pixel unit, one magenta pixel unit, one yellow pixel unit, one red pixel unit, one green pixel unit and one The blue pixel unit described above.

Optionally, the second pixel unit includes a first sub-pixel unit and a second sub-pixel unit, and the first sub-pixel unit includes at least one of the following: a cyan pixel unit, a magenta pixel unit, and a yellow pixel unit. The second sub-pixel unit includes at least one of the following: a red pixel unit, a green pixel unit, and a blue pixel unit;

Three of the first pixel units and three of the second sub-pixel units are arranged around each of the first sub-pixel units, and three of the first and second pixel units are arranged around each of the second sub-pixel units. Three of the first sub-pixel units.

Optionally, each pixel unit includes one, two, four or six photosensitive elements.

As shown in FIG. 14 , those skilled in the art can understand that the electronic device 1000 may also include a power supply (such as a battery) for supplying power to various components, and the power supply may be logically connected to the processor 1010 through the power management system, thereby realizing management through the power management system. Charging, discharging, and power management functions. The structure of the electronic device shown in FIG. 14 does not constitute a limitation to the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or arrange different components, and details will not be repeated here. .

Wherein, the processor 1010 is used for:

In one embodiment, in the pixel array of the image sensor, the second pixel unit includes a first sub-pixel unit and a second sub-pixel unit, and the first sub-pixel unit includes at least one of the following: a cyan pixel unit, A magenta pixel unit and a yellow pixel unit, the second sub-pixel unit includes at least one of the following: a red pixel unit, a green pixel unit, and a blue pixel unit, and the second image signal includes the second sub-pixel unit in the N+1th row An image signal corresponding to the sub-pixel unit, and an image signal corresponding to the first sub-pixel unit in row N+2.

In this embodiment, different exposure controls can be adopted for different color attributes to avoid overexposure and underexposure, and at the same time, the accuracy of object motion blur correction between pixels of the same color attribute can be improved.

It should be understood that, in the embodiment of the present application, the input unit 1004 may include a graphics processor (Graphics Processing Unit, GPU) 10041 and a microphone 10042, and the graphics processor 10041 is used for the image capture device ( Such as the image data of the still picture or video obtained by the camera) for processing. The display unit 1006 may include a display panel 10061, and the display panel 10061 may be configured in the form of a liquid crystal display, an organic light emitting diode, or the like. The user input unit 1007 includes at least one of a touch panel 10071 and other input devices 10072 . The touch panel 10071 is also called a touch screen. The touch panel 10071 may include two parts, a touch detection device and a touch controller. Other input devices 10072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, switch buttons, etc.), trackballs, mice, and joysticks, which will not be repeated here.

The memory 1009 can be used to store software programs as well as various data. The memory 1009 may mainly include a first storage area for storing programs or instructions and a second storage area for storing data, wherein the first storage area may store an operating system, an application program or instructions required by at least one function (such as a sound playing function, image playback function, etc.), etc. Furthermore, memory 1009 may include volatile memory or nonvolatile memory, or, memory 1009 may include both volatile and nonvolatile memory. Among them, the non-volatile memory can be read-only memory (Read-Only Memory, ROM), programmable read-only memory (Programmable ROM, PROM), erasable programmable read-only memory (Erasable PROM, EPROM), electronically programmable Erase Programmable Read-Only Memory (Electrically EPROM, EEPROM) or Flash. Volatile memory can be random access memory (Random Access Memory, RAM), static random access memory (Static RAM, SRAM), dynamic random access memory (Dynamic RAM, DRAM), synchronous dynamic random access memory (Synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (Double Data Rate SDRAM, DDRSDRAM), enhanced synchronous dynamic random access memory (Enhanced SDRAM, ESDRAM), synchronous connection dynamic random access memory (Synch link DRAM , SLDRAM) and Direct Memory Bus Random Access Memory (Direct Rambus RAM, DRRAM). The memory 1009 in the embodiment of the present application includes but is not limited to these and any other suitable types of memory.

The processor 1010 may include one or more processing units; optionally, the processor 1010 integrates an application processor and a modem processor, wherein the application processor mainly handles operations related to the operating system, user interface, and application programs, etc., Modem processors mainly process wireless communication signals, such as baseband processors. It can be understood that the foregoing modem processor may not be integrated into the processor 1010 .

It should be noted that, in this document, the term "comprising", "comprising" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article or apparatus comprising a set of elements includes not only those elements, It also includes other elements not expressly listed, or elements inherent in the process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not preclude the presence of additional identical elements in the process, method, article, or apparatus comprising that element. In addition, it should be pointed out that the scope of the methods and devices in the embodiments of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved. Functions are performed, for example, the described methods may be performed in an order different from that described, and various steps may also be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

Through the description of the above embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general-purpose hardware platform, and of course also by hardware, but in many cases the former is better implementation. Based on such an understanding, the technical solution of the present application can be embodied in the form of computer software products, which are stored in a storage medium (such as ROM/RAM, magnetic disk, etc.) , optical disc), including several instructions to enable a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present application.

In the description of this specification, references to the terms "one embodiment," "some embodiments," "exemplary embodiments," "example," "specific examples," or "some examples" are intended to mean that the implementation A specific feature, structure, material, or characteristic described by an embodiment or example is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

Although the embodiments of the present application have been shown and described, those skilled in the art can understand that various changes, modifications, substitutions and variations can be made to these embodiments without departing from the principle and spirit of the present application. The scope of the application is defined by the claims and their equivalents.

Claims

An image sensor comprising:

A pixel array, the pixel array includes a plurality of pixel groups, each of the pixel groups is composed of nine pixel units, and the nine pixel units include three first pixel units and six second pixel units, the The first pixel unit includes at least one of the following: a white pixel unit and an infrared pixel unit, and the photosensitive wavelength band of the second pixel unit is 400-700 nm.
The image sensor according to claim 1, wherein six of the second pixel units are arranged around one of the first pixel units in the pixel array.
The image sensor according to claim 2, wherein the photosensitive wavelength bands corresponding to the six second pixel units are all different.
The image sensor according to claim 1, wherein the shapes of the first pixel unit and the second pixel unit are regular hexagons.
The image sensor according to claim 1, wherein the second pixel unit comprises at least one of the following: a cyan pixel unit, a magenta pixel unit, a yellow pixel unit, a red pixel unit, a green pixel unit, and a blue pixel unit.
The image sensor according to claim 5, wherein each pixel group includes one cyan pixel unit, one magenta pixel unit, one yellow pixel unit, one red pixel unit, one The green pixel unit and one of the blue pixel units.
The image sensor according to claim 5, wherein the second pixel unit comprises a first sub-pixel unit and a second sub-pixel unit, and the first sub-pixel unit comprises at least one of the following: cyan pixel unit, magenta pixel unit A pixel unit, a yellow pixel unit, the second sub-pixel unit includes at least one of the following: a red pixel unit, a green pixel unit, and a blue pixel unit;

Three of the first pixel units and three of the second sub-pixel units are arranged around each of the first sub-pixel units, and three of the first and second pixel units are arranged around each of the second sub-pixel units. Three of the first sub-pixel units.
The image sensor according to claim 1, wherein the pixel group further includes a microlens, and one microlens is correspondingly provided for each pixel unit.
The image sensor according to claim 6, wherein each of the pixel units comprises one, two, four or six photosensitive elements.
An image signal processing method, applied to the image sensor according to any one of claims 1-9, said method comprising:

Acquiring a first image signal of an Nth row in the pixel array, where the first image signal is a signal corresponding to a first pixel unit in the Nth row;

Acquire a second image signal of the N+M row in the pixel array, the second image signal includes a signal corresponding to the second pixel unit in the N+M row, where N is a positive integer, and M is taken as Value is 1 or 2;

Image processing is performed based on the first image signal and the second image signal.
The image signal processing method according to claim 10, wherein the second pixel unit includes a first sub-pixel unit and a second sub-pixel unit, and the first sub-pixel unit includes at least one of the following: a cyan pixel unit, A magenta pixel unit and a yellow pixel unit, the second sub-pixel unit includes at least one of the following: a red pixel unit, a green pixel unit, and a blue pixel unit;

The second image signal includes an image signal corresponding to the second sub-pixel unit in the N+1th row, and an image signal corresponding to the first sub-pixel unit in the N+1th row.
An image signal processing device, applied to the image sensor according to any one of claims 1-9, said device comprising:

A first image signal acquisition module, configured to acquire a first image signal of an Nth row in the pixel array, where the first image signal is a signal corresponding to a first pixel unit in the Nth row;

The second image signal acquisition module is configured to acquire a second image signal of the N+Mth row in the pixel array, the second image signal includes a signal corresponding to the second pixel unit in the N+Mth row, wherein , N is a positive integer, and the value of M is 1 or 2;

An image processing module, configured to perform image processing based on the first image signal and the second image signal.
The image signal processing device according to claim 12, wherein the second pixel unit includes a first sub-pixel unit and a second sub-pixel unit, and the first sub-pixel unit includes at least one of the following: a cyan pixel unit, A magenta pixel unit and a yellow pixel unit, the second sub-pixel unit includes at least one of the following: a red pixel unit, a green pixel unit, and a blue pixel unit;

The second image signal includes an image signal corresponding to the second sub-pixel unit in the N+1th row, and an image signal corresponding to the first sub-pixel unit in the N+1th row.
A camera module, comprising the image sensor described in any one of claims 1-9.
The camera module according to claim 14, said camera module further comprising:

a circuit board, the image sensor is electrically connected to the circuit board;

A lens, the lens is arranged on a side of the image sensor away from the circuit board.
An electronic device comprising the camera module according to claim 14 or 15.
A readable storage medium, storing programs or instructions on the readable storage medium, and implementing the image signal processing method according to any one of claims 10-11 when the programs or instructions are executed by a processor.
A computer program product, the program product is stored in a non-transitory storage medium, and the program product is executed by at least one processor to implement the image signal processing method according to any one of claims 10-11.
An image signal processing device, comprising the device for executing the image signal processing method according to any one of claims 10-11.