WO2023109264A1 - 图像传感器、摄像模组、电子设备、图像生成方法和装置 - Google Patents

图像传感器、摄像模组、电子设备、图像生成方法和装置 Download PDF

Info

Publication number
WO2023109264A1
WO2023109264A1 PCT/CN2022/123995 CN2022123995W WO2023109264A1 WO 2023109264 A1 WO2023109264 A1 WO 2023109264A1 CN 2022123995 W CN2022123995 W CN 2022123995W WO 2023109264 A1 WO2023109264 A1 WO 2023109264A1
Authority
WO
WIPO (PCT)
Prior art keywords
color
filter
panchromatic
image
pixel value
Prior art date
Application number
PCT/CN2022/123995
Other languages
English (en)
French (fr)
Inventor
李小涛
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2023109264A1 publication Critical patent/WO2023109264A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/02—Constructional features of telephone sets
    • H04M1/0202—Portable telephone sets, e.g. cordless phones, mobile phones or bar type handsets
    • H04M1/026—Details of the structure or mounting of specific components
    • H04M1/0264—Details of the structure or mounting of specific components for a camera module assembly
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/51—Housings
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/57—Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/10—Circuitry of solid-state image sensors [SSIS]; Control thereof for transforming different wavelengths into image signals
    • H04N25/17—Colour separation based on photon absorption depth, e.g. full colour resolution obtained simultaneously at each pixel location

Definitions

  • the present application relates to the field of computer technology, in particular to an image sensor, camera module, electronic equipment, image generation method, device, computer-readable storage medium and computer program product.
  • An image sensor is arranged in the camera, and a color image is collected by the image sensor.
  • an optical filter array arranged in the form of a Bayer (Bayer) array is usually arranged in the image sensor, so that multiple pixels in the image sensor can receive light passing through the corresponding optical filter, thereby Generate pixel signals with different color channels to generate an image.
  • Various embodiments according to the present application provide an image sensor, a camera module, an electronic device, an image generating method, an apparatus, a computer-readable storage medium, and a computer program product.
  • An image sensor the image sensor includes a filter array and a pixel array
  • the filter array includes a minimum repeating unit, the minimum repeating unit includes a plurality of filter groups, each filter group includes a first A subunit and a second subunit; both the first subunit and the second subunit include a panchromatic filter and a color filter, and the color filters in the first subunit are arranged in On the diagonal in the first subunit where it is located, the color filter in the second subunit is arranged on the opposite diagonal in the second subunit where it is located; the panchromatic filter transmits The amount of light entering is greater than the amount of light passing through the color filter; each of the panchromatic filters includes N rows and N columns of panchromatic sub-filters, and each of the color filters includes N rows of N color sub-filters in rows, the color sub-filters in N rows and N columns are the same color as the color filter, and N is a positive integer; each pixel in the pixel array is connected to the color filter
  • a camera module the camera module includes a lens and the above-mentioned image sensor; the image sensor is used to receive light passing through the lens, and the pixels generate electrical signals according to the light.
  • An electronic device comprising:
  • a casing, the camera module is arranged on the casing.
  • the image sensor includes a filter array and a pixel array
  • the filter array includes a minimum repeating unit
  • the minimum repeating unit includes a plurality of filter groups
  • each filter group includes a first A subunit and a second subunit
  • both the first subunit and the second subunit include a panchromatic filter and a color filter
  • the color filters in the first subunit are arranged in the first subunit where they are located
  • the color filter in the second subunit is arranged on the anti-diagonal in the second sub-unit, which can make the color in the direction of the diagonal and in the direction of the anti-diagonal
  • the arrangement of the filters is more balanced, and the color channels have stronger resolution when imaging.
  • the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter, and more light can be obtained through the panchromatic filter during shooting, so that there is no need to adjust the shooting parameters, and it does not affect the shooting.
  • the clarity of imaging in low light is improved.
  • both stability and clarity can be taken into account, and the stability and clarity of imaging in dark light are both high.
  • An image generation method applied to an image sensor, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes a plurality of filter groups, each The filter group includes a first subunit and a second subunit; both of the first subunit and the second subunit include panchromatic filters and color filters, and the color filters in the first subunit
  • the optical filters are arranged on the diagonal in the first subunit, and the color filters in the second subunit are arranged on the anti-diagonal in the second subunit; the full The amount of light transmitted by the color filter is greater than the amount of light transmitted by the color filter; each of the panchromatic filters includes N rows and N columns of panchromatic sub-filters, and each of the color filters
  • the light sheet includes color sub-filters in N rows and N columns, the color sub-filters in N rows and N columns are the same color as the color filter, and N is a positive integer greater than or equal to 2; Each pixel in the pixel array corresponds to
  • the methods include:
  • the panchromatic pixel corresponding to each panchromatic sub-filter in the panchromatic filter is read out the full-resolution panchromatic pixel value, and each of the color filters is The color pixel corresponding to the color sub-filter reads out the full-resolution color pixel value;
  • a full resolution target image is generated based on each of said full resolution panchromatic pixel values and each of said full resolution color pixel values.
  • An image generating device applied to an image sensor, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes a plurality of filter groups, each The filter group includes a first subunit and a second subunit; both of the first subunit and the second subunit include panchromatic filters and color filters, and the color filters in the first subunit
  • the optical filters are arranged on the diagonal in the first subunit, and the color filters in the second subunit are arranged on the anti-diagonal in the second subunit; the full The amount of light transmitted by the color filter is greater than the amount of light transmitted by the color filter; each of the panchromatic filters includes N rows and N columns of panchromatic sub-filters, and each of the color filters
  • the light sheet includes color sub-filters in N rows and N columns, the color sub-filters in N rows and N columns are the same color as the color filter, and N is a positive integer greater than or equal to 2; Each pixel in the pixel array corresponds
  • the devices include:
  • the readout module is used to read out the full-resolution panchromatic pixel value from the panchromatic pixel corresponding to each panchromatic sub-filter in the panchromatic filter in the full-resolution mode, and read the full-resolution panchromatic pixel value of the color
  • the color pixel corresponding to each color sub-filter in the filter reads out the full-resolution color pixel value;
  • An image generating module configured to generate a full-resolution target image based on each of the full-resolution panchromatic pixel values and each of the full-resolution color pixel values.
  • An electronic device includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is made to execute the operation of the above-mentioned image generation method.
  • a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the operation of the above-mentioned method is realized.
  • a computer program product includes a computer program, and when the computer program is executed by a processor, the operations of the above-mentioned method are realized.
  • the panchromatic pixels corresponding to each panchromatic sub-filter in the panchromatic filter are read out in full Resolution panchromatic pixel value, and read out the full resolution color pixel value from the color pixel corresponding to each color sub-filter in the color filter; and the amount of light transmitted by the panchromatic filter is greater than that of the color filter
  • the amount of transmitted light can integrate the panchromatic channel information into the image to increase the overall light input, so that based on each full-resolution panchromatic pixel value and each full-resolution color pixel value, more information and detailed analysis can be generated. Sharper full-resolution target images.
  • the smallest repeating unit includes a plurality of filter groups, and each filter group includes a first subunit and a second subunit; both the first subunit and the second subunit include a panchromatic filter Light sheets and color filters, the color filters in the first subunit are arranged on the diagonal line of the first subunit, and the color filters in the second subunit are arranged on the first subunit
  • the arrangement of the color filters in the direction of the diagonal line and the direction of the anti-diagonal line can be more balanced, so that the color channel has a stronger color when generating a full-resolution target image resolution ability.
  • Fig. 1 is a schematic structural diagram of an electronic device in an embodiment.
  • FIG. 2 is an exploded schematic diagram of an image sensor in one embodiment.
  • Fig. 3 is a schematic diagram of connection between a pixel array and a readout circuit in one embodiment.
  • FIG. 4 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 1 in an embodiment.
  • FIG. 5 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 1 in another embodiment.
  • FIG. 6 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 1 in another embodiment.
  • FIG. 7 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 1 in another embodiment.
  • FIG. 8 is a schematic diagram of an arrangement of the smallest repeating unit in an optical filter array in which N is 2 in an embodiment.
  • FIG. 9 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 2 in another embodiment.
  • FIG. 10 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 2 in another embodiment.
  • FIG. 11 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 2 in another embodiment.
  • FIG. 12 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 3 in another embodiment.
  • Fig. 13 is a schematic flowchart of an image generation method in an embodiment.
  • Fig. 14 is a schematic flowchart of an image generation method in an embodiment.
  • Fig. 15 is a schematic diagram of a first target image in an embodiment.
  • Fig. 16 is a schematic flowchart of generating a second target image in an embodiment.
  • Figure 17 is a schematic diagram of a first color image and a first panchromatic image in one embodiment.
  • Fig. 18 is a schematic diagram of a second target image in one embodiment.
  • Fig. 19 is a schematic diagram of a second target image in another embodiment.
  • Fig. 20 is a schematic diagram of a second target image in another embodiment.
  • Fig. 21 is a schematic diagram of a second target image in another embodiment.
  • Fig. 22 is a schematic flowchart of generating a third target image in an embodiment.
  • Fig. 23 is a schematic diagram of a second full-color image, a dual-color second color image, and a single-color second color image in one embodiment.
  • Fig. 24 is a schematic diagram of a third target image in one embodiment.
  • Fig. 25 is a schematic diagram of a third target image in another embodiment.
  • Fig. 26 is a schematic flowchart of generating a fourth target image in an embodiment.
  • Fig. 27 is a schematic diagram of a fourth target image in an embodiment.
  • Fig. 28 is a structural block diagram of an image generating device in an embodiment.
  • Fig. 29 is a schematic diagram of the internal structure of an electronic device in one embodiment.
  • first, second and the like used in this application may be used to describe various elements herein, but these elements are not limited by these terms. These terms are only used to distinguish one element from another element.
  • a first object image could be termed a second object image, and, similarly, a second object image could be termed a first object image, without departing from the scope of the present application.
  • Both the first object image and the second object image are object images, but they are not the same object image.
  • the electronic device 100 includes a mobile phone, a tablet computer, a notebook computer, an ATM, a gate machine, a smart watch, a head-mounted display device, etc. It can be understood that the electronic device 100 can also be any other device with image processing functions.
  • the electronic device 100 includes a camera module 20 , a processor 30 and a casing 40 .
  • the camera module 20 and the processor 30 are both arranged in the housing 40, and the housing 40 can also be used to install functional modules such as power supply devices and communication devices of the electronic device 100, so that the housing 40 provides dustproof and drop-proof protection for the functional modules. , waterproof and other protection.
  • the camera module 20 may be a front camera module, a rear camera module, a side camera module, an under-screen camera module, etc., which is not limited here.
  • the camera module 20 includes a lens and an image sensor 21. When the camera module 20 captures an image, light passes through the lens and reaches the image sensor 21.
  • the image sensor 21 is used to convert the light signal irradiated on the image sensor 21 into an electrical signal.
  • the image sensor 21 includes a microlens array 22 , a filter array 23 , and a pixel array 24 .
  • the microlens array 22 includes a plurality of microlenses 221, the microlenses 221, the sub-filters in the filter array 23, and the pixels in the pixel array 24 are set in one-to-one correspondence, and the microlenses 221 are used to gather the incident light.
  • the collected light will pass through the corresponding sub-filter, and then projected onto the pixel, and be received by the corresponding pixel, and the pixel converts the received light into an electrical signal.
  • the filter array 23 includes a plurality of minimal repeating units 230 .
  • the minimum repeating unit 230 includes a plurality of filter groups, and each filter group includes a first subunit 233 and a second subunit 234; both the first subunit 233 and the second subunit 234 include a panchromatic filter 235 And the color filter 236, the color filter 236 in the first subunit 233 is arranged on the diagonal in the first subunit 233 where it is located, and the color filter 236 in the second subunit 234 is arranged On the anti-diagonal in the second subunit 234 where it is located; the amount of light transmitted by the panchromatic filter 235 is greater than the amount of light transmitted by the color filter 236; each panchromatic filter 235 includes N rows N Rows of panchromatic sub-filters, each color filter 236 includes N rows and N columns of color sub-filters, N rows and N columns of color sub-filters have the same color as the color filter, and N is positive integer.
  • N is 2.
  • the diagonal line may be a connecting line between the upper left corner and the lower right corner, or may be a connecting line between the upper right corner and the lower left corner.
  • the anti-diagonal line can be the line connecting the upper left corner and the lower right corner, or the connecting line between the upper right corner and the lower left corner. Diagonals and anti-diagonals are perpendicular to each other. That is to say, if the diagonal line is the line connecting the upper left corner and the lower right corner, then the anti-diagonal line is the line connecting the upper right corner and the lower left corner; if the diagonal line is the line connecting the upper right corner and the lower left corner, then the anti-diagonal line is is the line connecting the upper left and lower right corners.
  • the above image sensor includes a filter array 23 and a pixel array 24, the filter array 23 includes a minimum repeating unit 230, the minimum repeating unit 230 includes a plurality of filter groups, each filter group includes a first subunit 233 and The second subunit 234; the first subunit 233 and the second subunit 234 all include a panchromatic filter 235 and a color filter 236, and the color filter 236 in the first subunit 233 is arranged in the first subunit 233 where it is located.
  • the color filters in the second subunit 234 are arranged on the anti-diagonal in the second sub-unit 234, so that the direction of the diagonal and the anti-diagonal
  • the arrangement of the color filters 236 in the direction of the line is more balanced, and the color channels have stronger resolution ability during imaging.
  • the amount of light transmitted by the panchromatic filter 235 is greater than the amount of light transmitted by the color filter 236, and more light can be obtained through the panchromatic filter 236 during shooting, so that there is no need to adjust the shooting parameters, and there is no need to adjust the shooting parameters.
  • the clarity of imaging in dark light is improved. When imaging in dark light, both stability and clarity can be taken into account, and the stability and clarity of imaging in dark light are both high.
  • each row and each column of the minimum repeating unit 230 includes a color filter 236 of each color, that is, the color filters 236 of each color are arranged in a dispersed manner, which can improve color resolution and brightness variation.
  • the color filters 236 of each color are mixed and arranged, which also reduces the risk of false colors.
  • the panchromatic filter 235 and the color filter 236 are alternately arranged in each row and each column, that is, the panchromatic filter 235 is arranged in the smallest repeating unit, the first filter group or the second filter group. The number of 50% in the filter group can increase the amount of light entering each local area in the imaging.
  • the panchromatic filter 235 and the color filter 236 are alternately arranged in each row or column, which can improve the color resolution of each row or column of imaging, and make the color of imaging richer.
  • the minimum repeating unit 230 includes 2 first filter groups 231 and 2 second filter groups 232, and the 2 first filter groups 231 are on the diagonal of the minimum repeating unit 230 Arrangement, the two second filter groups 232 are arranged on the anti-diagonal line of the smallest repeating unit 230 . Two first filter sets 231 and two second filter sets 232 are arranged in matrix.
  • the two first filter groups are completely the same, and the two second filter groups are completely same.
  • each filter set includes only a panchromatic filter 235 and a color filter 236 of 2 colors.
  • the color filter 236 includes a first color filter, a second color filter and a third color filter.
  • the first color filter, the second color filter and the third color filter are filters of three different colors.
  • the colors of the first color filter, the second color filter and the third color filter can all be set as required.
  • the first color filter may be a red filter
  • the second color filter may be a green filter
  • the third color filter may be a blue filter.
  • the width of the wavelength band of the light transmitted by the color filter 236 is smaller than the width of the wavelength band of the light transmitted by the panchromatic filter 235, for example, the wavelength band of the transmitted light of the color filter 236 can correspond to the wavelength band of red light , the wavelength band of green light, or the wavelength band of blue light, the wavelength band of the light transmitted by the panchromatic filter 235 is the wavelength band of all visible light, that is to say, the color filter 236 only allows light of a specific color to transmit light, while the panchromatic filter The filter 235 can pass all colors of light.
  • the wavelength band of the light transmitted by the color filter 236 may also correspond to the wavelength band of other colored light, such as magenta light, purple light, cyan light, yellow light, etc., which is not limited here.
  • the color filter 236 in the first filter group 231 includes a first color filter and a second color filter
  • the color filter 236 in the second filter group 232 includes a second color filter filter and tertiary color filter.
  • each filter set includes 2 first subunits 233 and 2 second subunits 234; 2 first subunits 233 are arranged in the first row direction of the filter set , and two second subunits 234 are arranged in the second row direction of the filter group, and the first row direction and the second row direction are adjacently arranged; or, the two first subunits 233 are arranged in the filter group and the two second sub-units 234 are arranged in the second column direction of the filter group, and the first column direction and the second column direction are adjacently arranged.
  • the first filter group 231 includes 2 first subunits 233 and 2 second subunits 234, and the 2 first subunits 233 are in the first row direction of the first filter group 231 Arranged above, and the two second sub-units 234 are arranged in the direction of the second row of the filter group.
  • 2 first subunits 233 are arranged in the first column direction of the first filter group 231
  • 2 second subunits 234 are arranged in the second column direction of the filter group cloth.
  • first sub-unit 233 and the second sub-unit 234 containing filters of the same color in the same filter set are arranged on the diagonal of the same filter set.
  • the first subunit 233 and the second subunit 234 comprising the same color filter in the same filter group are arranged on the diagonal of the same filter group, and the color filter in the first subunit 236 is arranged on the diagonal line in the first subunit, and the color filter 236 in the second subunit is arranged on the anti-diagonal line in the second subunit. Then, they are arranged on the same
  • the color filter 236 in the first subunit 233 on the diagonal of the filter group is arranged on the diagonal of the first subunit where it is located, and the color filter 236 is arranged on the diagonal of the same filter group.
  • the color filters 236 in the second sub-unit 234 are arranged on the anti-diagonal of the second sub-unit, which can improve the distribution balance of the color filters 236 on the diagonal and anti-diagonal.
  • first subunit 233 and the second subunit 234 both include a panchromatic filter 235 and a color filter 236, so the first subunit 233 and the second subunit comprising the same color filter 234, that is, the first subunit 233 and the second subunit 234 including the color filter 236 of the same color.
  • the first filter group 231 includes 2 first subunits 233 and 2 second subunits 234, and the 2 first subunits 233 are arranged in the row direction of the first filter group 231 cloth, and two second subunits 234 are arranged in the row direction of the first filter group 231, and the first subunit 233 and the second subunit 234 containing the same color filter are arranged in the first filter group On the diagonal of the light sheet group 231 .
  • the color filter 236 includes a first color filter, a second color filter and a third color filter; one of the two first subunits 233 in the first filter group 231 One subunit 233 includes a first color filter, and the other first subunit 233 includes a second color filter; one of the two second subunits 234 in the first filter group 231 is a second subunit 234 includes a first color filter, and another second subunit 234 includes a second color filter.
  • the color filter 236 includes a first color filter, a second color filter and a third color filter; one of the two first subunits 233 in the second filter group 232 One subunit 233 includes a second color filter, and the other first subunit 233 includes a third color filter; one of the two second subunits 234 in the second filter group 232 is a second subunit 234 includes a second color filter, and another second subunit 234 includes a third color filter.
  • the pixel array 24 includes a plurality of pixels, and the pixels of the pixel array 24 are arranged corresponding to the sub-filters of the filter array 23 .
  • the pixel array 24 is configured to receive light passing through the filter array 23 to generate electrical signals.
  • the pixel array 24 is configured to receive the light passing through the filter array 23 to generate an electrical signal, which means that the pixel array 24 is used to detect a scene of a given set of subjects passing through the filter array 23
  • the light is photoelectrically converted to generate an electrical signal.
  • the light rays of the scene for a given set of subjects are used to generate image data.
  • the subject is a building
  • the scene of a given set of subjects refers to the scene where the building is located, which may also contain other objects.
  • the pixel array 24 includes a plurality of minimal repeating units 240, the smallest repeating unit 240 further includes a plurality of panchromatic pixels 241 and a plurality of color pixels 242 of different colors, and each row and each column includes each color
  • Each panchromatic pixel 242 corresponds to a sub-filter in the panchromatic filter 235, and the panchromatic pixel 242 receives light passing through the corresponding sub-filter to generate an electrical signal.
  • Each color pixel 242 corresponds to a sub-filter of the color filter 236, and the color pixel 242 receives light passing through the corresponding sub-filter to generate an electrical signal.
  • the readout circuit 25 is electrically connected to the pixel array 24 for controlling the exposure of the pixel array 24 and reading and outputting the pixel values of the pixels.
  • the readout circuit 25 includes a vertical drive unit 251 , a control unit 252 , a column processing unit 253 and a horizontal drive unit 254 .
  • the vertical driving unit 251 includes a shift register and an address decoder.
  • the vertical driving unit 251 includes readout scanning and reset scanning functions.
  • the control unit 252 configures timing signals according to the operation mode, and uses various timing signals to control the vertical driving unit 251 , the column processing unit 253 and the horizontal driving unit 254 to work together.
  • the column processing unit 253 may have an analog-to-digital (A/D) conversion function for converting an analog pixel signal into a digital format.
  • the horizontal driving unit 254 includes a shift register and an address decoder. The horizontal driving unit 254 sequentially scans the pixel array 24 column by column.
  • each panchromatic filter 235 includes 1 row and 1 column of panchromatic sub-filters, and each color filter 236 includes 1 row and 1 column of color sub-filters, that is to say, Each panchromatic sub-filter is a panchromatic filter 235 and each color sub-filter is a color filter 236 .
  • N is 1, and the minimum repeating unit includes 8 rows and 8 columns of 64 optical filters, arranged in the following manner:
  • w represents the panchromatic filter 235
  • a, b and c all represent the color filter 236 .
  • N is 1, and the minimum repeating unit includes 8 rows and 8 columns of 64 filters, arranged in the following manner:
  • w represents the panchromatic filter 235
  • a, b and c all represent the color filter 236 .
  • N is 1, and the minimum repeating unit includes 8 rows and 8 columns of 64 optical filters, arranged in the following manner:
  • w represents the panchromatic filter 235
  • a, b and c all represent the color filter 236 .
  • N is 1, and the minimum repeating unit includes 64 filters in 8 rows and 8 columns, arranged in the following manner:
  • w represents the panchromatic filter 235
  • a, b and c all represent the color filter 236 .
  • w can be a white filter, a a red filter, b a green filter, c a blue filter, or for example a a magenta filter, b a cyan filter, c It is a yellow filter or the like, and is not limited here.
  • the electronic device adjusts the minimum repeating unit to obtain a new minimum repeating unit.
  • the electronic device rotates the minimum repeating unit of FIG. 4 counterclockwise by 90 degrees to obtain the minimum repeating unit of FIG. 7; the a color filter and the c color filter in the minimum repeating unit of FIG. 4 are exchanged,
  • the minimum repeating unit in Figure 5 can be obtained;
  • the minimum repeating unit in Figure 6 can be obtained by exchanging the a color filter and the c color filter in the minimum repeating unit in Figure 4, and then rotating 90 degrees counterclockwise.
  • the b color filter takes the minimum area of 4 by 4 as the arrangement period, and the sampling rate of the b color filter in the diagonal and anti-diagonal directions is Consistent, the arrangement is more balanced, so the b channel has stronger resolution, so the b channel has stronger resolution in the horizontal direction, vertical direction and oblique direction. Similarly, the sampling rate of the a color filter and the c color filter in the diagonal and anti-diagonal directions of the local area is the same, and the arrangement is more balanced, so the a channel and the c channel have stronger resolution capabilities . Since the arrangement of a color filter, b color filter, c color filter and w panchromatic filter is more scattered, the filter array has the property of full arrangement in the horizontal and vertical directions, and takes into account Resolution in diagonal and diagonal directions.
  • N is 2, and the minimum repeating unit includes 256 sub-filters in 16 rows and 16 columns, and the arrangement is as follows:
  • w represents a panchromatic sub-filter
  • a, b, and c all represent color sub-filters.
  • N is 2, and the minimum repeating unit includes 256 sub-filters in 16 rows and 16 columns, and the arrangement is as follows:
  • w represents a panchromatic sub-filter
  • a, b, and c all represent color sub-filters.
  • N is 2, and the minimum repeating unit includes 256 sub-filters in 16 rows and 16 columns, and the arrangement is as follows:
  • w represents a panchromatic sub-filter
  • a, b, and c all represent color sub-filters.
  • N is 2, and the minimum repeating unit includes 256 sub-filters in 16 rows and 16 columns, and the arrangement is as follows:
  • w represents a panchromatic sub-filter
  • a, b, and c all represent color sub-filters.
  • w can be a white sub-filter, a a red sub-filter, b a green sub-filter, c a blue sub-filter, or for example a a magenta sub-filter and b a cyan Sub-filters, c is a yellow sub-filter, etc., which is not limited here.
  • the electronic device adjusts the minimum repeating unit to obtain a new minimum repeating unit.
  • the electronic device rotates the minimum repeating unit in FIG. 8 by 90 degrees counterclockwise to obtain the minimum repeating unit in FIG. 11 ; the a color filter and the c color filter in the minimum repeating unit in FIG. 8 are exchanged,
  • the minimum repeating unit in Figure 9 can be obtained;
  • the minimum repeating unit in Figure 10 can be obtained by exchanging the positions of a color filter and c color filter in the minimum repeating unit in Figure 8, and then rotating 90 degrees counterclockwise.
  • the b-color sub-filters take the minimum area of 8 by 8 as the arrangement period, and the sampling of the b-color sub-filters in the diagonal and anti-diagonal directions The rate is consistent, and the arrangement is more balanced, so the b channel has stronger resolution, so the b channel has stronger resolution in the horizontal direction, vertical direction and oblique direction.
  • the sampling rate of the a-color sub-filter and c-color sub-filter in the diagonal and anti-diagonal directions of the local area is the same, and the arrangement is more balanced, so the a channel and the c channel have stronger resolution.
  • the filter array Since the arrangement of a-color sub-filters, b-color sub-filters, c-color sub-filters and w panchromatic sub-filters is more dispersed, the filter array has the property of full arrangement in the horizontal and vertical directions , and take into account the resolution of the diagonal and oblique diagonal directions.
  • N can also be other positive integers such as 3, 4, or 5, and the arrangement method is similar to that of N being 1 or 2, which will not be repeated here.
  • Figure 12 shows the minimum repeating unit where N is 3 in an embodiment.
  • a camera module is also provided.
  • the camera module includes a lens and the above-mentioned image sensor; the image sensor is used to receive light passing through the lens, and the pixels generate electrical signals according to the light.
  • an electronic device comprising the above-mentioned camera module; and a casing, on which the camera module is arranged.
  • an image generation method is provided, which is applied to an image sensor.
  • the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes a plurality of filter groups, each A filter group includes a first subunit and a second subunit; both the first subunit and the second subunit include panchromatic filters and color filters, and the color filters in the first subunit are arranged On the diagonal in the first subunit where it is located, the color filters in the second subunit are arranged on the opposite diagonal in the second subunit where it is located; the amount of incoming light transmitted by the panchromatic filter Greater than the amount of light transmitted by the color filter; each panchromatic filter includes N rows and N columns of panchromatic sub-filters, and each color filter includes N rows and N columns of color sub-filters, N The color sub-filters in rows and N columns have the same color as the color filters, and N is a positive integer greater than or equal to 2; each pixel in the pixel array is set corresponding
  • the image generation method includes:
  • Operation 1302 in the full-resolution mode, read out the full-resolution panchromatic pixel value from the panchromatic pixel corresponding to each panchromatic sub-filter in the panchromatic filter, and read out the full-resolution panchromatic pixel value of each color sub-filter in the color filter The color pixel corresponding to the sub-filter reads out the full-resolution color pixel value.
  • Full resolution mode is a mode in which each sub-filter is read out as one pixel.
  • the color filter has a narrower spectral response than the panchromatic filter, so the amount of light transmitted by the panchromatic filter is greater than that of the color filter, that is, the light transmitted by the color filter
  • the band width is smaller than that of the light transmitted by the panchromatic filter, and the panchromatic filter transmits more light, and the corresponding panchromatic pixels obtained through the panchromatic filter have a higher signal-to-noise ratio.
  • Color pixels contain more information and can resolve more texture details.
  • the signal-to-noise ratio refers to the ratio between the normal signal and the noise signal. The higher the signal-to-noise ratio of a pixel, the higher the proportion of normal signals contained in the pixel, and the more information can be analyzed from the pixel.
  • the color pixels 242 may be G (Green, green) pixels, R (Red, red) pixels, B (Blue, blue) pixels, etc., but are not limited thereto.
  • the light transmitted by the panchromatic sub-filter in the panchromatic filter is projected onto the corresponding panchromatic pixel 241, and the panchromatic pixel 241 receives the light passing through the panchromatic sub-filter to Generate electrical signals.
  • the light transmitted by the color sub-filter in the color filter is projected onto the corresponding color pixel 242 , and the light of the color pixel 242 passes through the corresponding color sub-filter to generate an electrical signal.
  • Each panchromatic filter includes N rows and N columns of panchromatic sub-filters, and each panchromatic filter corresponds to N rows and N columns of panchromatic pixels 241 .
  • Each color filter includes N rows and N columns of color sub-filters of the same color, and each color filter corresponds to N rows and N columns of color pixels 242 .
  • N is a positive integer greater than or equal to 2.
  • N may also be 1, that is, each panchromatic filter corresponds to one panchromatic pixel 241 , and each color filter corresponds to one color pixel 242 .
  • a full resolution target image is generated based on the respective full resolution panchromatic pixel values and the respective full resolution color pixel values.
  • the electronic device can read pixel values from each full-resolution panchromatic pixel value and each full-resolution color pixel value according to a preset pixel reading method to generate a full-resolution target image.
  • the preset pixel reading mode is a preset pixel reading mode.
  • the panchromatic pixel corresponding to each panchromatic sub-filter in the panchromatic filter is read out the full resolution panchromatic pixel value, and the panchromatic pixel value of each panchromatic sub-filter in the color filter is The color pixel corresponding to each color sub-filter reads out the full-resolution color pixel value; and the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter, and the panchromatic channel information can be integrated into the image.
  • the overall light input amount is increased, so that based on each full-resolution panchromatic pixel value and each full-resolution color pixel value, a full-resolution target image with more information and clearer detail analysis can be generated.
  • the smallest repeating unit includes a plurality of filter groups, and each filter group includes a first subunit and a second subunit; both the first subunit and the second subunit include a panchromatic filter Light sheets and color filters, the color filters in the first subunit are arranged on the diagonal line of the first subunit, and the color filters in the second subunit are arranged on the first subunit
  • the arrangement of the color filters in the direction of the diagonal line and the direction of the anti-diagonal line can be more balanced, so that the color channel has a stronger color when generating a full-resolution target image resolution ability.
  • the above method further includes:
  • Operation 1402 in the first resolution mode, combine the panchromatic pixels corresponding to each panchromatic sub-filter in each panchromatic filter to read out the first panchromatic pixel value, and convert each color filter to The color pixels corresponding to each of the color sub-filters are combined to read out the first color pixel value; the resolution corresponding to the first resolution mode is smaller than the resolution corresponding to the full resolution mode.
  • the first resolution mode refers to a first-level pixel binning readout mode in which resolution, power consumption, signal-to-noise ratio, and frame rate are relatively balanced.
  • the first resolution mode may be a default mode for shooting images and videos.
  • the first resolution mode is used to respond to the shooting instruction.
  • the light transmitted by the panchromatic sub-filter in the panchromatic filter is projected onto the corresponding panchromatic pixel 241, and the panchromatic pixel 241 receives the light passing through the panchromatic sub-filter to generate electrical signals.
  • the light transmitted by the color sub-filter in the color filter is projected onto the corresponding color pixel 242 , and the light of the color pixel 242 passes through the corresponding color sub-filter to generate an electrical signal.
  • Combined readout refers to summing the pixel values of multiple pixels, or calculating the average value of the pixel values of multiple pixels.
  • the panchromatic pixels 241 corresponding to each panchromatic sub-filter are averaged, and the average value is read out as the first panchromatic pixel value.
  • the panchromatic pixel 241 corresponding to each panchromatic sub-filter is added, and the sum obtained by the addition is read out as the first panchromatic pixel value .
  • the electronic device may also use other methods to combine the panchromatic pixels 241 corresponding to the panchromatic sub-filters to read out the first panchromatic pixel value, which is not limited here.
  • the color pixels 242 corresponding to each color sub-filter are averaged, and the average value is read out as the first color pixel value.
  • the color pixels corresponding to each color sub-filter are summed, and the sum obtained by the addition is read out as the first color pixel value.
  • the electronic device may also combine the color pixels 242 corresponding to the color sub-filters in other ways to read out the first color pixel value, which is not limited here.
  • the manner of combining and reading out the first panchromatic pixel values may be the same or different.
  • the way of combining and reading out the first color pixel values may be the same or different.
  • the panchromatic filter and the color filter the way of combining and reading out the first panchromatic pixel value and the first color pixel value may be the same or different.
  • Operation 1404 generating a first target image based on each first panchromatic pixel value and each first color pixel value.
  • the electronic device can read pixel values from each first panchromatic pixel value and each first color pixel value according to a preset pixel reading method to generate a first target image.
  • the preset pixel reading mode is a preset pixel reading mode. Taking the arrangement of the smallest repeating unit of the filter array 23 as shown in FIG. 8 as an example, the generated first target image is shown in FIG. 15 .
  • w represents the first panchromatic pixel value
  • a, b and c represent the first color pixel values of three different colors.
  • the panchromatic pixels 241 corresponding to the panchromatic sub-filters in each panchromatic filter are combined to read out the first panchromatic pixel value, and each The color pixels 242 corresponding to each color sub-filter in the color filter are combined to read out the first color pixel value, and the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter.
  • the color channel information is fused into the image to increase the overall amount of incoming light, so that based on each first panchromatic pixel value and each first color pixel value, a first target image with more information and clearer detail analysis can be generated.
  • the above method further includes:
  • Operation 1602 in the second resolution mode, combine the multiple first panchromatic pixel values corresponding to each subunit in the first target image to read out the second panchromatic pixel value, and based on each second panchromatic pixel value The value generates the first full-color image; the resolution corresponding to the second resolution mode is smaller than the resolution corresponding to the first resolution mode, and the subunits include the first subunit and the second subunit.
  • the second resolution mode refers to the mode used in the scene where the resolution requirement is lower than that of the first resolution mode. out mode.
  • the resolution and power consumption corresponding to the second resolution mode are smaller than the resolution and power consumption corresponding to the first resolution mode.
  • the signal-to-noise ratio and frame rate corresponding to the second resolution mode are greater than the signal-to-noise ratio and frame rate corresponding to the first resolution mode.
  • the second resolution mode may specifically be a preview mode during image capture, a preview mode during video capture, or a scene with lower resolution requirements such as image capture and video capture under night scenes, but is not limited thereto.
  • the preview modes of video shooting include 1080p video preview, application video preview, etc.
  • the shooting instruction is preview shooting.
  • the second resolution mode is triggered.
  • the electronic device detects whether the current environment is a night scene, and if the current environment is a night scene, triggers the second resolution mode.
  • the readout mode corresponding to the second resolution mode is triggered.
  • the electronic device combines the multiple first panchromatic pixel values corresponding to each subunit in the first target image to read out the second panchromatic pixel value, and according to the preset pixel reading method, from each second panchromatic pixel value Color Pixel Values Pixel values are read to generate a first full-color image.
  • each pixel value in the first target image is obtained by combining the pixels corresponding to the sub-filters in each filter in the filter array in the first resolution mode, then the first target
  • Each pixel value in the image corresponds to each filter in the filter array, and also corresponds to multiple sub-filters in each filter.
  • the electronic device determines a plurality of pixel values of each subunit in the first target image, acquires a plurality of first panchromatic pixels from the plurality of pixel values and reads out a second panchromatic pixel value, And acquiring a plurality of first color pixel values of the same color from the plurality of pixel values and reading out a second color pixel value.
  • combining readout may include one of methods such as averaging, summing, or weighted averaging, which is not limited here.
  • Operation 1604 combining multiple first color pixel values corresponding to the same color in each subunit in the first target image to read out second color pixel values, and generating a first color image based on each second color pixel value.
  • the electronic device combines multiple first color pixel values corresponding to each subunit in the first target image to read out the second color pixel value, and calculates the second color pixel value according to the preset pixel value.
  • pixel values are read from respective second color pixel values to generate a first color image.
  • combining readout may include one of methods such as averaging, summing, or weighted averaging, which is not limited here.
  • the generated first color image is shown as 1702 in FIG. 17
  • the generated first full-color image is shown as 1704 .
  • w represents the second panchromatic pixel value
  • a, b, and c represent the second color pixel values of three different colors.
  • Operation 1606 based on the first panchromatic image and the first color image, generate a second target image.
  • the electronic device may generate a second target image based on the first full-color image and the first color image, and transmit the second target image.
  • the electronic device arranges each row of second panchromatic pixel values in the first full-color image alternately with each row of second color pixel values in the first color image to generate a second target image; or the first full-color image
  • Each column of second panchromatic pixel values in the first color image is alternately arranged with each column of second color pixel values in the first color image to generate a second target image.
  • Figure 18 and Figure 19 are schematic diagrams of the second target image obtained by alternately arranging each row of second panchromatic pixel values in the first full-color image and each row of second color pixel values in the first color image.
  • FIG. 20 and FIG. 21 are the second target image obtained by alternately arranging each column of second panchromatic pixel values in the first full-color image and each column of second color pixel values in the first color image.
  • the electronic device may also combine pixel values at the same position in the first full-color image and the first color image to obtain combined pixel values at corresponding positions, and form the second target image based on the combined pixel values.
  • the merging may adopt one of methods such as averaging, weighted averaging, or adding and summing.
  • the electronic device may also use other methods to generate the second target image, which is not limited here.
  • each first panchromatic pixel values corresponding to each subunit in the first target image are combined to read out the second panchromatic pixel value
  • the first target In the image multiple first color pixel values of the same color corresponding to each subunit are combined to read out the second color pixel value, and each different color pixel 242 can be mixed and arranged so that the generated second target image
  • the distribution of each second color pixel such as RGB pixel is more uniform, and the image quality is higher.
  • the resolution and image size of the obtained second target image are further reduced, and the panchromatic pixel 241 has a higher signal-to-noise ratio, and the frame rate of the image is high, so that the power consumption of the secondary pixel combination output is lower , Better signal-to-noise ratio image processing effect.
  • the panchromatic pixel 241 has a higher signal-to-noise ratio, and the frame rate of the image is high, so that the power consumption of the secondary pixel combination output is lower , Better signal-to-noise ratio image processing effect.
  • the second resolution mode there are full-arranged panchromatic pixels without interpolation, which improves the overall resolution.
  • the color pixels of each color such as the pixels of the first color and the pixels of the third color, are more dispersed and balanced in the diagonal direction or anti-diagonal direction.
  • the above method further includes: in the second resolution mode, corresponding to each panchromatic sub-filter of the multiple panchromatic filters in each first subunit or second subunit Combining panchromatic pixels to read out the fifth panchromatic pixel value, and generating a third panchromatic image based on each fifth panchromatic pixel value; filtering multiple colors of the same color in each first subunit or second subunit The color pixels corresponding to each color sub-filter of the sheet are combined to read out the fifth color pixel value, and a third color image is generated based on each fifth color pixel value; based on the third panchromatic image and the third color image, a fifth color image is generated target image.
  • the combined readout method may be one of averaging, weighted averaging, or addition.
  • the electronic device may generate a fifth target image based on the third panchromatic image and the third color image, and then transmit the fifth target image.
  • generating the fifth target image includes: combining the fifth panchromatic pixel value of each row in the third panchromatic image with the fifth color pixel value of each row in the third color image Arranging alternately to generate the fifth target image; or alternately arranging each column of fifth panchromatic pixel values in the third panchromatic image with each column of fifth color pixel values in the third color image to generate the fifth target image.
  • the electronic device may also combine pixel values at the same position in the third panchromatic image and the third color image to obtain combined pixel values at corresponding positions, and form a fifth target image based on the combined pixel values.
  • the combined readout may adopt one of methods such as averaging, weighted averaging, or adding and summing.
  • the electronic device may also use other methods to generate the fifth target image, which is not limited here.
  • the panchromatic pixels corresponding to the panchromatic sub-filters of the plurality of panchromatic filters in each first subunit or second subunit are combined and read out The fifth panchromatic pixel value, and combining the color pixels corresponding to each color sub-filter of a plurality of color filters of the same color in each first subunit or second subunit to read out the fifth color pixel value,
  • the third panchromatic image and the third color image can be generated more quickly, so that the fifth target image can be generated more quickly.
  • the above-mentioned embodiment can mix and arrange different color pixels, so that the distribution of each fourth color pixel value, such as RGB pixels, in the generated fifth target image is more uniform, and the image quality is higher. Moreover, the resolution and image size of the obtained fifth target image are further reduced, and the panchromatic pixel 241 has a higher signal-to-noise ratio, and the frame rate of the image is high, so that the power consumption of the secondary pixel combination output is lower , Better signal-to-noise ratio image processing effect.
  • the above method further includes:
  • Operation 2202 in the third resolution mode, combine the multiple second panchromatic pixel values corresponding to the same filter set in the first panchromatic image to read out the third panchromatic pixel value, and based on each third panchromatic pixel value
  • the panchromatic pixel values generate a second panchromatic image; the resolution corresponding to the third resolution mode is smaller than the resolution corresponding to the second resolution mode.
  • the third resolution mode refers to the mode used in the scene where the resolution requirement is lower than that of the second resolution mode. It is a three-level pixel binning read with low resolution, low power consumption, high SNR and high frame rate out mode.
  • the resolution and power consumption corresponding to the third resolution mode are smaller than the resolution and power consumption corresponding to the second resolution mode.
  • the signal-to-noise ratio and frame rate corresponding to the third resolution mode are greater than the signal-to-noise ratio and frame rate corresponding to the second resolution mode.
  • the third resolution mode may specifically be a preview mode during image capture, a preview mode during video capture, or a night scene mode for image capture and video capture under night scenes, etc., but is not limited thereto.
  • the preview modes of video shooting include 720p video preview, application video preview, etc.
  • the electronic device determines each filter set from the filter array, acquires multiple second panchromatic pixel values in the first panchromatic image obtained by each filter set, and converts the multiple second panchromatic pixel values The values are combined to read out the third panchromatic pixel value.
  • the electronic device reads pixel values from each third panchromatic pixel value to form a second panchromatic image according to a preset pixel reading method.
  • Operation 2204 combine and read out the third color pixel value of the first color and the third color pixel value of the second color corresponding to a plurality of second color pixel values of the same color in the same filter group in the first color image value and the third color pixel value of the third color, and based on the third color pixel value of the first color, the third color pixel value of the second color and the third color pixel value of the third color, generate the second color pixel value of the double color A color image and a single-color second-color image; a two-color second-color image includes third-color pixel values of the first color and third-color pixel values of the third color, and a single-color second-color image includes second-color The third color pixel value.
  • the third color pixel value of the first color is the pixel value read from the pixel corresponding to the first color filter
  • the third color pixel value of the second color is the pixel value read from the pixel corresponding to the second color filter
  • the third color pixel value of the third color is the pixel value read out from the pixel corresponding to the third color filter.
  • the electronic device determines each filter set from the filter array, and obtains multiple second color pixel values of the same color in the first color image obtained by each filter set, and second color pixel values of the same color
  • the color pixel value includes a second color pixel value of the first color, a second color pixel value of the second color, and a second color pixel value of the third color, and a plurality of second color pixel values of the first color are combined to read out the first color pixel value.
  • For the third panchromatic pixel value of one color combine multiple second color pixel values of the second color to read out the third panchromatic pixel value of the second color, combine and read multiple second color pixel values for the third color
  • the two-color second color image includes third color pixel values of the first color and third color pixel values of the third color.
  • the single-color second-color image includes third-color pixel values of the second color.
  • the third color pixel values of the first color are arranged on the diagonal of the two-color second color image
  • the third color pixel values of the third color are arranged on the anti-diagonal of the two-color second color image. cloth.
  • the electronic device converts the first panchromatic image 1704 into multiple second panchromatic images corresponding to the same filter set. Combine the pixel values to read out the third panchromatic pixel value, and generate the second panchromatic image 2302 in FIG.
  • each third panchromatic pixel value is smaller than that corresponding to the second resolution mode Resolution: combine the second color pixel values corresponding to multiple same colors in the same filter group in the first color image 1702 to read out the third color pixel value of the first color and the third color of the second color pixel value and the third color pixel value of the third color, and based on the third color pixel value of the first color, the third color pixel value of the second color and the third color pixel value of the third color, generate the A second color image 2304 of two colors and a second color image 2306 of one color.
  • w represents a third panchromatic pixel value
  • a, b, and c represent third color pixel values of three different colors.
  • a third target image is generated based on the second panchromatic image, the two-color second color image, and the single-color second color image.
  • the electronic device can The second color image is used to generate a third target image, and then the third target image is transmitted.
  • the electronic device converts the third panchromatic pixel value of each row in the second panchromatic image, the third color pixel value of each row in the dual-color second color image, and the third color pixel value of each row in the single-color second color image
  • the pixel values are arranged alternately to generate the second target image; or the third panchromatic pixel value of each column in the second panchromatic image, the third color pixel value of each column in the double-color second color image and the second color pixel value of single color
  • Each column of the third color pixel values in the color image is arranged alternately to generate the second target image.
  • FIG. 25 is In another embodiment, each column of the third panchromatic pixel value in the second panchromatic image, each column of the third color pixel value in the dual-color second color image, and each column of the third color pixel value in the single-color second color image The third target image generated by arranging pixel values alternately.
  • the third panchromatic pixel value in the second panchromatic image with the same coordinates, the third color pixel value in the double-color second color image, and the third color pixel value in the single-color second color image are arranged in The order is not limited.
  • the electronic device may also use other methods to generate the third target image, which is not limited here.
  • a plurality of second panchromatic pixel values corresponding to the same filter set in the first panchromatic image are combined to read out the third panchromatic pixel value, and the second panchromatic pixel value is read out.
  • the third color pixel value of the first color, the third color pixel value of the second color and the third color pixel value are combined and read out.
  • the value of the third color pixel can mix and arrange different color pixels, so that the distribution of the third color pixels in the generated third target image, such as RGB pixels, is more uniform and the image quality is higher.
  • the resolution and image size of the obtained third target image are further reduced, and the panchromatic pixels have a higher signal-to-noise ratio, and the frame rate of the image is high, so that the power consumption of the three-level pixel combination output is lower, Image processing effect with better signal-to-noise ratio.
  • the third target image includes fully arranged panchromatic pixels, which can improve the overall resolution. At the same time, in the third resolution mode, there is no need to merge pixels of the same color across cycles, and no interpolation is required, which improves the overall resolution.
  • the color pixels of each color such as the pixel values of the first color and the pixel values of the third color, are more dispersed and balanced on the diagonal or anti-diagonal.
  • the full arrangement means that each coordinate has the pixel, and interpolation estimation is not required.
  • the above method further includes: in the third resolution mode, combining the panchromatic pixels corresponding to the panchromatic sub-filters of the plurality of panchromatic filters in each filter set Read out the sixth panchromatic pixel value, and generate a fourth panchromatic image based on each sixth panchromatic pixel value; each color sub-filter of a plurality of color filters of the same color in each filter set Corresponding color pixels combine and read out the sixth color pixel value of the first color, the sixth color pixel value of the second color and the sixth color pixel value of the third color, and based on the sixth color pixel value of the first color, the sixth color pixel value of the first color, The sixth color pixel value of the two colors and the sixth color pixel value of the third color generate the fourth color image of two colors and the fourth color image of single color; the fourth color image of two colors includes the sixth color image of the first color The color pixel value and the sixth color pixel value of the third color, the fourth color image of the single color includes the sixth color pixel value of
  • the combined readout method may be one of averaging, weighted averaging, or addition.
  • the electronic device can The fourth color image is used to generate a sixth target image, and then the sixth target image is transmitted.
  • the sixth target image is generated, including: the sixth panchromatic pixel value of each row in the fourth panchromatic image, the double The sixth color pixel value of each row in the fourth color image of the color and the sixth color pixel value of each row in the fourth color image of the single color are arranged alternately to generate the sixth target image; or each column in the fourth panchromatic image The sixth panchromatic pixel value, the sixth color pixel value in each column of the dual-color fourth color image, and the sixth color pixel value in each column of the single-color fourth color image are arranged alternately to generate a sixth target image.
  • the electronic device may also use other methods to generate the sixth target image, which is not limited here.
  • the panchromatic pixels corresponding to the panchromatic sub-filters of the plurality of panchromatic filters in each filter set are combined to read out the fifth panchromatic Pixel values, so that the fourth panchromatic image can be generated more quickly;
  • the color pixels corresponding to the color sub-filters of multiple color filters of the same color in each filter group are combined to read out the color pixels of the first color
  • the fifth color pixel value, the fifth color pixel value of the second color, and the fifth color pixel value of the third color so that a fourth color image of two colors and a fourth color image of single color can be generated more quickly.
  • the above-mentioned embodiment can mix and arrange different color pixels, so that the distribution of each sixth color pixel value, such as RGB pixels, in the generated sixth target image is more uniform, and the image quality is higher. Moreover, the resolution and image size of the obtained sixth target image are further reduced, and the panchromatic pixel 241 has a higher signal-to-noise ratio, and the frame rate of the image is high, thereby achieving lower power consumption of the three-level pixel combination output , Better signal-to-noise ratio image processing effect.
  • each sixth color pixel value such as RGB pixels
  • the above method further includes:
  • Operation 2602 in the fourth resolution mode, combine the third panchromatic pixel values in the second panchromatic image to read out the fourth panchromatic pixel value; the resolution corresponding to the fourth resolution mode is smaller than that of the third resolution mode corresponding resolution.
  • the fourth resolution mode refers to the mode used in the scene where the resolution requirement is lower than that of the third resolution mode. out mode.
  • the resolution and power consumption corresponding to the fourth resolution mode are smaller than the resolution and power consumption corresponding to the third resolution mode.
  • the signal-to-noise ratio and frame rate corresponding to the fourth resolution mode are greater than the signal-to-noise ratio and frame rate corresponding to the third resolution mode.
  • the fourth resolution mode may specifically be a preview mode during image capture, a preview mode during video capture, or a night scene mode for image capture or video capture under night scenes, but is not limited to scenes with lower resolution requirements.
  • the preview modes of video shooting include 480p video preview, application video preview, etc.
  • the electronic device determines each smallest repeating unit from the filter array, acquires multiple third panchromatic pixel values in the second panchromatic image obtained by each smallest repeating unit, and combines the multiple third panchromatic pixel values Read out the fourth panchromatic pixel value.
  • Operation 2604 Combining the third color pixel values of the first color in the two-color second color image to read out the fourth color pixel value of the first color, and combining the third color pixel values in the two-color second color image Combine the third color pixel values of the third color to read the fourth color pixel value of the third color, and combine the third color pixel values of multiple second colors in the single-color second color image to read the fourth color of the second color Pixel values.
  • the electronic device determines each minimum repeating unit from the filter array, and obtains multiple third color pixel values of the same color in the second color image obtained by each minimum repeating unit, and the third color pixel values of the same color
  • the value includes the third color pixel value of the first color, the third color pixel value of the second color, and the third color pixel value of the third color, and the multiple third color pixel values of the first color are combined to read out the first color
  • the fourth panchromatic pixel value of the second color is combined to read the fourth panchromatic pixel value of the second color
  • the third color pixel value of the third color is combined to read the fourth panchromatic pixel value of the second color.
  • a fourth target image is generated based on the fourth panchromatic pixel value, the fourth color pixel value of the first color, the fourth color pixel value of the second color, and the fourth color pixel value of the third color.
  • the electronic device will correspond to the fourth pan-color pixel value of the same minimum repeating unit, the fourth color pixel value of the first color, the fourth color pixel value of the second color, and the fourth color pixel value of the third color alternately arranged to generate the fourth target image. It should be noted that, corresponding to the fourth panchromatic pixel value of the same minimum repeating unit, the fourth color pixel value of the first color, the fourth color pixel value of the second color, and the fourth color pixel value of the third color, There is no limit to the order of arrangement.
  • FIG. 27 is a schematic diagram of the fourth target image in an embodiment.
  • the third panchromatic pixel value in the second panchromatic image with the same coordinates, the third color pixel value in the double-color second color image, and the third color pixel value in the single-color second color image are arranged in The order is not limited.
  • the values of the third panchromatic pixels in the second panchromatic image are combined to read out the fourth panchromatic pixel value
  • the multiple second panchromatic pixel values in the two-color second color image are combined to read out the fourth panchromatic pixel value.
  • the third color pixel value of one color is combined to read the fourth color pixel value of the first color
  • the third color pixel values of multiple third colors in the two-color second color image are combined to read the fourth color pixel value of the third color.
  • the fourth target image can be matched with a high-pixel image sensor, taking into account both high resolution under high pixels and high signal-to-noise ratio under low pixels.
  • the fourth resolution mode there is no need to merge pixels of the same color across cycles, and no interpolation is required, which improves the overall resolution.
  • the above method further includes: in the fourth resolution mode, combining the panchromatic pixels corresponding to the panchromatic sub-filters of the plurality of panchromatic filters in the minimum repetition to read out the seventh panchromatic pixel color pixel value, the color pixels corresponding to each color sub-filter of a plurality of color filters of the same color in the smallest repeating unit are combined and read out the seventh color pixel value; based on each seventh panchromatic pixel value and each sixth Seven color pixel values to generate the seventh target image.
  • the combined readout method may be one of averaging, weighted averaging, or addition.
  • the color filter includes a first color filter, a second color filter and a third color filter.
  • the electronic device combines the color pixels corresponding to the sub-filters of the multiple first color filters in the minimum repeating unit to read out the seventh color pixel value of the first color, and filters the multiple second colors in the minimum repeating unit Combining the color pixels corresponding to each sub-filter of the chip to read the seventh color pixel value of the second color, and combining and reading the color pixels corresponding to each sub-filter of multiple third color filters in the minimum repeating unit Get the seventh color pixel value of the third color.
  • the panchromatic pixels corresponding to the panchromatic sub-filters of the multiple panchromatic filters in the minimum repetition are combined to read out the seventh panchromatic pixel value, and the minimum
  • the color pixels corresponding to each color sub-filter of a plurality of color filters of the same color in the repeating unit are combined and read out the seventh color pixel value, which is obtained based on each seventh panchromatic pixel value and each seventh color pixel value
  • the resolution and image size of the seventh target image are further reduced, and the panchromatic pixels have a higher signal-to-noise ratio, and the frame rate of the image is high, thereby achieving a four-level pixel combination output with lower power consumption and a higher signal-to-noise ratio.
  • the fourth target image can be matched with a high-pixel image sensor, taking into account both high resolution under high pixels and high signal-to-noise ratio under low pixels.
  • the fourth resolution mode there is no need to merge pixels of the same color across cycles, and no interpolation is required, which improves the overall resolution.
  • another image generation method is provided, which is applied to an image sensor, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes a plurality of filters
  • Each filter group includes a first subunit and a second subunit; both the first subunit and the second subunit include a panchromatic filter and a color filter, and the color filter in the first subunit
  • the slices are arranged on the diagonal in the first subunit, and the color filters in the second subunit are arranged on the anti-diagonal in the second subunit;
  • the panchromatic filter passes through The amount of incoming light is greater than the incoming light of the color filter; each pixel in the pixel array corresponds to the filter of the filter array, and the pixel array is configured to receive light passing through the filter array to generate electrical signal;
  • the image generating method includes: in the full resolution mode, reading out the full resolution panchromatic pixel value from the panchromatic pixel corresponding to each panchromatic filter, and reading out
  • the principle of generating the full-resolution target image in this embodiment is similar to the principle of generating the full-resolution target image in the embodiment of FIG. 12 , and will not be repeated here.
  • the optical filter array also has a first resolution mode, a second resolution mode and a third resolution mode, and N is The principle of the first resolution mode corresponding to 1 is similar to the principle of the second resolution mode corresponding to N greater than or equal to 2, and the principle of the second resolution mode corresponding to N being 1 is the third resolution corresponding to N greater than or equal to 2 The principle of the resolution mode is similar, and the principle of the third resolution mode corresponding to N being 1 is similar to the principle of the fourth resolution mode corresponding to N greater than or equal to 2, which will not be repeated here.
  • 26 may include multiple sub-operations or multiple stages, and these sub-operations or stages are not necessarily performed at the same time, but may be completed at the same time Executed at different times, the execution order of these sub-operations or stages is not necessarily sequential, but may be performed in turn or alternately with other operations or at least a part of sub-operations or stages of other operations.
  • Fig. 28 is a structural block diagram of an image generating device of an embodiment. As shown in FIG. 28, an image generating device is provided, which is applied to an image sensor.
  • the image sensor includes a filter array and a pixel array.
  • the filter array includes a minimum repeating unit, and the minimum repeating unit includes a plurality of filter groups.
  • Each filter group includes a first subunit and a second subunit; both the first subunit and the second subunit include a panchromatic filter and a color filter, and the color filter group in the first subunit
  • the color filter in the second subunit is arranged on the diagonal in the first subunit where it is located; the color filter in the second subunit where it is located is arranged on the opposite diagonal;
  • the amount of light is greater than the amount of light transmitted by the color filter;
  • each panchromatic filter includes N rows and N columns of panchromatic sub-filters, and each color filter includes N rows and N columns of color sub-filters, N rows and N columns of color sub-filters have the same color as the color filter, and N is a positive integer greater than or equal to 2;
  • each pixel in the pixel array corresponds to the sub-filter of the filter array, and the pixel array is configured to receive light passing through the filter array to generate an electrical signal;
  • the image generation device includes: a readout module 2802 and an image generation module 2804, wherein:
  • the readout module 2802 is configured to read out the full-resolution panchromatic pixel value from the panchromatic pixel corresponding to each panchromatic sub-filter in the panchromatic filter in the full-resolution mode, and read out the full-resolution panchromatic pixel value of the color filter The full-resolution color pixel value is read out from the color pixel corresponding to each color sub-filter.
  • An image generation module 2804 configured to generate a full-resolution target image based on each full-resolution panchromatic pixel value and each full-resolution color pixel value.
  • the above image generation device reads out the full-resolution panchromatic pixel value from the panchromatic pixel corresponding to each panchromatic sub-filter in the panchromatic filter in the full-resolution mode, and reads out the full-resolution panchromatic pixel value of each panchromatic sub-filter in the color filter.
  • the color pixel corresponding to each color sub-filter reads out the full-resolution color pixel value; and the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter, and the panchromatic channel information can be integrated into the image.
  • the overall light input amount is increased, so that based on each full-resolution panchromatic pixel value and each full-resolution color pixel value, a full-resolution target image with more information and clearer detail analysis can be generated.
  • the smallest repeating unit includes a plurality of filter groups, and each filter group includes a first subunit and a second subunit; both the first subunit and the second subunit include a panchromatic filter Light sheets and color filters, the color filters in the first subunit are arranged on the diagonal line of the first subunit, and the color filters in the second subunit are arranged on the first subunit
  • the arrangement of the color filters in the direction of the diagonal line and the direction of the anti-diagonal line can be more balanced, so that the color channel has a stronger color when generating a full-resolution target image resolution ability.
  • the above-mentioned readout module 2802 is also used to combine the panchromatic pixels corresponding to each panchromatic sub-filter in each panchromatic filter to read out the first panchromatic pixel in the first resolution mode.
  • the image generating module 2804 is further configured to generate a first target image based on each first panchromatic pixel value and each first color pixel value.
  • the above-mentioned readout module 2802 is also used to combine and read out the second panchromatic pixel values corresponding to each subunit in the first target image in the second resolution mode.
  • pixel value the subunit includes a first subunit and a second subunit;
  • the image generation module 2804 is also configured to generate a first panchromatic image based on each second panchromatic pixel value; the resolution corresponding to the second resolution mode is smaller than the first A resolution corresponding to a resolution mode;
  • the above-mentioned readout module 2802 is also used to combine and read out the second color pixel values corresponding to a plurality of first color pixel values of the same color in each subunit in the first target image, and the above-mentioned
  • the image generation module 2804 is further configured to generate a first color image based on each second color pixel value;
  • the image generation module 2804 is also configured to generate a second target image based on the first panchromatic image and the first color image.
  • the above-mentioned image generation module 2804 is further configured to alternately arrange the second panchromatic pixel values of each row in the first panchromatic image and the second color pixel values of each row in the first color image to generate the second target image; or alternately arrange each column of second panchromatic pixel values in the first panchromatic image with each column of second color pixel values in the first color image to generate a second target image.
  • the above-mentioned readout module 2802 is also used to combine and read out the second panchromatic pixel values corresponding to the same filter set in the first panchromatic image in the third resolution mode.
  • the image generating module 2804 is further configured to generate a second panchromatic image based on each third panchromatic pixel value; the resolution corresponding to the third resolution mode is smaller than the resolution corresponding to the second resolution mode; the above The readout module 2802 is also used to combine and read out the third color pixel value of the first color, the The third color pixel value and the third color pixel value of the third color, the above-mentioned image generation module 2804 is further configured to be based on the third color pixel value of the first color, the third color pixel value of the second color and the third color pixel value of the third color
  • the three-color pixel value generates a second color image of two colors and a second color image of a single color; the second color image of two colors includes a third color pixel value
  • the above-mentioned image generation module 2804 is further configured to convert each row of the third panchromatic pixel value in the second panchromatic image, each row of the third color pixel value in the dual-color second color image, and the single-color pixel value of each row In the two-color image, the third color pixel values of each row are arranged alternately to generate the second target image; The color pixel values and the third color pixel values of each column in the single-color second color image are arranged alternately to generate the second target image.
  • the above readout module 2802 is also used to read out the fourth panchromatic pixel value by combining the third panchromatic pixel values in the second panchromatic image in the fourth resolution mode; the fourth resolution The resolution corresponding to the mode is smaller than the resolution corresponding to the third resolution mode; the readout module 2802 is also used to combine multiple third color pixel values of the first color in the two-color second color image to read out the first color
  • the fourth color pixel value of the double-color second color image is combined to read the fourth color pixel value of the third color by combining the third color pixel values of the third color in the double-color second color image, and the second color image of the single color
  • a plurality of third color pixel values of the second color are combined to read out a fourth color pixel value of the second color;
  • the image generation module 2804 is further configured to base on the fourth panchromatic pixel value, the fourth color pixel value of the first color, The fourth color pixel value of the second color and the fourth color pixel value of the third color generate a
  • the above-mentioned readout module 2802 is also used to convert each panchromatic sub-filter of the plurality of panchromatic filters in each first subunit or second subunit to the second resolution mode. Combining the corresponding panchromatic pixels to read out the fifth panchromatic pixel value, the above image generation module 2804 is also used to generate a third panchromatic image based on each fifth panchromatic pixel value; the above readout module 2802 is also used to use each The color pixels corresponding to each color sub-filter of a plurality of color filters of the same color in a subunit or a second subunit are combined to read out the fifth color pixel value, and the above-mentioned image generation module 2804 is also used to The color pixel values generate a third color image; the image generating module 2804 is further configured to generate a fifth target image based on the third panchromatic image and the third color image.
  • the above-mentioned image generation module 2804 is further configured to alternately arrange the fifth panchromatic pixel values in each row in the third panchromatic image and the fifth color pixel values in each row in the third color image to generate the fifth target image; or alternately arrange each column of fifth panchromatic pixel values in the third panchromatic image with each column of fifth color pixel values in the third color image to generate a fifth target image.
  • the above-mentioned readout module 2802 is also used to convert the panchromatic sub-filters corresponding to the panchromatic sub-filters of the plurality of panchromatic filters in each filter group in the third resolution mode. Pixel combining to read out the sixth panchromatic pixel value, the image generating module 2804 is also used to generate a fourth panchromatic image based on each sixth panchromatic pixel value; the above readout module 2802 is also used to combine the The color pixels corresponding to each color sub-filter of a plurality of color filters of the same color are combined to read out the sixth color pixel value of the first color, the sixth color pixel value of the second color and the sixth color pixel value of the third color.
  • the above-mentioned image generation module 2804 is also used to generate a fourth color pixel value of two colors based on the sixth color pixel value of the first color, the sixth color pixel value of the second color and the sixth color pixel value of the third color image and a single-color fourth color image;
  • the double-color fourth color image includes the sixth color pixel value of the first color and the sixth color pixel value of the third color
  • the single-color fourth color image includes the second color Sixth color pixel value;
  • the image generating module 2804 is further configured to generate a sixth target image based on the fourth panchromatic image, the fourth color image of two colors and the fourth color image of single color.
  • the above-mentioned image generation module 2804 is further configured to convert the sixth panchromatic pixel value of each row in the fourth panchromatic image, the sixth color pixel value of each row in the fourth color image of two colors, and the sixth color pixel value of each row in the single-color In the four-color image, the sixth color pixel values of each row are arranged alternately to generate the sixth target image; The color pixel values and the sixth color pixel values in each column of the single-color fourth color image are arranged alternately to generate a sixth target image.
  • the above-mentioned readout module 2802 is also used to combine and read out the panchromatic pixels corresponding to each panchromatic sub-filter of multiple panchromatic filters in the minimum repetition in the fourth resolution mode. Seven panchromatic pixel values, the color pixels corresponding to the color sub-filters of a plurality of color filters of the same color in the smallest repeating unit are combined to read out the seventh color pixel value; the image generation module 2804 is also used to read out the seventh color pixel value based on Each seventh panchromatic pixel value and each seventh color pixel value generate a seventh target image.
  • each module in the above image generating device is only for illustration. In other embodiments, the image generating device can be divided into different modules according to needs, so as to complete all or part of the functions of the above image generating device.
  • Each module in the above-mentioned image generating device can be fully or partially realized by software, hardware and a combination thereof.
  • the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that the processor can invoke and execute the corresponding operations of the above-mentioned modules.
  • Fig. 29 is a schematic diagram of the internal structure of an electronic device in one embodiment.
  • the electronic device can be any terminal device such as mobile phone, tablet computer, notebook computer, desktop computer, PDA (Personal Digital Assistant, personal digital assistant), POS (Point of Sales, sales terminal), vehicle-mounted computer, wearable device, etc.
  • the electronic device includes a processor and memory connected by a system bus.
  • the processor may include one or more processing units.
  • the processor can be a CPU (Central Processing Unit, central processing unit) or a DSP (Digital Signal Processing, digital signal processor), etc.
  • the memory may include non-volatile storage media and internal memory. Nonvolatile storage media store operating systems and computer programs.
  • the computer program can be executed by a processor to implement an image generation method provided in the following embodiments.
  • the internal memory provides a high-speed running environment for the operating system computer program in the non-volatile storage medium.
  • each module in the image generation device provided in the embodiment of the present application may be in the form of a computer program.
  • the computer program can run on a terminal or a server.
  • the program modules constituted by the computer program can be stored in the memory of the electronic device.
  • the operations of the methods described in the embodiments of the present application are realized.
  • the embodiment of the present application also provides a computer-readable storage medium.
  • One or more non-transitory computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform the operations of the image generation method.
  • the embodiment of the present application also provides a computer program product containing instructions, which, when run on a computer, causes the computer to execute the image generation method.
  • Non-volatile memory can include ROM (Read-Only Memory, read-only memory), PROM (Programmable Read-only Memory, programmable read-only memory), EPROM (Erasable Programmable Read-Only Memory, erasable programmable read-only memory) Memory), EEPROM (Electrically Erasable Programmable Read-only Memory, Electrically Erasable Programmable Read-only Memory) or flash memory.
  • Volatile memory can include RAM (Random Access Memory, Random Access Memory), which is used as external cache memory.
  • RAM is available in various forms, such as SRAM (Static Random Access Memory, static random access memory), DRAM (Dynamic Random Access Memory, dynamic random access memory), SDRAM (Synchronous Dynamic Random Access Memory , synchronous dynamic random access memory), double data rate DDR SDRAM (Double Data Rate Synchronous Dynamic Random Access memory, double data rate synchronous dynamic random access memory), ESDRAM (Enhanced Synchronous Dynamic Random Access memory, enhanced synchronous dynamic random access memory access memory), SLDRAM (Sync Link Dynamic Random Access Memory, synchronous link dynamic random access memory), RDRAM (Rambus Dynamic Random Access Memory, bus dynamic random access memory), DRDRAM (Direct Rambus Dynamic Random Access Memory, interface dynamic random access memory) memory).
  • SRAM Static Random Access Memory, static random access memory
  • DRAM Dynanamic Random Access Memory, dynamic random access memory
  • SDRAM Synchronous Dynamic Random Access Memory , synchronous dynamic random access memory
  • double data rate DDR SDRAM Double Data Rate Synchronous Dynamic Random Access memory, double

Landscapes

  • Engineering & Computer Science (AREA)
  • Signal Processing (AREA)
  • Multimedia (AREA)
  • Color Television Image Signal Generators (AREA)

Abstract

一种图像传感器21、摄像模组20、电子设备100、图像生成方法、装置、计算机可读存储介质和计算机程序产品。图像传感器21包括滤光片阵列23和像素阵列24，滤光片阵列23包括最小重复单元230，最小重复单元230包括多个滤光片组，每个滤光片组包括第一子单元233和第二子单元234；每个子单元均包括全色滤光片235和彩色滤光片236，第一子单元233中的彩色滤光片236排布于所在的第一子单元233中的对角线上，第二子单元234中的彩色滤光片236排布于所在的第二子单元233中的反对角线上；全色滤光片235透过的进光量大于彩色滤光片236透过的进光量；每个滤光片包括N行N列个与滤光片同种颜色的子滤光片，N为正整数。

Description

图像传感器、摄像模组、电子设备、图像生成方法和装置
相关申请的交叉引用
本申请要求于2021年12月14日提交中国专利局、申请号为202111524625.4、发明名称为“图像传感器、摄像模组、电子设备、图像生成方法和装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机技术领域，特别是涉及一种图像传感器、摄像模组、电子设备、图像生成方法、装置、计算机可读存储介质和计算机程序产品。
背景技术
随着计算机技术的发展，手机等电子设备中大部分都配置有摄像头，以通过摄像头实现拍照功能。摄像头中设置有图像传感器，通过图像传感器采集彩色图像。为了实现彩色图像的采集，图像传感器中通常会设置以拜耳(Bayer)阵列形式排布的滤光片阵列，以使得图像传感器中的多个像素能够接收穿过对应的滤光片的光线，从而生成具有不同色彩通道的像素信号，进而生成图像。
然而，传统的图像传感器所生成的图像清晰度较低。
发明内容
根据本申请的各种实施例提供一种图像传感器、摄像模组、电子设备、图像生成方法、装置、计算机可读存储介质和计算机程序产品。
一种图像传感器，所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，所述最小重复单元包括多个滤光片组，每个滤光片组包括第一子单元和第二子单元；所述第一子单元和所述第二子单元均包括全色滤光片和彩色滤光片，所述第一子单元中的彩色滤光片排布于所在的第一子单元中的对角线上，所述第二子单元中的彩色滤光片排布于所在的第二子单元中的反对角线上；所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量；每个所述全色滤光片包括N行N列个全色子滤光片，每个所述彩色滤光片包括N行N列个彩色子滤光片，所述N行N列个彩色子滤光片与所述彩色滤光片的颜色相同，所述N为正整数；所述像素阵列中各个像素与所述滤光片阵列的子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号。
一种摄像模组，所述摄像模组包括镜头和上述的图像传感器；所述图像传感器用于接收穿过所述镜头的光线，所述像素根据所述光线生成电信号。
一种电子设备，包括：
上述的摄像模组；及
壳体，所述摄像模组设置在所述壳体上。
上述图像传感器、摄像模组和电子设备，图像传感器包括滤光片阵列和像素阵列，滤光片阵列包括最小重复单元，最小重复单元包括多个滤光片组，每个滤光片组包括第一子单元和第二子单元；第一子单元和第二子单元均包括全色滤光片和彩色滤光片，第一子单元中的彩色滤光片排布于所在的第一子单元中的对角线上，第二子单元中的彩色滤光片排布于所在的第二子单元中的反对角线上，可以使得对角线的方向上和反对角线的方向上的彩色滤光片的排布更加均衡，成像时彩色通道具有更强的分辨能力。
其中，全色滤光片透过的进光量大于彩色滤光片透过的进光量，可在拍摄时通过全色滤光片获得更多的光量，从而无需调节拍摄参数，在不影响拍摄的稳定性的情况下，提高暗光下的成像的清晰度。暗光下成像时，可兼顾稳定性和清晰度，暗光下成像的稳定性和清晰度均较高。
一种图像生成方法，应用于图像传感器，所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，所述最小重复单元包括多个滤光片组，每个滤光片组包括第一子单元和第二子 单元；所述第一子单元和所述第二子单元均包括全色滤光片和彩色滤光片，所述第一子单元中的彩色滤光片排布于所在的第一子单元中的对角线上，所述第二子单元中的彩色滤光片排布于所在的第二子单元中的反对角线上；所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量；每个所述全色滤光片包括N行N列个全色子滤光片，每个所述彩色滤光片包括N行N列个彩色子滤光片，所述N行N列个彩色子滤光片与所述彩色滤光片的颜色相同，所述N为大于或等于2的正整数；所述像素阵列中各个像素与所述滤光片阵列的子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号；
所述方法包括：
在全分辨率模式下，将所述全色滤光片中每个全色子滤光片对应的全色像素读出全分辨率全色像素值，以及将所述彩色滤光片中每个彩色子滤光片对应的彩色像素读出全分辨率彩色像素值；
基于各个所述全分辨率全色像素值和各个所述全分辨率彩色像素值，生成全分辨率目标图像。
一种图像生成装置，应用于图像传感器，所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，所述最小重复单元包括多个滤光片组，每个滤光片组包括第一子单元和第二子单元；所述第一子单元和所述第二子单元均包括全色滤光片和彩色滤光片，所述第一子单元中的彩色滤光片排布于所在的第一子单元中的对角线上，所述第二子单元中的彩色滤光片排布于所在的第二子单元中的反对角线上；所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量；每个所述全色滤光片包括N行N列个全色子滤光片，每个所述彩色滤光片包括N行N列个彩色子滤光片，所述N行N列个彩色子滤光片与所述彩色滤光片的颜色相同，所述N为大于或等于2的正整数；所述像素阵列中各个像素与所述滤光片阵列的子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号
所述装置包括：
读出模块，用于在全分辨率模式下，将所述全色滤光片中每个全色子滤光片对应的全色像素读出全分辨率全色像素值，以及将所述彩色滤光片中每个彩色子滤光片对应的彩色像素读出全分辨率彩色像素值；
图像生成模块，用于基于各个所述全分辨率全色像素值和各个所述全分辨率彩色像素值，生成全分辨率目标图像。
一种电子设备，包括存储器及处理器，所述存储器中储存有计算机程序，所述计算机程序被所述处理器执行时，使得所述处理器执行如上述的图像生成方法的操作。
一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现如上述的方法的操作。
一种计算机程序产品，包括计算机程序，该计算机程序被处理器执行时实现如上述的方法的操作。
上述图像生成方法、装置、电子设备、计算机可读存储介质和计算机程序产品，在全分辨率模式下，将全色滤光片中每个全色子滤光片对应的全色像素读出全分辨率全色像素值，以及将彩色滤光片中每个彩色子滤光片对应的彩色像素读出全分辨率彩色像素值；而全色滤光片透过的进光量大于彩色滤光片透过的进光量，能够将全色通道信息融合到图像中，提高整体的进光量，从而基于各个全分辨率全色像素值和各个全分辨率彩色像素值，能够生成信息更多、细节解析更清晰的全分辨率目标图像。
在该滤光片阵列中，最小重复单元包括多个滤光片组，每个滤光片组包括第一子单元和第二子单元；第一子单元和第二子单元均包括全色滤光片和彩色滤光片，第一子单元中的彩色滤光片排布于所在的第一子单元中的对角线上，第二子单元中的彩色滤光片排布于所在的第二子单元中的反对角线上，可以使得对角线的方向上和反对角线的方向上的彩色滤光片的排布更加均衡，从而在生成全分辨率目标图像时彩色通道具有更强的分辨能力。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域 普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为一个实施例中电子设备的结构示意图。
图2为一个实施例中图像传感器的分解示意图。
图3为一个实施例中像素阵列和读出电路的连接示意图。
图4为一个实施例中N为1的滤光片阵列中最小重复单元的排布方式示意图。
图5为另一个实施例中N为1的滤光片阵列中最小重复单元的排布方式示意图。
图6为另一个实施例中N为1的滤光片阵列中最小重复单元的排布方式示意图。
图7为另一个实施例中N为1的滤光片阵列中最小重复单元的排布方式示意图。
图8为一个实施例中N为2的滤光片阵列中最小重复单元的排布方式示意图。
图9为另一个实施例中N为2的滤光片阵列中最小重复单元的排布方式示意图。
图10为另一个实施例中N为2的滤光片阵列中最小重复单元的排布方式示意图。
图11为另一个实施例中N为2的滤光片阵列中最小重复单元的排布方式示意图。
图12为另一个实施例中N为3的滤光片阵列中最小重复单元的排布方式示意图。
图13为一个实施例中图像生成方法的流程示意图。
图14为一个实施例中图像生成方法的流程示意图。
图15为一个实施例中第一目标图像的示意图。
图16为一个实施例中生成第二目标图像的流程示意图。
图17为一个实施例中第一彩色图像和第一全色图像的示意图。
图18为一个实施例中第二目标图像的示意图。
图19为另一个实施例中第二目标图像的示意图。
图20为另一个实施例中第二目标图像的示意图。
图21为另一个实施例中第二目标图像的示意图。
图22为一个实施例中生成第三目标图像的流程示意图。
图23为一个实施例中第二全色图像、双颜色的第二彩色图像和单颜色的第二彩色图像的示意图。
图24为一个实施例中第三目标图像的示意图。
图25为另一个实施例中第三目标图像的示意图。
图26为一个实施例中生成第四目标图像的流程示意图。
图27为一个实施例中第四目标图像的示意图。
图28为一个实施例中图像生成装置的结构框图。
图29为一个实施例中电子设备的内部结构示意图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。
可以理解，本申请所使用的术语“第一”、“第二”等可在本文中用于描述各种元件，但这些元件不受这些术语限制。这些术语仅用于将第一个元件与另一个元件区分。举例来说，在不脱离本申请的范围的情况下，可以将第一目标图像称为第二目标图像，且类似地，可将第二目标图像称为第一目标图像。第一目标图像和第二目标图像两者都是目标图像，但其不是同一目标图像。
在一个实施例中，电子设备100包括手机、平板电脑、笔记本电脑、柜员机、闸机、智能手表、头显设备等，可以理解，电子设备100还可以是其他任意图像处理功能的装置。电子设备100包括摄像模组20、处理器30和壳体40。摄像模组20和处理器30均设置在壳体40内，壳体40还可用于安装电子设备100的供电装置、通信装置等功能模块，以使壳体40为功能模块提供防尘、防摔、防水等保护。
摄像模组20可以是前置摄像模组、后置摄像模组、侧置摄像模组、屏下摄像模组等，在此不做限制。摄像模组20包括镜头及图像传感器21，摄像模组20在拍摄图像时，光线穿过镜头并到达图像传感器21，图像传感器21用于将照射到图像传感器21上的光信号转化为电信号。
在一个实施例中，如图2所示，图像传感器21包括微透镜阵列22、滤光片阵列23、像素阵列24。
微透镜阵列22包括多个微透镜221，微透镜221、滤光片阵列23中的子滤光片和像素阵列24中的像素一一对应设置，微透镜221用于将入射的光线进行聚集，聚集之后的光线会穿过对应的子滤光片，然后投射至像素上，被对应的像素接收，像素再将接收的光线转化成电信号。
在一个实施例中，滤光片阵列23包括多个最小重复单元230。最小重复单元230包括多个滤光片组，每个滤光片组包括第一子单元233和第二子单元234；第一子单元233和第二子单元234均包括全色滤光片235和彩色滤光片236，第一子单元233中的彩色滤光片236排布于所在的第一子单元233中的对角线上，第二子单元234中的彩色滤光片236排布于所在的第二子单元234中的反对角线上；全色滤光片235透过的进光量大于彩色滤光片236透过的进光量；每个全色滤光片235包括N行N列个全色子滤光片，每个彩色滤光片236包括N行N列个彩色子滤光片，N行N列个彩色子滤光片与彩色滤光片的颜色相同，N为正整数。在图2中，N为2。
其中，对角线可以是左上角和右下角的连线，也可以是右上角和左下角的连线。反对角线可以是左上角和右下角的连线，也可以是右上角和左下角的连线。对角线和反对角线相互垂直。也就是说，若对角线是左上角和右下角的连线，则反对角线是右上角和左下角的连线；若对角线是右上角和左下角的连线，则反对角线是左上角和右下角的连线。
上述图像传感器包括滤光片阵列23和像素阵列24，滤光片阵列23包括最小重复单元230，最小重复单元230包括多个滤光片组，每个滤光片组包括第一子单元233和第二子单元234；第一子单元233和第二子单元234均包括全色滤光片235和彩色滤光片236，第一子单元233中的彩色滤光片236排布于所在的第一子单元233中的对角线上，第二子单元234中的彩色滤光片排布于所在的第二子单元234中的反对角线上，可以使得对角线的方向上和反对角线的方向上的彩色滤光片236的排布更加均衡，成像时彩色通道具有更强的分辨能力。
其中，全色滤光片235透过的进光量大于彩色滤光片236透过的进光量，可在拍摄时通过全色滤光片236获得更多的光量，从而无需调节拍摄参数，在不影响拍摄的稳定性的情况下，提高暗光下的成像的清晰度。暗光下成像时，可兼顾稳定性和清晰度，暗光下成像的稳定性和清晰度均较高。
在一个实施例中，最小重复单元230的每一行和每一列上均包括每种颜色的彩色滤光片236，即将各个颜色的彩色滤光片236分散排列，可以提高了彩色分辨能力和亮度变化分辨能力，各个颜色的彩色滤光片236混合排列，也降低了伪色的风险。可以理解的是，全色滤光片235与彩色滤光片236在每一行和每一列上交替排布，也即全色滤光片235在最小重复单元、第一滤光片组或第二滤光片组中占50％的数量，可以提高了成像中每个局部区域的进光量。
在一个实施例中，全色滤光片235与彩色滤光片236在每一行或者每一列上交替排布，可以提高成像的每一行或者每一列的彩色分辨率，使得成像的颜色更加丰富。
在一个实施例中，最小重复单元230包括2个第一滤光片组231和2个第二滤光片组232，2个第一滤光片组231在最小重复单元230的对角线上排布，2个第二滤光片组232在最小重复单元230的反对角线上排布。2个第一滤光片组231和2个第二滤光片组232呈矩阵排布。
其中，2个第一滤光片组完全相同，2个第二滤光片组完全相同。
在一个实施例中，每个滤光片组仅包括全色滤光片235和2种颜色的彩色滤光片236。彩色滤光片236包括第一颜色滤光片、第二颜色滤光片和第三颜色滤光片。第一颜色滤光片、第二颜色滤光片和第三颜色滤光片是三种不同彩色的滤光片。第一颜色滤光片、第二颜色滤光片和第三颜色滤光片的颜色均可以根据需要进行设置。例如，第一颜色滤光片可以为红色滤光片，第二颜色滤光片可以为绿色滤光片，第三颜色滤光片可以为蓝色滤光片。
彩色滤光片236的透过的光线的波段的宽度小于全色滤光片235透过的光线的波段的宽度，例如，彩色滤光片236的透过的光线的波段可对应红光的波段、绿光的波段、或蓝光的波段，全色滤光片235透过的光线的波段为所有可见光的波段，也即是说，彩色滤光片236仅允许特定颜色光线透光，而全色滤光片235可通过所有颜色的光线。当然，彩色滤光片236的透过的光线的波段还可对应其他色光的波段，如品红色光、紫色光、青色光、黄色光等，在此不作限制。
进一步地，第一滤光片组231中的彩色滤光片236包括第一颜色滤光片和第二颜色滤光片，第二滤光片组232中的彩色滤光片236包括第二颜色滤光片和第三颜色滤光片。
在一个实施例中，每个滤光片组均包括2个第一子单元233和2个第二子单元234；2个第一子单元233在滤光片组的第一行方向上排布，以及2个第二子单元234在滤光片组的第二行方向上排布，第一行方向和第二行方向相邻排布；或者，2个第一子单元233在滤光片组的第一列方向上排布，以及2个第二子单元234在滤光片组的第二列方向上排布，第一列方向和第二列方向相邻排布。
如图2所示，第一滤光片组231包括2个第一子单元233和2个第二子单元234，2个第一子单元233在第一滤光片组231的第一行方向上排布，以及2个第二子单元234在滤光片组的第二行方向上排布。在其他实施例中，2个第一子单元233在第一滤光片组231的第一列方向上排布，以及2个第二子单元234在滤光片组的第二列方向上排布。
进一步地，同一滤光片组中包含同种颜色滤光片的第一子单元233和第二子单元234排布于同一滤光片组的对角线上。
同一滤光片组中包含同种颜色滤光片的第一子单元233和第二子单元234排布于同一滤光片组的对角线上，并且第一子单元中的彩色滤光片236排布于所在的第一子单元中的对角线上，第二子单元中的彩色滤光片236排布于所在的第二子单元中的反对角线上，那么，排布于同一滤光片组的对角线上的第一子单元233中的彩色滤光片236排布于所在的第一子单元的对角线上，排布于同一滤光片组的对角线的第二子单元234中的彩色滤光片236排布于所在的第二子单元的反对角线上，可以提高彩色滤光片236在对角线和反对角线上的分布均衡性。
可以理解的是，第一子单元233和第二子单元234均包括全色滤光片235和彩色滤光片236，那么包含同种颜色滤光片的第一子单元233和第二子单元234，也即包含同种颜色的彩色滤光片236的第一子单元233和第二子单元234。如图2所示，第一滤光片组231中包括2个第一子单元233和2个第二子单元234，2个第一子单元233在第一滤光片组231的行方向上排布，以及2个第二子单元234在第一滤光片组231的行方向上排布，并且包含同种颜色滤光片的第一子单元233和第二子单元234排布于第一滤光片组231的对角线上。
进一步地，彩色滤光片236包括第一颜色滤光片、第二颜色滤光片和第三颜色滤光片；第一滤光片组231中的2个第一子单元233的其中一个第一子单元233包括第一颜色滤光片，另一个第一子单元233包括第二颜色滤光片；第一滤光片组231中的2个第二子单元234的其中一个第二子单元234包括第一颜色滤光片，另一个第二子单元234包括第二颜色滤光片。
同样的，彩色滤光片236包括第一颜色滤光片、第二颜色滤光片和第三颜色滤光片；第二滤光片组232中的2个第一子单元233的其中一个第一子单元233包括第二颜色滤光片，另一个第一子单元233包括第三颜色滤光片；第二滤光片组232中的2个第二子单元234的其中一个第二子单元234包括第二颜色滤光片，另一个第二子单元234包括第三颜色滤光片。
像素阵列24包括多个像素，像素阵列24的像素与滤光片阵列23的子滤光片对应设置。像素阵列24被配置成用于接收穿过滤光片阵列23的光线以生成电信号。
其中，像素阵列24被配置成用于接收穿过滤光片阵列23的光线以生成电信号，是指像素阵列24用于对穿过滤光片阵列23的被摄对象的给定集合的场景的光线进行光电转换，以生成电信号。被摄对象的给定集合的场景的光线用于生成图像数据。例如，被摄对象是建筑物，被摄对象的给定集合的场景是指该建筑物所处的场景，该场景中还可以包含其他对象。
在一个实施例中，像素阵列24包括多个最小重复单元240，最小重复单元240还包括多个全色像素241和多个不同颜色的彩色像素242，每一行和每一列上均包括每种颜色的彩色像素；每个全色像素242对应全色滤光片235中的一个子滤光片，全色像素242接收穿过对应的子滤光片的光线以生成电信号。每个彩色像素242对应彩色滤光片236的一个子滤光片，彩色像素242接收穿过对应的子滤光片的光线以生成电信号。
如图3所示，读出电路25与像素阵列24电连接，用于控制像素阵列24的曝光以及像素的像素值的读取和输出。读出电路25包括垂直驱动单元251、控制单元252、列处理单元253和水平驱动单元 254。垂直驱动单元251包括移位寄存器和地址译码器。垂直驱动单元251包括读出扫描和复位扫描功能。控制单元252根据操作模式配置时序信号，利用多种时序信号来控制垂直驱动单元251、列处理单元253和水平驱动单元254协同工作。列处理单元253可以具有用于将模拟像素信号转换为数字格式的模数(A/D)转换功能。水平驱动单元254包括移位寄存器和地址译码器。水平驱动单元254顺序逐列扫描像素阵列24。
当N为1时，每个全色滤光片235包括1行1列个全色子滤光片，每个彩色滤光片236包括1行1列个彩色子滤光片，也就是说，每个全色子滤光片为全色滤光片235，每个彩色子滤光片为彩色滤光片236。
在一个实施例中，如图4所示，N为1，最小重复单元包括8行8列64个滤光片，排布方式为：
Figure PCTCN2022123995-appb-000001
其中，w表示全色滤光片235，a、b和c均表示彩色滤光片236。
在一个实施例中，如图5所示，N为1，最小重复单元包括8行8列64个滤光片，排布方式为：
Figure PCTCN2022123995-appb-000002
其中，w表示全色滤光片235，a、b和c均表示彩色滤光片236。
在一个实施例中，如图6所示，N为1，最小重复单元包括8行8列64个滤光片，排布方式为：
Figure PCTCN2022123995-appb-000003
其中，w表示全色滤光片235，a、b和c均表示彩色滤光片236。
在一个实施例中，如图7所示，N为1，最小重复单元包括8行8列64个滤光片，排布方式为：
Figure PCTCN2022123995-appb-000004
Figure PCTCN2022123995-appb-000005
其中，w表示全色滤光片235，a、b和c均表示彩色滤光片236。
其中，w可以是白色滤光片，a为红色滤光片，b为绿色滤光片，c为蓝色滤光片，或者例如a为品红色滤光片，b为青色滤光片，c为黄色滤光片等，在此不做限制。
可以理解的是，电子设备将最小重复单元进行调整，可以得到新的最小重复单元。例如，电子设备将图4的最小重复单元逆时针方向旋转90度，可以得到图7的最小重复单元；将图4的最小重复单元中的a彩色滤光片和c彩色滤光片交换位置，可以得到图5的最小重复单元；将图4的最小重复单元中的a彩色滤光片和c彩色滤光片交换位置后，再逆时针方向旋转90度，可以得到图6的最小重复单元。
在上述的图4至图7的最小重复单元中，b彩色滤光片以4乘4的最小区域作为排布周期，并且b彩色滤光片在对角线和反对角线方向的采样率是一致的，排布更加均衡，因而b通道具有更强的分辨能力，从而b通道在水平方向、垂直方向和斜方向上均具有更强的分辨能力。同样的，a彩色滤光片和c彩色滤光片在局部区域的对角线和反对角线方向的采样率是一致的，排布更加均衡，因而a通道和c通道具有更强的分辨能力。由于a彩色滤光片、b彩色滤光片、c彩色滤光片和w全色滤光片的排列更加分散，使得滤光片阵列在水平和垂直方向的具有全排列的属性，并兼顾了对角线和斜对角线方向的分辨率。
在一个实施例中，如图8所示，N为2，最小重复单元包括16行16列256个子滤光片，排布方式为：
Figure PCTCN2022123995-appb-000006
其中，w表示全色子滤光片，a、b和c均表示彩色子滤光片。
在一个实施例中，如图9所示，N为2，最小重复单元包括16行16列256个子滤光片，排布方式为：
Figure PCTCN2022123995-appb-000007
Figure PCTCN2022123995-appb-000008
其中，w表示全色子滤光片，a、b和c均表示彩色子滤光片。
在一个实施例中，如图10所示，N为2，最小重复单元包括16行16列256个子滤光片，排布方式为：
Figure PCTCN2022123995-appb-000009
其中，w表示全色子滤光片，a、b和c均表示彩色子滤光片。
在一个实施例中，如图11所示，N为2，最小重复单元包括16行16列256个子滤光片，排布方式为：
Figure PCTCN2022123995-appb-000010
Figure PCTCN2022123995-appb-000011
其中，w表示全色子滤光片，a、b和c均表示彩色子滤光片。
其中，w可以是白色子滤光片，a为红色子滤光片，b为绿色子滤光片，c为蓝色子滤光片，或者例如a为品红色子滤光片，b为青色子滤光片，c为黄色子滤光片等，在此不做限制。
可以理解的是，电子设备将最小重复单元进行调整，可以得到新的最小重复单元。例如，电子设备将图8的最小重复单元逆时针方向旋转90度，可以得到图11的最小重复单元；将图8的最小重复单元中的a彩色滤光片和c彩色滤光片交换位置，可以得到图9的最小重复单元；将图8的最小重复单元中的a彩色滤光片和c彩色滤光片交换位置后，再逆时针方向旋转90度，可以得到图10的最小重复单元。
在上述的图8至图11的最小重复单元中，b彩色子滤光片以8乘8的最小区域作为排布周期，并且b彩色子滤光片在对角线和反对角线方向的采样率是一致的，排布更加均衡，因而b通道具有更强的分辨能力，从而b通道在水平方向、垂直方向和斜方向上均具有更强的分辨能力。同样的，a彩色子滤光片和c彩色子滤光片在局部区域的对角线和反对角线方向的采样率是一致的，排布更加均衡，因而a通道和c通道具有更强的分辨能力。由于a彩色子滤光片、b彩色子滤光片、c彩色子滤光片和w全色子滤光片的排列更加分散，使得滤光片阵列在水平和垂直方向的具有全排列的属性，并兼顾了对角线和斜对角线方向的分辨率。
需要说明的是，N还可以为3、4或5等其他正整数，排布方式与N为1或2类似，在此不做赘述。如图12为一个实施例中N为3的最小重复单元。
在一个实施例中，还提供了一种摄像模组，摄像模组包括镜头和上述的图像传感器；图像传感器用于接收穿过镜头的光线，像素根据光线生成电信号。
在一个实施例中，还提供了一种电子设备，包括上述的摄像模组；及壳体，摄像模组设置在壳体上。
在一个实施中，提供了一种图像生成方法，应用于图像传感器，图像传感器包括滤光片阵列和像素阵列，滤光片阵列包括最小重复单元，最小重复单元包括多个滤光片组，每个滤光片组包括第一子单元和第二子单元；第一子单元和第二子单元均包括全色滤光片和彩色滤光片，第一子单元中的彩色滤光片排布于所在的第一子单元中的对角线上，第二子单元中的彩色滤光片排布于所在的第二子单元中的反对角线上；全色滤光片透过的进光量大于彩色滤光片透过的进光量；每个全色滤光片包括N行N列个全色子滤光片，每个彩色滤光片包括N行N列个彩色子滤光片，N行N列个彩色子滤光片与彩色滤光片的颜色相同，N为大于或等于2的正整数；像素阵列中各个像素与滤光片阵列的子滤光片对应设置，像素阵列被配置成用于接收穿过滤光片阵列的光线以生成电信号。
如图13所示，该图像生成方法包括：
操作1302，在全分辨率模式下，将全色滤光片中每个全色子滤光片对应的全色像素读出全分辨率全色像素值，以及将彩色滤光片中每个彩色子滤光片对应的彩色像素读出全分辨率彩色像素值。
全分辨率模式是将每个子滤光片读出为一个像素的模式。
彩色滤光片具有比全色滤光片的更窄的光谱响应，则全色滤光片透过的进光量大于彩色滤光片透过的进光量，即彩色滤光片透过的光线的波段宽度小于全色滤光片透过的光线的波段宽度，全色滤光片透过更多的光线，通过全色滤光片得到相应的全色像素具有更高的信噪比，该全色像素包含有更多的信息，可以解析出更多的纹理细节。其中，信噪比是指正常信号与噪声信号之间的比值。像素的信噪比越高，则该像素包含的正常信号的比例越高，从该像素中解析到的信息也越多。
彩色像素242可以是G(Green，绿色)像素、R(Red，红色)像素和B(Blue，蓝色)像素等，但不限于此。
在接收到拍摄指令的情况下，检测用户是否选择所需使用的分辨率模式，当检测到用户选择使用全分辨率模式的情况下，或者，在用户未选择所需使用的分辨率模式，未使用预览拍摄、且当前环境非夜景模式的情况下，使用全分辨率模式响应该拍摄指令。
在全分辨率模式下，全色滤光片中的全色子滤光片透过的光线投射至对应的全色像素241上，全色像素241接收穿过全色子滤光片的光线以生成电信号。彩色滤光片中的彩色子滤光片透过的光线投射至对应的彩色像素242上，彩色像素242穿过对应的彩色子滤光片的光线以生成电信号。
每个全色滤光片包括N行N列个全色子滤光片，则每个全色滤光片对应N行N列个全色像素241。每个彩色滤光片包括N行N列个同个颜色的彩色子滤光片，则每个彩色滤光片对应N行N列个彩色像素242。N为大于或等于2的正整数。
在其他实施例中，N也可以为1，即每个全色滤光片对应1个全色像素241，每个彩色滤光片对应1个彩色像素242。
操作1304，基于各个全分辨率全色像素值和各个全分辨率彩色像素值，生成全分辨率目标图像。
电子设备可根据预设像素读取方式，从各个全分辨率全色像素值和各个全分辨率彩色像素值中读取像素值以生成全分辨率目标图像。预设像素读取方式是预先设置的像素读取方式。
上述图像生成方法，在全分辨率模式下，将全色滤光片中每个全色子滤光片对应的全色像素读出全分辨率全色像素值，以及将彩色滤光片中每个彩色子滤光片对应的彩色像素读出全分辨率彩色像素值；而全色滤光片透过的进光量大于彩色滤光片透过的进光量，能够将全色通道信息融合到图像中，提高整体的进光量，从而基于各个全分辨率全色像素值和各个全分辨率彩色像素值，能够生成信息更多、细节解析更清晰的全分辨率目标图像。
在该滤光片阵列中，最小重复单元包括多个滤光片组，每个滤光片组包括第一子单元和第二子单元；第一子单元和第二子单元均包括全色滤光片和彩色滤光片，第一子单元中的彩色滤光片排布于所在的第一子单元中的对角线上，第二子单元中的彩色滤光片排布于所在的第二子单元中的反对角线上，可以使得对角线的方向上和反对角线的方向上的彩色滤光片的排布更加均衡，从而在生成全分辨率目标图像时彩色通道具有更强的分辨能力。
在一个实施例中，如图14所示，上述方法还包括：
操作1402，在第一分辨率模式下，将每个全色滤光片中各个全色子滤光片对应的全色像素合并读出第一全色像素值，以及将每个彩色滤光片中各个彩色子滤光片对应的彩色像素合并读出第一彩色像素值；第一分辨率模式对应的分辨率小于全分辨率模式对应的分辨率。
第一分辨率模式是指分辨率、功耗、信噪比和帧率均比较均衡的一级像素合并读出模式。第一分辨率模式具体可以是图像、视频拍摄的默认模式。
在接收到拍摄指令的情况下，检测用户是否选择所需使用的分辨率模式，当检测到用户选择使用第一分辨率模式的情况下，或者，在用户未选择所需使用的分辨率模式，未使用预览拍摄、且当前环境非夜景模式的情况下，使用第一分辨率模式响应该拍摄指令。
在第一分辨率模式下，全色滤光片中的全色子滤光片透过的光线投射至对应的全色像素241上，全色像素241接收穿过全色子滤光片的光线以生成电信号。彩色滤光片中的彩色子滤光片透过的光线投射至对应的彩色像素242上，彩色像素242穿过对应的彩色子滤光片的光线以生成电信号。
合并读出是指将多个像素的像素值求和，或者计算出多个像素的像素值的均值。
在一种实施方式中，对于每个全色滤光片，对各个全色子滤光片对应的全色像素241求平均值，将该平均值读出为第一全色像素值。在另一种实施方式中，对于每个全色滤光片，对各个全色子滤光片对应的全色像素241进行相加，将相加得到的和读出为第一全色像素值。在其他实施例方式中，电子设备还可以采用其他方式将全色子滤光片对应的全色像素241合并读出第一全色像素值，在此不做限定。
在一种实施方式中，对于每个彩色滤光片，对各个彩色子滤光片对应的彩色像素242求平均值，将该平均值读出为第一彩色像素值。在另一种实施方式中，对于每个彩色滤光片，对各个彩色子滤光片对应的彩色像素进行相加，将相加得到的和读出为第一彩色像素值。在其他实施例方式中，电子设备还可以采用其他方式将彩色子滤光片对应的彩色像素242合并读出第一彩色像素值，在此不做限定。
需要说明的是，对于各个全色滤光片，合并读出第一全色像素值的方式可以相同，也可以不同。对于各个彩色滤光片，合并读出第一彩色像素值的方式可以相同，也可以不同。而对于全色滤光片和彩色滤光片，合并读出第一全色像素值和第一彩色像素值的方式可以相同，也可以不同。
操作1404，基于各个第一全色像素值和各个第一彩色像素值，生成第一目标图像。
电子设备可根据预设像素读取方式，从各个第一全色像素值和各个第一彩色像素值中读取像素值以生成第一目标图像。预设像素读取方式是预先设置的像素读取方式。以滤光片阵列23的最小重复单元 为图8的排布方式为例，则生成的第一目标图像如图15所示。在图15的第一目标图像中，w表示第一全色像素值，a、b和c表示三种不同颜色的第一彩色像素值。
在本实施例中，在第一分辨率模式下，将每个全色滤光片中各个全色子滤光片对应的全色像素241合并读出第一全色像素值，以及将每个彩色滤光片中各个彩色子滤光片对应的彩色像素242合并读出第一彩色像素值，而全色滤光片透过的进光量大于彩色滤光片透过的进光量，能够将全色通道信息融合到图像中，提高整体的进光量，从而基于各个第一全色像素值和各个第一彩色像素值，能够生成信息更多、细节解析更清晰的第一目标图像。
在一个实施例中，如图16所示，生成第一目标图像之后，上述方法还包括：
操作1602，在第二分辨率模式下，将第一目标图像中对应在每个子单元中的多个第一全色像素值合并读出第二全色像素值，并基于各个第二全色像素值生成第一全色图像；第二分辨率模式对应的分辨率小于第一分辨率模式对应的分辨率，子单元包括第一子单元和第二子单元。
第二分辨率模式是指对分辨率要求比第一分辨率模式要求低的场景下所使用的模式，是低分辨率、低功耗、高信噪比、高帧率的二级像素合并读出模式。该第二分辨率模式对应的分辨率、功耗小于第一分辨率模式对应的分辨率、功耗。该第二分辨率模式对应的信噪比、帧率大于第一分辨率模式对应的信噪比、帧率。
第二分辨率模式具体可以是图像拍摄时的预览模式、视频拍摄时的预览模式，或者在夜景下进行图像拍摄、视频拍摄的夜景模式等分辨率要求较低的场景，但不限于此。视频拍摄的预览模式例如1080p视频预览、应用视频预览等。
在接收到拍摄指令的情况下，确定该拍摄指令是否为预览拍摄。在该拍摄指令为预览拍摄的情况下，触发第二分辨率模式。或者，电子设备检测当前环境是否为夜景，在当前环境为夜景的情况下，触发第二分辨率模式。或者，在用户选择第二分辨率模式的情况下，触发第二分辨率模式对应的读出模式。
具体地，电子设备将第一目标图像中对应在每个子单元中的多个第一全色像素值合并读出第二全色像素值，并根据预设像素读取方式，从各个第二全色像素值读取像素值以生成第一全色图像。
可以理解的是，第一目标图像中各个像素值是在第一分辨率模式下，将滤光片阵列中每个滤光片中的子滤光片对应的像素合并得到的，则第一目标图像中每个像素值对应滤光片阵列中每个滤光片，也对应每个滤光片中的多个子滤光片。
在第二分辨率模式下，电子设备确定每个子单元在第一目标图像中的多个像素值，从多个像素值中获取多个第一全色像素合并读出第二全色像素值，以及从多个像素值中获取多个相同颜色的第一彩色像素值合并读出第二彩色像素值。
可以理解的是，合并读出可以包括求均值、求和或加权平均等其中一种方式，在此不做限定。
操作1604，将第一目标图像中对应在每个子单元中的多个相同颜色的第一彩色像素值合并读出第二彩色像素值，并基于各个第二彩色像素值生成第一彩色图像。
具体地，在第二分辨率模式下，电子设备将第一目标图像中对应在每个子单元中的多个相同颜色的第一彩色像素值合并读出第二彩色像素值，并根据预设像素读取方式，从各个第二彩色像素值读取像素值以生成第一彩色图像。
可以理解的是，合并读出可以包括求均值、求和或加权平均等其中一种方式，在此不做限定。
以图15为第一目标图像为例，则生成的第一彩色图像如图17中的1702所示，生成的第一全色图像如1704所示。在图17中，w表示第二全色像素值，a、b和c表示三种不同颜色的第二彩色像素值。
操作1606，基于第一全色图像和第一彩色图像，生成第二目标图像。
当需要将第一全色图像和第一彩色图像进行打包传输时，电子设备可以基于第一全色图像和第一彩色图像，生成第二目标图像，将第二目标图像进行传输。
具体地，电子设备将第一全色图像中每一行第二全色像素值与第一彩色图像中每一行第二彩色像素值相间排布，生成第二目标图像；或者将第一全色图像中每一列第二全色像素值与第一彩色图像中每一列第二彩色像素值相间排布，生成第二目标图像。
图18和图19为第一全色图像中每一行第二全色像素值与第一彩色图像中每一行第二彩色像素值相 间排布得到的第二目标图像的示意图。图20和图21为第一全色图像中每一列第二全色像素值与第一彩色图像中每一列第二彩色像素值相间排布得到的第二目标图像。
在另一实施例中，电子设备还可以将第一全色图像和第一彩色图像中同一位置的像素值进行合并，得到对应位置的合并像素值，基于各合并像素值构成第二目标图像。其中，合并可以采用求均值、加权求平均或相加求和等其中一种方式。
在其他实施例中，电子设备还可以采用其他方式生成第二目标图像，在此不做限定。
在本实施例中，在第二分辨率模式下，将第一目标图像中对应在每个子单元中的多个第一全色像素值合并读出第二全色像素值，以及将第一目标图像中对应在每个子单元中的多个相同颜色的第一彩色像素值合并读出第二彩色像素值，能够将各个不同的彩色像素242进行混合排列，使得所生成的第二目标图像中的各个第二彩色像素如RGB像素分布更均匀，图像质量更高。并且，所得到第二目标图像的分辨率和图像尺寸进一步减小，且全色像素241具有更高的信噪比，图像的帧率高，从而达到了二级像素合并输出的功耗更低、信噪比更佳的图像处理效果。并且，在第二分辨率模式下，具有全排列的全色像素，无需做插值，提高了整体的解析力。同时，在全尺寸情形下，各个颜色的彩色像素如第一颜色的像素和第三颜色的像素在对角方向或反对角方向更加分散均衡。
在另一个实施例中，上述方法还包括：在第二分辨率模式下，将每个第一子单元或者第二子单元中多个全色滤光片的各个全色子滤光片对应的全色像素合并读出第五全色像素值，并基于各个第五全色像素值生成第三全色图像；将每个第一子单元或者第二子单元中多个相同颜色的彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出第五彩色像素值，并基于各个第五彩色像素值生成第三彩色图像；基于第三全色图像和第三彩色图像，生成第五目标图像。
其中，合并读出的方式可以是求平均、加权平均或相加等其中一种。
当需要将第三全色图像和第三彩色图像进行打包传输时，电子设备可以基于第三全色图像和第三彩色图像，生成第五目标图像，再将第五目标图像进行传输。
其中，基于第三全色图像和第三彩色图像，生成第五目标图像，包括：将第三全色图像中每一行第五全色像素值与第三彩色图像中每一行第五彩色像素值相间排布，生成第五目标图像；或者将第三全色图像中每一列第五全色像素值与第三彩色图像中每一列第五彩色像素值相间排布，生成第五目标图像。
在另一实施例中，电子设备还可以将第三全色图像和第三彩色图像中同一位置的像素值进行合并，得到对应位置的合并像素值，基于各合并像素值构成第五目标图像。其中，合并读出可以采用求均值、加权求平均或相加求和等其中一种方式。
在其他实施例中，电子设备还可以采用其他方式生成第五目标图像，在此不做限定。
在本实施例中，在第二分辨率模式下，将每个第一子单元或者第二子单元中多个全色滤光片的各个全色子滤光片对应的全色像素合并读出第五全色像素值，以及将每个第一子单元或者第二子单元中多个相同颜色的彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出第五彩色像素值，可以更快速生成第三全色图像和第三彩色图像，从而更快速生成第五目标图像。
并且，上述实施例能够将各个不同的彩色像素进行混合排列，使得所生成的第五目标图像中的各个第四彩色像素值如RGB像素分布更均匀，图像质量更高。并且，所得到第五目标图像的分辨率和图像尺寸进一步减小，且全色像素241具有更高的信噪比，图像的帧率高，从而达到了二级像素合并输出的功耗更低、信噪比更佳的图像处理效果。
在一个实施例中，如图22所示，上述方法还包括：
操作2202，在第三分辨率模式下，将第一全色图像中对应在同一滤光片组中的多个第二全色像素值合并读出第三全色像素值，并基于各个第三全色像素值生成第二全色图像；第三分辨率模式对应的分辨率小于第二分辨率模式对应的分辨率。
第三分辨率模式是指对分辨率要求比第二分辨率模式要求低的场景下所使用的模式，是低分辨率、低功耗、高信噪比、高帧率的三级像素合并读出模式。该第三分辨率模式对应的分辨率、功耗小于第二分辨率模式对应的分辨率、功耗。该第三分辨率模式对应的信噪比、帧率大于第二分辨率模式对应的信噪比、帧率。
第三分辨率模式具体可以是图像拍摄时的预览模式、视频拍摄时的预览模式，或者在夜景下进行图像拍摄、视频拍摄的夜景模式等分辨率要求较低的场景，但不限于此。视频拍摄的预览模式例如720p视频预览、应用视频预览等。
电子设备从滤光片阵列中确定每个滤光片组，获取每个滤光片组所得到的第一全色图像中的多个第二全色像素值，将多个第二全色像素值合并读出第三全色像素值。电子设备根据预设像素读取方式，从各个第三全色像素值中读取像素值以第二全色图像。
操作2204，将第一彩色图像中对应在同一滤光片组中的多个同种颜色的第二彩色像素值合并读出第一颜色的第三彩色像素值、第二颜色的第三彩色像素值和第三颜色的第三彩色像素值，并基于第一颜色的第三彩色像素值、第二颜色的第三彩色像素值和第三颜色的第三彩色像素值，生成双颜色的第二彩色图像和单颜色的第二彩色图像；双颜色的第二彩色图像包括第一颜色的第三彩色像素值和第三颜色的第三彩色像素值，单颜色的第二彩色图像包括第二颜色的第三彩色像素值。
第一颜色的第三彩色像素值是第一颜色滤光片对应的像素读出的像素值，第二颜色的第三彩色像素值是第二颜色滤光片对应的像素读出的像素值，第三颜色的第三彩色像素值是第三颜色滤光片对应的像素读出的像素值。
电子设备从滤光片阵列中确定每个滤光片组，获取每个滤光片组所得到的第一彩色图像中的多个同种颜色的第二彩色像素值，同种颜色的第二彩色像素值包括第一颜色的第二彩色像素值、第二颜色的第二彩色像素值和第三颜色的第二彩色像素值，将多个第一颜色的第二彩色像素值合并读出第一颜色的第三全色像素值，将多个第二颜色的第二彩色像素值合并读出第二颜色的第三全色像素值，将多个第三颜色的第二彩色像素值合并读出第三颜色的第三全色像素值。
双颜色的第二彩色图像包括第一颜色的第三彩色像素值和第三颜色的第三彩色像素值。单颜色的第二彩色图像包括第二颜色的第三彩色像素值。其中，第一颜色的第三彩色像素值在双颜色的第二彩色图像的对角线上排布，第三颜色的第三彩色像素值在双颜色的第二彩色图像的反对角线上排布。
以图17的第一彩色图像和第一全色图像为例，在第三分辨率模式下，电子设备将第一全色图像1704中对应在同一滤光片组中的多个第二全色像素值合并读出第三全色像素值，并基于各个第三全色像素值生成图23中的第二全色图像2302；第三分辨率模式对应的分辨率小于第二分辨率模式对应的分辨率；将第一彩色图像1702中对应在同一滤光片组中的多个同种颜色的第二彩色像素值合并读出第一颜色的第三彩色像素值、第二颜色的第三彩色像素值和第三颜色的第三彩色像素值，并基于第一颜色的第三彩色像素值、第二颜色的第三彩色像素值和第三颜色的第三彩色像素值，生成图23中的双颜色的第二彩色图像2304和单颜色的第二彩色图像2306。在图23中，w表示第三全色像素值，a、b和c表示三种不同颜色的第三彩色像素值。
操作2206，基于第二全色图像、双颜色的第二彩色图像和单颜色的第二彩色图像，生成第三目标图像。
当需要将第二全色图像、双颜色的第二彩色图像和单颜色的第二彩色图像进行打包传输时，电子设备可以基于第二全色图像、双颜色的第二彩色图像和单颜色的第二彩色图像，生成第三目标图像，再将第三目标图像进行传输。
具体地，电子设备将第二全色图像中每一行第三全色像素值、双颜色的第二彩色图像中每一行第三彩色像素值和单颜色的第二彩色图像中每一行第三彩色像素值相间排布，生成第二目标图像；或者将第二全色图像中每一列第三全色像素值、双颜色的第二彩色图像中每一列第三彩色像素值和单颜色的第二彩色图像中每一列第三彩色像素值相间排布，生成第二目标图像。
以图23中的第二全色图像2302、双颜色的第二彩色图像2304和单颜色的第二彩色图像2306为例，图24是一种实施方式中第二全色图像中每一行第三全色像素值、双颜色的第二彩色图像中每一行第三彩色像素值和单颜色的第二彩色图像中每一行第三彩色像素值相间排布所生成的第三目标图像；图25是另一种实施方式中第二全色图像中每一列第三全色像素值、双颜色的第二彩色图像中每一列第三彩色像素值和单颜色的第二彩色图像中每一列第三彩色像素值相间排布所生成的第三目标图像。
需要说明的是，同一坐标的第二全色图像中第三全色像素值、双颜色的第二彩色图像中第三彩色像 素值和单颜色的第二彩色图像中第三彩色像素值在排列时并不限定顺序。
在其他实施例中，电子设备还可以采用其他方式生成第三目标图像，在此不做限定。
在本实施例中，在第三分辨率模式下，将第一全色图像中对应在同一滤光片组中的多个第二全色像素值合并读出第三全色像素值，将第一彩色图像中对应在同一滤光片组中的多个同种颜色的第二彩色像素值合并读出第一颜色的第三彩色像素值、第二颜色的第三彩色像素值和第三颜色的第三彩色像素值，能够将各个不同的彩色像素进行混合排列，使得所生成的第三目标图像中的第三彩色像素如RGB像素分布更均匀，图像质量更高。并且，所得到第三目标图像的分辨率和图像尺寸进一步减小，且全色像素具有更高的信噪比，图像的帧率高，从而达到了三级像素合并输出的功耗更低、信噪比更佳的图像处理效果。并且，在第三分辨率模式下，第三目标图像中包括有全排列的全色像素，可以提高整体的解析力。同时，在第三分辨率模式下，也无需跨越周期做同个颜色的像素的合并，无需做插值，提高了整体的解析力。在全尺寸情形下，各个颜色的彩色像素如第一颜色的像素值和第三颜色的像素值在对角线或反对角线更加分散均衡。其中，全排列指的是每一坐标均有该像素，不需要做插值估计。
在另一个实施例中，上述方法还包括：在第三分辨率模式下，将每个滤光片组中的多个全色滤光片的各个全色子滤光片对应的全色像素合并读出第六全色像素值，并基于各个第六全色像素值生成第四全色图像；将每个滤光片组中多个同种颜色的彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出第一颜色的第六彩色像素值、第二颜色的第六彩色像素值和第三颜色的第六彩色像素值，并基于第一颜色的第六彩色像素值、第二颜色的第六彩色像素值和第三颜色的第六彩色像素值，生成双颜色的第四彩色图像和单颜色的第四彩色图像；双颜色的第四彩色图像包括第一颜色的第六彩色像素值和第三颜色的第六彩色像素值，单颜色的第四彩色图像包括第二颜色的第六彩色像素值；基于第四全色图像、双颜色的第四彩色图像和单颜色的第四彩色图像，生成第六目标图像。
其中，合并读出的方式可以是求平均、加权平均或相加等其中一种。
当需要将第四全色图像、双颜色的第四彩色图像和单颜色的第四彩色图像进行打包传输时，电子设备可以基于第四全色图像、双颜色的第四彩色图像和单颜色的第四彩色图像，生成第六目标图像，再将第六目标图像进行传输。
其中，基于第四全色图像、双颜色的第四彩色图像和单颜色的第四彩色图像，生成第六目标图像，包括：将第四全色图像中每一行第六全色像素值、双颜色的第四彩色图像中每一行第六彩色像素值和单颜色的第四彩色图像中每一行第六彩色像素值相间排布，生成第六目标图像；或者将第四全色图像中每一列第六全色像素值、双颜色的第四彩色图像中每一列第六彩色像素值和单颜色的第四彩色图像中每一列第六彩色像素值相间排布，生成第六目标图像。
在其他实施例中，电子设备还可以采用其他方式生成第六目标图像，在此不做限定。
在本实施例中，在第三分辨率模式下，将每个滤光片组中的多个全色滤光片的各个全色子滤光片对应的全色像素合并读出第五全色像素值，从而可以更快速生成第四全色图像；将每个滤光片组中多个同种颜色的彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出第一颜色的第五彩色像素值、第二颜色的第五彩色像素值和第三颜色的第五彩色像素值，从而可以更快速生成双颜色的第四彩色图像和单颜色的第四彩色图像。并且，上述实施例能够将各个不同的彩色像素进行混合排列，使得所生成的第六目标图像中的各个第六彩色像素值如RGB像素分布更均匀，图像质量更高。并且，所得到第六目标图像的分辨率和图像尺寸进一步减小，且全色像素241具有更高的信噪比，图像的帧率高，从而达到了三级像素合并输出的功耗更低、信噪比更佳的图像处理效果。
在一个实施例中，如图26所示，上述方法还包括：
操作2602，在第四分辨率模式下，将第二全色图像中各个第三全色像素值合并读出第四全色像素值；第四分辨率模式对应的分辨率小于第三分辨率模式对应的分辨率。
第四分辨率模式是指对分辨率要求比第三分辨率模式要求低的场景下所使用的模式，是低分辨率、低功耗、高信噪比、高帧率的四级像素合并读出模式。该第四分辨率模式对应的分辨率、功耗小于第三分辨率模式对应的分辨率、功耗。该第四分辨率模式对应的信噪比、帧率大于第三分辨率模式对应的信噪比、帧率。
第四分辨率模式具体可以是图像拍摄时的预览模式、视频拍摄时的预览模式，或者在夜景下进行图像拍摄、视频拍摄的夜景模式等分辨率要求较低的场景，但不限于此。视频拍摄的预览模式例如480p视频预览、应用视频预览等。
电子设备从滤光片阵列中确定每个最小重复单元，获取每个最小重复单元所得到的第二全色图像中的多个第三全色像素值，将多个第三全色像素值合并读出第四全色像素值。
操作2604，将双颜色的第二彩色图像中多个第一颜色的第三彩色像素值合并读出第一颜色的第四彩色像素值，将双颜色的第二彩色图像中多个第三颜色的第三彩色像素值合并读出第三颜色的第四彩色像素值，以及将单颜色的第二彩色图像中多个第二颜色的第三彩色像素值合并读出第二颜色的第四彩色像素值。
电子设备从滤光片阵列中确定每个最小重复单元，获取每个最小重复单元所得到的第二彩色图像中的多个同种颜色的第三彩色像素值，同种颜色的第三彩色像素值包括第一颜色的第三彩色像素值、第二颜色的第三彩色像素值和第三颜色的第三彩色像素值，将多个第一颜色的第三彩色像素值合并读出第一颜色的第四全色像素值，将多个第二颜色的第三彩色像素值合并读出第二颜色的第四全色像素值，将多个第三颜色的第三彩色像素值合并读出第三颜色的第四全色像素值。
操作2606，基于第四全色像素值、第一颜色的第四彩色像素值、第二颜色的第四彩色像素值和第三颜色的第四彩色像素值，生成第四目标图像。
具体地，电子设备将对应于同个最小重复单元的第四全色像素值、第一颜色的第四彩色像素值、第二颜色的第四彩色像素值和第三颜色的第四彩色像素值相间排布，生成第四目标图像。需要说明的是，对应于同个最小重复单元的第四全色像素值、第一颜色的第四彩色像素值、第二颜色的第四彩色像素值和第三颜色的第四彩色像素值，在排布时并不限定顺序。
以图23中的第二全色图像2302、双颜色的第二彩色图像2304和单颜色的第二彩色图像2306为例，图27是一个实施例中第四目标图像的示意图。
需要说明的是，同一坐标的第二全色图像中第三全色像素值、双颜色的第二彩色图像中第三彩色像素值和单颜色的第二彩色图像中第三彩色像素值在排列时并不限定顺序。
在本实施例中，在第四分辨率模式下，将第二全色图像中各个第三全色像素值合并读出第四全色像素值，将双颜色的第二彩色图像中多个第一颜色的第三彩色像素值合并读出第一颜色的第四彩色像素值，将双颜色的第二彩色图像中多个第三颜色的第三彩色像素值合并读出第三颜色的第四彩色像素值，以及将单颜色的第二彩色图像中多个第二颜色的第三彩色像素值合并读出第二颜色的第四彩色像素值，所得到第四目标图像的分辨率和图像尺寸进一步减小，且全色像素具有更高的信噪比，图像的帧率高，从而达到了四级像素合并输出的功耗更低、信噪比更佳的图像处理效果。并且，在第四分辨率模式下，第四目标图像能够与高像素图像传感器相匹配，兼顾高像素下的高解析力和低像素下的高信噪比。同时，在第四分辨率模式下，也无需跨越周期做同个颜色的像素的合并，无需做插值，提高了整体的解析力。
在另一个实施例中，上述方法还包括：在第四分辨率模式下，将最小重复中多个全色滤光片的各个全色子滤光片对应的全色像素合并读出第七全色像素值，将最小重复单元中多个同种颜色的彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出第七彩色像素值；基于各个第七全色像素值和各个第七彩色像素值，生成第七目标图像。
其中，合并读出的方式可以是求平均、加权平均或相加等其中一种。
需要说明的是，对应于同个最小重复单元的第七全色像素值和各个第七彩色像素值，在排布时并不限定顺序。其中，彩色滤光片包括第一颜色滤光片、第二颜色滤光片和第三颜色滤光片。电子设备将最小重复单元中多个第一颜色滤光片的各个子滤光片对应的彩色像素合并读出第一颜色的第七彩色像素值、将最小重复单元中多个第二颜色滤光片的各个子滤光片对应的彩色像素合并读出第二颜色的第七彩色像素值，以及将最小重复单元中多个第三颜色滤光片的各个子滤光片对应的彩色像素合并读出第三颜色的第七彩色像素值。
在本实施例中，在第四分辨率模式下，将最小重复中多个全色滤光片的各个全色子滤光片对应的全色像素合并读出第七全色像素值，将最小重复单元中多个同种颜色的彩色滤光片的各个彩色子滤光片对 应的彩色像素合并读出第七彩色像素值，基于各个第七全色像素值和各个第七彩色像素值所得到第七目标图像的分辨率和图像尺寸进一步减小，且全色像素具有更高的信噪比，图像的帧率高，从而达到了四级像素合并输出的功耗更低、信噪比更佳的图像处理效果。并且，在第四分辨率模式下，第四目标图像能够与高像素图像传感器相匹配，兼顾高像素下的高解析力和低像素下的高信噪比。同时，在第四分辨率模式下，也无需跨越周期做同个颜色的像素的合并，无需做插值，提高了整体的解析力。
在一个实施例中，还提供了另一种图像生成方法，应用于图像传感器，图像传感器包括滤光片阵列和像素阵列，滤光片阵列包括最小重复单元，最小重复单元包括多个滤光片组，每个滤光片组包括第一子单元和第二子单元；第一子单元和第二子单元均包括全色滤光片和彩色滤光片，第一子单元中的彩色滤光片排布于所在的第一子单元中的对角线上，第二子单元中的彩色滤光片排布于所在的第二子单元中的反对角线上；全色滤光片透过的进光量大于彩色滤光片透过的进光量；像素阵列中各个像素与滤光片阵列的滤光片对应设置，像素阵列被配置成用于接收穿过滤光片阵列的光线以生成电信号；该图像生成方法包括：在全分辨率模式下，将每个全色滤光片对应的全色像素读出全分辨率全色像素值，以及将每个彩色滤光片对应的彩色像素读出全分辨率彩色像素值；基于各个全分辨率全色像素值和各个全分辨率彩色像素值，生成全分辨率目标图像。
本实施例中生成全分辨率目标图像的原理，与图12的实施例中生成全分辨率目标图像的原理类似，在此不做赘述。
需要说明的是，若N为1，即滤光片不包含子滤光片，那么该滤光片阵列还具有第一分辨率模式、第二分辨率模式和第三分辨率模式，并且N为1对应的第一分辨率模式的原理与N大于或等于2对应的第二分辨率模式的原理类似，N为1对应的第二分辨率模式的原理与N大于或等于2对应的第三分辨率模式的原理类似，N为1对应的第三分辨率模式的原理与N大于或等于2对应的第四分辨率模式的原理类似，在此不做赘述。
应该理解的是，虽然图13、图14、图16、图22和图26的流程图中的各个操作按照箭头的指示依次显示，但是这些操作并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些操作的执行并没有严格的顺序限制，这些操作可以以其它的顺序执行。而且，图13、图14、图16、图22和图26中的至少一部分操作可以包括多个子操作或者多个阶段，这些子操作或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些子操作或者阶段的执行顺序也不必然是依次进行，而是可以与其它操作或者其它操作的子操作或者阶段的至少一部分轮流或者交替地执行。
图28为一个实施例的图像生成装置的结构框图。如图28所示，提供了一种图像生成装置，应用于图像传感器，图像传感器包括滤光片阵列和像素阵列，滤光片阵列包括最小重复单元，最小重复单元包括多个滤光片组，每个滤光片组包括第一子单元和第二子单元；第一子单元和第二子单元均包括全色滤光片和彩色滤光片，第一子单元中的彩色滤光片排布于所在的第一子单元中的对角线上，第二子单元中的彩色滤光片排布于所在的第二子单元中的反对角线上；全色滤光片透过的进光量大于彩色滤光片透过的进光量；每个全色滤光片包括N行N列个全色子滤光片，每个彩色滤光片包括N行N列个彩色子滤光片，N行N列个彩色子滤光片与彩色滤光片的颜色相同，N为大于或等于2的正整数；像素阵列中各个像素与滤光片阵列的子滤光片对应设置，像素阵列被配置成用于接收穿过滤光片阵列的光线以生成电信号；
该图像生成装置包括：读出模块2802和图像生成模块2804，其中：
读出模块2802，用于在全分辨率模式下，将全色滤光片中每个全色子滤光片对应的全色像素读出全分辨率全色像素值，以及将彩色滤光片中每个彩色子滤光片对应的彩色像素读出全分辨率彩色像素值。
图像生成模块2804，用于基于各个全分辨率全色像素值和各个全分辨率彩色像素值，生成全分辨率目标图像。
上述图像生成装置，在全分辨率模式下，将全色滤光片中每个全色子滤光片对应的全色像素读出全 分辨率全色像素值，以及将彩色滤光片中每个彩色子滤光片对应的彩色像素读出全分辨率彩色像素值；而全色滤光片透过的进光量大于彩色滤光片透过的进光量，能够将全色通道信息融合到图像中，提高整体的进光量，从而基于各个全分辨率全色像素值和各个全分辨率彩色像素值，能够生成信息更多、细节解析更清晰的全分辨率目标图像。
在该滤光片阵列中，最小重复单元包括多个滤光片组，每个滤光片组包括第一子单元和第二子单元；第一子单元和第二子单元均包括全色滤光片和彩色滤光片，第一子单元中的彩色滤光片排布于所在的第一子单元中的对角线上，第二子单元中的彩色滤光片排布于所在的第二子单元中的反对角线上，可以使得对角线的方向上和反对角线的方向上的彩色滤光片的排布更加均衡，从而在生成全分辨率目标图像时彩色通道具有更强的分辨能力。
在一个实施例中，上述读出模块2802还用于在第一分辨率模式下，将每个全色滤光片中各个全色子滤光片对应的全色像素合并读出第一全色像素值，以及将每个彩色滤光片中各个彩色子滤光片对应的彩色像素合并读出第一彩色像素值；第一分辨率模式对应的分辨率小于全分辨率模式对应的分辨率；上述图像生成模块2804还用于基于各个第一全色像素值和各个第一彩色像素值，生成第一目标图像。
在一个实施例中，上述读出模块2802还用于在第二分辨率模式下，将第一目标图像中对应在每个子单元中的多个第一全色像素值合并读出第二全色像素值，子单元包括第一子单元和第二子单元；上述图像生成模块2804还用于基于各个第二全色像素值生成第一全色图像；第二分辨率模式对应的分辨率小于第一分辨率模式对应的分辨率；上述读出模块2802还用于将第一目标图像中对应在每个子单元中的多个相同颜色的第一彩色像素值合并读出第二彩色像素值，上述图像生成模块2804还用于基于各个第二彩色像素值生成第一彩色图像；上述图像生成模块2804还用于基于第一全色图像和第一彩色图像，生成第二目标图像。
在一个实施例中，上述图像生成模块2804还用于将第一全色图像中每一行第二全色像素值与第一彩色图像中每一行第二彩色像素值相间排布，生成第二目标图像；或者将第一全色图像中每一列第二全色像素值与第一彩色图像中每一列第二彩色像素值相间排布，生成第二目标图像。
在一个实施例中，上述读出模块2802还用于在第三分辨率模式下，将第一全色图像中对应在同一滤光片组中的多个第二全色像素值合并读出第三全色像素值，上述图像生成模块2804还用于基于各个第三全色像素值生成第二全色图像；第三分辨率模式对应的分辨率小于第二分辨率模式对应的分辨率；上述读出模块2802还用于将第一彩色图像中对应在同一滤光片组中的多个同种颜色的第二彩色像素值合并读出第一颜色的第三彩色像素值、第二颜色的第三彩色像素值和第三颜色的第三彩色像素值，上述图像生成模块2804还用于基于第一颜色的第三彩色像素值、第二颜色的第三彩色像素值和第三颜色的第三彩色像素值，生成双颜色的第二彩色图像和单颜色的第二彩色图像；双颜色的第二彩色图像包括第一颜色的第三彩色像素值和第三颜色的第三彩色像素值，单颜色的第二彩色图像包括第二颜色的第三彩色像素值；上述图像生成模块2804还用于基于第二全色图像、双颜色的第二彩色图像和单颜色的第二彩色图像，生成第三目标图像。
在一个实施例中，上述图像生成模块2804还用于将第二全色图像中每一行第三全色像素值、双颜色的第二彩色图像中每一行第三彩色像素值和单颜色的第二彩色图像中每一行第三彩色像素值相间排布，生成第二目标图像；或者将第二全色图像中每一列第三全色像素值、双颜色的第二彩色图像中每一列第三彩色像素值和单颜色的第二彩色图像中每一列第三彩色像素值相间排布，生成第二目标图像。
在一个实施例中，上述读出模块2802还用于在第四分辨率模式下，将第二全色图像中各个第三全色像素值合并读出第四全色像素值；第四分辨率模式对应的分辨率小于第三分辨率模式对应的分辨率；上述读出模块2802还用于将双颜色的第二彩色图像中多个第一颜色的第三彩色像素值合并读出第一颜色的第四彩色像素值，将双颜色的第二彩色图像中多个第三颜色的第三彩色像素值合并读出第三颜色的第四彩色像素值，以及将单颜色的第二彩色图像中多个第二颜色的第三彩色像素值合并读出第二颜色的第四彩色像素值；上述图像生成模块2804还用于基于第四全色像素值、第一颜色的第四彩色像素值、第二颜色的第四彩色像素值和第三颜色的第四彩色像素值，生成第四目标图像。
在一个实施例中，上述读出模块2802还用于在第二分辨率模式下，将每个第一子单元或者第二子 单元中多个全色滤光片的各个全色子滤光片对应的全色像素合并读出第五全色像素值，上述图像生成模块2804还用于基于各个第五全色像素值生成第三全色图像；上述读出模块2802还用于将每个第一子单元或者第二子单元中多个相同颜色的彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出第五彩色像素值，上述图像生成模块2804还用于基于各个第五彩色像素值生成第三彩色图像；上述图像生成模块2804还用于基于第三全色图像和第三彩色图像，生成第五目标图像。
在一个实施例中，上述图像生成模块2804还用于将第三全色图像中每一行第五全色像素值与第三彩色图像中每一行第五彩色像素值相间排布，生成第五目标图像；或者将第三全色图像中每一列第五全色像素值与第三彩色图像中每一列第五彩色像素值相间排布，生成第五目标图像。
在一个实施例中，上述读出模块2802还用于在第三分辨率模式下，将每个滤光片组中的多个全色滤光片的各个全色子滤光片对应的全色像素合并读出第六全色像素值，上述图像生成模块2804还用于基于各个第六全色像素值生成第四全色图像；上述读出模块2802还用于将每个滤光片组中多个同种颜色的彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出第一颜色的第六彩色像素值、第二颜色的第六彩色像素值和第三颜色的第六彩色像素值，上述图像生成模块2804还用于基于第一颜色的第六彩色像素值、第二颜色的第六彩色像素值和第三颜色的第六彩色像素值，生成双颜色的第四彩色图像和单颜色的第四彩色图像；双颜色的第四彩色图像包括第一颜色的第六彩色像素值和第三颜色的第六彩色像素值，单颜色的第四彩色图像包括第二颜色的第六彩色像素值；上述图像生成模块2804还用于基于第四全色图像、双颜色的第四彩色图像和单颜色的第四彩色图像，生成第六目标图像。
在一个实施例中，上述图像生成模块2804还用于将第四全色图像中每一行第六全色像素值、双颜色的第四彩色图像中每一行第六彩色像素值和单颜色的第四彩色图像中每一行第六彩色像素值相间排布，生成第六目标图像；或者将第四全色图像中每一列第六全色像素值、双颜色的第四彩色图像中每一列第六彩色像素值和单颜色的第四彩色图像中每一列第六彩色像素值相间排布，生成第六目标图像。
在一个实施例中，上述读出模块2802还用于在第四分辨率模式下，将最小重复中多个全色滤光片的各个全色子滤光片对应的全色像素合并读出第七全色像素值，将最小重复单元中多个同种颜色的彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出第七彩色像素值；上述图像生成模块2804还用于基于各个第七全色像素值和各个第七彩色像素值，生成第七目标图像。
上述图像生成装置中各个模块的划分仅仅用于举例说明，在其他实施例中，可将图像生成装置按照需要划分为不同的模块，以完成上述图像生成装置的全部或部分功能。
关于图像生成装置的具体限定可以参见上文中对于图像生成方法的限定，在此不再赘述。上述图像生成装置中的各个模块可全部或部分通过软件、硬件及其组合来实现。上述各模块可以硬件形式内嵌于或独立于计算机设备中的处理器中，也可以以软件形式存储于计算机设备中的存储器中，以便于处理器调用执行以上各个模块对应的操作。
图29为一个实施例中电子设备的内部结构示意图。该电子设备可以是手机、平板电脑、笔记本电脑、台式电脑、PDA(Personal Digital Assistant，个人数字助理)、POS(Point of Sales，销售终端)、车载电脑、穿戴式设备等任意终端设备。该电子设备包括通过系统总线连接的处理器和存储器。其中，该处理器可以包括一个或多个处理单元。处理器可为CPU(Central Processing Unit,中央处理单元)或DSP(Digital Signal Processing，数字信号处理器)等。存储器可包括非易失性存储介质及内存储器。非易失性存储介质存储有操作系统和计算机程序。该计算机程序可被处理器所执行，以用于实现以下各个实施例所提供的一种图像生成方法。内存储器为非易失性存储介质中的操作系统计算机程序提供高速缓存的运行环境。
本申请实施例中提供的图像生成装置中的各个模块的实现可为计算机程序的形式。该计算机程序可在终端或服务器上运行。该计算机程序构成的程序模块可存储在电子设备的存储器上。该计算机程序被处理器执行时，实现本申请实施例中所描述方法的操作。
本申请实施例还提供了一种计算机可读存储介质。一个或多个包含计算机可执行指令的非易失性计算机可读存储介质，当所述计算机可执行指令被一个或多个处理器执行时，使得所述处理器执行图像生成方法的操作。
本申请实施例还提供了一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行图像生成方法。
本申请所使用的对存储器、存储、数据库或其它介质的任何引用可包括非易失性和/或易失性存储器。非易失性存储器可包括ROM(Read-Only Memory，只读存储器)、PROM(Programmable Read-only Memory，可编程只读存储器)、EPROM(Erasable Programmable Read-Only Memory，可擦除可编程只读存储器)、EEPROM(Electrically Erasable Programmable Read-only Memory，电可擦除可编程只读存储器)或闪存。易失性存储器可包括RAM(Random Access Memory，随机存取存储器)，它用作外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如SRAM(Static Random Access Memory，静态随机存取存储器)、DRAM(Dynamic Random Access Memory，动态随机存取存储器)、SDRAM(Synchronous Dynamic Random Access Memory，同步动态随机存取存储器)、双数据率DDR SDRAM(Double Data Rate Synchronous Dynamic Random Access memory，双数据率同步动态随机存取存储器)、ESDRAM(Enhanced Synchronous Dynamic Random Access memory，增强型同步动态随机存取存储器)、SLDRAM(Sync Link Dynamic Random Access Memory，同步链路动态随机存取存储器)、RDRAM(Rambus Dynamic Random Access Memory，总线式动态随机存储器)、DRDRAM(Direct Rambus Dynamic Random Access Memory，接口动态随机存储器)。
以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对本申请专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。

Claims (30)

  1. 一种图像传感器，其特征在于，所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，所述最小重复单元包括多个滤光片组，每个滤光片组包括第一子单元和第二子单元；所述第一子单元和所述第二子单元均包括全色滤光片和彩色滤光片，所述第一子单元中的彩色滤光片排布于所在的第一子单元中的对角线上，所述第二子单元中的彩色滤光片排布于所在的第二子单元中的反对角线上；所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量；每个所述全色滤光片包括N行N列个全色子滤光片，每个所述彩色滤光片包括N行N列个彩色子滤光片，所述N行N列个彩色子滤光片与所述彩色滤光片的颜色相同，所述N为正整数；所述像素阵列中各个像素与所述滤光片阵列的子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号。
  2. 根据权利要求1所述的图像传感器，其特征在于，每一行和所述每一列上均包括每种颜色的彩色滤光片。
  3. 根据权利要求1所述的图像传感器，其特征在于，所述最小重复单元包括2个第一滤光片组和2个第二滤光片组，所述2个第一滤光片组在所述最小重复单元的对角线上排布，所述2个第二滤光片组在所述最小重复单元的反对角线上排布。
  4. 根据权利要求3所述的图像传感器，其特征在于，所述2个第一滤光片组完全相同，所述2个第二滤光片组完全相同。
  5. 根据权利要求3所述的图像传感器，其特征在于，所述每个滤光片组仅包括全色滤光片和2种颜色的彩色滤光片。
  6. 根据权利要求5所述的图像传感器，其特征在于，所述第一滤光片组中的彩色滤光片包括第一颜色滤光片和第二颜色滤光片，所述第二滤光片组中的彩色滤光片包括第二颜色滤光片和第三颜色滤光片。
  7. 根据权利要求3所述的图像传感器，其特征在于，所述每个滤光片组均包括2个第一子单元和2个第二子单元；所述2个第一子单元在所述滤光片组的第一行方向上排布，以及所述2个第二子单元在所述滤光片组的第二行方向上排布，所述第一行方向和所述第二行方向相邻排布；或者，所述2个第一子单元在所述滤光片组的第一列方向上排布，以及所述2个第二子单元在所述滤光片组的第二列方向上排布，所述第一列方向和所述第二列方向相邻排布。
  8. 根据权利要求7所述的图像传感器，其特征在于，所述彩色滤光片包括第一颜色滤光片、第二颜色滤光片和第三颜色滤光片；所述第一滤光片组中的2个第一子单元的其中一个第一子单元包括第一颜色滤光片，另一个第一子单元包括第二颜色滤光片；所述第一滤光片组中的2个第二子单元的其中一个第二子单元包括第一颜色滤光片，另一个第二子单元包括第二颜色滤光片。
  9. 根据权利要求7所述的图像传感器，其特征在于，彩色滤光片包括第一颜色滤光片、第二颜色滤光片和第三颜色滤光片；所述第二滤光片组中的2个第一子单元的其中一个第一子单元包括第二颜色滤光片，另一个第一子单元包括第三颜色滤光片；所述第二滤光片组中的2个第二子单元的其中一个第二子单元包括第二颜色滤光片，另一个第二子单元包括第三颜色滤光片。
  10. 根据权利要求7所述的图像传感器，其特征在于，同一滤光片组中包含同种颜色滤光片的第一子单元和第二子单元排布于所述同一滤光片组的对角线或反对角线上。
  11. 根据权利要求1所述的图像传感器，其特征在于，所述N为1，所述最小重复单元包括8行8列64个滤光片，排布方式为：
    Figure PCTCN2022123995-appb-100001
    Figure PCTCN2022123995-appb-100002
    或者
    Figure PCTCN2022123995-appb-100003
    或者
    Figure PCTCN2022123995-appb-100004
    或者
    Figure PCTCN2022123995-appb-100005
    其中，w表示全色滤光片，a、b和c均表示彩色滤光片。
  12. 根据权利要求1所述的图像传感器，其特征在于，所述N为2，所述最小重复单元包括16行16列256个子滤光片，排布方式为：
    Figure PCTCN2022123995-appb-100006
    Figure PCTCN2022123995-appb-100007
    或者
    Figure PCTCN2022123995-appb-100008
    或者
    Figure PCTCN2022123995-appb-100009
    或者
    Figure PCTCN2022123995-appb-100010
    Figure PCTCN2022123995-appb-100011
    其中，w表示全色子滤光片，a、b和c均表示彩色子滤光片。
  13. 一种摄像模组，其特征在于，所述摄像模组包括镜头和权利要求1-12中任一项所述的图像传感器；所述图像传感器用于接收穿过所述镜头的光线，所述像素根据所述光线生成电信号。
  14. 一种电子设备，其特征在于，包括：
    权利要求13所述的摄像模组；及
    壳体，所述摄像模组设置在所述壳体上。
  15. 一种图像生成方法，应用于图像传感器，其特征在于，所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，所述最小重复单元包括多个滤光片组，每个滤光片组包括第一子单元和第二子单元；所述第一子单元和所述第二子单元均包括全色滤光片和彩色滤光片，所述第一子单元中的彩色滤光片排布于所在的第一子单元中的对角线上，所述第二子单元中的彩色滤光片排布于所在的第二子单元中的反对角线上；所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量；每个所述全色滤光片包括N行N列个全色子滤光片，每个所述彩色滤光片包括N行N列个彩色子滤光片，所述N行N列个彩色子滤光片与所述彩色滤光片的颜色相同，所述N为大于或等于2的正整数；所述像素阵列中各个像素与所述滤光片阵列的子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号；
    所述方法包括：
    在全分辨率模式下，将所述全色滤光片中每个全色子滤光片对应的全色像素读出全分辨率全色像素值，以及将所述彩色滤光片中每个彩色子滤光片对应的彩色像素读出全分辨率彩色像素值；及
    基于各个所述全分辨率全色像素值和各个所述全分辨率彩色像素值，生成全分辨率目标图像。
  16. 根据权利要求15所述的方法，其特征在于，所述方法还包括：
    在第一分辨率模式下，将每个所述全色滤光片中各个全色子滤光片对应的全色像素合并读出第一全色像素值，以及将每个所述彩色滤光片中各个彩色子滤光片对应的彩色像素合并读出第一彩色像素值；所述第一分辨率模式对应的分辨率小于所述全分辨率模式对应的分辨率；及
    基于各个所述第一全色像素值和各个所述第一彩色像素值，生成第一目标图像。
  17. 根据权利要求16所述的方法，其特征在于，所述生成第一目标图像之后，还包括：
    在第二分辨率模式下，将所述第一目标图像中对应在每个子单元中的多个第一全色像素值合并读出第二全色像素值，并基于各个所述第二全色像素值生成第一全色图像；所述第二分辨率模式对应的分辨率小于所述第一分辨率模式对应的分辨率，所述子单元包括所述第一子单元和所述第二子单元；
    将所述第一目标图像中对应在每个子单元中的多个相同颜色的第一彩色像素值合并读出第二彩色像素值，并基于各个所述第二彩色像素值生成第一彩色图像；及
    基于所述第一全色图像和所述第一彩色图像，生成第二目标图像。
  18. 根据权利要求17所述的方法，其特征在于，所述基于所述第一全色图像和所述第一彩色图像，生成第二目标图像，包括：
    将所述第一全色图像中每一行第二全色像素值与所述第一彩色图像中每一行第二彩色像素值相间排布，生成第二目标图像；或者
    将所述第一全色图像中每一列第二全色像素值与所述第一彩色图像中每一列第二彩色像素值相间排布，生成第二目标图像。
  19. 根据权利要求17所述的方法，其特征在于，所述方法还包括：
    在第三分辨率模式下，将所述第一全色图像中对应在同一滤光片组中的多个第二全色像素值合并读出第三全色像素值，并基于各个第三全色像素值生成第二全色图像；所述第三分辨率模式对应的分辨率小于所述第二分辨率模式对应的分辨率；
    将所述第一彩色图像中对应在同一滤光片组中的多个同种颜色的第二彩色像素值合并读出第一颜色的第三彩色像素值、第二颜色的第三彩色像素值和第三颜色的第三彩色像素值，并基于所述第一颜色的第三彩色像素值、第二颜色的第三彩色像素值和第三颜色的第三彩色像素值，生成双颜色的第二彩色图像和单颜色的第二彩色图像；所述双颜色的第二彩色图像包括第一颜色的第三彩色像素值和第三颜色的第三彩色像素值，所述单颜色的第二彩色图像包括第二颜色的第三彩色像素值；及
    基于所述第二全色图像、双颜色的第二彩色图像和单颜色的第二彩色图像，生成第三目标图像。
  20. 根据权利要求19所述的方法，其特征在于，所述基于所述第二全色图像、双颜色的第二彩色图像和单颜色的第二彩色图像，生成第三目标图像，包括：
    将所述第二全色图像中每一行第三全色像素值、所述双颜色的第二彩色图像中每一行第三彩色像素值和所述单颜色的第二彩色图像中每一行第三彩色像素值相间排布，生成第二目标图像；或者
    将所述第二全色图像中每一列第三全色像素值、所述双颜色的第二彩色图像中每一列第三彩色像素值和所述单颜色的第二彩色图像中每一列第三彩色像素值相间排布，生成第二目标图像。
  21. 根据权利要求19所述的方法，其特征在于，所述方法还包括：
    在第四分辨率模式下，将所述第二全色图像中各个第三全色像素值合并读出第四全色像素值；所述第四分辨率模式对应的分辨率小于所述第三分辨率模式对应的分辨率；
    将所述双颜色的第二彩色图像中多个第一颜色的第三彩色像素值合并读出第一颜色的第四彩色像素值，将所述双颜色的第二彩色图像中多个第三颜色的第三彩色像素值合并读出第三颜色的第四彩色像素值，以及将所述单颜色的第二彩色图像中多个第二颜色的第三彩色像素值合并读出第二颜色的第四彩色像素值；及
    基于所述第四全色像素值、所述第一颜色的第四彩色像素值、所述第二颜色的第四彩色像素值和所述第三颜色的第四彩色像素值，生成第四目标图像。
  22. 根据权利要求15所述的方法，其特征在于，所述方法还包括：
    在第二分辨率模式下，将每个第一子单元或者第二子单元中多个全色滤光片的各个全色子滤光片对应的全色像素合并读出第五全色像素值，并基于各个所述第五全色像素值生成第三全色图像；
    将每个第一子单元或者第二子单元中多个相同颜色的彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出第五彩色像素值，并基于各个所述第五彩色像素值生成第三彩色图像；及
    基于所述第三全色图像和所述第三彩色图像，生成第五目标图像。
  23. 根据权利要求22所述的方法，其特征在于，所述基于所述第三全色图像和所述第三彩色图像，生成第五目标图像，包括：
    将所述第三全色图像中每一行第五全色像素值与所述第三彩色图像中每一行第五彩色像素值相间排布，生成第五目标图像；或者
    将所述第三全色图像中每一列第五全色像素值与所述第三彩色图像中每一列第五彩色像素值相间排布，生成第五目标图像。
  24. 根据权利要求15所述的方法，其特征在于，所述方法还包括：
    在第三分辨率模式下，将所述每个滤光片组中的多个全色滤光片的各个全色子滤光片对应的全色像素合并读出第六全色像素值，并基于各个第六全色像素值生成第四全色图像；
    将所述每个滤光片组中多个同种颜色的彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出第一颜色的第六彩色像素值、第二颜色的第六彩色像素值和第三颜色的第六彩色像素值，并基于第一颜色的第六彩色像素值、第二颜色的第六彩色像素值和第三颜色的第六彩色像素值，生成双颜色的第四彩色图像和单颜色的第四彩色图像；所述双颜色的第四彩色图像包括第一颜色的第六彩色像素值和第三颜色的第六彩色像素值，所述单颜色的第四彩色图像包括第二颜色的第六彩色像素值；及
    基于所述第四全色图像、所述双颜色的第四彩色图像和单颜色的第四彩色图像，生成第六目标图像。
  25. 根据权利要求24所述的方法，其特征在于，所述基于所述第四全色图像、所述双颜色的第四彩色图像和单颜色的第四彩色图像，生成第六目标图像，包括：
    将所述第四全色图像中每一行第六全色像素值、所述双颜色的第四彩色图像中每一行第六彩色像素值和单颜色的第四彩色图像中每一行第六彩色像素值相间排布，生成第六目标图像；或者
    将所述第四全色图像中每一列第六全色像素值、所述双颜色的第四彩色图像中每一列第六彩色像素值和单颜色的第四彩色图像中每一列第六彩色像素值相间排布，生成第六目标图像。
  26. 根据权利要求15所述的方法，其特征在于，所述方法还包括：
    在第四分辨率模式下，将所述最小重复中多个全色滤光片的各个全色子滤光片对应的全色像素合并读出第七全色像素值，将所述最小重复单元中多个同种颜色的彩色滤光片的各个彩色子滤光片对应的彩色像素合并读出第七彩色像素值；及
    基于各个所述第七全色像素值和各个第七彩色像素值，生成第七目标图像。
  27. 一种图像生成装置，应用于图像传感器，其特征在于，所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，所述最小重复单元包括多个滤光片组，每个滤光片组包括第一子单元和第二子单元；所述第一子单元和所述第二子单元均包括全色滤光片和彩色滤光片，所述第一子单元中的彩色滤光片排布于所在的第一子单元中的对角线上，所述第二子单元中的彩色滤光片排布于所在的第二子单元中的反对角线上；所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量；每个所述全色滤光片包括N行N列个全色子滤光片，每个所述彩色滤光片包括N行N列个彩色子滤光片，所述N行N列个彩色子滤光片与所述彩色滤光片的颜色相同，所述N为大于或等于2的正整数；所述像素阵列中各个像素与所述滤光片阵列的子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号；
    所述装置包括：
    读出模块，用于在全分辨率模式下，将所述全色滤光片中每个全色子滤光片对应的全色像素读出全分辨率全色像素值，以及将所述彩色滤光片中每个彩色子滤光片对应的彩色像素读出全分辨率彩色像素值；及
    图像生成模块，用于基于各个所述全分辨率全色像素值和各个所述全分辨率彩色像素值，生成全分辨率目标图像。
  28. 一种电子设备，包括存储器及处理器，所述存储器中储存有计算机程序，其特征在于，所述计算机程序被所述处理器执行时，使得所述处理器执行如权利要求15至26中任一项所述的图像生成方法的操作。
  29. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求16至26中任一项所述的方法的操作。
  30. 一种计算机程序产品，包括计算机程序，其特征在于，该计算机程序被处理器执行时实现如权利要求15至26中任一项所述的方法的操作。
PCT/CN2022/123995 2021-12-14 2022-10-09 图像传感器、摄像模组、电子设备、图像生成方法和装置 WO2023109264A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111524625.4 2021-12-14
CN202111524625.4A CN114363486B (zh) 2021-12-14 2021-12-14 图像传感器、摄像模组、电子设备、图像生成方法和装置

Publications (1)

Publication Number Publication Date
WO2023109264A1 true WO2023109264A1 (zh) 2023-06-22

Family

ID=81099862

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/123995 WO2023109264A1 (zh) 2021-12-14 2022-10-09 图像传感器、摄像模组、电子设备、图像生成方法和装置

Country Status (2)

Country Link
CN (1) CN114363486B (zh)
WO (1) WO2023109264A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114554046B (zh) * 2021-12-01 2024-08-13 Oppo广东移动通信有限公司 图像传感器、摄像模组、电子设备、图像生成方法和装置
CN114363486B (zh) * 2021-12-14 2024-08-02 Oppo广东移动通信有限公司 图像传感器、摄像模组、电子设备、图像生成方法和装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009237321A (ja) * 2008-03-27 2009-10-15 Fujifilm Corp 画像露光装置
CN104280803A (zh) * 2013-07-01 2015-01-14 全视科技有限公司 彩色滤光片阵列、彩色滤光片阵列设备及图像传感器
CN111818314A (zh) * 2020-06-23 2020-10-23 北京迈格威科技有限公司 一种滤波器阵列及图像传感器
CN112118378A (zh) * 2020-10-09 2020-12-22 Oppo广东移动通信有限公司 图像获取方法及装置、终端和计算机可读存储介质
CN213279832U (zh) * 2020-10-09 2021-05-25 Oppo广东移动通信有限公司 图像传感器、相机和终端
CN114363486A (zh) * 2021-12-14 2022-04-15 Oppo广东移动通信有限公司 图像传感器、摄像模组、电子设备、图像生成方法和装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
TWI422020B (zh) * 2008-12-08 2014-01-01 Sony Corp 固態成像裝置
RU2551649C2 (ru) * 2011-02-28 2015-05-27 Фуджифилм Корпорэйшн Устройство формирования цветного изображения
CN111756972A (zh) * 2020-05-15 2020-10-09 深圳市汇顶科技股份有限公司 图像传感器和电子设备

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009237321A (ja) * 2008-03-27 2009-10-15 Fujifilm Corp 画像露光装置
CN104280803A (zh) * 2013-07-01 2015-01-14 全视科技有限公司 彩色滤光片阵列、彩色滤光片阵列设备及图像传感器
CN111818314A (zh) * 2020-06-23 2020-10-23 北京迈格威科技有限公司 一种滤波器阵列及图像传感器
CN112118378A (zh) * 2020-10-09 2020-12-22 Oppo广东移动通信有限公司 图像获取方法及装置、终端和计算机可读存储介质
CN213279832U (zh) * 2020-10-09 2021-05-25 Oppo广东移动通信有限公司 图像传感器、相机和终端
CN114363486A (zh) * 2021-12-14 2022-04-15 Oppo广东移动通信有限公司 图像传感器、摄像模组、电子设备、图像生成方法和装置

Also Published As

Publication number Publication date
CN114363486B (zh) 2024-08-02
CN114363486A (zh) 2022-04-15

Similar Documents

Publication Publication Date Title
WO2023109264A1 (zh) 图像传感器、摄像模组、电子设备、图像生成方法和装置
WO2023109265A1 (zh) 图像传感器、摄像模组、电子设备、图像生成方法和装置
WO2023098284A1 (zh) 图像传感器、摄像模组、电子设备、图像生成方法和装置
WO2021196554A1 (zh) 图像传感器、处理系统及方法、电子设备和存储介质
WO2023098282A1 (zh) 图像传感器、摄像模组、电子设备、图像生成方法和装置
WO2021196553A1 (zh) 高动态范围图像处理系统及方法、电子设备和可读存储介质
WO2021212763A1 (zh) 高动态范围图像处理系统及方法、电子设备和可读存储介质
WO2023124607A1 (zh) 图像生成方法、装置、电子设备和计算机可读存储介质
CN113170061B (zh) 图像传感器、成像装置、电子设备、图像处理系统及信号处理方法
WO2023082766A1 (zh) 图像传感器、摄像模组、电子设备、图像生成方法和装置
EP2720455B1 (en) Image pickup device imaging three-dimensional moving image and two-dimensional moving image, and image pickup apparatus mounting image pickup device
CN113840067B (zh) 图像传感器、图像生成方法、装置和电子设备
WO2008069920A1 (en) Improved light sensitivity in image sensors
CN113676708B (zh) 图像生成方法、装置、电子设备和计算机可读存储介质
US10764507B1 (en) Image processing system capable of generating a snapshot image with high image quality by using a zero-shutter-lag snapshot operation
CN113573030B (zh) 图像生成方法、装置、电子设备和计算机可读存储介质
US20230007191A1 (en) Image sensor, imaging apparatus, electronic device, image processing system, and signal processing method
WO2023098230A1 (zh) 图像传感器、摄像模组、电子设备、图像生成方法和装置
JP2012015766A (ja) ３次元撮像装置および光透過板
JP7298020B2 (ja) 画像キャプチャ方法、カメラアセンブリ及び移動端末
RU2710779C1 (ru) Устройство "кольцевого" фотоприёмника цветного изображения для панорамного телевизионно-компьютерного наблюдения
CN114080795A (zh) 图像传感器和电子设备
RU2570348C1 (ru) Компьютерная система панорамного телевизионного наблюдения цветного изображения
KR20230067491A (ko) 이미지 센서, 어플리케이션 프로세서 및 이미지 센싱 장치

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE