WO2023082766A1 - 图像传感器、摄像模组、电子设备、图像生成方法和装置 - Google Patents

图像传感器、摄像模组、电子设备、图像生成方法和装置 Download PDF

Info

Publication number
WO2023082766A1
WO2023082766A1 PCT/CN2022/114371 CN2022114371W WO2023082766A1 WO 2023082766 A1 WO2023082766 A1 WO 2023082766A1 CN 2022114371 W CN2022114371 W CN 2022114371W WO 2023082766 A1 WO2023082766 A1 WO 2023082766A1
Authority
WO
WIPO (PCT)
Prior art keywords
color
filter
pixel value
filters
image
Prior art date
Application number
PCT/CN2022/114371
Other languages
English (en)
French (fr)
Inventor
李小涛
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2023082766A1 publication Critical patent/WO2023082766A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/90—Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof

Definitions

  • the present application relates to the field of image technology, and in particular to an image sensor, camera module, electronic equipment, image generation method, device, computer-readable storage medium and computer program product.
  • An image sensor is arranged in the camera, and a color image is collected by the image sensor.
  • an optical filter array arranged in the form of a Bayer (Bayer) array is usually arranged in the image sensor, so that multiple pixels in the image sensor can receive light passing through the corresponding optical filter, thereby Generate pixel signals with different color channels to generate an image.
  • Embodiments of the present application provide an image sensor, an image generation method, device, electronic equipment, and a computer-readable storage medium, which can improve the definition of imaging.
  • An image sensor the image sensor includes a filter array and a pixel array
  • the filter array includes a minimum repeating unit, the minimum repeating unit includes a plurality of filter groups, each filter group includes a color An optical filter and a panchromatic filter, the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter, and both the color filter and the panchromatic filter It includes 4 sub-filters, and the multiple filter groups include at least the first filter group, and at least one of the color filters in the first filter group includes the first color sub-filter and the second color sub-filter A sub-filter; the pixel array includes a plurality of pixels, the pixels of the pixel array correspond to the sub-filters of the filter array, and the pixel array is configured to receive light from an array of light sheets to generate electrical signals.
  • a camera module the camera module includes a lens and an image sensor; the image sensor is used to receive light passing through the lens, and the pixels generate electrical signals according to the light; the image sensor includes a light filter A sheet array and a pixel array, the filter array includes a minimum repeating unit, the minimum repeating unit includes a plurality of filter groups, each filter group includes a color filter and a panchromatic filter, the The amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter, and both the color filter and the panchromatic filter include 4 sub-filters, a plurality of filters
  • the group includes at least a first filter group, and at least one of the color filters in the first filter group includes a first color sub-filter and a second color sub-filter; the pixel array includes a plurality of Pixels, the pixels of the pixel array are arranged corresponding to the sub-filters of the filter array, and the pixel array is configured to receive light passing through the filter array to generate electrical signals.
  • An electronic device the electronic device includes a camera module and a housing, the camera module is arranged on the housing; the camera module includes a lens and an image sensor; the image sensor is used to receive The light of the lens, the pixel generates an electrical signal according to the light; the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes a plurality of filters A light sheet group, each filter group includes a color filter and a panchromatic filter, the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter, and the color filter Both the filter and the panchromatic filter include 4 sub-filters, and the multiple filter groups include at least the first filter group, and at least one of the color filters in the first filter group Including a first color sub-filter and a second color sub-filter; the pixel array includes a plurality of pixels, and the pixels of the pixel array correspond to the sub-filters of the filter array, and the pixels The array is configured to receive light
  • the above image sensor includes a filter array and a pixel array
  • the filter array includes a minimum repeating unit
  • the minimum repeating unit includes a plurality of filter groups
  • the filter groups include panchromatic filters and color filters, panchromatic
  • panchromatic The amount of light transmitted by the filter is greater than that of the color filter, so that more light can be obtained during shooting, so there is no need to adjust the shooting parameters, and the darkness can be improved without affecting the stability of the shooting. Clarity of imaging under light. When imaging in dark light, both stability and clarity can be taken into account, and the stability and clarity of imaging in dark light are both high.
  • each panchromatic filter includes 4 sub-filters
  • each color filter includes 4 sub-filters
  • the plurality of filter groups includes at least the first filter group
  • the first filter group At least one color filter in the sheet group includes a first color sub-filter and a second color sub-filter
  • the pixel array includes a plurality of panchromatic pixels and a plurality of color pixels
  • each panchromatic pixel corresponds to a panchromatic filter A sub-filter of the light sheet
  • each photosensitive pixel of the first color corresponds to a sub-filter of the first color of the color filter
  • each photosensitive pixel of the second color corresponds to a sub-filter of the second color of the color filter
  • the light sheet can mix and arrange different color photosensitive pixels in the same color filter, effectively improve the resolution ability of the color channel, and reduce the problem of false colors in the generated image.
  • An image generation method applied to an image sensor, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes a plurality of filter groups, each The filter set includes a color filter and a panchromatic filter, the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter, and the color filter and the Panchromatic filters all include 4 sub-filters, and a plurality of filter groups include a first filter group and a second filter group, at least one of the color filters in the first filter group Including a first color sub-filter and a second color sub-filter; the color filter in the second filter group includes at least a third color sub-filter; the pixel array includes a plurality of pixels, and the pixel The photosensitive pixels of the first color of the array correspond to the first color sub-filters of the filter array, and the photosensitive pixels of the second color of the pixel array correspond to the second color sub-filters of the filter array.
  • the methods include:
  • the first pixel value read out is merged, and the multiple corresponding to the color filters Combining and reading out the second pixel value of the photosensitive pixels of the first color and combining and reading the third pixel value of the plurality of photosensitive pixels of the third color corresponding to the color filter to obtain the first combined image;
  • a first target image is obtained based on the first combined image and the second combined image.
  • An image generating device applied to an image sensor, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, the minimum repeating unit includes a plurality of filter groups, each The filter set includes a color filter and a panchromatic filter, the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter, and the color filter and the Panchromatic filters all include 4 sub-filters, and a plurality of filter groups include a first filter group and a second filter group, at least one of the color filters in the first filter group Including a first color sub-filter and a second color sub-filter; the color filter in the second filter group includes at least a third color sub-filter; the pixel array includes a plurality of pixels, and the pixel The photosensitive pixels of the first color of the array correspond to the first color sub-filters of the filter array, and the photosensitive pixels of the second color of the pixel array correspond to the second color sub-filters of the filter array.
  • the devices include:
  • the first combining module is configured to combine the first pixel values read out according to the plurality of panchromatic pixels corresponding to the panchromatic filters in the filter set in the first resolution mode, and the Combining and reading out the second pixel value of the plurality of photosensitive pixels of the first color corresponding to the color filter, and combining and reading the third pixel value of the plurality of photosensitive pixels of the third color corresponding to the color filter to obtain the first merge images;
  • the second merging module is configured to combine and read out the fourth pixel values of a plurality of photosensitive pixels of the second color corresponding to the color filter to obtain a second merging image
  • An image generating module configured to obtain a first target image based on the first combined image and the second combined image.
  • An electronic device including a memory, a processor, and an image sensor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor performs the following operations:
  • the first pixel value read out is merged, and the multiple corresponding to the color filters
  • the color filter Combining the read-out fourth pixel values of the plurality of photosensitive pixels of the second color corresponding to the filter to obtain a second combined image; based on the first combined image and the second combined image, a first target image is obtained.
  • a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the following operations are realized:
  • the first pixel value read out is merged, and the multiple corresponding to the color filters
  • the color filter Combining the read-out fourth pixel values of the plurality of photosensitive pixels of the second color corresponding to the filter to obtain a second combined image; based on the first combined image and the second combined image, a first target image is obtained.
  • a computer program product including a computer program that, when executed by a processor, implements the following operations:
  • the first pixel value read out is merged, and the multiple corresponding to the color filters
  • the color filter Combining the read-out fourth pixel values of the plurality of photosensitive pixels of the second color corresponding to the filter to obtain a second combined image; based on the first combined image and the second combined image, a first target image is obtained.
  • the image generation method, device, electronic equipment, computer-readable storage medium, and computer program product described above in the first resolution mode, combine and read out a plurality of panchromatic pixels corresponding to the panchromatic filters in the filter set
  • the first pixel value of the color filter, and the second pixel value of the multiple first color photosensitive pixels corresponding to the color filter combined and read out, and the third pixel value of the multiple third color photosensitive pixels corresponding to the color filter combined and read out value, the panchromatic channel information can be integrated into the image, and the overall amount of incoming light can be increased, so that the first merged image with more information and clearer detail analysis can be generated.
  • the photosensitive pixels of the second color can be separated to form a second combined image, so that the photosensitive pixels of the second color are in the second combined image and The positions of the photosensitive pixels of the first color in the first merged image are consistent.
  • the first target image is obtained, and the photosensitive pixels of the first color in the first combined image and the photosensitive pixels of the second color in the second combined image can be mixed and arranged, so that the generated first A target image color is clearer.
  • pixel combination readout reduces the size of the generated first target image and consumes low power consumption for generating the image.
  • Fig. 1 is a schematic structural diagram of an electronic device in an embodiment.
  • FIG. 2 is an exploded schematic diagram of an image sensor in one embodiment.
  • Fig. 3 is a schematic diagram of connection between a pixel array and a readout circuit in one embodiment.
  • Figure 4a is a schematic diagram of a first diagonal direction and a second diagonal direction in one embodiment.
  • Fig. 4b is a schematic diagram of a third diagonal direction and a fourth diagonal direction in one embodiment.
  • Figure 5 is a schematic diagram of the arrangement of the smallest repeating units in an embodiment.
  • Figure 6 is a schematic diagram of the arrangement of the smallest repeating units in one of the embodiments.
  • Fig. 7 is a schematic diagram of the arrangement of the smallest repeating units in another embodiment.
  • Fig. 8 is a schematic diagram of the arrangement of the smallest repeating units in another embodiment.
  • Fig. 9 is a flowchart of an image generation method in one embodiment.
  • Fig. 10 is a schematic diagram of obtaining a first target image in a first resolution mode in an embodiment.
  • Fig. 11 is a schematic diagram of obtaining a first target image in a first resolution mode in another embodiment.
  • Fig. 12 is a schematic diagram of obtaining a second target image in a second resolution mode in an embodiment.
  • Fig. 13 is a schematic diagram of obtaining a second target image in a second resolution mode in another embodiment.
  • Fig. 14 is a schematic diagram of an image generation method in the second resolution mode in another embodiment.
  • Fig. 15 is a structural block diagram of an image generating device in an embodiment.
  • Fig. 16 is a block diagram of the internal structure of an electronic device in one embodiment.
  • first, second and the like used in this application may be used to describe various elements herein, but these elements are not limited by these terms. These terms are only used to distinguish one element from another element.
  • a first object image could be termed a second object image, and, similarly, a second object image could be termed a first object image, without departing from the scope of the present application.
  • Both the first and the second are target images, but they are not the same target image.
  • a plurality refers to at least two, such as 2, 4, but not limited thereto.
  • the electronic device 100 includes a mobile phone, a tablet computer, a notebook computer, an ATM, a gate machine, a smart watch, a head-mounted display device, etc. It can be understood that the electronic device 100 can also be any other device with image processing functions.
  • the electronic device 100 includes a camera module 20 , a processor 30 and a casing 40 .
  • the camera module 20 and the processor 30 are both arranged in the housing 40, and the housing 40 can also be used to install functional modules such as power supply devices and communication devices of the electronic device 100, so that the housing 40 provides dustproof and drop-proof protection for the functional modules. , waterproof and other protection.
  • the camera module 20 may be a front camera module, a rear camera module, a side camera module, an under-screen camera module, etc., which is not limited here.
  • the camera module 20 includes a lens and an image sensor 21. When the camera module 20 captures an image, light passes through the lens and reaches the image sensor 21.
  • the image sensor 21 is used to convert the light signal irradiated on the image sensor 21 into an electrical signal.
  • the image sensor 21 includes a microlens array 22 , a filter array 23 , and a pixel array 24 .
  • the microlens array 22 includes a plurality of microlenses 221, the microlenses 221, the sub-filters in the filter array 23, and the pixels in the pixel array 24 are set in one-to-one correspondence, and the microlenses 221 are used to gather the incident light.
  • the collected light will pass through the corresponding sub-filter, and then projected onto the pixel, and be received by the corresponding pixel, and the pixel converts the received light into an electrical signal.
  • the filter array 23 includes a plurality of minimal repeating units 230 .
  • the minimum repeating unit 230 may include a plurality of filter groups.
  • Each filter set includes a panchromatic filter 233 and a color filter 234 , the amount of light transmitted by the panchromatic filter 233 is greater than the amount of light transmitted by the color filter 234 .
  • Each panchromatic filter 233 includes 4 sub-filters 2331, each color filter 234 includes 4 sub-filters, and a plurality of filter groups includes at least the first filter group 231, the first At least one color filter 234 in the filter set 231 includes a first color sub-filter 2341 and a second color sub-filter 2342 .
  • Different color filters 234 are also included in different filter sets.
  • the plurality of filter sets further include a second filter set 232 , and the color filters 234 in the second filter set 232 include at least a third color sub-filter 2343 .
  • the colors corresponding to the wavelength bands of the transmitted light of the color filters 234 of the filter set in the minimum repeating unit 230 include color a, color b and/or color c.
  • the color corresponding to the wavelength band of the transmitted light of the color filter 234 of the filter group 232 includes color a, color b and color c, or color a, color b or color c, or color a and color b, or color b and color c, or color a and color c.
  • the color a is red
  • the color b is green
  • the color c is blue, or for example, the color a is magenta, the color b is cyan, and the color c is yellow, etc., which are not limited here.
  • the width of the wavelength band of the light transmitted by the color filter 234 is smaller than the width of the wavelength band of the light transmitted by the panchromatic filter 233, for example, the wavelength band of the light transmitted by the color filter 234 It can correspond to the wavelength band of red light, the wavelength band of green light, or the wavelength band of blue light.
  • the wavelength band of the light transmitted by the panchromatic filter 233 is the wavelength band of all visible light, that is to say, the color filter 234 only allows specific color light
  • the panchromatic filter 233 can pass light of all colors.
  • the wavelength band of the light transmitted by the color filter 234 may also correspond to the wavelength band of other colored light, such as magenta light, purple light, cyan light, yellow light, etc., which is not limited here.
  • the ratio of the number of color filters 234 to the number of panchromatic filters 233 in the filter set may be 1:3, 1:1 or 3:1.
  • the ratio of the number of color filters 234 to the number of panchromatic filters 233 is 1:3, then the number of color filters 234 is 1, and the number of panchromatic filters 233 is 3.
  • Color filter 233 quantity is more, and the imaging quality under dark light is better;
  • the ratio of the quantity of color filter 234 and the quantity of panchromatic filter 233 is 1:1, then color filter 234
  • the number of color filters 234 is 2, and the number of panchromatic filters 233 is 2.
  • the imaging quality under dark light is also better; or, the number of color filters 234 and the full color filter
  • the ratio of the number of color filters 233 is 3:1, then the number of color filters 234 is 3, and the number of panchromatic filters 233 is 1. At this time, better color performance can be obtained, and dark Image quality under light.
  • the pixel array 24 includes a plurality of pixels, and the pixels of the pixel array 24 are arranged corresponding to the sub-filters of the filter array 23 .
  • the pixel array 24 is configured to receive light passing through the filter array 23 to generate electrical signals.
  • the pixel array 24 is configured to receive the light passing through the filter array 23 to generate an electrical signal, which means that the pixel array 24 is used to detect a scene of a given set of subjects passing through the filter array 23
  • the light is photoelectrically converted to generate an electrical signal.
  • the light rays of the scene for a given set of subjects are used to generate image data.
  • the subject is a building
  • the scene of a given set of subjects refers to the scene where the building is located, which may also contain other objects.
  • the pixel array 24 includes a plurality of minimal repeating units 240 , the smallest repeating unit 240 includes a plurality of pixel groups, and the plurality of pixel groups include a panchromatic pixel group 243 and a color pixel group 244 .
  • Each panchromatic pixel group 243 includes 4 panchromatic pixels 2431, each color pixel group 244 includes 4 color pixels, a plurality of pixel groups at least includes the first pixel group 241, at least one of the first pixel groups 241
  • the color pixel group 244 includes first color photosensitive pixels 2441 and second color photosensitive pixels 2442 .
  • Each panchromatic pixel 2431 corresponds to a sub-filter 2331 in the panchromatic filter 233, and the panchromatic pixel 2431 receives light passing through the corresponding sub-filter 2331 to generate an electrical signal.
  • Each first-color photosensitive pixel 2441 corresponds to a first-color sub-filter 2341 of the color filter 234, and the first-color photosensitive pixel 2441 receives light passing through the corresponding first-color sub-filter 2341 to generate an electrical signal .
  • Each second-color photosensitive pixel 2442 corresponds to a second-color sub-filter 2342 of the color filter 234, and the second-color photosensitive pixel 2442 receives light passing through the corresponding second-color sub-filter 2342 to generate an electrical signal .
  • the plurality of pixel groups further includes a second pixel group 242 , and the color pixel group 244 in the second pixel group 242 includes at least a third color photosensitive pixel 2443 .
  • Each third-color photosensitive pixel 2443 corresponds to a third-color sub-filter 2343 of the color filter 234, and the third-color photosensitive pixel 2443 receives light passing through the corresponding third-color sub-filter 2343 to generate an electrical signal .
  • the image sensor 21 in this implementation includes a filter array 23 and a pixel array 24, the filter array 23 includes a minimum repeating unit 230, the minimum repeating unit 230 includes a plurality of filter groups, and each filter group includes a panchromatic Filter 233 and color filter 234, color filter 234 has narrower spectral response than panchromatic filter 233, the amount of light that panchromatic filter 233 passes through is greater than color filter 234 and passes through
  • the amount of incoming light can get more light when shooting, so there is no need to adjust shooting parameters, and the clarity of imaging in dark light can be improved without affecting the stability of shooting.
  • both stability and clarity can be taken into account, and the stability and clarity of imaging in dark light are both high.
  • each panchromatic filter 233 includes 4 sub-filters 2331
  • each color filter 234 includes 4 sub-filters
  • the plurality of filter groups includes at least the first filter group 231
  • At least one color filter 234 in the first filter group 231 includes a first color sub-filter 2341 and a second color sub-filter 2342
  • the pixel array 24 includes a plurality of panchromatic pixels 2431 and a plurality of color pixels 2441
  • each panchromatic pixel 2431 corresponds to a sub-filter 2331 of the panchromatic filter 233
  • each photosensitive pixel 2441 of the first color corresponds to a first color sub-filter 2341 of the color filter 234
  • each The second color photosensitive pixel 2442 corresponds to a second color sub-filter 2342 of the color filter 234, which can mix and arrange different color photosensitive pixels in the same color filter, effectively improving the resolution of the color channel and reducing the There is a problem with false colors in the generated image.
  • the minimum repeating unit 230 in the filter array 23 includes 4 filter groups, and the 4 filter groups include 2 first filter groups 231 and 2 A second filter group 232, two first filter groups 231 and two second filter groups 232 are arranged in a matrix.
  • Each first filter group 231 includes a panchromatic filter 233 and a color filter 234, each panchromatic filter 233 and each color filter 234 have 4 sub-filters, and at least one The color filter 234 includes a first color sub-filter 2341 and a second color sub-filter 2342 , and the second filter set 232 includes 16 sub-filters in total.
  • Each second filter group 232 includes a panchromatic filter 233 and a color filter 234, each panchromatic filter 233 and each color filter 234 have 4 sub-filters, and the color filter The light sheet 234 includes at least the third color sub-filter 2343 , and the second filter set 232 includes 16 sub-filters in total.
  • the pixel array 24 includes a plurality of minimum repeating units 240 corresponding to the plurality of minimum repeating units 230 of the filter array 23 .
  • Each minimum repeating unit 240 includes 4 pixel groups, and the 4 pixel groups include 2 first pixel groups 241 and 2 second pixel groups 242, and the 2 first pixel groups 241 and 2 second pixel groups 242 form Matrix arrangement.
  • Each first pixel group 241 corresponds to a first filter group 231
  • each second pixel group 242 corresponds to a second filter group 232 .
  • the readout circuit 25 is electrically connected to the pixel array 24 for controlling the exposure of the pixel array 24 and reading and outputting the pixel values of the pixel points.
  • the readout circuit 25 includes a vertical drive unit 251 , a control unit 252 , a column processing unit 253 , and a horizontal drive unit 254 .
  • the vertical driving unit 251 includes a shift register and an address decoder.
  • the vertical driving unit 251 includes readout scanning and reset scanning functions.
  • the control unit 252 configures timing signals according to the operation mode, and uses various timing signals to control the vertical driving unit 251 , the column processing unit 253 and the horizontal driving unit 254 to work together.
  • the column processing unit 253 may have an analog-to-digital (A/D) conversion function for converting an analog pixel signal into a digital format.
  • the horizontal driving unit 254 includes a shift register and an address decoder. The horizontal driving unit 254 sequentially scans the pixel array 24 column by column.
  • the first filter group 231 in the minimum repeating unit 230 is arranged in the direction of the first diagonal line D1
  • the second filter group 232 is arranged in the direction of the second diagonal line D2.
  • the direction of the first diagonal line D1 is different from the direction of the second diagonal line D2.
  • the direction of the first diagonal line D1 and the direction of the second diagonal line D2 are different, which can give consideration to both color performance and low-light imaging quality.
  • the direction of the first diagonal line D1 is different from the direction of the second diagonal line D2. Specifically, the direction of the first diagonal line D1 is not parallel to the direction of the second diagonal line D2, or the direction of the first diagonal line D1 is not parallel to the direction of the second diagonal line.
  • the direction of the diagonal line D2 is vertical, etc.
  • each filter set includes a color filter 234 and a panchromatic filter 233, and the color filter 234 and panchromatic filter of each filter set
  • the sheets 233 are arranged in a matrix.
  • Each panchromatic filter 233 in the filter set is arranged in the third diagonal direction D3, and each color filter 234 in the filter set is arranged in the fourth diagonal D4 direction.
  • the direction of the third diagonal line D3 is different from the direction of the fourth diagonal line D4, which can take into account both color performance and low-light imaging quality.
  • the direction of the third diagonal line D3 is different from the direction of the fourth diagonal line D4. Specifically, the direction of the third diagonal line D3 is not parallel to the direction of the fourth diagonal line D4, or the direction of the third diagonal line D3 is not parallel to the direction of the fourth diagonal line.
  • the direction of the diagonal line D4 is vertical, etc.
  • one color filter 234 and one panchromatic filter 233 can be located on the third diagonal line D3, and another color filter 234 and another panchromatic filter 233 can be located on the fourth pair. Corner line D4.
  • the color filter 234 may be disposed in a direction parallel to the third diagonal line D3, or the color filter 234 may be disposed in a direction parallel to the fourth diagonal line D4.
  • At least one color filter 234 in the first filter set 231 includes a first color sub-filter 2341 and a second color sub-filter 2342 .
  • the first color sub-filter 2341 is arranged in one direction of the fifth diagonal direction or the sixth diagonal direction
  • the second color sub-filter 2342 is arranged in the fifth diagonal direction or the sixth diagonal direction in the other direction of the direction.
  • the sub-filters 2331 of the panchromatic filter 233 in the first filter group 231 are arranged in the fifth diagonal direction and the sixth diagonal direction.
  • the minimum repeating unit 230 includes 2 first filter groups 231 and 2 second filter groups 232, 2 first filter groups 231 and 2 second filter groups 232 Arranged in a matrix.
  • the first color sub-filter 2341 in the color filter 234 of one of the first filter group 231 is arranged on the fifth diagonal direction, and the second color sub-filter 2341 is arranged on the fifth diagonal direction.
  • the filter 2342 is arranged on the sixth diagonal direction; and, the first color sub-filter 2341 in the color filter 234 of the other first filter group 231 is arranged on the sixth diagonal direction , the second color sub-filter 2342 is disposed in the fifth diagonal direction.
  • the first color sub-filter 2341 is arranged in the direction of the fifth diagonal.
  • the sheet 2341 is in a direction parallel to the fifth diagonal direction, and the other diagonal directions are similar.
  • a plurality of filter groups includes a first filter group 231 and a second filter group 232
  • at least one color filter 234 in the first filter group 231 includes a first color sub-filter 2341 and a second color sub-filter 2342.
  • the color filter 234 in the second filter set 232 includes at least a third color sub-filter 2343 .
  • the third color sub-filter 2343 in the second filter group 232 is arranged in the direction of the seventh diagonal line D7 and the direction of the eighth diagonal line D8, and the direction of the seventh diagonal line D7 and the direction of the eighth diagonal line D8 The direction is different.
  • the direction of the seventh diagonal line D7 is different from the direction of the eighth diagonal line D8. Specifically, the direction of the seventh diagonal line D7 is not parallel to the direction of the eighth diagonal line D8, or the direction of the seventh diagonal line D7 is not parallel to the direction of the eighth diagonal line.
  • the direction of the diagonal line D8 is vertical, etc.
  • the minimum repeating unit 230 in the filter array 23 includes 4 filter groups, and the 4 filter groups are arranged in a matrix.
  • the four filter sets include a first filter set 231 and a second filter set 232 .
  • Each first filter group 231 includes 2 panchromatic filters 233 and 2 color filters 234, and each second filter group 232 includes 2 panchromatic filters 233 and 2 color filters.
  • the panchromatic filter 233 includes 4 sub-filters 2331, and the same color filter 234 of the first filter group 231 includes 2 first color sub-filters 2341 and 2 second color sub-filters.
  • Sheet 2342, the same color filter 234 of the second filter group 232 includes 4 third color sub-filters 2343, then the minimum repeating unit is 8 rows and 8 columns of 64 sub-filters, the arrangement is:
  • w panchromatic sub-filter 2331
  • a, b and c all represent color sub-filters.
  • the color sub-filters include a first color sub-filter 2341 , a second color sub-filter 2342 and a third color sub-filter 2343 .
  • the panchromatic sub-filter 2331 refers to a sub-filter that can filter out all light rays other than the visible light band
  • the first color sub-filter 2341, the second color sub-filter 2342 and the third color sub-filter Sheet 2343 may be a red sub-filter, a green sub-filter, a blue sub-filter, a magenta sub-filter, a cyan sub-filter, and a yellow sub-filter.
  • the red sub-filter is a sub-filter for filtering all light except red light
  • the green sub-filter is a sub-filter for filtering all light except green light
  • the blue sub-filter is a sub-filter for filtering A sub-filter for all light except blue
  • a magenta sub-filter for all light except magenta and a cyan sub-filter for all light except cyan A sub-filter for all light rays
  • the yellow sub-filter is a sub-filter for filtering out all light rays except yellow light.
  • a can be red sub-filter, green sub-filter, blue sub-filter, magenta sub-filter, cyan sub-filter or yellow sub-filter
  • b can be red sub-filter, Green sub-filter, blue sub-filter, magenta sub-filter, cyan sub-filter or yellow sub-filter
  • c can be red sub-filter, green sub-filter, blue sub-filter filter, magenta sub-filter, cyan sub-filter, or yellow sub-filter.
  • b is the red sub-filter, a is the green sub-filter, c is the blue sub-filter; or, c is the red sub-filter, a is the green sub-filter, b is the blue sub-filter Filter; another example, c is a red sub-filter, a is a green sub-filter, b is a blue sub-filter; or, a is a red sub-filter, b is a blue sub-filter , c is a green sub-filter, etc., which are not limited here; for another example, b is a magenta sub-filter, a is a cyan sub-filter, b is a yellow sub-filter, etc.
  • the color filter may further include sub-filters of other colors, such as an orange sub-filter, a purple sub-filter, etc., which are not limited here.
  • the smallest repeating unit 230 in the filter array 23 includes 4 filter groups, and the 4 filter groups are arranged in a matrix.
  • the four filter sets include a first filter set 231 and a second filter set 232 .
  • Each first filter group 231 includes 2 panchromatic filters 233 and 2 color filters 234, and each second filter group 232 includes 2 panchromatic filters 233 and 2 color filters.
  • the panchromatic filter 233 includes 4 sub-filters 2331, and the same color filter 234 of the first filter group 231 includes 2 first color sub-filters 2341 and 2 second color sub-filters.
  • Sheet 2342, the same color filter 234 of the second filter group 232 includes 4 third color sub-filters 2343, then the minimum repeating unit is 8 rows and 8 columns of 64 sub-filters, the arrangement is:
  • w panchromatic sub-filter 2331
  • a, b and c all represent color sub-filters.
  • the color sub-filters include a first color sub-filter 2341 , a second color sub-filter 2342 and a third color sub-filter 2343 .
  • photosensitive pixels of different colors are mixed and arranged, such as red pixels and blue pixels are mixed and arranged, so that each row and column of the smallest repeating unit has RGB pixels, which can effectively improve the resolution of the color channel. Reduce the problem of false colors in generated images.
  • panchromatic pixels are introduced to effectively increase the amount of incoming light, thereby improving the clarity of the image.
  • the image sensor in this embodiment also has the advantage of two-level pixel binning and reading. There are RGB pixels in each row and column of the smallest repeating unit of the image sensor, and there is no need to cross different smallest repeating units for the same pixel. Merging can effectively reduce processing power consumption.
  • a plurality of filter groups includes a first filter group 231 and a second filter group 232
  • at least one color filter 234 in the first filter group 231 includes a first color sub-filter 2341 and a second color sub-filter 2342.
  • the color filter 234 in the second filter group 232 includes a third color sub-filter 2343, and the color filter 234 in the second filter group 232 also includes a first color sub-filter 2341 or a second color filter.
  • Color sub-filter 2342 is
  • the same color filter 234 in the second filter group 232 includes a third color sub-filter 2343 and a first color sub-filter 2341, or the same color filter in the second filter group 232
  • the light sheet 234 includes a third color sub-filter 2343 and a second color sub-filter 2342 .
  • the second filter group 232 includes two color filters 234, and the sub-filters of the same color in each color filter 234 are arranged diagonally, The sub-filters of the same color are arranged in opposite directions in the two color filters 234 .
  • the second filter group 232 includes two color filters 234 , and the same color filter 234 includes a third color sub-filter 2343 and a first color sub-filter 2341 .
  • the third color sub-filters 2343 in the same color filter 234 are arranged diagonally, and the first color sub-filters 2341 in the same color filter 234 are arranged diagonally.
  • the arrangement directions of the first color sub-filters 2341 in the two color filters 234 are opposite, and the arrangement directions of the third color sub-filters 2343 in the two color filters 234 are opposite.
  • the third color sub-filters 2343 are respectively arranged in the direction of the diagonal line A1 and the direction of the diagonal line A2 among the two color filters 234 .
  • the second filter set 232 includes two color filters 234 , and the same color filter 234 includes a third color sub-filter 2343 and a second color sub-filter 2342 .
  • the third color sub-filters 2343 in the same color filter 234 are arranged diagonally, and the second color sub-filters 2342 in the same color filter 234 are arranged diagonally.
  • the arrangement directions of the second color sub-filters 2342 in the two color filters 234 are opposite, and the arrangement directions of the third color sub-filters 2343 in the two color filters 234 are opposite.
  • the smallest repeating unit 230 in the filter array 23 includes 4 filter groups, and the 4 filter groups are arranged in a matrix.
  • Each filter set includes 2 panchromatic filters 233 and 2 color filters 234 .
  • the panchromatic filter 233 includes 4 sub-filters 2331, and the same color filter 234 of the first filter group 231 includes 2 first color sub-filters 2341 and 2 second color sub-filters.
  • Sheet 2342 there are two first color sub-filters 2341 and two third color sub-filters 2343 in the same color filter 234 in the second filter group 232, and there is a second filter group
  • the same color filter 234 of 232 includes 2 second color sub-filters 2342 and 2 third color sub-filters 2343, then the minimum repeating unit is 8 rows and 8 columns of 64 sub-filters, the arrangement for:
  • w panchromatic sub-filter 2331
  • a, b and c all represent color sub-filters.
  • the color sub-filters include a first color sub-filter 2341 , a second color sub-filter 2342 and a third color sub-filter 2343 .
  • the smallest repeating unit 230 in the filter array 23 includes 4 filter groups, and the 4 filter groups are arranged in a matrix.
  • Each filter set includes 2 panchromatic filters 233 and 2 color filters 234 .
  • the panchromatic filter 233 includes 4 sub-filters 2331, and the same color filter 234 of the first filter group 231 includes 2 first color sub-filters 2341 and 2 second color sub-filters.
  • the same color filter 234 in the second filter group 232 includes two first color sub-filters 2341 and two third color sub-filters 2343, there is a second filter group 232
  • the same color filter 234 includes two second color sub-filters 2342 and two third color sub-filters 2343, the minimum repeating unit is 8 rows and 8 columns of 64 sub-filters, and the arrangement is :
  • w panchromatic sub-filter 2331
  • a, b and c all represent color sub-filters.
  • the color sub-filters include a first color sub-filter 2341 , a second color sub-filter 2342 and a third color sub-filter 2343 .
  • an image generation method is provided, which is applied to an image sensor 21 as shown in FIG.
  • the minimum repeating unit 230 includes a plurality of filter groups, each filter group includes a color filter 234 and a panchromatic filter 233, the amount of light transmitted by the panchromatic filter 233 is greater than that of the color filter 234
  • the amount of light that passes through, the color filter 234 has a narrower spectral response than the panchromatic filter 233
  • each panchromatic filter 233 includes 4 sub-filters 2331
  • each color filter 234 Including 4 sub-filters, a plurality of filter groups include a first filter group 231 and a second filter group 232, at least one color filter 234 in the first filter group 231 includes the first A color sub-filter 2341 and a second color sub-filter 2342.
  • the color filters 234 of the second filter set 232 include at least a third color sub-filter 2343 .
  • the pixel array 24 includes a plurality of pixels, the first color photosensitive pixel 2441 of the pixel array 24 corresponds to the first color sub-filter 2341 of the filter array 23, and the second color photosensitive pixel 2442 of the pixel array 24 is connected to the filter sub-filter 2341.
  • the second color sub-filter 2342 of the array 23 is set correspondingly, the third color photosensitive pixel 2443 of the pixel array 24 is set correspondingly to the third color sub-filter 2343 of the filter array 23, and the pixel array 24 is configured for receiving light passing through the filter array 23 to generate electrical signals;
  • the pixel array 24 includes a plurality of panchromatic pixels 2431 and a plurality of color pixels 2441, each panchromatic pixel 2431 corresponds to a sub-filter 2331 of the panchromatic filter 233, and each first color photosensitive A pixel 2441 corresponds to a first color sub-filter 2341 of the color filter 234, each second color photosensitive pixel 2442 corresponds to a second color sub-filter 2342 of the color filter 234, and each third color photosensitive The pixel 2443 corresponds to a third color sub-filter 2343 of the color filter 234;
  • the image generation method includes:
  • Operation 902 in the first resolution mode, combine and read out the first pixel values according to the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, and the multiple first pixel values corresponding to the color filters Combining and reading out the second pixel values of the color photosensitive pixels and combining and reading out the third pixel values of the plurality of third color photosensitive pixels corresponding to the color pixels to obtain a first combined image.
  • the first resolution mode refers to a first-level pixel binning readout mode in which resolution, power consumption, signal-to-noise ratio, and frame rate are relatively balanced.
  • the first resolution mode may be a default mode for shooting images and videos.
  • Combined readout refers to summing the pixel values of multiple pixels, or calculating the average value of the pixel values of multiple pixels.
  • the first pixel value read out by combining multiple panchromatic pixels may be the sum of the pixel values of multiple panchromatic pixels to obtain the first pixel value; or, calculating the average value of the pixel values of multiple panchromatic pixels, as the first pixel value.
  • the color filter 234 has a narrower spectral response than that of the panchromatic filter 233, so the amount of light transmitted by the panchromatic filter 233 is greater than the amount of light transmitted by the color filter 234, that is, the color filter 234
  • the wavelength band width of the transmitted light is smaller than the wavelength band width of the light transmitted by the panchromatic filter 233, and the panchromatic filter 233 transmits more light, and the corresponding panchromatic pixel 2431 has
  • the panchromatic pixels contain more information and can resolve more texture details.
  • the signal-to-noise ratio refers to the ratio between the normal signal and the noise signal. The higher the signal-to-noise ratio of a pixel, the higher the proportion of normal signals contained in the pixel, and the more information can be analyzed from the pixel.
  • the color pixels may be G (Green, green) pixels, R (Red, red) pixels, B (Blue, blue) pixels, etc., but not limited thereto.
  • the first resolution mode is used to respond to the shooting instruction.
  • the light transmitted by the sub-filter 2331 in the panchromatic filter 233 is projected onto the corresponding panchromatic pixel 2431, and the panchromatic pixel 2431 receives the light passing through the sub-filter 2331 to Generate electrical signals.
  • the light transmitted by the first color sub-filter 2341 in the color filter 234 is projected onto the corresponding first color photosensitive pixel 2441, and the first color photosensitive pixel 2441 passes through the corresponding first color photosensitive pixel 2341. light to generate electrical signals.
  • the light transmitted by the second color sub-filter 2342 in the color filter 234 is projected onto the corresponding second color photosensitive pixel 2442, and the second color photosensitive pixel 2442 passes through the corresponding second color sub filter 2342. light to generate electrical signals.
  • the light transmitted by the third color sub-filter 2343 in the color filter 234 is projected onto the corresponding third color photosensitive pixel 2443, and the third color photosensitive pixel 2443 passes through the corresponding third color sub filter 2343. light to generate electrical signals.
  • At least one color filter 234 in the first filter group 231 includes a first color sub-filter 2341 and a second color sub-filter 2342, then at least one color filter in the first filter group 231 234 corresponds to the photosensitive pixel 2441 of the first color and the photosensitive pixel 2442 of the second color.
  • the electronic device combines the multiple panchromatic pixels 2431 corresponding to the same panchromatic filter 233 to read out the first pixel value.
  • a plurality of first color sub-filters 2341 and a plurality of second color sub-filters 2342 may be included in the same color filter 234 at the same time, and the electronic device filters light of a plurality of first color sub-filters in the same color filter 234
  • the photosensitive pixels 2441 of the first color corresponding to the slice 2341 combine to read out the second pixel value.
  • the same color filter 234 may include multiple third color sub-filters 2343, and the electronic device combines the third color photosensitive pixels 2443 corresponding to the multiple third color sub-filters 2343 in the same color filter 234 Read out the third pixel value.
  • the electronic device obtains a first combined image according to each first pixel value, each second pixel value, and each third pixel value.
  • a second combined image is obtained according to the fourth pixel values read out by combining the plurality of second-color photosensitive pixels corresponding to the color filter.
  • At least one color filter 234 in the first filter group 231 includes a first color sub-filter 2341 and a second color sub-filter 2342, then at least one color filter in the first filter group 231 234 corresponds to the photosensitive pixel 2441 of the first color and the photosensitive pixel 2442 of the second color.
  • the electronic device reads the second color photosensitive pixels 2442 corresponding to each second color sub-filter 2342 in the same color filter 234 and reads out fourth pixel values, and generate a second merged image based on each fourth pixel value.
  • the first merged image is an image composed of the first pixel value, the second pixel value and the third pixel value
  • the second merged image is an image composed of the fourth pixel value and empty pixels.
  • empty pixels are pixels without any information.
  • the same panchromatic filter 233 corresponds to four panchromatic pixels 2431, and in different filter groups, the same color filter 234 can simultaneously correspond to two photosensitive pixels 2441 of the first color and two photosensitive pixels 2441 of the second color.
  • the two-color photosensitive pixels 2442 and the color filter 234 can correspond to four third-color photosensitive pixels 2443 .
  • Operation 906 Obtain a first target image based on the first combined image and the second combined image.
  • the electronic device can read pixel values from the first combined image and the second combined image according to a preset pixel reading method to generate the first target image.
  • the preset pixel reading mode is a preset pixel reading mode.
  • the first pixel values read out by combining the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, and the multiple panchromatic pixels corresponding to the color filters can integrate the panchromatic channel information into the image and improve the overall image quality.
  • the amount of incoming light makes it possible to generate the first merged image with more information and clearer detail analysis.
  • the photosensitive pixels of the second color can be separated to form a second combined image, so that the photosensitive pixels of the second color are in the second combined image and The positions of the photosensitive pixels of the first color in the first merged image are consistent.
  • the first target image is obtained, and the photosensitive pixels of the first color in the first combined image and the photosensitive pixels of the second color in the second combined image can be mixed and arranged, so that the generated first A target image color is clearer.
  • pixel combination readout reduces the size of the generated first target image and consumes low power consumption for generating the image.
  • the first target image is obtained, including:
  • each second pixel value in the first combined image is the same as the position of each fourth pixel value in the second combined image.
  • the electronic device may traverse each pixel value in the first merged image, and in each traverse, determine whether the currently traversed pixel value is the second pixel value. If the currently traversed pixel value is not the second pixel value, continue to traverse the next pixel value. In the case that the currently traversed pixel value is the second pixel value, determine the position of the currently traversed second pixel value, and read the second pixel value at the same position as the currently traversed second pixel value from the second merged image. Four pixel values, and adjust the read fourth pixel value to be adjacent to the second pixel value, and then continue to traverse the next pixel value until the pixel values in the first merged image are traversed to generate the first target image.
  • the second pixel value and the fourth pixel value are adjacent to each other, indicating that the second pixel value and the fourth pixel value have the same position in the first target image, that is, the second pixel value and the fourth pixel value are in the same position.
  • the coordinates of the fourth pixel value in the first target image are the same.
  • the currently traversed pixel value is not the second pixel value, continue to traverse the next pixel value.
  • the currently traversed pixel value is the second pixel value, determine the position of the currently traversed second pixel value, and read the second pixel value at the same position as the currently traversed second pixel value from the second merged image.
  • four pixel values and adjust the read fourth pixel value to be horizontally adjacent to the second pixel value.
  • the fourth pixel value is horizontally adjacent to the second pixel value, and the fourth pixel value may be taken as the previous pixel value adjacent to the second pixel value in the horizontal direction, or the fourth pixel value may be taken as the second pixel value in the horizontal direction. Next pixel value in the direction.
  • the pixel values in the first merged image are traversed, and when the traversed pixel value is the second pixel value, the fourth pixel value at the same position as the second pixel value is read from the second merged image , and adjust the read fourth pixel value to be adjacent to the second pixel value, so that the photosensitive pixels of the first color and the photosensitive pixels of the second color are mixed and arranged, and the color resolution ability is improved, so that the generated first target image
  • photosensitive pixels of the first color, photosensitive pixels of the second color and photosensitive pixels of the third color in each row and column that is, each row and column of the target image has RGB pixels, which can effectively reduce the risk of false color.
  • the original image 1002 is obtained by the image sensor, the local identical pixels in the original image 1002 are averaged, and binning is performed, that is, the four locally identical panchromatic pixels are combined and read out the first
  • the pixel value, the locally identical 2 photosensitive pixels of the first color are combined to read the second pixel value, and the locally identical 4 photosensitive pixels of the third color are combined to read the third pixel value to obtain the first combined image 1004 .
  • the locally identical two photosensitive pixels of the second color are combined to read out the fourth pixel value to obtain a second combined image 1006 .
  • the second merged image 1006 may be generated by fourth pixel values and empty pixels.
  • a panchromatic pixel is a w pixel
  • a photosensitive pixel of a first color is an R pixel
  • a photosensitive pixel of a third color is a G pixel
  • a photosensitive pixel of a second color is a B pixel.
  • the pixel values are read from the first merged image 1004 and the second merged image 1006 , so that each fourth pixel value is used as the next pixel value adjacent to the second pixel value, and the first target image 1008 is obtained.
  • the color filters of the second filter group further include a first color sub-filter or a second color sub-filter; in the first resolution mode, according to the The first pixel value read out by combining the multiple panchromatic pixels corresponding to the panchromatic filter, and the second pixel value read out by combining the multiple first color photosensitive pixels corresponding to the color filter, and the color filter corresponding A plurality of photosensitive pixels of the third color are combined to read out the third pixel value to obtain the first combined image, including:
  • the first resolution mode according to the first pixel value combined and read out by the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, according to the multiple first color sub-colors in the first filter set Combine and read out the second pixel value of the first color photosensitive pixel corresponding to the filter, according to the third color corresponding to a plurality of third color sub-filters in the second filter group containing the first color sub-filter Combining and reading out the third pixel value of the photosensitive pixels, and combining and reading out the fourth pixel value of the photosensitive pixels of the second color corresponding to the plurality of second color sub-filters in the second filter set to obtain the first combined image ;
  • the fourth pixel value read out by combining the light-sensitive pixels of the second color is based on the second pixel value read out by combining the light-sensitive pixels of the first color corresponding to the plurality of sub-filters of the first color in the second filter group, according to the second pixel value including the first color light-sensitive pixels combined and read out.
  • the third color photosensitive pixels corresponding to the plurality of third color sub-filters in the second filter group of the two-color sub-filters combine the read-out third pixel values to obtain a second combined image.
  • a plurality of filter sets include a first filter set 231 and a second filter set 232, and at least one color filter 234 in the first filter set 231 includes a first color sub-filter 2341 and the second color sub-filter 2342.
  • the color filter 234 of the second filter set 232 includes a third color sub-filter 2343 , and also includes a first color sub-filter 2341 or a second color sub-filter 2342 .
  • the electronic device In the first resolution mode, the electronic device combines multiple panchromatic pixels 2431 corresponding to the same panchromatic filter 233 to read out the first pixel value.
  • the electronic device For the color filters 234 in the first filter group 231, the electronic device combines the first color photosensitive pixels 2441 corresponding to the multiple first color sub-filters 2341 of the same color filter 234 to read out the second pixel value, and the second color photosensitive pixels 2442 corresponding to a plurality of second color sub-filters 2342 of the same color filter 234 are combined to read out the fourth pixel value, then the same color filter 234 corresponds to the second pixel value and fourth pixel value.
  • the electronic device uses multiple second color filters of the same color filter 234
  • the photosensitive pixels 2441 of the first color corresponding to one color sub-filter 2341 are combined to read the second pixel value
  • the photosensitive pixels 2443 of the third color corresponding to the multiple third color sub-filters 2343 are combined to read the third pixel value.
  • pixel value, the same color filter 234 corresponds to the second pixel value and the third pixel value.
  • the electronic device uses multiple color filters of the same color filter 234
  • Each second-color photosensitive pixel 2442 corresponding to the two-color sub-filter 2342 is combined to read out the fourth pixel value, and each third-color photosensitive pixel corresponding to a plurality of third-color sub-filters 2343 of the same color filter 234
  • the pixels 2443 combine to read out the third pixel value, and the same color filter 234 corresponds to the fourth pixel value and the third pixel value.
  • the electronic device can read the second pixel value; for the color filter 234 corresponding to the second pixel value and the third pixel value, the electronic device can read Take the third pixel value; for the color filter 234 corresponding to the fourth pixel value and the third pixel value, the electronic device can read the fourth pixel value; , the fourth pixel value and all the first pixel values to generate the first merged image.
  • the electronic device can obtain the fourth pixel value; for the color filter 234 corresponding to the second pixel value and the third pixel value, the electronic device can obtain the fourth pixel value Two pixel values; for the color filter 234 corresponding to the fourth pixel value and the third pixel value, the electronic device can obtain the third pixel value; value and null pixels to generate a second merged image.
  • the electronic device reads pixel values from the first merged image and the second merged image to generate a first target image.
  • the color filter 234 of the second filter group 232 includes a third color sub-filter 2343, and also includes a first color sub-filter 2341 or a second color sub-filter 2342.
  • the first resolution mode according to the first pixel value combined and read out by multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, according to the multiple first color sub-filters in the first filter set Combine and read out the second pixel value of the photosensitive pixels of the first color corresponding to the light sheet, according to the photosensitivity of the third color corresponding to the multiple third color sub-filters in the second filter group containing the first color sub-filters
  • the third pixel value read out by pixel combination and the fourth pixel value combined and read out according to the second color light-sensitive pixels corresponding to the second color sub-filters in the second filter group, so that the combined readout obtained
  • the first merged image contains pixels of all colors, which can improve the resolution of color channels.
  • the third pixel value combined and read out by the multiple photosensitive pixels of the third color corresponding to the color pixel, and the value corresponding to the color filter Combining the read-out fourth pixel values of multiple photosensitive pixels of the second color to obtain a second combined image, so that the distribution of pixels of each color in the first target image generated according to the first combined image and the second combined image is more uniform and effective. Improve image clarity.
  • the original image 1102 is obtained by the image sensor, and the local four adjacent panchromatic pixels in the original image 1102 are combined to read out the first pixel value, and the diagonal
  • the two photosensitive pixels of the first color are combined to read the second pixel value
  • the two photosensitive pixels of the third color on the diagonal are combined to read the third pixel value
  • the two photosensitive pixels of the second color on the diagonal are combined to read the third pixel value.
  • the electronic device may generate a first target image based on the first merged image 1104 and the second merged image 1106 .
  • the method further includes:
  • Operation 1202 in the second resolution mode, combine and read out the first pixel values according to the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, and the multiple first pixel values corresponding to the color filters.
  • the second resolution mode refers to the mode used in scenes with relatively low resolution requirements, which is a secondary pixel binning readout mode with low resolution, low power consumption, high signal-to-noise ratio, and high frame rate.
  • the resolution and power consumption corresponding to the second resolution mode are smaller than the resolution and power consumption corresponding to the first resolution mode.
  • the signal-to-noise ratio and frame rate corresponding to the second resolution mode are greater than the signal-to-noise ratio and frame rate corresponding to the first resolution mode.
  • the second resolution mode may specifically be a preview mode during image capture, a preview mode during video capture, or a scene with lower resolution requirements such as image capture and video capture under night scenes, but is not limited thereto.
  • the preview modes of video shooting are 1080p video preview, application video preview, etc.
  • the shooting instruction is preview shooting.
  • the second resolution mode is triggered.
  • the electronic device detects whether the current environment is a night scene, and if the current environment is a night scene, triggers the second resolution mode.
  • the readout mode corresponding to the second resolution mode is triggered.
  • the light transmitted by the sub-filter 2331 in the panchromatic filter 233 is projected onto the corresponding panchromatic pixel 2431, and the panchromatic pixel 2431 receives the light passing through the sub-filter 2331 to Generate electrical signals.
  • the light transmitted by the first color sub-filter 2341 in the color filter 234 is projected onto the corresponding first color photosensitive pixel 2441, and the first color photosensitive pixel 2441 passes through the corresponding first color photosensitive pixel 2341. light to generate electrical signals.
  • the light transmitted by the second color sub-filter 2342 in the color filter 234 is projected onto the corresponding second color photosensitive pixel 2442, and the second color photosensitive pixel 2442 passes through the corresponding second color sub filter 2342. light to generate electrical signals.
  • the light transmitted by the third color sub-filter 2343 in the color filter 234 is projected onto the corresponding third color photosensitive pixel 2443, and the third color photosensitive pixel 2443 passes through the corresponding third color sub filter 2343. light to generate electrical signals.
  • At least one color filter 234 in the first filter group 231 includes a first color sub-filter 2341 and a second color sub-filter 2342, then at least one color filter in the first filter group 231 234 corresponds to the photosensitive pixel 2441 of the first color and the photosensitive pixel 2442 of the second color.
  • the electronic device combines the multiple panchromatic pixels 2431 corresponding to the same panchromatic filter 233 to read out the first pixel value.
  • a plurality of first color sub-filters 2341 and a plurality of second color sub-filters 2342 may be included in the same color filter 234 at the same time, and the electronic device filters light of a plurality of first color sub-filters in the same color filter 234
  • the photosensitive pixels 2441 of the first color corresponding to the slice 2341 combine to read out the second pixel value.
  • the same color filter 234 may include multiple third color sub-filters 2343, and the electronic device combines the third color photosensitive pixels 2443 corresponding to the multiple third color sub-filters 2343 in the same color filter 234 Read out the third pixel value.
  • the electronic device obtains a first combined image according to each first pixel value, each second pixel value, and each third pixel value.
  • a second combined image is obtained according to the fourth pixel values read out by combining the plurality of photosensitive pixels of the second color corresponding to the color filter.
  • the electronic device reads the second color photosensitive pixels 2442 corresponding to each second color sub-filter 2342 in the same color filter 234 and reads out fourth pixel values, and generate a second merged image based on each fourth pixel value.
  • a panchromatic combined image is obtained.
  • the electronic device combines the plurality of panchromatic pixels in the ninth diagonal direction in the first combined image to read fifth pixel values, and generates a panchromatic combined image according to each fifth pixel value. All of the panchromatic pixels in the panchromatic combined image are panchromatic.
  • Operation 1208 combine and read out the sixth pixel value based on the plurality of photosensitive pixels of the first color in the direction of the tenth diagonal in the first merged image, and the plurality of photosensitive pixels of the third color in the direction of the tenth diagonal Combining the read-out seventh pixel values to obtain a third combined image, the direction of the ninth diagonal is different from the direction of the tenth diagonal.
  • the electronic device combines and reads out the sixth pixel value from a plurality of light-sensitive pixels of the first color in the direction of the tenth diagonal in the first combined image, and combines and reads out a plurality of light-sensitive pixels of the third color in the direction of the tenth diagonal.
  • a seventh pixel value is obtained, and a third combined image is generated according to the sixth pixel value and the seventh pixel value.
  • the third combined image includes photosensitive pixels of the first color and photosensitive pixels of the third color.
  • the direction of the ninth diagonal is different from the direction of the tenth diagonal. Specifically, the direction of the seventh diagonal is not parallel to the direction of the tenth diagonal, or the direction of the ninth diagonal is perpendicular to the direction of the tenth diagonal, etc. .
  • a fourth combined image is obtained based on combined and read eighth pixel values of a plurality of photosensitive pixels of the second color in the tenth diagonal direction in the second combined image.
  • the electronic device combines the plurality of light-sensitive pixels of the second color in the direction of the tenth diagonal in the second combined image to read out an eighth pixel value. Based on the respective eighth pixel values, a fourth merged image is generated.
  • the electronic device combines and reads out the eighth pixel value from the two light-sensitive pixels of the second color in the tenth diagonal direction in the second combined image, and generates the eighth pixel value based on each eighth pixel value and the empty pixel.
  • Four merged images are used to generate and read the eighth pixel value from the two light-sensitive pixels of the second color in the tenth diagonal direction in the second combined image, and generates the eighth pixel value based on each eighth pixel value and the empty pixel.
  • a second target image is obtained based on the panchromatic combined image, the third combined image and the fourth combined image.
  • the electronic device can generate the second target image according to the pixel values read from the full-color combined image, the third combined image and the fourth combined image.
  • the first pixel values read out are combined according to the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, and the multiple corresponding to the color filters
  • Combining the read-out fourth pixel values of the two photosensitive pixels of the second color to obtain a second combined image can separate the photosensitive pixels of the second color as a separate image and reduce the resolution of the image.
  • a panchromatic combined image is obtained based on the fifth pixel values read out by combining a plurality of panchromatic pixels in the ninth diagonal direction in the first combined image, which can separate the panchromatic pixels from the first combined image while reducing the Image size to reduce power consumption.
  • the seventh pixel value the third merged image including the photosensitive pixels of the first color and the photosensitive pixels of the third color is obtained, and the multi-in-one pixel readout method makes the generated image noise less.
  • the resolution of the obtained second target image is further reduced, and the panchromatic pixels have a higher signal-to-noise ratio, and the frame rate of the image is high, thereby achieving lower power consumption and higher signal-to-noise ratio of the secondary pixel combination output.
  • Better image processing effect is achieved.
  • the second target image is obtained, including:
  • each sixth pixel value in the third combined image is the same as the position of each eighth pixel value in the fourth combined image.
  • the position of each pan-color pixel in the pan-color combined image is the same as the position of each sixth pixel value in the third combined image and the position of each eighth pixel value in the fourth combined image.
  • the electronic device may traverse each pixel value in the third merged image, and in each traverse, determine whether the currently traversed pixel value is the sixth pixel value or the seventh pixel value. In the case that the currently traversed pixel value is not the sixth pixel value and not the seventh pixel value, continue to traverse the next pixel value.
  • the currently traversed pixel value is the sixth pixel value or the seventh pixel value
  • determine the position of the currently traversed pixel value and read the fifth pixel value corresponding to the traversed pixel value from the pan-color combined image , and adjust the read fifth pixel value to be adjacent to the currently traversed pixel value.
  • the eighth pixel value identical to the sixth pixel position is read from the fourth merged image, and the read eighth pixel value is adjusted to be the same as the sixth pixel value.
  • the sixth pixel value is adjacent, and then continue to traverse the next pixel value until the pixel values in the pan-color combined image, the third combined image and the fourth combined image are all read, and the second target image is obtained.
  • the sixth pixel value and the eighth pixel value are adjacent in the second target image, it means that the positions of the sixth pixel value and the eighth pixel value in the second target image are the same, that is, the sixth pixel value The same coordinates as the eighth pixel value in the second target image.
  • the fifth pixel value is adjacent to the currently traversed pixel value, which may be that the fifth pixel value is adjacent to the currently traversed pixel value in the horizontal direction, specifically, the fifth pixel value may be used as the current pixel value.
  • the eighth pixel value is adjacent to the sixth pixel value, and may be horizontally adjacent to the sixth pixel value.
  • the eighth pixel value is horizontally adjacent to the sixth pixel value, which may be the eighth pixel value as the sixth pixel value adjacent to the previous pixel value in the horizontal direction, or the eighth pixel value as the sixth pixel value in the horizontal direction Next pixel value in the direction.
  • the eighth pixel value is used as the sixth pixel value adjacent to the sixth pixel value in the horizontal direction.
  • a pixel value when the fifth pixel value is the sixth pixel value adjacent to the previous pixel value in the horizontal direction, the eighth pixel value is used as the sixth pixel value adjacent to the sixth pixel value in the horizontal direction.
  • the pixels in the third merged image are traversed, and when the traversed pixel value is the sixth pixel value or the seventh pixel value, the pixel value corresponding to the traversed pixel value is read from the full-color merged image.
  • Five pixel values are adjusted, and the read fifth pixel value is adjusted to be adjacent to the traversed pixel value, so that panchromatic pixels are introduced into the image and the amount of light entering is increased.
  • the traversed pixel value is the sixth pixel value
  • the eighth pixel value that is the same as the sixth pixel position is read from the fourth combined image, and the read eighth pixel value is adjusted to be the same as the sixth pixel value.
  • the pixel values are adjacent, so that the photosensitive pixels of the first color and the photosensitive pixels of the second color are mixed and arranged, which improves the color resolution capability, so that each row and column of the generated second target image has photosensitive pixels of the first color and photosensitive pixels of the second color.
  • photosensitive pixels and second-color photosensitive pixels that is, each row and column of the target image has RGB pixels, which can effectively reduce the risk of false colors.
  • the original image 1302 is obtained by the image sensor, and the locally identical 4 panchromatic pixels in the original image 1302 are combined to read out the first pixel value, and the locally identical 2 first
  • the color photosensitive pixels are combined to read out the second pixel value, and the locally identical four photosensitive pixels of the third color are combined to read out the third pixel value to obtain the first combined image 1304 .
  • the locally identical two photosensitive pixels of the second color are combined to read a fourth pixel value to obtain a second combined image 1306 .
  • the second merged image 1306 may be generated by fourth pixel values and empty pixels.
  • a fourth combined image 1312 is obtained based on combined and read eighth pixel values of the two light-sensitive pixels of the second color in the direction of the tenth diagonal line D10 in the second combined image 1306 .
  • the traversed pixel value being the sixth pixel value or the seventh pixel value
  • reading the fifth pixel value corresponding to the traversed pixel value from the full-color merged image 1308 and use the read fifth pixel value as the previous pixel value adjacent to the traversed pixel value.
  • the traversed pixel value is the sixth pixel value
  • the eighth pixel value identical to the sixth pixel position is read from the fourth merged image 1312, and the read eighth pixel value is used as the same as
  • the second target image 1314 is obtained after traversing the pixel values adjacent to the sixth pixel value until the pixel values in the third merged image.
  • the color filters of the second filter group further include a first color sub-filter or a second color sub-filter;
  • the first pixel value read out by combining the multiple panchromatic pixels corresponding to the color filter, and the second pixel value read out by combining the multiple first color light-sensitive pixels corresponding to the color filter, and the second pixel value corresponding to the color filter Combining and reading out the third pixel values of a plurality of photosensitive pixels of the third color to obtain a first combined image, including:
  • the second resolution mode according to the first pixel value combined and read out by the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, according to the multiple first color sub-pixels in the first filter set Combine and read out the second pixel value of the first color photosensitive pixel corresponding to the filter, according to the third color corresponding to a plurality of third color sub-filters in the second filter group containing the first color sub-filter Combining and reading out the third pixel value of the photosensitive pixels, and combining and reading out the fourth pixel value of the photosensitive pixels of the second color corresponding to the plurality of second color sub-filters in the second filter set to obtain the first combined image ;
  • a second combined image is obtained by combining and reading out the fourth pixel values of a plurality of light-sensitive pixels of the second color corresponding to the color filter, including:
  • the second pixel value read out by combining the corresponding first-color photosensitive pixels is combined according to the third-color photosensitive pixels corresponding to multiple third-color sub-filters in the second filter group containing the second-color sub-filters read out the third pixel value to obtain the second merged image;
  • the seventh pixel value of to obtain the third combined image, including: the fifth pixel value combined and read out based on a plurality of panchromatic pixels in the direction of the ninth diagonal in the first combined image, in the direction of the tenth diagonal
  • the sixth pixel value combined and read out by a plurality of photosensitive pixels of the first color on the direction of the tenth diagonal, the seventh pixel value read out in combination of a plurality of photosensitive pixels of the third color in the direction of the tenth diagonal, and Combining and reading the eighth pixel values of multiple light-sensitive pixels of the second color in the direction to obtain a panchromatic combined image and a third combined image respectively;
  • the fourth combined image based on the eighth pixel values read out by combining and reading out a plurality of light-sensitive pixels of the second color in the tenth diagonal direction in the second combined image including: based on the second combined image, in the tenth pair
  • a plurality of light-sensitive pixels of the second color in the diagonal direction combine the read-out eighth pixel values to obtain a fourth combined image.
  • a plurality of filter sets include a first filter set 231 and a second filter set 232, and the color filter 234 in the first filter set 231 includes a first color sub-filter 2341 and the second color sub-filter 2342.
  • the color filter 234 of the second filter set 232 includes a third color sub-filter 2343 , and also includes a first color sub-filter 2341 or a second color sub-filter 2342 .
  • the electronic device In the second resolution mode, the electronic device combines multiple panchromatic pixels 2431 corresponding to the same panchromatic filter 233 to read out the first pixel value.
  • the electronic device For the color filters 234 in the first filter group 231, the electronic device combines the first color photosensitive pixels 2441 corresponding to the multiple first color sub-filters 2341 of the same color filter 234 to read out the second pixel value, and the second color photosensitive pixels 2442 corresponding to a plurality of second color sub-filters 2342 of the same color filter 234 are combined to read out the fourth pixel value, then the same color filter 234 corresponds to the second pixel value and fourth pixel value.
  • the electronic device uses multiple second color filters of the same color filter 234
  • the photosensitive pixels 2441 of the first color corresponding to one color sub-filter 2341 are combined to read the second pixel value
  • the photosensitive pixels 2443 of the third color corresponding to the multiple third color sub-filters 2343 are combined to read the third pixel value.
  • pixel value, the same color filter 234 corresponds to the second pixel value and the third pixel value.
  • the electronic device uses multiple color filters of the same color filter 234
  • Each second-color photosensitive pixel 2442 corresponding to the two-color sub-filter 2342 is combined to read out the fourth pixel value, and each third-color photosensitive pixel corresponding to a plurality of third-color sub-filters 2343 of the same color filter 234
  • the pixels 2443 combine to read out the third pixel value, and the same color filter 234 corresponds to the fourth pixel value and the third pixel value.
  • the electronic device can read the second pixel value; for the color filter 234 corresponding to the second pixel value and the third pixel value, the electronic device can read Take the third pixel value; for the color filter 234 corresponding to the fourth pixel value and the third pixel value, the electronic device can read the fourth pixel value; pixel values, each fourth pixel value and all first pixel values to generate a first merged image.
  • the first combined image includes panchromatic pixels, photosensitive pixels of the first color, photosensitive pixels of the second color and photosensitive pixels of the third color.
  • the electronic device For the first combined image, the electronic device combines a plurality of panchromatic pixels in the ninth diagonal direction to read fifth pixel values, and generates a panchromatic combined image according to each fifth pixel value. All of the panchromatic pixels in the panchromatic combined image are panchromatic.
  • the electronic device For the first merged image, the electronic device combines and reads out the sixth pixel value from a plurality of light-sensitive pixels of the first color in the direction of the tenth diagonal in the first merged image, and reads out the value of the plurality of light-sensitive pixels in the direction of the tenth diagonal.
  • the third color photosensitive pixels are combined to read the seventh pixel value, and the plurality of second color photosensitive pixels in the direction of the tenth diagonal are combined to read the eighth pixel value.
  • a third combined image is generated according to the read sixth pixel value, seventh pixel value and eighth pixel value.
  • the third combined image includes photosensitive pixels of the first color, photosensitive pixels of the second color and photosensitive pixels of the third color.
  • the electronic device can obtain the fourth pixel value; for the color filter 234 corresponding to the second pixel value and the third pixel value, the electronic device can obtain the fourth pixel value Two pixel values; for the color filter 234 corresponding to the fourth pixel value and the third pixel value, the electronic device can obtain the third pixel value; value and null pixels to generate a second merged image.
  • the electronic device For the second merged image, the electronic device combines and reads out the sixth pixel value from the plurality of light-sensitive pixels of the first color in the direction of the tenth diagonal in the second merged image, and reads out the value of the plurality of photosensitive pixels of the first color in the direction of the tenth diagonal in the second merged image.
  • the photosensitive pixels of the three colors are combined to read out the seventh pixel value
  • the plurality of photosensitive pixels of the second color in the tenth diagonal direction are combined to read out the eighth pixel value.
  • a fourth combined image is generated according to the read sixth pixel value, seventh pixel value, and eighth pixel value, and empty pixels.
  • the fourth combined image includes photosensitive pixels of the first color, photosensitive pixels of the second color, photosensitive pixels of the third color and empty pixels.
  • the electronic device can generate the second target image according to the panchromatic combined image, the third combined image and the fourth combined image.
  • the second resolution mode according to the first pixel value combined and read out by a plurality of panchromatic pixels corresponding to the panchromatic filters in the filter set, according to the number of panchromatic pixels in the first filter set
  • the second pixel value combined and read out from the first color photosensitive pixels corresponding to the first color sub-filters is combined and read out according to the combined readout value of the third color photosensitive pixels in the second filter group containing the first color sub-filters
  • the third pixel value, and the fourth pixel value combined and read out according to the third color photosensitive pixels corresponding to the plurality of second color sub-filters in the second filter group are combined to obtain the first combined image, and the first combined image will be obtained according to the first combined image.
  • the fourth pixel value read out by combining the second-color light-sensitive pixels corresponding to the multiple second-color sub-filters in the filter set is based on the fourth pixel value corresponding to the multiple first-color sub-filters in the second filter set.
  • the second pixel value read out by combination of photosensitive pixels of one color is combined and read out according to the combined readout value of the third color photosensitive pixels corresponding to multiple third color sub-filters in the second filter group including the second color sub-filters.
  • the third pixel value, to obtain the second merged image can separate the first color photosensitive pixel, the second color photosensitive pixel and the third color photosensitive pixel corresponding to each color filter to form two merged images, so that the two merged images
  • the photosensitive pixels of each color in are in the same position in the respective merged images.
  • a panchromatic combined image is obtained based on the fifth pixel values read out by combining a plurality of panchromatic pixels in the ninth diagonal direction in the first combined image, and the panchromatic pixels can be separated from the first combined image while subtracting Small image size to reduce power consumption.
  • the combined readout of the plurality of photosensitive pixels of the third color in the direction of the tenth diagonal Based on the sixth pixel value combined and read out by the plurality of photosensitive pixels of the first color in the direction of the tenth diagonal in the second combined image, the combined readout of the plurality of photosensitive pixels of the third color in the direction of the tenth diagonal.
  • the dimensions of the image, pan-merged image remain the same.
  • the third combined image and the fourth combined image it is possible to mix and arrange different first color photosensitive pixels, second color photosensitive pixels and third color photosensitive pixels, so that in the generated second target image, RGB The pixel distribution is more even and the image quality is higher. Moreover, the resolution of the obtained second target image is further reduced, and the panchromatic pixels have a higher signal-to-noise ratio, and the frame rate of the image is high, thereby achieving lower power consumption and higher signal-to-noise ratio of the secondary pixel combination output. Better image processing effect.
  • the original image 1402 is obtained by the image sensor, and the local four adjacent panchromatic pixels in the original image 1402 are combined to read out the first pixel value, and the The two photosensitive pixels of the first color are combined to read the second pixel value, the two photosensitive pixels of the third color on the diagonal are combined to read the third pixel value, and the two photosensitive pixels of the second color on the diagonal are combined to read the third pixel value.
  • a panchromatic combined image 1408 Combine the two panchromatic pixels in the direction of the ninth diagonal line D9 in the first combined image 1404 and read out the fifth pixel value to obtain a panchromatic combined image 1408 .
  • the two photosensitive pixels of the first color in the direction of the tenth diagonal line D10 are combined to read out the sixth pixel value, and the two photosensitive pixels of the third color in the direction of the tenth diagonal line D10
  • the seventh pixel value is read out in combination
  • the eighth pixel value is read out in combination with the two light-sensitive pixels of the second color in the direction of the tenth diagonal line D10. According to the read sixth pixel value, seventh pixel value and eighth pixel value, a third combined image 1410 is generated.
  • the sixth pixel value of the two photosensitive pixels of the first color in the direction of the tenth diagonal line D10 in the second combined image 1406 is combined to read out the sixth pixel value, and
  • the 2 photosensitive pixels of the third color are combined to read out the seventh pixel value, and the 2 photosensitive pixels of the second color in the direction of the tenth diagonal line D10 are combined to read out the eighth pixel value.
  • a fourth combined image 1412 is generated.
  • the electronic device may generate a second target image based on the full-color combined image 1408 , the third combined image 1410 and the fourth combined image 1412 .
  • the method further includes: in the second resolution mode, combining and reading out the first pixel values according to a plurality of panchromatic pixels corresponding to the filter set to obtain a panchromatic image; Combining and reading out the second pixel values of the corresponding plurality of photosensitive pixels of the first color, and combining and reading the third pixel values of the plurality of photosensitive pixels of the third color corresponding to the filter group to obtain a fifth combined image; Combining the read-out fourth pixel values of the plurality of photosensitive pixels of the second color corresponding to the sheet group to obtain a sixth combined image; based on the panchromatic image, the fifth combined image and the sixth combined image, a third target image is obtained.
  • a plurality of filter sets includes a first filter set 231 and a second filter set 232, and at least one color filter 234 in the first filter set 231 includes a first color sub-filter 2341 and a second color filter set 2341.
  • the two-color sub-filters 2342 at least one color filter 234 in the first filter group 231 corresponds to the photosensitive pixels 2441 of the first color and the photosensitive pixels 2442 of the second color.
  • the color filter 234 in the second filter set 232 includes at least a third color sub-filter 2343 , and the color filter 234 in the second filter set 232 at least corresponds to the third color photosensitive pixel 2443 .
  • the electronic device In the second resolution mode, the electronic device combines the panchromatic pixels 2431 corresponding to all the panchromatic filters 233 in the same filter set to read out the first pixel value, and obtains a panchromatic image according to each first pixel value . Further, the electronic device combines the panchromatic pixels 2431 corresponding to all the panchromatic filters 233 in each first filter set 231 to read out the first pixel value, so that one first filter set 231 corresponds to one first filter set 231 A pixel value, so as to obtain the first pixel value corresponding to each first filter set 231 respectively.
  • the electronic device combines the panchromatic pixels 2431 corresponding to all panchromatic filters 233 in each second filter group 232 to read out the first pixel value, so that one second filter group 232 corresponds to one first pixel value , so as to obtain the first pixel values corresponding to the second filter sets 232 respectively.
  • a full-color image is obtained according to the first pixel values corresponding to the first filter groups 231 and the first pixel values corresponding to the second filter groups 232 .
  • the electronic device combines the photosensitive pixels 2441 of the first color corresponding to the plurality of sub-filters 2341 of the first color of all the color filters 234 in the same filter set to read out the second pixel value.
  • the electronic device combines the light-sensitive pixels 2443 of the third color corresponding to the multiple third-color sub-filters 2343 of all the color filters 234 in the same filter set to read out the third pixel value.
  • the electronic device obtains a fifth combined image according to each second pixel value and each third pixel value.
  • the electronic device combines the photosensitive pixels 2441 of the first color corresponding to the multiple first color sub-filters 2341 of all the color filters 234 in each first filter group 231 to read out the second pixel value, so that one first filter set 231 corresponds to one second pixel value, so as to obtain the second pixel values corresponding to each first filter set 231 respectively.
  • the electronic device combines and reads out the third pixel value from the third color photosensitive pixels 2443 corresponding to the multiple third color sub-filters 2343 of all the color filters 234 in each second filter group 232, so that one The second filter group 232 corresponds to a third pixel value, so as to obtain the third pixel value corresponding to each second filter group 232 respectively.
  • the fifth combined image is obtained according to the second pixel values corresponding to the first filter sets 231 and the third pixel values corresponding to the second filter sets 232 respectively.
  • the electronic device combines the light-sensitive pixels 2442 of the second color corresponding to the multiple second-color sub-filters 2342 of all the color filters 234 in the same filter set to read out the fourth pixel value. Further, the electronic device combines the light-sensitive pixels 2442 of the second color corresponding to the plurality of second-color sub-filters 2342 of all the color filters 234 in each first filter group 231 to read out the fourth pixel value. , so that one first filter set 231 corresponds to one fourth pixel value, so as to obtain fourth pixel values corresponding to each first filter set 231 respectively. The electronic device obtains the sixth combined image according to the fourth pixel values corresponding to the first filter sets 231 .
  • the electronic device can read pixel values from the full-color image, the fifth combined image, and the sixth combined image according to a preset pixel reading method to generate a third target image.
  • a panchromatic image in the second resolution mode, can be separated by combining the first pixel values read out by multiple panchromatic pixels corresponding to the filter set;
  • the second pixel value combined and read out from the photosensitive pixels of the first color, and the third pixel value read out from the multiple photosensitive pixels of the third color corresponding to the filter group can combine the photosensitive pixels of the first color and the photosensitive pixels of the third color Separated to form the fifth merged image, which can quickly and effectively reduce the size of the image.
  • the photosensitive pixels of the second color According to the fourth pixel value combined and read out by multiple photosensitive pixels of the second color corresponding to the filter group, the photosensitive pixels of the second color can be separated to form the sixth combined image, so that the photosensitive pixels of the second color can be combined in the sixth combined image.
  • the third target image is obtained based on the panchromatic image, the fifth merged image, and the sixth merged image, which can integrate panchromatic channel information into the image, improve the overall light input, and generate images with more information and clearer detail analysis
  • the third target image is obtained through combining and reading out multiple pixels corresponding to the filter set, the size of the generated third target image is reduced, and the power consumption required for generating the image is low.
  • the color filter of the second filter group further includes a first color sub-filter or a second color sub-filter; according to a plurality of photosensitive pixels of the first color corresponding to the filter group are combined The second pixel value read out, and the third pixel value read out by combining the plurality of photosensitive pixels of the third color corresponding to the filter group, to obtain a fifth combined image, including:
  • the second pixel value combined and read out by the first color photosensitive pixels corresponding to the multiple first color sub-filters in the first filter set, according to the second pixel value in the second filter set containing the first color sub-filters
  • the third color photosensitive pixels corresponding to the multiple third color sub-filters combine and read out the third pixel value, and the second color photosensitive pixels corresponding to the multiple second color sub-filters in the second filter group Combining the read-out fourth pixel values to obtain a fifth combined image
  • the sixth combined image is obtained according to the combined and read-out fourth pixel values of a plurality of second-color photosensitive pixels corresponding to the filter set, including:
  • the combined readout of the fourth pixel value of the second color photosensitive pixels corresponding to the multiple second color sub-filters in the first filter set according to the multiple first color sub-filters in the second filter set
  • the second pixel value read out by combining the corresponding first-color photosensitive pixels is combined according to the third-color photosensitive pixels corresponding to multiple third-color sub-filters in the second filter group containing the second-color sub-filters read out the third pixel value to obtain a sixth merged image.
  • a plurality of filter sets includes a first filter set 231 and a second filter set 232, and at least one color filter 234 in the first filter set 231 includes a first color sub-filter 2341 and a second color filter set 2341.
  • the two-color sub-filters 2342 at least one color filter 234 in the first filter group 231 corresponds to the photosensitive pixels 2441 of the first color and the photosensitive pixels 2442 of the second color.
  • the color filter 234 in the second filter set 232 includes a third color sub-filter 2343 , and also includes a first color sub-filter 2341 or a second color sub-filter 2342 .
  • the electronic device In the second resolution mode, the electronic device combines the panchromatic pixels 2431 corresponding to all the panchromatic filters 233 in the same filter set to read out the first pixel value, and obtains a panchromatic image according to each first pixel value . Further, the electronic device combines the panchromatic pixels 2431 corresponding to all the panchromatic filters 233 in each first filter set 231 to read out the first pixel value, so that one first filter set 231 corresponds to one first filter set 231 A pixel value, so as to obtain the first pixel value corresponding to each first filter set 231 respectively.
  • the electronic device combines the panchromatic pixels 2431 corresponding to all panchromatic filters 233 in each second filter group 232 to read out the first pixel value, so that one second filter group 232 corresponds to one first pixel value , so as to obtain the first pixel values corresponding to the second filter sets 232 respectively.
  • a full-color image is obtained according to the first pixel values corresponding to the first filter groups 231 and the first pixel values corresponding to the second filter groups 232 .
  • the electronic device combines and reads out the second pixel value from the first color photosensitive pixels 2441 corresponding to the multiple first color sub-filters 2341 of all the color filters 234 in each first filter group 231, so that One first filter set 231 corresponds to one second pixel value, so as to obtain the second pixel values corresponding to each first filter set 231 respectively.
  • the electronic device For the second filter group 232 that includes the first color sub-filter 2341 and the third color sub-filter 2343, the electronic device respectively converts the multiples of all the color filters 234 in each second color filter group 232 Each third color photosensitive pixel 2443 corresponding to each third color sub-filter 2343 is combined to read out the third pixel value, so that a second filter group 232 including the first color sub-filter 2341 corresponds to a third pixel value, so as to obtain the third pixel value corresponding to each second filter group 232 including the first color sub-filter 2341.
  • the electronic device For the second filter group 232 that includes the second color sub-filter 2342 and the third color sub-filter 2343, the electronic device respectively converts the multiples of all the color filters 234 in each second color filter group 232 Each second color photosensitive pixel 2442 corresponding to each second color sub-filter 2342 is combined to read out the fourth pixel value, so that a second filter group 232 including the second color sub-filter 2342 corresponds to a fourth pixel value. pixel values to obtain fourth pixel values corresponding to each second filter group 232 including the second color sub-filter 2342 .
  • the fourth pixel value corresponding to each second filter group 232 of the color sub-filter 2342 obtains the fifth merged image.
  • the electronic device combines and reads out the fourth pixel value from the second color photosensitive pixels 2442 corresponding to the multiple second color sub-filters 2342 of all the color filters 234 in each first filter group 231, so that One first filter set 231 corresponds to one fourth pixel value, so as to obtain the fourth pixel value corresponding to each first filter set 231 respectively.
  • the electronic device For the second filter group 232 that includes the first color sub-filter 2341 and the third color sub-filter 2343, the electronic device respectively converts the multiples of all the color filters 234 in each second color filter group 232 Each first color photosensitive pixel 2441 corresponding to each first color sub-filter 2341 is combined to read out the second pixel value, so that a second filter group 232 including the first color sub-filter 2341 corresponds to a second Pixel values to obtain second pixel values corresponding to each second filter group 232 including the first color sub-filter 2341 .
  • the electronic device For the second filter group 232 that includes the second color sub-filter 2342 and the third color sub-filter 2343, the electronic device respectively converts the multiples of all the color filters 234 in each second color filter group 232 Each third color photosensitive pixel 2443 corresponding to each third color sub-filter 2343 is combined to read out the third pixel value, so that a second filter group 232 including a second color sub-filter 2342 corresponds to a third pixel value, so as to obtain the third pixel value corresponding to each second filter group 232 including the second color sub-filter 2342 respectively.
  • the fourth pixel value corresponding to each first filter group 231 corresponds to the third pixel value to obtain the sixth combined image.
  • the electronic device can read pixel values from the full-color image, the fifth combined image, and the sixth combined image according to a preset pixel reading method to generate a third target image.
  • the second pixel values read out by combining the first color photosensitive pixels corresponding to the multiple first color sub-filters in the first filter group, according to the The third pixel value read out by combining the third color photosensitive pixels corresponding to the multiple third color sub-filters in the second filter set of one color sub-filter, and according to the multiple The fourth pixel value read out by combining the second color photosensitive pixels corresponding to the second color sub-filter can separate part of the first color photosensitive pixels, the third color photosensitive pixels and the second color photosensitive pixels to form a fifth combined image , capable of reducing image size through pixel binning to reduce processing power.
  • the multiple first color sub-filters in the second filter set correspond to The second pixel value read out by combining the photosensitive pixels of the first color, according to the combined reading of the photosensitive pixels of the third color corresponding to the multiple third color sub-filters in the second filter group containing the second color sub-filters
  • the obtained third pixel value can separate part of the photosensitive pixels of the first color, the photosensitive pixels of the third color and the photosensitive pixels of the second color to form the sixth combined image, so that the photosensitive pixels of the second color can be combined in the sixth combined image and The coordinate positions of the photosensitive pixels of the first color in the fifth merged image are consistent.
  • the third target image is obtained based on the panchromatic image, the fifth merged image, and the sixth merged image, which can integrate panchromatic channel information into the image, improve the overall light input, and generate images with more information and clearer detail analysis
  • the third target image is obtained through combining and reading out multiple pixels corresponding to the filter set, the size of the generated third target image is reduced, and the power consumption required for generating the image is low.
  • an image generation method is provided, which is applied to an image sensor of an electronic device, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes a plurality of filter arrays
  • Each filter group includes a color filter and a panchromatic filter, the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter, and the color filter and the panchromatic filter
  • Each light sheet includes 4 sub-filters, and the plurality of filter groups includes at least a first filter group, and at least one color filter in the first filter group includes a first color sub-filter and a second color sub-filter.
  • the color filter in the second filter group includes a third color sub-filter, and the color filter in the second filter group also includes a first color sub-filter or a second color sub-filter Optical filter;
  • the first color photosensitive pixel of the pixel array corresponds to the first color sub-filter of the optical filter array, and the second color photosensitive pixel of the pixel array corresponds to the second color sub-filter of the optical filter array Setting, the photosensitive pixel of the third color of the pixel array is set corresponding to the sub-filter of the third color of the filter array, and the pixel array is configured to receive light passing through the filter array to generate an electrical signal;
  • the method includes:
  • the first resolution mode combine and read out the first pixel value according to the multiple panchromatic pixels corresponding to the panchromatic filter in the filter set, and the multiple first color photosensitive pixels corresponding to the color filter Combining the read-out second pixel values and the third pixel values read out by combining the multiple third color photosensitive pixels corresponding to the color filter to obtain a first combined image; according to the multiple second color values corresponding to the color filter Combine and read out the fourth pixel value of the photosensitive pixel to obtain the second combined image; traverse the pixel values in the first combined image, and read from the second combined image when the traversed pixel value is the second pixel value The fourth pixel value at the same position as the second pixel value, and adjust the read fourth pixel value to be adjacent to the second pixel value until all the pixel values in the first merged image are traversed to obtain the first target image.
  • the first resolution mode according to the first pixel value combined and read out by multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, according to the multiple The first color photosensitive pixel corresponding to the first color sub-filter combines and reads out the second pixel value, according to the correspondence between multiple third color sub-filters in the second filter group including the first color sub-filter The third pixel value combined and read out from the photosensitive pixels of the third color, and the fourth pixel value combined and read out according to the photosensitive pixels of the second color corresponding to the multiple second color sub-filters in the second filter group are obtained.
  • the second pixel value read out by combining the first color photosensitive pixels corresponding to the color sub-filters is based on the second pixel value corresponding to the plurality of third color sub-filters in the second filter group including the second color sub-filters.
  • the three-color photosensitive pixels combine the read-out third pixel values to obtain a second combined image; based on the first combined image and the second combined image, a first target image is obtained.
  • the first pixel values read out are combined according to the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, and the multiple first pixel values corresponding to the color filters Combining and reading out the second pixel value of a photosensitive pixel of one color, and combining and reading out the third pixel value of a plurality of photosensitive pixels of a third color corresponding to the color filter to obtain a first combined image;
  • the resolution corresponding to the second resolution mode The ratio is lower than the resolution corresponding to the first resolution mode;
  • the second combined image is obtained according to the fourth pixel value read out by combining and reading out a plurality of light-sensitive pixels of the second color corresponding to the color filter; based on the ninth pair in the first combined image Combining and reading the fifth pixel value of multiple panchromatic pixels in the diagonal direction to obtain a panchromatic combined image;
  • the sixth pixel value, and the seventh pixel value combined and read out by a plurality of photosensitive pixels of the third color in the direction of the tenth diagonal
  • the first pixel value read out by combining the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set according to the multiple panchromatic pixels in the first filter set
  • the first color photosensitive pixel corresponding to the first color sub-filter combines and reads out the second pixel value, according to the correspondence between multiple third color sub-filters in the second filter group including the first color sub-filter
  • the third pixel value combined and read out from the photosensitive pixels of the third color, and the fourth pixel value combined and read out according to the photosensitive pixels of the second color corresponding to the multiple second color sub-filters in the second filter group are obtained.
  • the first merged image; according to the fourth pixel value read out in combination with the second color photosensitive pixels corresponding to the second color sub-filters in the first filter set, according to the multiple first color sub-filters in the second filter set The second pixel value read out by combining the first color photosensitive pixels corresponding to the color sub-filters is based on the second pixel value corresponding to the plurality of third color sub-filters in the second filter group including the second color sub-filters.
  • the first pixel values read out are combined according to the multiple panchromatic pixels corresponding to the filter set to obtain a panchromatic image; according to the multiple first color pixels corresponding to the filter set Combining the second pixel value read out from the photosensitive pixels, and combining the third pixel values read out from the plurality of photosensitive pixels of the third color corresponding to the filter group to obtain a fifth combined image; combining the read-out fourth pixel values of the two-color photosensitive pixels to obtain a sixth combined image; and obtaining a third target image based on the panchromatic image, the fifth combined image and the sixth combined image.
  • a panchromatic image is obtained by combining the first pixel values read out according to the multiple panchromatic pixels corresponding to the filter set; according to the multiple first pixel values in the first filter set
  • the second pixel value read out by combining the first color photosensitive pixels corresponding to the color sub-filters, according to the second pixel value corresponding to the plurality of third color sub-filters in the second filter group including the first color sub-filters
  • the third pixel value combined and read out from the three-color light-sensitive pixels and the fourth pixel value combined and read out from the second-color light-sensitive pixels corresponding to the second color sub-filters in the second filter group obtain the fifth Combining images; combining and reading out the fourth pixel value according to the second color photosensitive pixels corresponding to the multiple second color sub-filters in the first filter set, and according to the multiple first color sub-filters in the second filter set Combine and read out the second pixel value of the first color photosensitive pixel corresponding to the filter, according to the third color corresponding to
  • an image sensor structure supporting output of two resolutions is provided, thereby providing image output modes of two resolutions, which can adapt to different application scenarios.
  • the first resolution mode is used in scenes with relatively high resolution requirements, and the first pixel value read out by combining multiple panchromatic pixels corresponding to the panchromatic filter in the filter set, and the color filter
  • the second pixel value read out by combining the corresponding plurality of photosensitive pixels of the first color, and the third pixel value read out by combining the plurality of photosensitive pixels of the third color corresponding to the color filter can integrate the panchromatic channel information into the image In this method, the overall light input amount is increased, so that the first merged image with more information and clearer detail analysis can be generated.
  • the second combined image is obtained according to the fourth pixel values read out by combining multiple second color photosensitive pixels corresponding to the color filter, so that the size of the second combined image is consistent with that of the first combined image, thereby ensuring that each second color
  • the positions of the photosensitive pixels in the second combined image are the same as the positions of the photosensitive pixels of each first color in the first combined image, that is, the positions of the fourth pixel values in the second combined image are guaranteed to be the same as the positions of the second pixel values in the second combined image.
  • the positions in a merged image are the same.
  • each row and each column of the generated first target image has the first color photosensitive pixel, the second color photosensitive pixel and the third color photosensitive pixel, that is, each row and each column of the target image has There are RGB pixels, which can effectively reduce the risk of false colors.
  • the first resolution mode combine and read out the first pixel values according to the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, and the multiple first color photosensitizers corresponding to the color filters
  • the fourth pixel value makes the first combined image obtained by combined reading include pixels of all colors, which can improve the resolution capability of the color channels.
  • the third pixel value combined and read out by the multiple photosensitive pixels of the third color corresponding to the color pixel, and the value corresponding to the color filter Combining the read-out fourth pixel values of multiple photosensitive pixels of the second color to obtain a second combined image, so that the distribution of pixels of each color in the first target image generated according to the first combined image and the second combined image is more uniform and effective. Improve image clarity.
  • the second resolution mode in scenes with lower resolution requirements such as night scene shooting, and combine the first pixel values read out according to the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, and the color filter Combining the second pixel values read out from the plurality of color pixels corresponding to the light sheet, and combining the third pixel values read out from the plurality of third color light-sensitive pixels corresponding to the color filter to obtain the first combined image, which will be combined according to the color
  • the plurality of photosensitive pixels of the second color corresponding to the filter combine the read-out fourth pixel values to obtain a second combined image, which can separate the photosensitive pixels of the second color as separate images and reduce the resolution of the image.
  • a panchromatic combined image is obtained based on the fifth pixel values read out by combining a plurality of panchromatic pixels in the ninth diagonal direction in the first combined image, and the panchromatic pixels can be separated from the first combined image while subtracting Small image size to reduce power consumption.
  • the obtained seventh pixel value is used to obtain a third merged image including the photosensitive pixels of the first color and the photosensitive pixels of the third color, and the all-in-one pixel readout method makes the generated image noise less.
  • the eighth pixel value read out based on the combination of a plurality of light-sensitive pixels of the second color in the eighth diagonal direction in the second combined image, so that the size of the fourth combined image is the same as the size of the third combined image and the size of the full-color combined image be consistent.
  • Traverse the pixels in the third merged image and when the traversed pixel value is the sixth pixel value or the seventh pixel value, read the fifth pixel value corresponding to the traversed pixel value from the full-color merged image, and The read fifth pixel value is adjusted to be adjacent to the traversed pixel value, so that panchromatic pixels are introduced into the image, and the amount of incoming light is increased.
  • the eighth pixel value that is the same as the sixth pixel position is read from the fourth combined image, and the read eighth pixel value is adjusted to be the same as the sixth pixel value.
  • the pixel values are adjacent, so that the photosensitive pixels of the first color and the photosensitive pixels of the second color are mixed and arranged, which improves the color resolution capability, so that each row and column of the generated second target image has photosensitive pixels of the first color and photosensitive pixels of the second color.
  • photosensitive pixels and second-color photosensitive pixels that is, each row and column of the target image has RGB pixels, which can effectively reduce the risk of false colors.
  • the resolution of the obtained second target image is further reduced, and the panchromatic pixels have a higher signal-to-noise ratio, and the frame rate of the image is high, thereby achieving lower power consumption and higher signal-to-noise ratio of the secondary pixel combination output.
  • Better image processing effect is achieved.
  • the multiple color pixels corresponding to the color filter are combined and read
  • the third pixel value and the fourth pixel value read out by combining the plurality of photosensitive pixels of the second color corresponding to the color filter to obtain a second combined image can combine the photosensitive pixels of the first color and the photosensitive pixel of the second color corresponding to each color filter
  • the photosensitive pixels of the second color and the photosensitive pixels of the third color are
  • a panchromatic combined image is obtained based on the fifth pixel values read out by combining a plurality of panchromatic pixels in the ninth diagonal direction in the first combined image, and the panchromatic pixels can be separated from the first combined image while subtracting Small image size to reduce power consumption.
  • the combined readout of the plurality of photosensitive pixels of the third color in the direction of the tenth diagonal are combined to obtain the fourth combined image, so that the size of the fourth combined image is the same as that of the third combined image
  • the dimensions of the image, pan-merged image remain the same.
  • the third combined image and the fourth combined image it is possible to mix and arrange different first color photosensitive pixels, second color photosensitive pixels and third color photosensitive pixels, so that in the generated second target image, RGB The pixel distribution is more even and the image quality is higher. Moreover, the resolution of the obtained second target image is further reduced, and the panchromatic pixels have a higher signal-to-noise ratio, and the frame rate of the image is high, thereby achieving lower power consumption and higher signal-to-noise ratio of the secondary pixel combination output. Better image processing effect.
  • FIGS. 10-14 may include multiple sub-operations or multiple stages. These sub-operations or stages are not necessarily performed at the same time, but may be performed at different times. These sub-operations or The execution order of the phases is not necessarily performed sequentially, but may be performed alternately or alternately with other operations or sub-operations of other operations or at least a part of phases.
  • Fig. 15 is a structural block diagram of an image generating device of an embodiment.
  • the image generating device 1500 is applied to an image sensor, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, the minimum repeating unit includes a plurality of filter groups, each filter The film group includes a color filter and a panchromatic filter. The amount of light transmitted by the panchromatic filter is greater than that of the color filter.
  • Both the color filter and the panchromatic filter include 4 sub-filters A light sheet, a plurality of filter groups including a first filter group and a second filter group, at least one color filter in the first filter group includes a first color sub-filter and a second color sub-filter Sub-filter; the color filter in the second filter group includes at least a third color sub-filter; the pixel array includes a plurality of pixels, the first color photosensitive pixel of the pixel array and the first color of the filter array The sub-filters are correspondingly arranged, the second color photosensitive pixels of the pixel array are correspondingly arranged with the second color sub-filters of the filter array, the third color photosensitive pixels of the pixel array are connected with the third color sub-filters of the filter array The light sheet is correspondingly arranged, and the pixel array is configured to receive light passing through the light filter array to generate an electrical signal;
  • the image generating device 1500 includes:
  • the first combining module 1502 is configured to combine the first pixel values read out according to the plurality of panchromatic pixels corresponding to the panchromatic filters in the filter set in the first resolution mode, and the color filter corresponding
  • the first combined image is obtained by combining the second pixel values read out from the plurality of photosensitive pixels of the first color and the third pixel values read out from the plurality of photosensitive pixels of the third color corresponding to the color filter.
  • the second combining module 1504 is configured to combine and read out the fourth pixel values according to the plurality of second color photosensitive pixels corresponding to the color filter to obtain a second combined image.
  • An image generating module 1506, configured to obtain a first target image based on the first combined image and the second combined image.
  • the first pixel values read out by combining the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, and the multiple panchromatic pixels corresponding to the color filters can integrate the panchromatic channel information into the image and improve the overall image quality.
  • the amount of incoming light makes it possible to generate the first merged image with more information and clearer detail analysis.
  • the second combined image is obtained according to the fourth pixel values read out by combining the plurality of second color photosensitive pixels corresponding to the color filter, and the first target image is obtained based on the first combined image and the second combined image, and the second combined image can be obtained.
  • the light-sensitive pixels of the first color in the combined image and the light-sensitive pixels of the second color in the second combined image are mixed and arranged, so that the color of the generated first target image is clearer.
  • pixel combination readout reduces the size of the generated first target image and consumes low power consumption for generating the image.
  • the image generation module 1506 is further configured to traverse the pixel values in the first merged image, and read from the second merged image when the traversed pixel value is the second pixel value.
  • the pixel values in the first merged image are traversed, and when the traversed pixel value is the second pixel value, the fourth pixel value at the same position as the second pixel value is read from the second merged image , and adjust the read fourth pixel value to be adjacent to the second pixel value, so that the photosensitive pixels of the first color and the photosensitive pixels of the second color are mixed and arranged, and the color resolution ability is improved, so that the generated first target image
  • photosensitive pixels of the first color, photosensitive pixels of the second color and photosensitive pixels of the third color in each row and column that is, each row and column of the target image has RGB pixels, which can effectively reduce the risk of false color.
  • the color filters of the second filter group further include a first color sub-filter or a second color sub-filter; the first combining module 1502 is also used for the first resolution mode Next, according to the first pixel value combined and read out from the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, according to the first pixel value corresponding to the multiple first color sub-filters in the first filter set
  • the combined readout of the second pixel value of a color photosensitive pixel is based on the combined readout of the third color photosensitive pixel corresponding to a plurality of third color sub-filters in the second filter group including the first color sub-filter.
  • the third pixel value and the fourth pixel value combined and read out according to the second color photosensitive pixels corresponding to the plurality of second color sub-filters in the second filter group are combined to obtain a first combined image;
  • the second combination module 1504 is also used to combine and read out the fourth pixel value according to the second color photosensitive pixels corresponding to the second color sub-filters in the first filter set, and according to the second color sub-filter in the second filter set Combine and read out the second pixel values of the first color photosensitive pixels corresponding to the plurality of first color sub-filters, according to the plurality of third color sub-filters in the second filter group containing the second color sub-filters The third color photosensitive pixels corresponding to the slice combine the read-out third pixel values to obtain a second combined image.
  • the color filter 234 of the second filter group 232 includes a third color sub-filter 2343, and also includes a first color sub-filter 2341 or a second color sub-filter 2342.
  • the first resolution mode according to the first pixel value combined and read out by multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, according to the multiple first color sub-filters in the first filter set Combine and read out the second pixel value of the photosensitive pixels of the first color corresponding to the light sheet, according to the photosensitivity of the third color corresponding to the multiple third color sub-filters in the second filter group containing the first color sub-filters
  • the third pixel value read out by pixel combination and the fourth pixel value combined and read out according to the second color light-sensitive pixels corresponding to the second color sub-filters in the second filter group, so that the combined readout obtained
  • the first merged image contains pixels of all colors, which can improve the resolution of color channels.
  • the third pixel value combined and read out by the multiple photosensitive pixels of the third color corresponding to the color pixel, and the value corresponding to the color filter Combining the read-out fourth pixel values of multiple photosensitive pixels of the second color to obtain a second combined image, so that the distribution of pixels of each color in the first target image generated according to the first combined image and the second combined image is more uniform and effective. Improve image clarity.
  • the device also includes a third merging module
  • the first combining module 1502 is also used to combine the first pixel values read out according to the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set in the second resolution mode, and the color filter Combining and reading out the second pixel values of the corresponding multiple photosensitive pixels of the first color, and combining and reading the third pixel values of the multiple photosensitive pixels of the third color corresponding to the color filter to obtain the first combined image; the second resolution The resolution corresponding to the first resolution mode is smaller than the resolution corresponding to the first resolution mode;
  • the second combining module 1504 is further configured to combine and read out the fourth pixel values according to the plurality of second color photosensitive pixels corresponding to the color filter to obtain a second combined image;
  • the third merging module is used to combine and read out the fifth pixel value of a plurality of panchromatic pixels in the ninth diagonal direction in the first combined image to obtain a panchromatic combined image; based on the tenth pair in the first combined image Combining and reading the sixth pixel value of a plurality of photosensitive pixels of the first color in the direction of the diagonal line, and combining and reading the seventh pixel value of a plurality of photosensitive pixels of the third color in the direction of the tenth diagonal line to obtain the third combined image, the ninth diagonal direction is different from the tenth diagonal direction;
  • the third merging module is further configured to obtain a fourth merging image based on merging and reading eighth pixel values of a plurality of light-sensitive pixels of the second color in the tenth diagonal direction in the second merging image;
  • the image generating module 1506 is further configured to obtain a second target image based on the panchromatic combined image, the third combined image and the fourth combined image.
  • the first pixel values read out are combined according to the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, and the multiple corresponding to the color filters
  • Combining the read-out fourth pixel values of the two photosensitive pixels of the second color to obtain a second combined image can separate the photosensitive pixels of the second color as a separate image and reduce the resolution of the image.
  • a panchromatic combined image is obtained based on the fifth pixel values read out by combining a plurality of panchromatic pixels in the ninth diagonal direction in the first combined image, and the panchromatic pixels can be separated from the first combined image while subtracting Small image size to reduce power consumption.
  • the obtained seventh pixel value is used to obtain a third merged image including the photosensitive pixels of the first color and the photosensitive pixels of the third color, and the all-in-one pixel readout method makes the generated image noise less.
  • the size of the fourth combined image is the same as that of the third combined image and the size of the full-color combined image be consistent.
  • the third merged image and the fourth merged image different photosensitive pixels of the first color and photosensitive pixels of the second color can be mixed and arranged, so that the distribution of RGB pixels in the generated second target image is more uniform. Higher quality.
  • the resolution of the obtained second target image is further reduced, and the panchromatic pixels have a higher signal-to-noise ratio, and the frame rate of the image is high, thereby achieving lower power consumption and higher signal-to-noise ratio of the secondary pixel combination output.
  • Better image processing effect is achieved.
  • the image generation module 1506 is further configured to traverse the pixels in the third merged image; when the traversed pixel value is the sixth pixel value or the seventh pixel value, from the full-color merged image Read the fifth pixel value corresponding to the traversed pixel value, and adjust the read fifth pixel value to be adjacent to the traversed pixel value; in the case of the traversed pixel value of the sixth pixel value, from In the fourth merged image, the eighth pixel value that is the same as the sixth pixel position is read, and the read eighth pixel value is adjusted to be adjacent to the sixth pixel value, until the full-color merged image, the third merged image and After all the pixel values in the fourth merged image are read, the second target image is obtained.
  • the pixels in the third merged image are traversed, and when the traversed pixel value is the sixth pixel value or the seventh pixel value, the pixel value corresponding to the traversed pixel value is read from the full-color merged image.
  • Five pixel values are adjusted, and the read fifth pixel value is adjusted to be adjacent to the traversed pixel value, so that panchromatic pixels are introduced into the image and the amount of light entering is increased.
  • the traversed pixel value is the sixth pixel value
  • the eighth pixel value that is the same as the sixth pixel position is read from the fourth combined image, and the read eighth pixel value is adjusted to be the same as the sixth pixel value.
  • the pixel values are adjacent, so that the photosensitive pixels of the first color and the photosensitive pixels of the second color are mixed and arranged, which improves the color resolution capability, so that each row and column of the generated second target image has photosensitive pixels of the first color and photosensitive pixels of the second color.
  • photosensitive pixels and second-color photosensitive pixels that is, each row and column of the target image has RGB pixels, which can effectively reduce the risk of false colors.
  • the color filters of the second filter group further include a first color sub-filter or a second color sub-filter; the first combining module 1502 is also used for the second resolution mode Next, according to the first pixel value combined and read out from the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, according to the first pixel value corresponding to the multiple first color sub-filters in the first filter set
  • the combined readout of the second pixel value of a color photosensitive pixel is based on the combined readout of the third color photosensitive pixel corresponding to a plurality of third color sub-filters in the second filter group including the first color sub-filter.
  • the third pixel value and the fourth pixel value combined and read out according to the second color photosensitive pixels corresponding to the plurality of second color sub-filters in the second filter group are combined to obtain a first combined image;
  • the second combination module 1504 is also used to combine and read out the fourth pixel value according to the second color photosensitive pixels corresponding to the second color sub-filters in the first filter set, and according to the second color sub-filter in the second filter set Combine and read out the second pixel values of the first color photosensitive pixels corresponding to the plurality of first color sub-filters, according to the plurality of third color sub-filters in the second filter group containing the second color sub-filters Combining and reading out the third pixel value of the third color photosensitive pixel corresponding to the slice to obtain the second combined image;
  • the third merging module is further configured to combine and read out the sixth pixel values of the multiple first-color light-sensitive pixels in the tenth diagonal direction in the first combined image, and the multiple sixth pixel values in the tenth diagonal direction.
  • the third merging module is further configured to combine and read out the sixth pixel values of the multiple photosensitive pixels of the first color in the direction of the tenth diagonal in the second combined image, and the multiple values in the direction of the tenth diagonal combining and reading out the seventh pixel value of the third-color photosensitive pixels and the combined and read-out eighth pixel value of the plurality of second-color photosensitive pixels in the tenth diagonal direction to obtain a fourth combined image.
  • the second resolution mode according to the first pixel value combined and read out by a plurality of panchromatic pixels corresponding to the panchromatic filters in the filter set, according to the number of panchromatic pixels in the first filter set
  • the second pixel value combined and read out from the first color photosensitive pixels corresponding to the first color sub-filters is combined and read out according to the combined readout value of the third color photosensitive pixels in the second filter group containing the first color sub-filters
  • the third pixel value, and the fourth pixel value combined and read out according to the third color photosensitive pixels corresponding to the plurality of second color sub-filters in the second filter group are combined to obtain the first combined image, and the first combined image will be obtained according to the first combined image.
  • the fourth pixel value read out by combining the second-color light-sensitive pixels corresponding to the multiple second-color sub-filters in the filter set is based on the fourth pixel value corresponding to the multiple first-color sub-filters in the second filter set.
  • the second pixel value read out by combination of photosensitive pixels of one color is combined and read out according to the combined readout value of the third color photosensitive pixels corresponding to multiple third color sub-filters in the second filter group including the second color sub-filters.
  • the third pixel value, to obtain the second merged image can separate the first color photosensitive pixel, the second color photosensitive pixel and the third color photosensitive pixel corresponding to each color filter to form two merged images, so that the two merged images
  • the photosensitive pixels of each color in are in the same position in the respective merged images.
  • a panchromatic combined image is obtained based on the fifth pixel values read out by combining a plurality of panchromatic pixels in the ninth diagonal direction in the first combined image, and the panchromatic pixels can be separated from the first combined image while subtracting Small image size to reduce power consumption.
  • the combined readout of the plurality of photosensitive pixels of the third color in the direction of the tenth diagonal are combined to obtain the fourth combined image, so that the size of the fourth combined image is the same as that of the third combined image
  • the dimensions of the image, pan-merged image remain the same.
  • the third combined image and the fourth combined image it is possible to mix and arrange different first color photosensitive pixels, second color photosensitive pixels and third color photosensitive pixels, so that in the generated second target image, RGB The pixel distribution is more even and the image quality is higher. Moreover, the resolution of the obtained second target image is further reduced, and the panchromatic pixels have a higher signal-to-noise ratio, and the frame rate of the image is high, thereby achieving lower power consumption and higher signal-to-noise ratio of the secondary pixel combination output. Better image processing effect.
  • the first merging module 1502 is further configured to combine the first pixel values read out according to the plurality of panchromatic pixels corresponding to the filter set in the second resolution mode to obtain a panchromatic image; Combining and reading out the second pixel value of a plurality of photosensitive pixels of the first color corresponding to the filter group, and combining and reading out the third pixel value of a plurality of photosensitive pixels of the third color corresponding to the filter group to obtain a fifth combined image ;
  • the second merging module 1504 is further configured to combine and read out the fourth pixel values according to the plurality of photosensitive pixels of the second color corresponding to the filter group to obtain a sixth combined image;
  • the image generation module 1506 is further configured to obtain a third target image based on the panchromatic image, the fifth combined image and the sixth combined image.
  • a panchromatic image in the second resolution mode, can be separated by combining the first pixel values read out by multiple panchromatic pixels corresponding to the filter set;
  • the second pixel value combined and read out from the photosensitive pixels of the first color, and the third pixel value read out from the multiple photosensitive pixels of the third color corresponding to the filter group can combine the photosensitive pixels of the first color and the photosensitive pixels of the third color Separated to form the fifth merged image, which can quickly and effectively reduce the size of the image.
  • the photosensitive pixels of the second color According to the fourth pixel value combined and read out by multiple photosensitive pixels of the second color corresponding to the filter group, the photosensitive pixels of the second color can be separated to form the sixth combined image, so that the photosensitive pixels of the second color can be combined in the sixth combined image.
  • the third target image is obtained based on the panchromatic image, the fifth merged image, and the sixth merged image, which can integrate panchromatic channel information into the image, improve the overall light input, and generate images with more information and clearer detail analysis
  • the third target image is obtained through combining and reading out multiple pixels corresponding to the filter set, the size of the generated third target image is reduced, and the power consumption required for generating the image is low.
  • the color filters of the second filter group further include a first color sub-filter or a second color sub-filter; the first combining module 1502 is also used to The second pixel value read out by combining the first color photosensitive pixels corresponding to the multiple first color sub-filters in the group, according to the multiple third color sub-filters in the second filter group containing the first color sub-filters The third pixel value combined and read out by the photosensitive pixels of the third color corresponding to the filter, and the fourth pixel value combined and read out according to the photosensitive pixels of the second color corresponding to the multiple second color sub-filters in the second filter group pixel value, to obtain the fifth merged image;
  • the second combination module 1504 is also used to combine and read out the fourth pixel value according to the second color photosensitive pixels corresponding to the second color sub-filters in the first filter set, and according to the second color sub-filter in the second filter set Combine and read out the second pixel values of the first color photosensitive pixels corresponding to the plurality of first color sub-filters, according to the plurality of third color sub-filters in the second filter group containing the second color sub-filters The third color photosensitive pixels corresponding to the slice combine the read-out third pixel values to obtain a sixth combined image.
  • the second pixel values read out by combining the first color photosensitive pixels corresponding to the multiple first color sub-filters in the first filter group, according to the The third pixel value read out by combining the third color photosensitive pixels corresponding to the multiple third color sub-filters in the second filter set of one color sub-filter, and according to the multiple The fourth pixel value read out by combining the second color photosensitive pixels corresponding to the second color sub-filter can separate part of the first color photosensitive pixels, the third color photosensitive pixels and the second color photosensitive pixels to form a fifth combined image , capable of reducing image size through pixel binning to reduce processing power.
  • the multiple first color sub-filters in the second filter set correspond to The second pixel value read out by combining the photosensitive pixels of the first color, according to the combined reading of the photosensitive pixels of the third color corresponding to the multiple third color sub-filters in the second filter group containing the second color sub-filters
  • the obtained third pixel value can separate part of the photosensitive pixels of the first color, the photosensitive pixels of the third color and the photosensitive pixels of the second color to form the sixth combined image, so that the photosensitive pixels of the second color can be combined in the sixth combined image and The coordinate positions of the photosensitive pixels of the first color in the fifth merged image are consistent.
  • the third target image is obtained based on the panchromatic image, the fifth merged image, and the sixth merged image, which can integrate panchromatic channel information into the image, improve the overall light input, and generate images with more information and clearer detail analysis
  • the third target image is obtained through combining and reading out multiple pixels corresponding to the filter set, the size of the generated third target image is reduced, and the power consumption required for generating the image is low.
  • each module in the above image generating device is only for illustration. In other embodiments, the image generating device can be divided into different modules as required to complete all or part of the functions of the above image generating device.
  • Fig. 16 is a schematic diagram of the internal structure of an electronic device in one embodiment.
  • the electronic device includes a processor and a memory connected through a system bus.
  • the processor is used to provide computing and control capabilities to support the operation of the entire electronic device.
  • the memory may include non-volatile storage media and internal memory. Nonvolatile storage media store operating systems and computer programs.
  • the computer program can be executed by a processor to implement an image generation method provided in the following embodiments.
  • the internal memory provides a high-speed running environment for the operating system computer program in the non-volatile storage medium.
  • the electronic device may be a mobile phone, a tablet computer, a personal digital assistant, or a wearable device.
  • each module in the image generation device provided in the embodiment of the present application may be in the form of a computer program.
  • the computer program can run on a terminal or a server.
  • the program modules constituted by the computer program can be stored in the memory of the terminal or server.
  • the computer program is executed by the processor, the operations of the methods described in the embodiments of the present application are realized.
  • the embodiment of the present application also provides a computer-readable storage medium.
  • One or more non-transitory computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform the operations of the image generation method.
  • a computer program product comprising instructions which, when run on a computer, cause the computer to perform an image generation method.
  • Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory can include random access memory (RAM), which acts as external cache memory.
  • RAM is available in many forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchronous Link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).
  • SRAM Static RAM
  • DRAM Dynamic RAM
  • SDRAM Synchronous DRAM
  • DDR SDRAM Double Data Rate SDRAM
  • ESDRAM Enhanced SDRAM
  • SLDRAM Synchronous Link (Synchlink) DRAM
  • SLDRAM Synchronous Link (Synchlink) DRAM
  • Rambus direct RAM
  • DRAM direct memory bus dynamic RAM
  • RDRAM memory bus dynamic RAM

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Color Television Image Signal Generators (AREA)

Abstract

一种图像传感器21，包括滤光片阵列23和像素阵列24，滤光片阵列23包括最小重复单元230，最小重复单元230包括多个滤光片组，每个滤光片组包括全色滤光片233和彩色滤光片234，全色滤光片233透过的进光量大于彩色滤光片234透过的进光量，彩色滤光片234和全色滤光片233均包括4个子滤光片，多个滤光片组至少包括第一滤光片组231，第一滤光片组231中至少一个彩色滤光片234中包括第一颜色子滤光片2341和第二颜色子滤光片2342；像素阵列24包括多个像素，像素阵列24的像素与滤光片阵列23的子滤光片对应设置，像素阵列24被配置成用于接收穿过滤光片阵列23的光线以生成电信号。

Description

图像传感器、摄像模组、电子设备、图像生成方法和装置
相关申请的交叉引用
本申请要求于2021年11月12日提交中国专利局、申请号为2021113399842、发明名称为“图像传感器、摄像模组、电子设备、图像生成方法和装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及影像技术领域，特别是涉及一种图像传感器、摄像模组、电子设备、图像生成方法、装置、计算机可读存储介质和计算机程序产品。
背景技术
随着计算机技术的发展，手机等电子设备中大部分都配置有摄像头，以通过摄像头实现拍照功能。摄像头中设置有图像传感器，通过图像传感器采集彩色图像。为了实现彩色图像的采集，图像传感器中通常会设置以拜耳(Bayer)阵列形式排布的滤光片阵列，以使得图像传感器中的多个像素能够接收穿过对应的滤光片的光线，从而生成具有不同色彩通道的像素信号，进而生成图像。
然而，传统的图像传感器所生成的图像清晰度较低。
发明内容
本申请实施例提供一种图像传感器、图像生成方法、装置、电子设备、计算机可读存储介质，可以提高成像的清晰度。
一种图像传感器，所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，所述最小重复单元包括多个滤光片组，每个滤光片组包括彩色滤光片和全色滤光片，所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量，所述彩色滤光片和所述全色滤光片均包括4个子滤光片，多个滤光片组至少包括第一滤光片组，第一滤光片组中至少一个所述彩色滤光片中包括第一颜色子滤光片和第二颜色子滤光片；所述像素阵列包括多个像素，所述像素阵列的像素与所述滤光片阵列的子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号。
一种摄像模组，所述摄像模组包括镜头和图像传感器；所述图像传感器用于接收穿过所述镜头的光线，所述像素根据所述光线生成电信号；所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，所述最小重复单元包括多个滤光片组，每个滤光片组包括彩色滤光片和全色滤光片，所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量，所述彩色滤光片和所述全色滤光片均包括4个子滤光片，多个滤光片组至少包括第一滤光片组，第一滤光片组中至少一个所述彩色滤光片中包括第一颜色子滤光片和第二颜色子滤光片；所述像素阵列包括多个像素，所述像素阵列的像素与所述滤光片阵列的子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号。
一种电子设备，所述电子设备包括摄像模组及壳体，所述摄像模组设置在所述壳体上；所述摄像模组包括镜头和图像传感器；所述图像传感器用于接收穿过所述镜头的光线，所述像素根据所述光线生成电信号；所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，所述最小重复单元包括多个滤光片组，每个滤光片组包括彩色滤光片和全色滤光片，所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量，所述彩色滤光片和所述全色滤光片均包括4个子滤光片，多个滤光片组至少包括第一滤光片组，第一滤光片组中至少一个所述彩色滤光片中包括第一颜色子滤光片和第二颜色子滤光片；所述像素阵列包括多个像素，所述像素阵列的像素与所述滤光片阵列的子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号。
上述图像传感器包括滤光片阵列和像素阵列，滤光片阵列包括最小重复单元，最小重复单元包括多个滤光片组，滤光片组包括全色滤光片和彩色滤光片，全色滤光片透过的进光量大于彩色滤光片透过的进光量，可在拍摄时可获取到更多的光量，从而无需调节拍摄参数，在不影响拍摄的稳定性的情况下，提高暗光下的成像的清晰度。暗光下成像时，可兼顾稳定性和清晰度，暗光下成像的稳定性和清晰度均较高。并且，每个全色滤光片中包括4个子滤光片，每个彩色滤光片中包括4个子滤光片，多个滤光片 组至少包括第一滤光片组，第一滤光片组中至少一个彩色滤光片中包括第一颜色子滤光片和第二颜色子滤光片，像素阵列包括多个全色像素和多个彩色像素，每个全色像素对应全色滤光片的一个子滤光片，每个第一颜色感光像素对应彩色滤光片的一个第一颜色子滤光片，每个第二颜色感光像素对应彩色滤光片的一个第二颜色子滤光片，能够将不同的颜色感光像素在同一个彩色滤光片中混合排列，有效提高彩色通道的分辨能力，降低所生成的图像中存在伪色的问题。
一种图像生成方法，应用于图像传感器，所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，所述最小重复单元包括多个滤光片组，每个滤光片组包括彩色滤光片和全色滤光片，所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量，所述彩色滤光片和所述全色滤光片均包括4个子滤光片，多个滤光片组包括第一滤光片组和第二滤光片组，第一滤光片组中至少一个所述彩色滤光片中包括第一颜色子滤光片和第二颜色子滤光片；第二滤光片组中彩色滤光片至少包括第三颜色子滤光片；所述像素阵列包括多个像素，所述像素阵列的第一颜色感光像素与所述滤光片阵列的第一颜色子滤光片对应设置，像素阵列的第二颜色感光像素与滤光片阵列的第二颜色子滤光片对应设置，像素阵列的第三颜色感光像素与滤光片阵列的第三颜色子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号；
所述方法包括：
在第一分辨率模式下，根据所述滤光片组中的所述全色滤光片对应的多个全色像素合并读出的第一像素值，以及所述彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及所述彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像；
根据所述彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像；
基于所述第一合并图像和所述第二合并图像，得到第一目标图像。
一种图像生成装置，应用于图像传感器，所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，最小重复单元包括多个滤光片组，所述每个滤光片组包括彩色滤光片和全色滤光片，所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量，所述彩色滤光片和所述全色滤光片均包括4个子滤光片，多个滤光片组包括第一滤光片组和第二滤光片组，第一滤光片组中至少一个所述彩色滤光片中包括第一颜色子滤光片和第二颜色子滤光片；第二滤光片组中彩色滤光片至少包括第三颜色子滤光片；所述像素阵列包括多个像素，所述像素阵列的第一颜色感光像素与所述滤光片阵列的第一颜色子滤光片对应设置，像素阵列的第二颜色感光像素与滤光片阵列的第二颜色子滤光片对应设置，像素阵列的第三颜色感光像素与滤光片阵列的第三颜色子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号；
所述装置包括：
第一合并模块，用于在第一分辨率模式下，根据所述滤光片组中的所述全色滤光片对应的多个全色像素合并读出的第一像素值，以及所述彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及所述彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像；
第二合并模块，用于根据所述彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像；
图像生成模块，用于基于所述第一合并图像和所述第二合并图像，得到第一目标图像。
一种电子设备，包括存储器、处理器及图像传感器，所述存储器中储存有计算机程序，所述计算机程序被所述处理器执行时，使得所述处理器执行如下操作：
在第一分辨率模式下，根据所述滤光片组中的所述全色滤光片对应的多个全色像素合并读出的第一像素值，以及所述彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及所述彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像；根据所述彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像；基于所述第一合并图像和所述第二合并图像，得到第一目标图像。
一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现如下操作：
在第一分辨率模式下，根据所述滤光片组中的所述全色滤光片对应的多个全色像素合并读出的第一 像素值，以及所述彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及所述彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像；根据所述彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像；基于所述第一合并图像和所述第二合并图像，得到第一目标图像。
一种计算机程序产品，包括计算机程序，该计算机程序被处理器执行时实现如下操作：
在第一分辨率模式下，根据所述滤光片组中的所述全色滤光片对应的多个全色像素合并读出的第一像素值，以及所述彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及所述彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像；根据所述彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像；基于所述第一合并图像和所述第二合并图像，得到第一目标图像。
上述图像生成方法、装置、电子设备、计算机可读存储介质和计算机程序产品，在第一分辨率模式下，通过滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，以及彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，能够将全色通道信息融合到图像中，提高整体的进光量，使得能够生成信息更多、细节解析更清晰的第一合并图像。根据彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，能够将第二颜色感光像素分离出来形成第二合并图像，使得第二颜色感光像素在第二合并图像和第一颜色感光像素在第一合并图像中的位置保持一致。基于第一合并图像和第二合并图像，得到第一目标图像，能够将第一合并图像中的第一颜色感光像素和第二合并图像中的第二颜色感光像素混合排列，使得所生成的第一目标图像色彩更清晰。并且像素合并读出使得生成的第一目标图像尺寸减小、生成图像所需消耗的功耗低。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他实施例的附图。
图1为一个实施例中电子设备的结构示意图。
图2为一个实施例中图像传感器的分解示意图。
图3为一个实施例中像素阵列和读出电路的连接示意图。
图4a为一个实施例中第一对角线方向和第二对角线方向的示意图。
图4b为一个实施例中第三对角线方向和第四对角线方向的示意图。
图5为一个实施例中最小重复单元的排布方式示意图。
图6为其中一个实施例中最小重复单元的排布方式示意图。
图7为另一个实施例中最小重复单元的排布方式示意图。
图8为另一个实施例中最小重复单元的排布方式示意图。
图9为一个实施例中图像生成方法的流程图。
图10为一个实施例中在第一分辨率模式下得到第一目标图像的示意图。
图11为另一个实施例中在第一分辨率模式下得到第一目标图像的示意图。
图12为一个实施例中在第二分辨率模式下得到第二目标图像的示意图。
图13为另一个实施例中在第二分辨率模式下得到第二目标图像的示意图。
图14为另一个实施例中在第二分辨率模式下的图像生成方法的示意图。
图15为一个实施例中图像生成装置的结构框图。
图16为一个实施例中电子设备的内部结构框图。
具体实施方式
为了便于理解本申请，下面将参照相关附图对本申请进行更全面的描述。附图中给出了本申请的较佳实施例。但是，本申请可以以许多不同的形式来实现，并不限于本文所描述的实施例。相反地，提供这些实施例的目的是使对本申请的公开内容的理解更加透彻全面。
可以理解，本申请所使用的术语“第一”、“第二”等可在本文中用于描述各种元件，但这些元件不受这些术语限制。这些术语仅用于将第一个元件与另一个元件区分。举例来说，在不脱离本申请的范围的情况下，可以将第一目标图像称为第二目标图像，且类似地，可将第二目标图像称为第一目标图像。第一和第二两者都是目标图像，但其不是同一目标图像。多个指至少两个，例如2个、4个，但不限于此。
在一个实施例中，电子设备100包括手机、平板电脑、笔记本电脑、柜员机、闸机、智能手表、头显设备等，可以理解，电子设备100还可以是其他任意图像处理功能的装置。电子设备100包括摄像模组20、处理器30和壳体40。摄像模组20和处理器30均设置在壳体40内，壳体40还可用于安装电子设备100的供电装置、通信装置等功能模块，以使壳体40为功能模块提供防尘、防摔、防水等保护。
摄像模组20可以是前置摄像模组、后置摄像模组、侧置摄像模组、屏下摄像模组等，在此不做限制。摄像模组20包括镜头及图像传感器21，摄像模组20在拍摄图像时，光线穿过镜头并到达图像传感器21，图像传感器21用于将照射到图像传感器21上的光信号转化为电信号。
在一个实施例中，如图2所示，图像传感器21包括微透镜阵列22、滤光片阵列23、像素阵列24。
微透镜阵列22包括多个微透镜221，微透镜221、滤光片阵列23中的子滤光片和像素阵列24中的像素一一对应设置，微透镜221用于将入射的光线进行聚集，聚集之后的光线会穿过对应的子滤光片，然后投射至像素上，被对应的像素接收，像素再将接收的光线转化成电信号。
滤光片阵列23包括多个最小重复单元230。最小重复单元230可包括多个滤光片组。每个滤光片组包括全色滤光片233和彩色滤光片234，全色滤光片233透过的进光量大于彩色滤光片234透过的进光量。每个全色滤光片233中包括4个子滤光片2331，每个彩色滤光片234中包括4个子滤光片，多个滤光片组至少包括第一滤光片组231，第一滤光片组231中至少一个彩色滤光片234中包括第一颜色子滤光片2341和第二颜色子滤光片2342。在不同的滤光片组中还包括有不同的彩色滤光片234。
在一个实施例中，多个滤光片组还包括第二滤光片组232，第二滤光片组232中的彩色滤光片234至少包括第三颜色子滤光片2343。
最小重复单元230中滤光片组的彩色滤光片234的透过的光线的波段对应的颜色包括颜色a、颜色b和/或颜色c。滤光片组232的彩色滤光片234的透过的光线的波段对应的颜色包括颜色a、颜色b和颜色c、或者颜色a、颜色b或颜色c、或者颜色a和颜色b、或者颜色b和颜色c、或者颜色a和颜色c。其中，颜色a为红色，颜色b为绿色，颜色c为蓝色，或者例如颜色a为品红色，颜色b为青色，颜色c为黄色等，在此不做限制。
在一个实施例中，彩色滤光片234的透过的光线的波段的宽度小于全色滤光片233透过的光线的波段的宽度，例如，彩色滤光片234的透过的光线的波段可对应红光的波段、绿光的波段、或蓝光的波段，全色滤光片233透过的光线的波段为所有可见光的波段，也即是说，彩色滤光片234仅允许特定颜色光线透光，而全色滤光片233可通过所有颜色的光线。当然，彩色滤光片234的透过的光线的波段还可对应其他色光的波段，如品红色光、紫色光、青色光、黄色光等，在此不作限制。
在一个实施例中，滤光片组中彩色滤光片234的数量和全色滤光片233的数量的比例可以是1:3、1:1或3:1。例如，彩色滤光片234的数量和全色滤光片233的数量的比例为1:3，则彩色滤光片234的数量为1，全色滤光片233的数量为3，此时全色滤光片233数量较多，在暗光下的成像质量更好；或者，彩色滤光片234的数量和全色滤光片233的数量的比例为1:1，则彩色滤光片234的数量为2，全色滤光片233的数量为2，此时既可以获得较好的色彩表现的同时，暗光下的成像质量也较好；或者，彩色滤光片234的数量和全色滤光片233的数量的比例为3:1，则彩色滤光片234的数量为3，全色滤光片233的数量为1，此时可获得更好的色彩表现，且能提高暗光下的成像质量。
像素阵列24包括多个像素，像素阵列24的像素与滤光片阵列23的子滤光片对应设置。像素阵列24被配置成用于接收穿过滤光片阵列23的光线以生成电信号。
其中，像素阵列24被配置成用于接收穿过滤光片阵列23的光线以生成电信号，是指像素阵列24用于对穿过滤光片阵列23的被摄对象的给定集合的场景的光线进行光电转换，以生成电信号。被摄对象的给定集合的场景的光线用于生成图像数据。例如，被摄对象是建筑物，被摄对象的给定集合的场景 是指该建筑物所处的场景，该场景中还可以包含其他对象。
在一个实施例中，像素阵列24包括多个最小重复单元240，最小重复单元240包括多个像素组，多个像素组包括全色像素组243和彩色像素组244。每个全色像素组243中包括4个全色像素2431，每个彩色像素组244中包括4个彩色像素，多个像素组至少包括第一像素组241，第一像素组241中的至少一个彩色像素组244中包括第一颜色感光像素2441和第二颜色感光像素2442。每个全色像素2431对应全色滤光片233中的一个子滤光片2331，全色像素2431接收穿过对应的子滤光片2331的光线以生成电信号。每个第一颜色感光像素2441对应彩色滤光片234的一个第一颜色子滤光片2341，第一颜色感光像素2441接收穿过对应的第一颜色子滤光片2341的光线以生成电信号。每个第二颜色感光像素2442对应彩色滤光片234的一个第二颜色子滤光片2342，第二颜色感光像素2442接收穿过对应的第二颜色子滤光片2342的光线以生成电信号。
在一个实施例中，多个像素组还包括第二像素组242，第二像素组242中的彩色像素组244至少包括第三颜色感光像素2443。每个第三颜色感光像素2443对应彩色滤光片234的一个第三颜色子滤光片2343，第三颜色感光像素2443接收穿过对应的第三颜色子滤光片2343的光线以生成电信号。
本实施中的图像传感器21包括滤光片阵列23和像素阵列24，滤光片阵列23包括最小重复单元230，最小重复单元230包括多个滤光片组，每个滤光片组包括全色滤光片233和彩色滤光片234，彩色滤光片234具有比全色滤光片233的更窄的光谱响应，全色滤光片233透过的进光量大于彩色滤光片234透过的进光量，可在拍摄时可获取到更多的光量，从而无需调节拍摄参数，在不影响拍摄的稳定性的情况下，提高暗光下的成像的清晰度。暗光下成像时，可兼顾稳定性和清晰度，暗光下成像的稳定性和清晰度均较高。并且，每个全色滤光片233中包括4个子滤光片2331，每个彩色滤光片234中包括4个子滤光片，多个滤光片组至少包括第一滤光片组231，第一滤光片组231中至少一个彩色滤光片234中包括第一颜色子滤光片2341和第二颜色子滤光片2342，像素阵列24包括多个全色像素2431和多个彩色像素2441，每个全色像素2431对应全色滤光片233的一个子滤光片2331，每个第一颜色感光像素2441对应彩色滤光片234的一个第一颜色子滤光片2341，每个第二颜色感光像素2442对应彩色滤光片234的一个第二颜色子滤光片2342，能够将不同的颜色感光像素在同一个彩色滤光片中混合排列，有效提高彩色通道的分辨能力，降低所生成的图像中存在伪色的问题。
在一个实施例中，如图2所示，滤光片阵列23中的最小重复单元230包括4个滤光片组，并且4个滤光片组包括2个第一滤光片组231和2个第二滤光片组232，2个第一滤光片组231和2个第二滤光片组232呈矩阵排列。
每个第一滤光片组231包括全色滤光片233和彩色滤光片234，每个全色滤光片233和每个彩色滤光片234均有4个子滤光片，且至少一个彩色滤光片234中包括第一颜色子滤光片2341和第二颜色子滤光片2342，则该第二滤光片组232共包括16个子滤光片。
每个第二滤光片组232包括全色滤光片233和彩色滤光片234，每个全色滤光片233和每个彩色滤光片234均有4个子滤光片，且彩色滤光片234中至少包括第三颜色子滤光片2343，则该第二滤光片组232共包括16个子滤光片。
同样的，像素阵列24包括多个最小重复单元240，与滤光片阵列23的多个最小重复单元230一一对应。每个最小重复单元240包括4个像素组，并且4个像素组包括2个第一像素组241和2个第二像素组242，2个第一像素组241和2个第二像素组242呈矩阵排列。每个第一像素组241对应一个第一滤光片组231，每个第二像素组242对应一个第二滤光片组232。
如图3所示，读出电路25与像素阵列24电连接，用于控制像素阵列24的曝光以及像素点的像素值的读取和输出。读出电路25包括垂直驱动单元251、控制单元252、列处理单元253和水平驱动单元254。垂直驱动单元251包括移位寄存器和地址译码器。垂直驱动单元251包括读出扫描和复位扫描功能。控制单元252根据操作模式配置时序信号，利用多种时序信号来控制垂直驱动单元251、列处理单元253和水平驱动单元254协同工作。列处理单元253可以具有用于将模拟像素信号转换为数字格式的模数(A/D)转换功能。水平驱动单元254包括移位寄存器和地址译码器。水平驱动单元254顺序逐列扫描像素阵列24。
在一个实施例中，如图4a所示，最小重复单元230中第一滤光片组231设置在第一对角线D1方向，第二滤光片组232设置在第二对角线D2方向，第一对角线D1方向与第二对角线D2方向不同。第一对角线D1方向和第二对角线D2方向不同，能够兼顾色彩表现和暗光成像质量。
第一对角线D1方向与第二对角线D2方向不同，具体可以是第一对角线D1方向与第二对角线D2方向不平行，或者，第一对角线D1方向与第二对角线D2方向垂直等。
在一个实施例中，如图4b所示，每个滤光片组均包括彩色滤光片234和全色滤光片233，每个滤光片组的彩色滤光片234和全色滤光片233呈矩阵排列。滤光片组中的各个全色滤光片233设置在第三对角线方向D3，滤光片组中的各个彩色滤光片234设置在第四对角线D4方向。第三对角线D3方向和第四对角线D4方向不同，能够兼顾色彩表现和暗光成像质量。
第三对角线D3方向与第四对角线D4方向不同，具体可以是第三对角线D3方向与第四对角线D4方向不平行，或者，第三对角线D3方向与第四对角线D4方向垂直等。
在其他实施方式中，一个彩色滤光片234和一个全色滤光片233可位于第三对角线D3，另一个彩色滤光片234和另一个全色滤光片233可位于第四对角线D4。
在一个实施例中，彩色滤光片234可设置在与第三对角线D3相平行的方向上，或彩色滤光片234可设置在与第四对角线D4相平行的方向上。
在一个实施例中，第一滤光片组231中至少一个彩色滤光片234中包括第一颜色子滤光片2341和第二颜色子滤光片2342。第一颜色子滤光片2341设置在第五对角线方向或第六对角线方向的一个方向上，第二颜色子滤光片2342设置在第五对角线方向或第六对角线方向的另一个方向上。第一滤光片组231中全色滤光片233的子滤光片2331置在第五对角线方向和第六对角线方向上。
在一个实施例中，最小重复单元230包括2个第一滤光片组231和2个第二滤光片组232，2个第一滤光片组231和2个第二滤光片组232呈矩阵排列。
2个第一滤光片组231中，其中一个第一滤光片组231的彩色滤光片234中的第一颜色子滤光片2341设置在第五对角线方向上，第二颜色子滤光片2342设置在第六对角线方向上；并且，另一个第一滤光片组231的彩色滤光片234中的第一颜色子滤光片2341设置在第六对角线方向上，第二颜色子滤光片2342设置在第五对角线方向上。
可以理解的是，第一颜色子滤光片2341设置在第五对角线方向，可以是第一颜色子滤光片2341处于在第五对角线上，也可以是第一颜色子滤光片2341处于与第五对角线方向相平行的方向上，其他对角线方向类似。
在一个实施例中，如图5所示，多个滤光片组包括第一滤光片组231和第二滤光片组232，第一滤光片组231中至少一个彩色滤光片234中包括第一颜色子滤光片2341和第二颜色子滤光片2342。第二滤光片组232中彩色滤光片234至少包括第三颜色子滤光片2343。第二滤光片组232中的第三颜色子滤光片2343设置在第七对角线D7方向和第八对角线D8方向上，第七对角线D7方向与第八对角线D8方向不同。
第七对角线D7方向与第八对角线D8方向不同，具体可以是第七对角线D7方向与第八对角线D8方向不平行，或者，第七对角线D7方向与第八对角线D8方向垂直等。
在一个实施例中，如图5所示，滤光片阵列23中的最小重复单元230包括4个滤光片组，并且4个滤光片组呈矩阵排列。4个滤光片组中包括第一滤光片组231和第二滤光片组232。每个第一滤光片组231中包含2个全色滤光片233和2个彩色滤光片234，每个第二滤光片组232中包含2个全色滤光片233和2个彩色滤光片234。全色滤光片233中包括4个子滤光片2331，第一滤光片组231的同一彩色滤光片234中包括2个第一颜色子滤光片2341和2个第二颜色子滤光片2342，第二滤光片组232的同一彩色滤光片234中包括4个第三颜色子滤光片2343，则最小重复单元为8行8列64个子滤光片，排布方式为：
w w a c w w b b
w w c a w w b b
a c w w b b w w
c a w w b b w w
w w b b w w c a
w w b b w w a c
b b w w c a w w
b b w w a c w w
其中，w表示全色子滤光片2331，a、b和c均表示彩色子滤光片。彩色子滤光片包括第一颜色子滤光片2341、第二颜色子滤光片2342和第三颜色子滤光片2343。
a、b和c分别表示第一颜色子滤光片2341、第二颜色子滤光片2342和第三颜色子滤光片2343中的一种。全色子滤光片2331指的是可滤除可见光波段之外的所有光线的子滤光片，第一颜色子滤光片2341、第二颜色子滤光片2342和第三颜色子滤光片2343可为红色子滤光片、绿色子滤光片、蓝色子滤光片、品红色子滤光片、青色子滤光片和黄色子滤光片。红色子滤光片为滤除红光之外的所有光线的子滤光片，绿色子滤光片为滤除绿光之外的所有光线的子滤光片，蓝色子滤光片为滤除蓝光之外的所有光线的子滤光片，品红色子滤光片为滤除品红色光之外的所有光线的子滤光片，青色色子滤光片为滤除青光之外的所有光线的子滤光片，黄色子滤光片为滤除黄光之外的所有光线的子滤光片。
a可以是红色子滤光片、绿色子滤光片、蓝色子滤光片、品红色子滤光片、青色子滤光片或黄色子滤光片，b可以是红色子滤光片、绿色子滤光片、蓝色子滤光片、品红色子滤光片、青色子滤光片或黄色子滤光片，c可以是红色子滤光片、绿色子滤光片、蓝色子滤光片、品红色子滤光片、青色子滤光片或黄色子滤光片。例如，b为红色子滤光片、a为绿色子滤光片、c为蓝色子滤光片；或者，c为红色子滤光片、a为绿色子滤光片、b为蓝色子滤光片；再例如，c为红色子滤光片、a为绿色子滤光片、b为蓝色子滤光片；或者，a为红色子滤光片、b为蓝色子滤光片、c为绿色子滤光片等，在此不作限制；再例如，b为品红色子滤光片、a为青色子滤光片、b为黄色子滤光片等。在其他实施方式中，彩色滤光片还可包括其他颜色的子滤光片，如橙色子滤光片、紫色子滤光片等，在此不作限制。
在一个实施例中，如图6所示，滤光片阵列23中的最小重复单元230包括4个滤光片组，并且4个滤光片组呈矩阵排列。4个滤光片组中包括第一滤光片组231和第二滤光片组232。每个第一滤光片组231中包含2个全色滤光片233和2个彩色滤光片234，每个第二滤光片组232中包含2个全色滤光片233和2个彩色滤光片234。全色滤光片233中包括4个子滤光片2331，第一滤光片组231的同一彩色滤光片234中包括2个第一颜色子滤光片2341和2个第二颜色子滤光片2342，第二滤光片组232的同一彩色滤光片234中包括4个第三颜色子滤光片2343，则最小重复单元为8行8列64个子滤光片，排布方式为：
b b w w a c w w
b b w w c a w w
w w b b w w a c
w w b b w w c a
c a w w b b w w
a c w w b b w w
w w c a w w b b
w w a c w w b b
其中，w表示全色子滤光片2331，a、b和c均表示彩色子滤光片。彩色子滤光片包括第一颜色子滤光片2341、第二颜色子滤光片2342和第三颜色子滤光片2343。
本实施例中，将不同颜色的感光像素混合排列，如将红色像素和蓝色像素混合排列，使得最小重复单元的每行每列均有RGB像素的存在，能够有效提高彩色通道的分辨能力，降低生成图像存在伪色的问题。并且，引入了全色像素，有效提高进光量，从而能够提高图像的清晰度。同时，本实施例中的图像传感器还具备二级像素合并读取的优势，图像传感器的最小重复单元的每行每列均有RGB像素的存在，无需跨越不同的最小重复单元进行相同像素的像素合并，能够有效降低处理的功耗。
在一个实施例中，如图7所示，多个滤光片组包括第一滤光片组231和第二滤光片组232，第一滤 光片组231中至少一个彩色滤光片234中包括第一颜色子滤光片2341和第二颜色子滤光片2342。第二滤光片组232中彩色滤光片234包括第三颜色子滤光片2343，且第二滤光片组232中彩色滤光片234还包括第一颜色子滤光片2341或第二颜色子滤光片2342。
具体地，第二滤光片组232中的同一彩色滤光片234包括第三颜色子滤光片2343和第一颜色子滤光片2341，或者第二滤光片组232中的同一彩色滤光片234包括第三颜色子滤光片2343和第二颜色子滤光片2342。
在一个实施例中，如图7所示，第二滤光片组232包括2个彩色滤光片234，每个彩色滤光片234中的同一颜色的子滤光片呈对角排布，同一颜色的子滤光片在两个彩色滤光片234中的排布方向相反。
具体地，第二滤光片组232包括两个彩色滤光片234，同一彩色滤光片234包括第三颜色子滤光片2343和第一颜色子滤光片2341。同一彩色滤光片234中的第三颜色子滤光片2343呈对角排布，同一彩色滤光片234中的第一颜色子滤光片2341呈对角排布。第一颜色子滤光片2341在2个彩色滤光片234中的排布方向相反，第三颜色子滤光片2343在2个彩色滤光片234中的排布方向相反。例如，第三颜色子滤光片2343在2个彩色滤光片234中分别设置在对角线A1方向和对角线A2方向上。
或者，第二滤光片组232包括两个彩色滤光片234，同一彩色滤光片234包括第三颜色子滤光片2343和第二颜色子滤光片2342。同一彩色滤光片234中的第三颜色子滤光片2343呈对角排布，同一彩色滤光片234中的第二颜色子滤光片2342呈对角排布。第二颜色子滤光片2342在2个彩色滤光片234中的排布方向相反，第三颜色子滤光片2343在2个彩色滤光片234中的排布方向相反。
在一个实施例中，如图7所示，滤光片阵列23中的最小重复单元230包括4个滤光片组，并且4个滤光片组呈矩阵排列。每个滤光片组中包含2个全色滤光片233和2个彩色滤光片234。全色滤光片233中包括4个子滤光片2331，第一滤光片组231的同一彩色滤光片234中包括2个第一颜色子滤光片2341和2个第二颜色子滤光片2342，存在第二滤光片组232的同一彩色滤光片234中包括2个第一颜色子滤光片2341和2个第三颜色子滤光片2343，以及存在第二滤光片组232的同一彩色滤光片234中包括2个第二颜色子滤光片2342和2个第三颜色子滤光片2343，则最小重复单元为8行8列64个子滤光片，排布方式为：
w w a c w w c b
w w c a w w b c
a c w w b c w w
c a w w c b w w
w w b a w w c a
w w a b w w a c
a b w w c a w w
b a w w a c w w
其中，w表示全色子滤光片2331，a、b和c均表示彩色子滤光片。彩色子滤光片包括第一颜色子滤光片2341、第二颜色子滤光片2342和第三颜色子滤光片2343。
在一个实施例中，如图8所示，滤光片阵列23中的最小重复单元230包括4个滤光片组，并且4个滤光片组呈矩阵排列。每个滤光片组中包含2个全色滤光片233和2个彩色滤光片234。全色滤光片233中包括4个子滤光片2331，第一滤光片组231的同一彩色滤光片234中包括2个第一颜色子滤光片2341和2个第二颜色子滤光片2342，存在第二滤光片组232的同一彩色滤光片234中包括2个第一颜色子滤光片2341和2个第三颜色子滤光片2343，存在第二滤光片组232的同一彩色滤光片234中包括2个第二颜色子滤光片2342和2个第三颜色子滤光片2343，则最小重复单元为8行8列64个子滤光片，排布方式为：
b c w w a c w w
c b w w c a w w
w w c b w w a c
w w b c w w c a
c a w w a b w w
a c w w b a w w
w w c a w w b a
w w a c w w a b
其中，w表示全色子滤光片2331，a、b和c均表示彩色子滤光片。彩色子滤光片包括第一颜色子滤光片2341、第二颜色子滤光片2342和第三颜色子滤光片2343。
在一个实施例中，提供了一种图像生成方法，应用于如图2所示的图像传感器21，图像传感器21包括滤光片阵列23和像素阵列24，滤光片阵列23包括最小重复单元230，最小重复单元230包括多个滤光片组，每个滤光片组包括彩色滤光片234和全色滤光片233，全色滤光片233透过的进光量大于彩色滤光片234透过的进光量，彩色滤光片234具有比全色滤光片233的更窄的光谱响应，每个全色滤光片233中包括4个子滤光片2331，每个彩色滤光片234中包括4个子滤光片，多个滤光片组包括第一滤光片组231和第二滤光片组232，第一滤光片组231中至少一个彩色滤光片234中包括第一颜色子滤光片2341和第二颜色子滤光片2342。第二滤光片组232的彩色滤光片234中至少包括第三颜色子滤光片2343。
像素阵列24包括多个像素，像素阵列24的第一颜色感光像素2441与滤光片阵列23的第一颜色子滤光片2341对应设置，像素阵列24的第二颜色感光像素2442与滤光片阵列23的第二颜色子滤光片2342对应设置，像素阵列24的第三颜色感光像素2443与滤光片阵列23的第三颜色子滤光片2343对应设置，像素阵列24被配置成用于接收穿过滤光片阵列23的光线以生成电信号；
在一个实施例中，像素阵列24包括多个全色像素2431和多个彩色像素2441，每个全色像素2431对应全色滤光片233的一个子滤光片2331，每个第一颜色感光像素2441对应彩色滤光片234的一个第一颜色子滤光片2341，每个第二颜色感光像素2442对应彩色滤光片234的一个第二颜色子滤光片2342，每个第三颜色感光像素2443对应彩色滤光片234的一个第三颜色子滤光片2343；
如图9所示，该图像生成方法包括：
操作902，在第一分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，以及彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及彩色像素对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像。
第一分辨率模式是指分辨率、功耗、信噪比和帧率均比较均衡的一级像素合并读出模式。第一分辨率模式具体可以是图像、视频拍摄的默认模式。合并读出是指将多个像素的像素值求和，或者计算出多个像素的像素值的均值。例如，多个全色像素合并读出的第一像素值，可以是将多个全色像素的像素值求和，得到第一像素值；或者，计算多个全色像素的像素值的均值，作为第一像素值。
彩色滤光片234具有比全色滤光片233的更窄的光谱响应，则全色滤光片233透过的进光量大于彩色滤光片234透过的进光量，即彩色滤光片234透过的光线的波段宽度小于全色滤光片233透过的光线的波段宽度，全色滤光片233透过更多的光线，通过全色滤光片233得到相应的全色像素2431具有更高的信噪比，该全色像素包含有更多的信息，可以解析出更多的纹理细节。其中，信噪比是指正常信号与噪声信号之间的比值。像素的信噪比越高，则该像素包含的正常信号的比例越高，从该像素中解析到的信息也越多。
彩色像素可以是G(Green，绿色)像素、R(Red，红色)像素和B(Blue，蓝色)像素等，但不限于此。
在接收到拍摄指令的情况下，检测用户是否选择所需使用的分辨率模式，当检测到用户选择使用第一分辨率模式的情况下，或者，在用户未选择所需使用的分辨率模式，未使用预览拍摄、且当前环境非夜景模式的情况下，使用第一分辨率模式响应该拍摄指令。
在第一分辨率模式下，全色滤光片233中的子滤光片2331透过的光线投射至对应的全色像素2431上，全色像素2431接收穿过子滤光片2331的光线以生成电信号。彩色滤光片234中的第一颜色子滤光片2341透过的光线投射至对应的第一颜色感光像素2441上，第一颜色感光像素2441穿过对应的第一颜色子滤光片2341的光线以生成电信号。彩色滤光片234中的第二颜色子滤光片2342透过的光线投射 至对应的第二颜色感光像素2442上，第二颜色感光像素2442穿过对应的第二颜色子滤光片2342的光线以生成电信号。彩色滤光片234中的第三颜色子滤光片2343透过的光线投射至对应的第三颜色感光像素2443上，第三颜色感光像素2443穿过对应的第三颜色子滤光片2343的光线以生成电信号。
第一滤光片组231中至少一个彩色滤光片234中包括第一颜色子滤光片2341和第二颜色子滤光片2342，则第一滤光片组231中至少一个彩色滤光片234对应第一颜色感光像素2441和第二颜色感光像素2442。
电子设备将同一全色滤光片233对应的多个全色像素2431合并读出第一像素值。同一彩色滤光片234中可同时包括多个第一颜色子滤光片2341和多个第二颜色子滤光片2342，电子设备将同一彩色滤光片234中多个第一颜色子滤光片2341对应的各第一颜色感光像素2441合并读出第二像素值。同一彩色滤光片234中可包括多个第三颜色子滤光片2343，电子设备将同一彩色滤光片234中多个第三颜色子滤光片2343对应的各第三颜色感光像素2443合并读出第三像素值。
电子设备根据各第一像素值、各第二像素值和各第三像素值，得到第一合并图像。
操作904，根据彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像。
第一滤光片组231中至少一个彩色滤光片234中包括第一颜色子滤光片2341和第二颜色子滤光片2342，则第一滤光片组231中至少一个彩色滤光片234对应第一颜色感光像素2441和第二颜色感光像素2442。
对于包括第二颜色子滤光片2342的彩色滤光片234，电子设备读取在同一彩色滤光片234中的各第二颜色子滤光片2342对应的第二颜色感光像素2442合并读出第四像素值，并基于各第四像素值生成第二合并图像。
在一个实施例中，第一合并图像是由第一像素值、第二像素值和第三像素值构成的图像，第二合并图像是由第四像素值和空像素构成的图像。其中，空像素是无任何信息的像素。
在一个实施例中，同一全色滤光片233对应4个全色像素2431，在不同滤光片组中，同一彩色滤光片234可同时对应2个第一颜色感光像素2441和2个第二颜色感光像素2442，彩色滤光片234可对应4个第三颜色感光像素2443。
操作906，基于第一合并图像和第二合并图像，得到第一目标图像。
电子设备可根据预设像素读取方式，从第一合并图像和第二合并图像中读取像素值以生成第一目标图像。预设像素读取方式是预先设置的像素读取方式。
本实施例中，在第一分辨率模式下，通过滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，以及彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，能够将全色通道信息融合到图像中，提高整体的进光量，使得能够生成信息更多、细节解析更清晰的第一合并图像。根据彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，能够将第二颜色感光像素分离出来形成第二合并图像，使得第二颜色感光像素在第二合并图像和第一颜色感光像素在第一合并图像中的位置保持一致。基于第一合并图像和第二合并图像，得到第一目标图像，能够将第一合并图像中的第一颜色感光像素和第二合并图像中的第二颜色感光像素混合排列，使得所生成的第一目标图像色彩更清晰。并且像素合并读出使得生成的第一目标图像尺寸减小、生成图像所需消耗的功耗低。
在一个实施例中，基于第一合并图像和第二合并图像，得到第一目标图像，包括：
遍历第一合并图像中的像素值，在所遍历的像素值为第二像素值的情况下，从第二合并图像中读取与第二像素值位置相同的第四像素值，并将所读取的第四像素值调整至与第二像素值相邻，直至第一合并图像中的像素值均被遍历后，得到第一目标图像。
具体地，各第二像素值在第一合并图像中的位置和各第四像素值在第二合并图像中的位置相同。电子设备可遍历第一合并图像中的各像素值，在每次遍历中，确定当前所遍历的像素值是否为第二像素值。在当前所遍历的像素值非第二像素值的情况下，继续遍历下一像素值。在当前所遍历的像素值为第二像素值的情况下，确定当前所遍历的第二像素值的位置，并从第二合并图像中读取与当前所遍历的第二像 素值位置相同的第四像素值，并将所读取的第四像素值调整至与第二像素值相邻，接着继续遍历下一像素值，直至第一合并图像中的像素值均被遍历后，生成第一目标图像。
本实施例中，在第一目标图像中，第二像素值和第四像素值相邻表示第二像素值和第四像素值在第一目标图像中的位置相同，即表示第二像素值和第四像素值在第一目标图像中的坐标相同。
在一个实施例中，在当前所遍历的像素值非第二像素值的情况下，继续遍历下一像素值。在当前所遍历的像素值为第二像素值的情况下，确定当前所遍历的第二像素值的位置，并从第二合并图像中读取与当前所遍历的第二像素值位置相同的第四像素值，并将所读取的第四像素值调整至与第二像素值水平相邻。第四像素值与第二像素值水平相邻，可以是将第四像素值作为第二像素值在水平方向上的相邻前一像素值，或者将第四像素值作为第二像素值在水平方向上的相邻后一像素值。
本实施例中，遍历第一合并图像中的像素值，在所遍历的像素值为第二像素值的情况下，从第二合并图像中读取与第二像素值位置相同的第四像素值，并将所读取的第四像素值调整至与第二像素值相邻，使得第一颜色感光像素和第二颜色感光像素混合排列，提高了彩色分辨能力，使得所生成第一目标图像的每行和每列均有第一颜色感光像素、第二颜色感光像素和第三颜色感光像素存在，即目标图像的每行每列均具备有RGB像素，能够有效降低伪色的风险。
如图10所示，在第一分辨率模式下，通过图像传感器获得原始图像1002，将原始图像1002中局部相同像素取平均，做binning，即将局部相同的4个全色像素合并读出第一像素值、局部相同的2个第一颜色感光像素合并读出第二像素值，以及局部相同的4个第三颜色感光像素合并读出第三像素值，得到第一合并图像1004。并且，将局部相同的2个第二颜色感光像素合并读出第四像素值，得到第二合并图像1006。为了保持第二合并图像1006和第一合并图像1004的尺寸相同，可通过第四像素值和空像素生成第二合并图像1006。
例如，全色像素为w像素、第一颜色感光像素为R像素、第三颜色感光像素为G像素、第二颜色感光像素为B像素。
从第一合并图像1004和第二合并图像1006读取像素值，使得每个第四像素值作为与第二像素值相邻的后一像素值，得到第一目标图像1008。
在一个实施例中，第二滤光片组的彩色滤光片还包括第一颜色子滤光片或第二颜色子滤光片；在第一分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，以及彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像，包括：
在第一分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，根据第一滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值、根据包含第一颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值、以及根据第二滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，得到第一合并图像；
根据彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像，包括：根据第一滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，根据第二滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值，根据包含第二颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值，得到第二合并图像。
具体地，多个滤光片组包括第一滤光片组231和第二滤光片组232，第一滤光片组231中至少一个彩色滤光片234中包括第一颜色子滤光片2341和第二颜色子滤光片2342。第二滤光片组232的彩色滤光片234中包括第三颜色子滤光片2343，还包括第一颜色子滤光片2341或第二颜色子滤光片2342。
在第一分辨率模式下，电子设备将同一全色滤光片233对应的多个全色像素2431合并读出第一像素值。
对于第一滤光片组231中的彩色滤光片234，电子设备将同一彩色滤光片234的多个第一颜色子滤光片2341对应的各第一颜色感光像素2441合并读出第二像素值，并将同一彩色滤光片234的多个第二 颜色子滤光片2342对应的各第二颜色感光像素2442合并读出第四像素值，则该同一彩色滤光片234对应第二像素值和第四像素值。
在第二滤光片组232中的彩色滤光片234包括第一颜色子滤光片2341和第三颜色子滤光片2343的情况下，电子设备将同一彩色滤光片234的多个第一颜色子滤光片2341对应的各第一颜色感光像素2441合并读出第二像素值，并将多个第三颜色子滤光片2343对应的各第三颜色感光像素2443合并读出第三像素值，则该同一彩色滤光片234对应第二像素值和第三像素值。
在第二滤光片组232中的彩色滤光片234包括第二颜色子滤光片2342和第三颜色子滤光片2343的情况下，电子设备将同一彩色滤光片234的多个第二颜色子滤光片2342对应的各第二颜色感光像素2442合并读出第四像素值，并将同一彩色滤光片234的多个第三颜色子滤光片2343对应的各第三颜色感光像素2443合并读出第三像素值，则该同一彩色滤光片234对应第四像素值和第三像素值。
对于对应第二像素值和第四像素值的彩色滤光片234，电子设备可读取第二像素值；对于对应第二像素值和第三像素值的彩色滤光片234，电子设备可读取第三像素值；对于对应第四像素值和第三像素值的彩色滤光片234，电子设备可读取第四像素值；电子设备根据所读取的第二像素值、第三像素值、第四像素值和所有的第一像素值，生成第一合并图像。
对于对应第二像素值和第四像素值的彩色滤光片234，电子设备可获取第四像素值；对于对应第二像素值和第三像素值的彩色滤光片234，电子设备可获取第二像素值；对于对应第四像素值和第三像素值的彩色滤光片234，电子设备可获取第三像素值；电子设备根据所获取的第四像素值、第二像素值和第三像素值和空像素，生成第二合并图像。
电子设备从第一合并图像和第二合并图像读取像素值，生成第一目标图像。
本实施例中，第二滤光片组232的彩色滤光片234中包括第三颜色子滤光片2343，还包括第一颜色子滤光片2341或第二颜色子滤光片2342，在第一分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，根据第一滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值、根据包含第一颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值、以及根据第二滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，使得合并读取所得到的第一合并图像中包含了所有颜色的像素，能够提高彩色通道的分辨能力。根据彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值、彩色像素对应的多个第三颜色感光像素合并读出的第三像素值、以及彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像，使得根据第一合并图像和第二合并图像生成的第一目标图像中的各颜色的像素分布更均匀，有效提高图像的清晰度。
如图11所示，在第一分辨率模式下，通过图像传感器获得原始图像1102，将原始图像1102中的局部4个相邻的全色像素合并读出第一像素值，将对角线上的2个第一颜色感光像素合并读出第二像素值、将对角线上的2个第三颜色感光像素合并读出第三像素值、将对角线上的2个第二颜色感光像素合并读出的第四像素值；读取第二像素值、第三像素值、第四像素值和所有的第一像素值，生成第一合并图像1104；读取其余的第四像素值、第二像素值和第三像素值，并结合空像素，生成第二合并图像1106。电子设备可基于第一合并图像1104和第二合并图像1106生成第一目标图像。
在一个实施例中，如图12所示，该方法还包括：
操作1202，在第二分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，以及彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像；该第二分辨率模式对应的分辨率小于第一分辨率模式对应的分辨率。
第二分辨率模式是指对分辨率要求比较低的场景下所使用的模式，是低分辨率、低功耗、高信噪比、高帧率的二级像素合并读出模式。该第二分辨率模式对应的分辨率、功耗小于第一分辨率模式对应的分辨率、功耗。该第二分辨率模式对应的信噪比、帧率大于第一分辨率模式对应的信噪比、帧率。
第二分辨率模式具体可以是图像拍摄时的预览模式、视频拍摄时的预览模式，或者在夜景下进行图像拍摄、视频拍摄的夜景模式等分辨率要求较低的场景，但不限于此。视频拍摄的预览模式例如1080p 视频预览、应用视频预览等。
在接收到拍摄指令的情况下，确定该拍摄指令是否为预览拍摄。在该拍摄指令为预览拍摄的情况下，触发第二分辨率模式。或者，电子设备检测当前环境是否为夜景，在当前环境为夜景的情况下，触发第二分辨率模式。或者，在用户选择第二分辨率模式的情况下，触发第二分辨率模式对应的读出模式。
在第二分辨率模式下，全色滤光片233中的子滤光片2331透过的光线投射至对应的全色像素2431上，全色像素2431接收穿过子滤光片2331的光线以生成电信号。彩色滤光片234中的第一颜色子滤光片2341透过的光线投射至对应的第一颜色感光像素2441上，第一颜色感光像素2441穿过对应的第一颜色子滤光片2341的光线以生成电信号。彩色滤光片234中的第二颜色子滤光片2342透过的光线投射至对应的第二颜色感光像素2442上，第二颜色感光像素2442穿过对应的第二颜色子滤光片2342的光线以生成电信号。彩色滤光片234中的第三颜色子滤光片2343透过的光线投射至对应的第三颜色感光像素2443上，第三颜色感光像素2443穿过对应的第三颜色子滤光片2343的光线以生成电信号。
第一滤光片组231中至少一个彩色滤光片234中包括第一颜色子滤光片2341和第二颜色子滤光片2342，则第一滤光片组231中至少一个彩色滤光片234对应第一颜色感光像素2441和第二颜色感光像素2442。
电子设备将同一全色滤光片233对应的多个全色像素2431合并读出第一像素值。同一彩色滤光片234中可同时包括多个第一颜色子滤光片2341和多个第二颜色子滤光片2342，电子设备将同一彩色滤光片234中多个第一颜色子滤光片2341对应的各第一颜色感光像素2441合并读出第二像素值。同一彩色滤光片234中可包括多个第三颜色子滤光片2343，电子设备将同一彩色滤光片234中多个第三颜色子滤光片2343对应的各第三颜色感光像素2443合并读出第三像素值。
电子设备根据各第一像素值、各第二像素值和各第三像素值，得到第一合并图像。
操作1204，根据彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像。
对于包括第二颜色子滤光片2342的彩色滤光片234，电子设备读取在同一彩色滤光片234中的各第二颜色子滤光片2342对应的第二颜色感光像素2442合并读出第四像素值，并基于各第四像素值生成第二合并图像。
操作1206，基于第一合并图像中第九对角线方向上的多个全色像素合并读出的第五像素值，得到全色合并图像。
电子设备将第一合并图像中在第九对角线方向上的多个全色像素合并读出第五像素值，根据各第五像素值生成全色合并图像。该全色合并图像中均为全色像素。
操作1208，基于第一合并图像中第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值，以及在第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值，得到第三合并图像，第九对角线方向不同于第十对角线方向。
电子设备将第一合并图像中第十对角线方向上的多个第一颜色感光像素合并读出第六像素值，并将第十对角线方向上的多个第三颜色感光像素合并读出第七像素值，根据第六像素值和第七像素值，生成第三合并图像。该第三合并图像中包括第一颜色感光像素和第三颜色感光像素。第九对角线方向与第十对角线方向不同，具体可以是七对角线方向与第十对角线方向不平行，或者，第九对角线方向与第十对角线方向垂直等。
操作1210，基于第二合并图像中第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，得到第四合并图像。
电子设备将第二合并图像中处于第十对角线方向上的多个第二颜色感光像素合并读出第八像素值。基于各第八像素值，生成第四合并图像。
在一个实施例中，电子设备将第二合并图像中处于第十对角线方向上的2个第二颜色感光像素合并读出第八像素值，基于各第八像素值和空像素，生成第四合并图像。
操作1212，基于全色合并图像、第三合并图像和第四合并图像，得到第二目标图像。
电子设备可根据从全色合并图像、第三合并图像和第四合并图像中读取像素值以生成第二目标图 像。
本实施例中，在第二分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，以及彩色滤光片对应的多个彩色像素合并读出的第二像素值，以及彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像，并将根据彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像，能够将第二颜色感光像素分离出来作为单独的图像并降低图像的分辨率。基于第一合并图像中第九对角线方向上的多个全色像素合并读出的第五像素值得到全色合并图像，能够将全色像素从第一合并图像中分离出来的同时减小图像的尺寸，以降低功耗。基于第一合并图像中第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值，以及第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值，得到包含第一颜色感光像素和第三颜色感光像素的第三合并图像，多合一的像素读出方式使得生成的图像噪声少。基于第二合并图像中第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，使得第四合并图像的尺寸与第三合并图像、全色合并图像的尺寸保持一致。基于全色合并图像、第三合并图像和第四合并图像，能够将不同第一颜色感光像素和第二颜色感光像素进行混合排列，使得所生成的第二目标图像中RGB像素分布更均匀，图像质量更高。并且，所得到第二目标图像的分辨率进一步减小，且全色像素具有更高的信噪比，图像的帧率高，从而达到了二级像素合并输出的功耗更低、信噪比更佳的图像处理效果。
在一个实施例中，基于全色合并图像、第三合并图像和第四合并图像，得到第二目标图像，包括：
遍历第三合并图像中的像素；在所遍历的像素值为第六像素值或第七像素值的情况下，从全色合并图像中读取与遍历的像素值对应的第五像素值，并将所读取的第五像素值调整至与遍历的像素值相邻；在所遍历的像素值为第六像素值的情况下，从第四合并图像中读取与第六像素位置相同的第八像素值，并将所读取的第八像素值调整至与第六像素值相邻，直至全色合并图像、第三合并图像和第四合并图像中的像素值均读取后，得到第二目标图像。
具体地，各第六像素值在第三合并图像中的位置和各第八像素值在第四合并图像中的位置相同。各全色像素在全色合并图像中的位置和各第六像素值在第三合并图像中的位置、各第八像素值在第四合并图像中的位置相同。
电子设备可遍历第三合并图像中的各像素值，在每次遍历中，确定当前所遍历的像素值是否为第六像素值或第七像素值。在当前所遍历的像素值非第六像素值且非第七像素值的情况下，继续遍历下一像素值。
在当前所遍历的像素值为第六像素值或第七像素值的情况下，确定当前所遍历的像素值的位置，从全色合并图像中读取与遍历的像素值对应的第五像素值，并将所读取的第五像素值调整至与当前所遍历的像素值相邻。并且，在当前所遍历的像素值为第六像素值的情况下，从第四合并图像中读取与第六像素位置相同的第八像素值，将所读取的第八像素值调整至与第六像素值相邻，接着继续遍历下一像素值，直至全色合并图像、第三合并图像和第四合并图像中的像素值均读取后，得到第二目标图像。
本实施例中，在第二目标图像中第六像素值和第八像素值相邻，则表示第六像素值和第八像素值在第二目标图像中的位置相同，即表示第六像素值和第八像素值在第二目标图像中的坐标相同。
在一个实施例中，第五像素值与当前所遍历的像素值相邻，可以是第五像素值与当前所遍历的像素值在水平方向上相邻，具体可以是将第五像素值作为当前所遍历的像素值在水平方向上的相邻前一像素值或相邻后一像素值。
第八像素值与第六像素值相邻，可以是第八像素值与第六像素值水平相邻。第八像素值与第六像素值水平相邻，可以是将第八像素值作为第六像素值在水平方向上的相邻前一像素值，或者将第八像素值作为第六像素值在水平方向上的相邻后一像素值。
在一个实施例中，在第五像素值作为第六像素值在水平方向上的相邻前一像素值的情况下，将第八像素值作为该第六像素值在水平方向上的相邻后一像素值。
本实施例中，遍历第三合并图像中的像素，在所遍历的像素值为第六像素值或第七像素值的情况下，从全色合并图像中读取与遍历的像素值对应的第五像素值，并将所读取的第五像素值调整至与遍历的像素值相邻，使得在图像中引入了全色像素，提高了进光量。在所遍历的像素值为第六像素值的情况下， 从第四合并图像中读取与第六像素位置相同的第八像素值，并将所读取的第八像素值调整至与第六像素值相邻，使得第一颜色感光像素和第二颜色感光像素混合排列，提高了彩色分辨能力，使得所生成第二目标图像的每行和每列均有第一颜色感光像素、第二颜色感光像素和第二颜色感光像素存在，即目标图像的每行每列均具备有RGB像素，能够有效降低伪色的风险。
如图13所示，在第二分辨率模式下，通过图像传感器获得原始图像1302，将原始图像1302中局部相同的4个全色像素合并读出第一像素值、局部相同的2个第一颜色感光像素合并读出第二像素值，以及局部相同的4个第三颜色感光像素合并读出第三像素值，得到第一合并图像1304。并且，将局部相同的2个第二颜色感光像素合并读出第四像素值，得到第二合并图像1306。为了保持第二合并图像1306和第一合并图像1304的尺寸相同，可通过第四像素值和空像素生成第二合并图像1306。
将第一合并图像1304中在第九对角线D9方向上的2个全色像素合并读出第五像素值，得到全色合并图像1308。将第一合并图像1304中在第十对角线D10方向上的2个第一颜色感光像素合并读出第六像素值，以及在第十对角线D10方向上的2个第三颜色感光像素合并读出第七像素值，得到第三合并图像1310。
基于第二合并图像1306中在第十对角线D10方向上的2个第二颜色感光像素合并读出的第八像素值，得到第四合并图像1312。
遍历第三合并图像1310中的像素，在所遍历的像素值为第六像素值或第七像素值的情况下，从全色合并图像1308中读取与遍历的像素值对应的第五像素值，并将所读取的第五像素值作为与遍历的像素值相邻的前一像素值。并且，在所遍历的像素值为第六像素值的情况下，从第四合并图像1312中读取与第六像素位置相同的第八像素值，并将所读取的第八像素值作为与第六像素值相邻的后一像素值，直至第三合并图像中的像素值均遍历后，得到第二目标图像1314。
在一个实施中，在第二分辨率模式下，第二滤光片组的彩色滤光片还包括第一颜色子滤光片或第二颜色子滤光片；根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，以及彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像，包括：
在第二分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，根据第一滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值、根据包含第一颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值、以及根据第二滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，得到第一合并图像；
根据彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像，包括：
根据第一滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，根据第二滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值，根据包含第二颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值，得到第二合并图像；
基于第一合并图像中第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值，以及在第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值，得到第三合并图像，包括：基于第一合并图像中在第九对角线方向上的多个全色像素合并读出的第五像素值、在第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值、在第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值、以及在第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，分别得到全色合并图像和第三合并图像；
基于第二合并图像中在第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，得到第四合并图像，包括：基于第二合并图像中，在第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值、在第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值、以及在第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，得到第四合并图像。
具体地，多个滤光片组包括第一滤光片组231和第二滤光片组232，第一滤光片组231中的彩色滤 光片234中包括第一颜色子滤光片2341和第二颜色子滤光片2342。第二滤光片组232的彩色滤光片234中包括第三颜色子滤光片2343，还包括第一颜色子滤光片2341或第二颜色子滤光片2342。
在第二分辨率模式下，电子设备将同一全色滤光片233对应的多个全色像素2431合并读出第一像素值。
对于第一滤光片组231中的彩色滤光片234，电子设备将同一彩色滤光片234的多个第一颜色子滤光片2341对应的各第一颜色感光像素2441合并读出第二像素值，并将同一彩色滤光片234的多个第二颜色子滤光片2342对应的各第二颜色感光像素2442合并读出第四像素值，则该同一彩色滤光片234对应第二像素值和第四像素值。
在第二滤光片组232中的彩色滤光片234包括第一颜色子滤光片2341和第三颜色子滤光片2343的情况下，电子设备将同一彩色滤光片234的多个第一颜色子滤光片2341对应的各第一颜色感光像素2441合并读出第二像素值，并将多个第三颜色子滤光片2343对应的各第三颜色感光像素2443合并读出第三像素值，则该同一彩色滤光片234对应第二像素值和第三像素值。
在第二滤光片组232中的彩色滤光片234包括第二颜色子滤光片2342和第三颜色子滤光片2343的情况下，电子设备将同一彩色滤光片234的多个第二颜色子滤光片2342对应的各第二颜色感光像素2442合并读出第四像素值，并将同一彩色滤光片234的多个第三颜色子滤光片2343对应的各第三颜色感光像素2443合并读出第三像素值，则该同一彩色滤光片234对应第四像素值和第三像素值。
对于对应第二像素值和第四像素值的彩色滤光片234，电子设备可读取第二像素值；对于对应第二像素值和第三像素值的彩色滤光片234，电子设备可读取第三像素值；对于对应第四像素值和第三像素值的彩色滤光片234，电子设备可读取第四像素值；电子设备根据所读取的各第二像素值、各第三像素值、各第四像素值和所有的第一像素值，生成第一合并图像。该第一合并图像中包括全色像素、第一颜色感光像素、第二颜色感光像素和第三颜色感光像素。
对于第一合并图像，电子设备将第九对角线方向上的多个全色像素合并读出第五像素值，根据各第五像素值生成全色合并图像。该全色合并图像中均为全色像素。
对于第一合并图像，电子设备将第一合并图像中在第十对角线方向上的多个第一颜色感光像素合并读出第六像素值，并将在第十对角线方向上的多个第三颜色感光像素合并读出第七像素值，将在第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值。根据读出的第六像素值、第七像素值和第八像素值，生成第三合并图像。该第三合并图像中包括第一颜色感光像素、第二颜色感光像素和第三颜色感光像素。
对于对应第二像素值和第四像素值的彩色滤光片234，电子设备可获取第四像素值；对于对应第二像素值和第三像素值的彩色滤光片234，电子设备可获取第二像素值；对于对应第四像素值和第三像素值的彩色滤光片234，电子设备可获取第三像素值；电子设备根据所获取的第四像素值、第二像素值和第三像素值和空像素，生成第二合并图像。
对于第二合并图像，电子设备将第二合并图像中在第十对角线方向上的多个第一颜色感光像素合并读出第六像素值、在第十对角线方向上的多个第三颜色感光像素合并读出第七像素值、以及在第十对角线方向上的多个第二颜色感光像素合并读出第八像素值。根据读出的第六像素值、第七像素值和第八像素值，以及空像素，生成第四合并图像。该第四合并图像中包括第一颜色感光像素、第二颜色感光像素、第三颜色感光像素和空像素。
电子设备根据全色合并图像、第三合并图像和第四合并图像，可生成第二目标图像。
本实施例中，在第二分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，根据第一滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值，根据包含第一颜色子滤光片的第二滤光片组中第三颜色感光像素合并读出的第三像素值，以及根据第二滤光片组中多个第二颜色子滤光片对应的第三颜色感光像素合并读出的第四像素值，得到第一合并图像，并将根据第一滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，根据第二滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值，根据包含第二颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的 第三像素值，得到第二合并图像，能够将各个彩色滤光片对应的第一颜色感光像素、第二颜色感光像素和第三颜色感光像素分离出来形成两张合并图像，使得两张合并图像中各颜色的感光像素在各自合并图像中的位置相同。
基于第一合并图像中在第九对角线方向上的多个全色像素合并读出的第五像素值得到全色合并图像，能够将全色像素从第一合并图像中分离出来的同时减小图像的尺寸，以降低功耗。
基于第一合并图像中在第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值，以及在第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值，以及在第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，得到包含第一颜色感光像素、第二颜色感光像素和第三颜色感光像素的第三合并图像，使得RGB像素在图像中的分布更均匀。基于第二合并图像中，在第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值、在第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值、以及在第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，得到第四合并图像，使得第四合并图像的尺寸与第三合并图像、全色合并图像的尺寸保持一致。基于全色合并图像、第三合并图像和第四合并图像，能够将不同第一颜色感光像素、第二颜色感光像素和第三颜色感光像素进行混合排列，使得所生成的第二目标图像中RGB像素分布更均匀，图像质量更高。并且，所得到第二目标图像的分辨率进一步减小，且全色像素具有更高的信噪比，图像的帧率高，从而达到了二级像素合并输出的功耗更低、信噪比更佳的图像处理效果。
如图14所示，在第二分辨率模式下，通过图像传感器获得原始图像1402，将原始图像1402中的局部4个相邻的全色像素合并读出第一像素值，将对角线上的2个第一颜色感光像素合并读出第二像素值、将对角线上的2个第三颜色感光像素合并读出第三像素值、将对角线上的2个第二颜色感光像素合并读出的第四像素值；读取第二像素值、第三像素值、第四像素值和所有的第一像素值，生成第一合并图像1404；读取其余的第四像素值、第二像素值和第三像素值，并结合空像素，生成第二合并图像1406。
将第一合并图像1404中在第九对角线D9方向上的2个全色像素合并读出第五像素值，得到全色合并图像1408。将第一合并图像1404中在第十对角线D10方向上的2个第一颜色感光像素合并读出第六像素值，以及在第十对角线D10方向上的2个第三颜色感光像素合并读出第七像素值，以及在第十对角线D10方向上的2个第二颜色感光像素合并读出第八像素值。根据读出的第六像素值、第七像素值和第八像素值，生成第三合并图像1410。
对于第二合并图像1406，将第二合并图像1406中在第十对角线D10方向上的2个第一颜色感光像素合并读出第六像素值，以及在第十对角线D10方向上的2个第三颜色感光像素合并读出第七像素值，以及在第十对角线D10方向上的2个第二颜色感光像素合并读出第八像素值。根据读出的第六像素值、第七像素值和第八像素值，生成第四合并图像1412。
电子设备可基于全色合并图像1408、第三合并图像1410和第四合并图像1412，生成第二目标图像。
在一个实施例中，该方法还包括：在第二分辨率模式下，根据滤光片组对应的多个全色像素合并读出的第一像素值，得到全色图像；根据滤光片组对应的多个第一颜色感光像素合并读出的第二像素值，以及滤光片组对应的多个第三颜色感光像素合并读出的第三像素值，得到第五合并图像；根据滤光片组对应的多个第二颜色感光像素合并读出的第四像素值，得到第六合并图像；基于全色图像、第五合并图像和第六合并图像，得到第三目标图像。
多个滤光片组包括第一滤光片组231和第二滤光片组232，第一滤光片组231中至少一个彩色滤光片234中包括第一颜色子滤光片2341和第二颜色子滤光片2342，则第一滤光片组231中至少一个彩色滤光片234对应第一颜色感光像素2441和第二颜色感光像素2442。第二滤光片组232中彩色滤光片234至少包括第三颜色子滤光片2343，则第二滤光片组232中彩色滤光片234至少对应第三颜色感光像素2443。
在第二分辨率模式下，电子设备将同一滤光片组中的所有全色滤光片233对应的全色像素2431合并读出第一像素值，并根据各第一像素值得到全色图像。进一步地，电子设备分别将每个第一滤光片组231中所有全色滤光片233对应的全色像素2431合并读出第一像素值，使得一个第一滤光片组231对应一个第一像素值，以得到各第一滤光片组231分别对应的第一像素值。电子设备分别将每个第二滤光 片组232中所有全色滤光片233对应的全色像素2431合并读出第一像素值，使得一个第二滤光片组232对应一个第一像素值，以得到各第二滤光片组232分别对应的第一像素值。根据各第一滤光片组231分别对应的第一像素值和各第二滤光片组232分别对应的第一像素值得到全色图像。
电子设备将同一滤光片组中所有彩色滤光片234的多个第一颜色子滤光片2341所对应的各第一颜色感光像素2441合并读出第二像素值。电子设备将同一滤光片组中所有彩色滤光片234的多个第三颜色子滤光片2343对应的各第三颜色感光像素2443合并读出第三像素值。电子设备根据各第二像素值和各第三像素值，得到第五合并图像。
进一步地，电子设备分别将每个第一滤光片组231中所有彩色滤光片234的多个第一颜色子滤光片2341所对应的各第一颜色感光像素2441合并读出第二像素值，使得一个第一滤光片组231对应一个第二像素值，以得到各第一滤光片组231分别对应的第二像素值。电子设备分别将每个第二滤光片组232中所有彩色滤光片234的多个第三颜色子滤光片2343对应的各第三颜色感光像素2443合并读出第三像素值，使得一个第二滤光片组232对应一个第三像素值，以得到各第二滤光片组232分别对应的第三像素值。根据各第一滤光片组231分别对应的第二像素值和各第二滤光片组232分别对应的第三像素值得到第五合并图像。
电子设备将同一滤光片组中所有彩色滤光片234的多个第二颜色子滤光片2342所对应的各第二颜色感光像素2442合并读出第四像素值。进一步地，电子设备分别将每个第一滤光片组231中所有彩色滤光片234的多个第二颜色子滤光片2342对应的各第二颜色感光像素2442合并读出第四像素值，使得一个第一滤光片组231对应一个第四像素值，以得到各第一滤光片组231分别对应的第四像素值。电子设备根据各第一滤光片组231分别对应的第四像素值，得到第六合并图像。
电子设备可根据预设像素读取方式，从全色图像、第五合并图像和第六合并图像中读取像素值以生成第三目标图像。
本实施例中，在第二分辨率模式下，通过滤光片组对应的多个全色像素合并读出的第一像素值，能够分离出全色图像；根据滤光片组对应的多个第一颜色感光像素合并读出的第二像素值，以及滤光片组对应的多个第三颜色感光像素合并读出的第三像素值，能够将第一颜色感光像素和第三颜色感光像素分离出来形成第五合并图像，能够快速有效降低图像尺寸。根据滤光片组对应的多个第二颜色感光像素合并读出的第四像素值，能够将第二颜色感光像素分离出来形成第六合并图像，使得第二颜色感光像素在第六合并图像和第一颜色感光像素在第五合并图像中的坐标位置保持一致。基于全色图像、第五合并图像和第六合并图像得到第三目标图像，能够将全色通道信息融合到图像中，提高整体的进光量，使得能够生成信息更多、细节解析更清晰的第三目标图像。并且通过滤光片组对应的多个像素合并读出使得生成的第三目标图像尺寸减小、生成图像所需消耗的功耗低。
在一个实施例中，第二滤光片组的彩色滤光片还包括第一颜色子滤光片或第二颜色子滤光片；根据滤光片组对应的多个第一颜色感光像素合并读出的第二像素值，以及滤光片组对应的多个第三颜色感光像素合并读出的第三像素值，得到第五合并图像，包括：
根据第一滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值、根据包含第一颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值、以及根据第二滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，得到第五合并图像；
根据滤光片组对应的多个第二颜色感光像素合并读出的第四像素值，得到第六合并图像，包括：
根据第一滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，根据第二滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值，根据包含第二颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值，得到第六合并图像。
多个滤光片组包括第一滤光片组231和第二滤光片组232，第一滤光片组231中至少一个彩色滤光片234中包括第一颜色子滤光片2341和第二颜色子滤光片2342，则第一滤光片组231中至少一个彩色滤光片234对应第一颜色感光像素2441和第二颜色感光像素2442。第二滤光片组232中彩色滤光片234 包括第三颜色子滤光片2343，还包括第一颜色子滤光片2341或第二颜色子滤光片2342。
在第二分辨率模式下，电子设备将同一滤光片组中的所有全色滤光片233对应的全色像素2431合并读出第一像素值，并根据各第一像素值得到全色图像。进一步地，电子设备分别将每个第一滤光片组231中所有全色滤光片233对应的全色像素2431合并读出第一像素值，使得一个第一滤光片组231对应一个第一像素值，以得到各第一滤光片组231分别对应的第一像素值。电子设备分别将每个第二滤光片组232中所有全色滤光片233对应的全色像素2431合并读出第一像素值，使得一个第二滤光片组232对应一个第一像素值，以得到各第二滤光片组232分别对应的第一像素值。根据各第一滤光片组231分别对应的第一像素值和各第二滤光片组232分别对应的第一像素值得到全色图像。
电子设备分别将每个第一滤光片组231中所有彩色滤光片234的多个第一颜色子滤光片2341所对应的各第一颜色感光像素2441合并读出第二像素值，使得一个第一滤光片组231对应一个第二像素值，以得到各第一滤光片组231分别对应的第二像素值。
对于包含第一颜色子滤光片2341和第三颜色子滤光片2343的第二滤光片组232，电子设备分别将每个第二滤光片组232中所有彩色滤光片234的多个第三颜色子滤光片2343对应的各第三颜色感光像素2443合并读出第三像素值，使得一个包含第一颜色子滤光片2341的第二滤光片组232对应一个第三像素值，以得到包含第一颜色子滤光片2341的每个第二滤光片组232分别对应的第三像素值。
对于包含第二颜色子滤光片2342和第三颜色子滤光片2343的第二滤光片组232，电子设备分别将每个第二滤光片组232中所有彩色滤光片234的多个第二颜色子滤光片2342所对应的各第二颜色感光像素2442合并读出第四像素值，使得一个包含第二颜色子滤光片2342的第二滤光片组232对应一个第四像素值，以得到包含第二颜色子滤光片2342的每个第二滤光片组232分别对应的第四像素值。
电子设备根据各第一滤光片组231分别对应的第二像素值、包含第一颜色子滤光片2341的每个第二滤光片组232分别对应的第三像素值、以及包含第二颜色子滤光片2342的每个第二滤光片组232分别对应的第四像素值得到第五合并图像。
电子设备分别将每个第一滤光片组231中所有彩色滤光片234的多个第二颜色子滤光片2342所对应的各第二颜色感光像素2442合并读出第四像素值，使得一个第一滤光片组231对应一个第四像素值，以得到每个第一滤光片组231分别对应的第四像素值。
对于包含第一颜色子滤光片2341和第三颜色子滤光片2343的第二滤光片组232，电子设备分别将每个第二滤光片组232中所有彩色滤光片234的多个第一颜色子滤光片2341所对应的各第一颜色感光像素2441合并读出第二像素值，使得一个包含第一颜色子滤光片2341的第二滤光片组232对应一个第二像素值，以得到包含第一颜色子滤光片2341的每个第二滤光片组232分别对应的第二像素值。
对于包含第二颜色子滤光片2342和第三颜色子滤光片2343的第二滤光片组232，电子设备分别将每个第二滤光片组232中所有彩色滤光片234的多个第三颜色子滤光片2343对应的各第三颜色感光像素2443合并读出第三像素值，使得一个包含第二颜色子滤光片2342的第二滤光片组232对应一个第三像素值，以得到包含第二颜色子滤光片2342的每个第二滤光片组232分别对应的第三像素值。
电子设备根据各第一滤光片组231分别对应的第四像素值、包含第一颜色子滤光片2341的每个第二滤光片组232分别对应的第二像素值，以及包含第二颜色子滤光片2342的每个第二滤光片组232分别对应的第三像素值，得到第六合并图像。
电子设备可根据预设像素读取方式，从全色图像、第五合并图像和第六合并图像中读取像素值以生成第三目标图像。
本实施例中，在第二分辨率模式下，通过根据第一滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值、根据包含第一颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值、以及根据第二滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，能够将部分第一颜色感光像素、第三颜色感光像素和第二颜色感光像素分离出来形成第五合并图像，能够通过像素合并减小图像尺寸，以减少处理功耗。根据第一滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值、第二滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值，根据包含第二颜色子滤光片 的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值，能够将部分第一颜色感光像素、第三颜色感光像素和第二颜色感光像素分离出来出来形成第六合并图像，使得第二颜色感光像素在第六合并图像和第一颜色感光像素在第五合并图像中的坐标位置保持一致。基于全色图像、第五合并图像和第六合并图像得到第三目标图像，能够将全色通道信息融合到图像中，提高整体的进光量，使得能够生成信息更多、细节解析更清晰的第三目标图像。并且通过滤光片组对应的多个像素合并读出使得生成的第三目标图像尺寸减小、生成图像所需消耗的功耗低。
在一个实施例中，提供了一种图像生成方法，应用于电子设备的图像传感器，图像传感器包括滤光片阵列和像素阵列，滤光片阵列包括最小重复单元，最小重复单元包括多个滤光片组，每个滤光片组包括彩色滤光片和全色滤光片，全色滤光片透过的进光量大于彩色滤光片透过的进光量，彩色滤光片和全色滤光片均包括4个子滤光片，多个滤光片组至少包括第一滤光片组，第一滤光片组中至少一个彩色滤光片包括第一颜色子滤光片和第二颜色子滤光片；第二滤光片组中彩色滤光片包括第三颜色子滤光片，第二滤光片组的彩色滤光片还包括第一颜色子滤光片或第二颜色子滤光片；像素阵列的第一颜色感光像素与滤光片阵列的第一颜色子滤光片对应设置，像素阵列的第二颜色感光像素与滤光片阵列的第二颜色子滤光片对应设置，像素阵列的第三颜色感光像素与滤光片阵列的第三颜色子滤光片对应设置，像素阵列被配置成用于接收穿过滤光片阵列的光线以生成电信号；
该方法包括：
在第一分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，以及彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像；根据彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像；遍历第一合并图像中的像素值，在所遍历的像素值为第二像素值的情况下，从第二合并图像中读取与第二像素值位置相同的第四像素值，并将所读取的第四像素值调整至与第二像素值相邻，直至第一合并图像中的像素值均被遍历后，得到第一目标图像。
可选地，在第一分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，根据第一滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值、根据包含第一颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值、以及根据第二滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，得到第一合并图像；根据第一滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，根据第二滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值，根据包含第二颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值，得到第二合并图像；基于第一合并图像和第二合并图像，得到第一目标图像。
可选地，在第二分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，以及彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像；第二分辨率模式对应的分辨率小于第一分辨率模式对应的分辨率；根据彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像；基于第一合并图像中第九对角线方向上的多个全色像素合并读出的第五像素值，得到全色合并图像；基于第一合并图像中第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值，以及第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值，得到第三合并图像，第九对角线方向不同于第十对角线方向；基于第二合并图像中在第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，得到第四合并图像；遍历第三合并图像中的像素；在所遍历的像素值为第六像素值或第七像素值的情况下，从全色合并图像中读取与遍历的像素值对应的第五像素值，并将所读取的第五像素值调整至与遍历的像素值相邻；在所遍历的像素值为第六像素值的情况下，从第四合并图像中读取与第六像素位置相同的第八像素值，并将所读取的第八像素值调整至与第六像素值相邻，直至全色合并图像、第三合并图像和第四合并图像中的像素值均读取后，得到第二目标图像。
可选地，在第二分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一 像素值，根据第一滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值、根据包含第一颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值、以及根据第二滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，得到第一合并图像；根据第一滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，根据第二滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值，根据包含第二颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值，得到第二合并图像；基于第一合并图像中第九对角线方向上的多个全色像素合并读出的第五像素值、在第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值、在第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值、以及在第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，分别得到全色合并图像和第三合并图像；基于第二合并图像中，在第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值、在第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值、以及在第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，得到第四合并图像；基于全色合并图像、第三合并图像和第四合并图像，得到第二目标图像。
可选地，在第二分辨率模式下，根据滤光片组对应的多个全色像素合并读出的第一像素值，得到全色图像；根据滤光片组对应的多个第一颜色感光像素合并读出的第二像素值，以及滤光片组对应的多个第三颜色感光像素合并读出的第三像素值，得到第五合并图像；根据滤光片组对应的多个第二颜色感光像素合并读出的第四像素值，得到第六合并图像；基于全色图像、第五合并图像和第六合并图像，得到第三目标图像。
可选地，在第二分辨率模式下，根据滤光片组对应的多个全色像素合并读出的第一像素值，得到全色图像；根据第一滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值、根据包含第一颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值、以及根据第二滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，得到第五合并图像；根据第一滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，根据第二滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值，根据包含第二颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值，得到第六合并图像；基于全色图像、第五合并图像和第六合并图像，得到第三目标图像。
本实施例中，提供了支持两种分辨率输出的图像传感器结构，从而提供了两种分辨率的图像输出模式，能够适配不同的应用场景。
在对分辨率要求比较高的场景下使用第一分辨率模式，通过滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，以及彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，能够将全色通道信息融合到图像中，提高整体的进光量，使得能够生成信息更多、细节解析更清晰的第一合并图像。根据彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像，使得第二合并图像与第一合并图像的尺寸保持一致，从而保证各第二颜色感光像素在第二合并图像中的位置与各第一颜色感光像素在第一合并图像中的位置相同，即保证各第四像素值在第二合并图像中的位置与各第二像素值在第一合并图像中的位置相同。
遍历第一合并图像中的像素值，以将第二合并图像中的第四像素值调整至与位置相同的第二像素值相邻，使得第一颜色感光像素和第二颜色感光像素混合排列，提高了彩色分辨能力，使得所生成第一目标图像的每行和每列均有第一颜色感光像素、第二颜色感光像素和第三颜色感光像素存在，即目标图像的每行每列均具备有RGB像素，能够有效降低伪色的风险。
或者在第一分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，以及彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值、彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值、以及彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，使得合并读取所得到的第一合并图像中包含了所有颜色的像素，能够提高彩色通道的分辨能力。 根据彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值、彩色像素对应的多个第三颜色感光像素合并读出的第三像素值、以及彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像，使得根据第一合并图像和第二合并图像生成的第一目标图像中的各颜色的像素分布更均匀，有效提高图像的清晰度。
在夜景拍摄等分辨率要求较低的场景下使用第二分辨率模式，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，以及彩色滤光片对应的多个彩色像素合并读出的第二像素值，以及彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像，并将根据彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像，能够将第二颜色感光像素分离出来作为单独的图像并降低图像的分辨率。基于第一合并图像中在第九对角线方向上的多个全色像素合并读出的第五像素值得到全色合并图像，能够将全色像素从第一合并图像中分离出来的同时减小图像的尺寸，以降低功耗。基于第一合并图像中在第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值，以及在第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值，得到包含第一颜色感光像素和第三颜色感光像素的第三合并图像，多合一的像素读出方式使得生成的图像噪声少。基于第二合并图像中在第八对角线方向上的多个第二颜色感光像素合并读出的第八像素值，使得第四合并图像的尺寸与第三合并图像、全色合并图像的尺寸保持一致。遍历第三合并图像中的像素，在所遍历的像素值为第六像素值或第七像素值的情况下，从全色合并图像中读取与遍历的像素值对应的第五像素值，并将所读取的第五像素值调整至与遍历的像素值相邻，使得在图像中引入了全色像素，提高了进光量。在所遍历的像素值为第六像素值的情况下，从第四合并图像中读取与第六像素位置相同的第八像素值，并将所读取的第八像素值调整至与第六像素值相邻，使得第一颜色感光像素和第二颜色感光像素混合排列，提高了彩色分辨能力，使得所生成第二目标图像的每行和每列均有第一颜色感光像素、第二颜色感光像素和第二颜色感光像素存在，即目标图像的每行每列均具备有RGB像素，能够有效降低伪色的风险。并且，所得到第二目标图像的分辨率进一步减小，且全色像素具有更高的信噪比，图像的帧率高，从而达到了二级像素合并输出的功耗更低、信噪比更佳的图像处理效果。
或者，在第二分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，彩色滤光片对应的多个彩色像素合并读出的第二像素值，彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，以及彩色滤光片对应的多个第三颜色感光像素合并读出的第四像素值，得到第一合并图像，并将根据彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值、彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值、以及彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像，能够将各个彩色滤光片对应的第一颜色感光像素、第二颜色感光像素和第三颜色感光像素分离出来形成两张合并图像，使得两张合并图像中各颜色的感光像素在各自合并图像中的位置相同。
基于第一合并图像中在第九对角线方向上的多个全色像素合并读出的第五像素值得到全色合并图像，能够将全色像素从第一合并图像中分离出来的同时减小图像的尺寸，以降低功耗。基于第一合并图像中在第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值，以及在第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值，以及在第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，得到包含第一颜色感光像素、第二颜色感光像素和第三颜色感光像素的第三合并图像，使得RGB像素在图像中的分布更均匀。基于第二合并图像中，在第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值、在第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值、以及在第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，得到第四合并图像，使得第四合并图像的尺寸与第三合并图像、全色合并图像的尺寸保持一致。基于全色合并图像、第三合并图像和第四合并图像，能够将不同第一颜色感光像素、第二颜色感光像素和第三颜色感光像素进行混合排列，使得所生成的第二目标图像中RGB像素分布更均匀，图像质量更高。并且，所得到第二目标图像的分辨率进一步减小，且全色像素具有更高的信噪比，图像的帧率高，从而达到了二级像素合并输出的功耗更低、信噪比更佳的图像处理效果。
应该理解的是，虽然图10-图14的流程图中的各个操作按照箭头的指示依次显示，但是这些操作并不是必然按照箭头指示的顺序依次执行。除非本文中有明确的说明，这些操作的执行并没有严格的顺序限制，这些操作可以以其它的顺序执行。而且，图10-图14中的至少一部分操作可以包括多个子操作或者多个阶段，这些子操作或者阶段并不必然是在同一时刻执行完成，而是可以在不同的时刻执行，这些子操作或者阶段的执行顺序也不必然是依次进行，而是可以与其它操作或者其它操作的子操作或者阶段的至少一部分轮流或者交替地执行。
图15为一个实施例的图像生成装置装置的结构框图。如图15所示，图像生成装置1500应用于图像传感器，图像传感器包括滤光片阵列和像素阵列，滤光片阵列包括最小重复单元，最小重复单元包括多个滤光片组，每个滤光片组包括彩色滤光片和全色滤光片，全色滤光片透过的进光量大于彩色滤光片透过的进光量，彩色滤光片和全色滤光片均包括4个子滤光片，多个滤光片组包括第一滤光片组和第二滤光片组，第一滤光片组中至少一个彩色滤光片中包括第一颜色子滤光片和第二颜色子滤光片；第二滤光片组中彩色滤光片至少包括第三颜色子滤光片；像素阵列包括多个像素，像素阵列的第一颜色感光像素与滤光片阵列的第一颜色子滤光片对应设置，像素阵列的第二颜色感光像素与滤光片阵列的第二颜色子滤光片对应设置，像素阵列的第三颜色感光像素与滤光片阵列的第三颜色子滤光片对应设置，像素阵列被配置成用于接收穿过滤光片阵列的光线以生成电信号；
该图像生成装置1500包括：
第一合并模块1502，用于在第一分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，以及彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像。
第二合并模块1504，用于根据彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像。
图像生成模块1506，用于基于第一合并图像和第二合并图像，得到第一目标图像。
本实施例中，在第一分辨率模式下，通过滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，以及彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，能够将全色通道信息融合到图像中，提高整体的进光量，使得能够生成信息更多、细节解析更清晰的第一合并图像。根据彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像，并基于第一合并图像和第二合并图像，得到第一目标图像，能够将第一合并图像中的第一颜色感光像素和第二合并图像中的第二颜色感光像素混合排列，使得所生成的第一目标图像色彩更清晰。并且像素合并读出使得生成的第一目标图像尺寸减小、生成图像所需消耗的功耗低。
在一个实施例中，图像生成模块1506，还用于遍历第一合并图像中的像素值，在所遍历的像素值为第二像素值的情况下，从第二合并图像中读取与第二像素值位置相同的第四像素值，并将所读取的第四像素值调整至与第二像素值相邻，直至第一合并图像中的像素值均被遍历后，得到第一目标图像。
本实施例中，遍历第一合并图像中的像素值，在所遍历的像素值为第二像素值的情况下，从第二合并图像中读取与第二像素值位置相同的第四像素值，并将所读取的第四像素值调整至与第二像素值相邻，使得第一颜色感光像素和第二颜色感光像素混合排列，提高了彩色分辨能力，使得所生成第一目标图像的每行和每列均有第一颜色感光像素、第二颜色感光像素和第三颜色感光像素存在，即目标图像的每行每列均具备有RGB像素，能够有效降低伪色的风险。
在一个实施例中，第二滤光片组的彩色滤光片还包括第一颜色子滤光片或第二颜色子滤光片；第一合并模块1502，还用于在第一分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，根据第一滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值、根据包含第一颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值、以及根据第二滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，得到第一合并图像；
第二合并模块1504，还用于根据第一滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素 合并读出的第四像素值，根据第二滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值，根据包含第二颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值，得到第二合并图像。
本实施例中，第二滤光片组232的彩色滤光片234中包括第三颜色子滤光片2343，还包括第一颜色子滤光片2341或第二颜色子滤光片2342，在第一分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，根据第一滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值、根据包含第一颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值、以及根据第二滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，使得合并读取所得到的第一合并图像中包含了所有颜色的像素，能够提高彩色通道的分辨能力。根据彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值、彩色像素对应的多个第三颜色感光像素合并读出的第三像素值、以及彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像，使得根据第一合并图像和第二合并图像生成的第一目标图像中的各颜色的像素分布更均匀，有效提高图像的清晰度。
在一个实施例中，该装置还包括第三合并模块；
第一合并模块1502，还用于在第二分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，以及彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像；第二分辨率模式对应的分辨率小于第一分辨率模式对应的分辨率；
第二合并模块1504，还用于根据彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像；
第三合并模块，用于基于第一合并图像中第九对角线方向上的多个全色像素合并读出的第五像素值，得到全色合并图像；基于第一合并图像中第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值，以及第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值，得到第三合并图像，第九对角线方向不同于第十对角线方向；
第三合并模块，还用于基于第二合并图像中第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，得到第四合并图像；
图像生成模块1506，还用于基于全色合并图像、第三合并图像和第四合并图像，得到第二目标图像。
本实施例中，在第二分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，以及彩色滤光片对应的多个彩色像素合并读出的第二像素值，以及彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像，并将根据彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像，能够将第二颜色感光像素分离出来作为单独的图像并降低图像的分辨率。基于第一合并图像中在第九对角线方向上的多个全色像素合并读出的第五像素值得到全色合并图像，能够将全色像素从第一合并图像中分离出来的同时减小图像的尺寸，以降低功耗。基于第一合并图像中在第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值，以及在第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值，得到包含第一颜色感光像素和第三颜色感光像素的第三合并图像，多合一的像素读出方式使得生成的图像噪声少。基于第二合并图像中在第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，使得第四合并图像的尺寸与第三合并图像、全色合并图像的尺寸保持一致。基于全色合并图像、第三合并图像和第四合并图像，能够将不同第一颜色感光像素和第二颜色感光像素进行混合排列，使得所生成的第二目标图像中RGB像素分布更均匀，图像质量更高。并且，所得到第二目标图像的分辨率进一步减小，且全色像素具有更高的信噪比，图像的帧率高，从而达到了二级像素合并输出的功耗更低、信噪比更佳的图像处理效果。
在一个实施例中，该图像生成模块1506，还用于遍历第三合并图像中的像素；在所遍历的像素值为第六像素值或第七像素值的情况下，从全色合并图像中读取与遍历的像素值对应的第五像素值，并将所读取的第五像素值调整至与遍历的像素值相邻；在所遍历的像素值为第六像素值的情况下，从第四合 并图像中读取与第六像素位置相同的第八像素值，并将所读取的第八像素值调整至与第六像素值相邻，直至全色合并图像、第三合并图像和第四合并图像中的像素值均读取后，得到第二目标图像。
本实施例中，遍历第三合并图像中的像素，在所遍历的像素值为第六像素值或第七像素值的情况下，从全色合并图像中读取与遍历的像素值对应的第五像素值，并将所读取的第五像素值调整至与遍历的像素值相邻，使得在图像中引入了全色像素，提高了进光量。在所遍历的像素值为第六像素值的情况下，从第四合并图像中读取与第六像素位置相同的第八像素值，并将所读取的第八像素值调整至与第六像素值相邻，使得第一颜色感光像素和第二颜色感光像素混合排列，提高了彩色分辨能力，使得所生成第二目标图像的每行和每列均有第一颜色感光像素、第二颜色感光像素和第二颜色感光像素存在，即目标图像的每行每列均具备有RGB像素，能够有效降低伪色的风险。
在一个实施例中，第二滤光片组的彩色滤光片还包括第一颜色子滤光片或第二颜色子滤光片；第一合并模块1502，还用于在第二分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，根据第一滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值、根据包含第一颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值、以及根据第二滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，得到第一合并图像；
第二合并模块1504，还用于根据第一滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，根据第二滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值，根据包含第二颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值，得到第二合并图像；
第三合并模块，还用于基于第一合并图像中第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值、在第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值、以及在第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，第三合并图像；
第三合并模块，还用于基于第二合并图像中，在第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值、在第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值、以及在第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，得到第四合并图像。
本实施例中，在第二分辨率模式下，根据滤光片组中的全色滤光片对应的多个全色像素合并读出的第一像素值，根据第一滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值，根据包含第一颜色子滤光片的第二滤光片组中第三颜色感光像素合并读出的第三像素值，以及根据第二滤光片组中多个第二颜色子滤光片对应的第三颜色感光像素合并读出的第四像素值，得到第一合并图像，并将根据第一滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，根据第二滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值，根据包含第二颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值，得到第二合并图像，能够将各个彩色滤光片对应的第一颜色感光像素、第二颜色感光像素和第三颜色感光像素分离出来形成两张合并图像，使得两张合并图像中各颜色的感光像素在各自合并图像中的位置相同。
基于第一合并图像中在第九对角线方向上的多个全色像素合并读出的第五像素值得到全色合并图像，能够将全色像素从第一合并图像中分离出来的同时减小图像的尺寸，以降低功耗。
基于第一合并图像中在第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值，以及在第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值，以及在第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，得到包含第一颜色感光像素、第二颜色感光像素和第三颜色感光像素的第三合并图像，使得RGB像素在图像中的分布更均匀。基于第二合并图像中，在第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值、在第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值、以及在第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，得到第四合并图像，使得第四合并图像的尺寸与第三合并图像、全色合并图像的尺寸保持一致。基于全色合并图像、第三合并图像和第四合并图像，能够将不同第一颜色感光像素、第二颜色感光像素和第三颜 色感光像素进行混合排列，使得所生成的第二目标图像中RGB像素分布更均匀，图像质量更高。并且，所得到第二目标图像的分辨率进一步减小，且全色像素具有更高的信噪比，图像的帧率高，从而达到了二级像素合并输出的功耗更低、信噪比更佳的图像处理效果。
在一个实施例中，第一合并模块1502，还用于在第二分辨率模式下，根据滤光片组对应的多个全色像素合并读出的第一像素值，得到全色图像；根据滤光片组对应的多个第一颜色感光像素合并读出的第二像素值，以及滤光片组对应的多个第三颜色感光像素合并读出的第三像素值，得到第五合并图像；
第二合并模块1504，还用于根据滤光片组对应的多个第二颜色感光像素合并读出的第四像素值，得到第六合并图像；
图像生成模块1506，还用于基于全色图像、第五合并图像和第六合并图像，得到第三目标图像。
本实施例中，在第二分辨率模式下，通过滤光片组对应的多个全色像素合并读出的第一像素值，能够分离出全色图像；根据滤光片组对应的多个第一颜色感光像素合并读出的第二像素值，以及滤光片组对应的多个第三颜色感光像素合并读出的第三像素值，能够将第一颜色感光像素和第三颜色感光像素分离出来形成第五合并图像，能够快速有效降低图像尺寸。根据滤光片组对应的多个第二颜色感光像素合并读出的第四像素值，能够将第二颜色感光像素分离出来形成第六合并图像，使得第二颜色感光像素在第六合并图像和第一颜色感光像素在第五合并图像中的坐标位置保持一致。基于全色图像、第五合并图像和第六合并图像得到第三目标图像，能够将全色通道信息融合到图像中，提高整体的进光量，使得能够生成信息更多、细节解析更清晰的第三目标图像。并且通过滤光片组对应的多个像素合并读出使得生成的第三目标图像尺寸减小、生成图像所需消耗的功耗低。
在一个实施例中，第二滤光片组的彩色滤光片还包括第一颜色子滤光片或第二颜色子滤光片；第一合并模块1502，还用于根据第一滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值、根据包含第一颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值、以及根据第二滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，得到第五合并图像；
第二合并模块1504，还用于根据第一滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，根据第二滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值，根据包含第二颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值，得到第六合并图像。
本实施例中，在第二分辨率模式下，通过根据第一滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值、根据包含第一颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值、以及根据第二滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，能够将部分第一颜色感光像素、第三颜色感光像素和第二颜色感光像素分离出来形成第五合并图像，能够通过像素合并减小图像尺寸，以减少处理功耗。根据第一滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值、第二滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值，根据包含第二颜色子滤光片的第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值，能够将部分第一颜色感光像素、第三颜色感光像素和第二颜色感光像素分离出来出来形成第六合并图像，使得第二颜色感光像素在第六合并图像和第一颜色感光像素在第五合并图像中的坐标位置保持一致。基于全色图像、第五合并图像和第六合并图像得到第三目标图像，能够将全色通道信息融合到图像中，提高整体的进光量，使得能够生成信息更多、细节解析更清晰的第三目标图像。并且通过滤光片组对应的多个像素合并读出使得生成的第三目标图像尺寸减小、生成图像所需消耗的功耗低。
上述图像生成装置中各个模块的划分仅用于举例说明，在其他实施例中，可将图像生成装置按照需要划分为不同的模块，以完成上述图像生成装置的全部或部分功能。
图16为一个实施例中电子设备的内部结构示意图。如图16所示，该电子设备包括通过系统总线连接的处理器和存储器。其中，该处理器用于提供计算和控制能力，支撑整个电子设备的运行。存储器可包括非易失性存储介质及内存储器。非易失性存储介质存储有操作系统和计算机程序。该计算机程序可 被处理器所执行，以用于实现以下各个实施例所提供的一种图像生成方法。内存储器为非易失性存储介质中的操作系统计算机程序提供高速缓存的运行环境。该电子设备可以是手机、平板电脑或者个人数字助理或穿戴式设备等。
本申请实施例中提供的图像生成装置中的各个模块的实现可为计算机程序的形式。该计算机程序可在终端或服务器上运行。该计算机程序构成的程序模块可存储在终端或服务器的存储器上。该计算机程序被处理器执行时，实现本申请实施例中所描述方法的操作。
本申请实施例还提供了一种计算机可读存储介质。一个或多个包含计算机可执行指令的非易失性计算机可读存储介质，当计算机可执行指令被一个或多个处理器执行时，使得处理器执行图像生成方法的操作。
一种包含指令的计算机程序产品，当其在计算机上运行时，使得计算机执行图像生成方法。
本申请实施例所使用的对存储器、存储、数据库或其它介质的任何引用可包括非易失性和/或易失性存储器。合适的非易失性存储器可包括只读存储器(ROM)、可编程ROM(PROM)、电可编程ROM(EPROM)、电可擦除可编程ROM(EEPROM)或闪存。易失性存储器可包括随机存取存储器(RAM)，它用作外部高速缓冲存储器。作为说明而非局限，RAM以多种形式可得，诸如静态RAM(SRAM)、动态RAM(DRAM)、同步DRAM(SDRAM)、双数据率SDRAM(DDR SDRAM)、增强型SDRAM(ESDRAM)、同步链路(Synchlink)DRAM(SLDRAM)、存储器总线(Rambus)直接RAM(RDRAM)、直接存储器总线动态RAM(DRDRAM)、以及存储器总线动态RAM(RDRAM)。
以上所述实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。

Claims (27)

  1. 一种图像传感器，其特征在于，所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，所述最小重复单元包括多个滤光片组，每个滤光片组包括彩色滤光片和全色滤光片，所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量，所述彩色滤光片和所述全色滤光片均包括4个子滤光片，所述多个滤光片组至少包括第一滤光片组，所述第一滤光片组中至少一个所述彩色滤光片包括第一颜色子滤光片和第二颜色子滤光片；所述像素阵列包括多个像素，所述像素阵列的像素与所述滤光片阵列的子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号。
  2. 根据权利要求1所述的图像传感器，其特征在于，所述多个滤光片组还包括第二滤光片组，所述第二滤光片组中所述彩色滤光片至少包括第三颜色子滤光片。
  3. 根据权利要求2所述的图像传感器，其特征在于，所述最小重复单元包括4个滤光片组，所述4个滤光片组包括2个所述第一滤光片组和2个所述第二滤光片组，所述2个所述第一滤光片组和2个所述第二滤光片组呈矩阵排列。
  4. 根据权利要求3所述的图像传感器，其特征在于，所述最小重复单元中所述第一滤光片组设置在第一对角线方向，所述第二滤光片组设置在第二对角线方向，所述第一对角线方向与所述第二对角线方向不同。
  5. 根据权利要求1至4中任一项所述的图像传感器，其特征在于，在每个所述滤光片组中，所述彩色滤光片和全色滤光片呈矩阵排列，所述全色滤光片设置在第三对角线方向，所述彩色滤光片设置在第四对角线方向，所述第三对角线方向与所述第四对角线方向不同。
  6. 根据权利要求1所述的图像传感器，其特征在于，所述第一颜色子滤光片设置在第五对角线方向或第六对角线方向的一个方向上，所述第二颜色子滤光片设置在所述第五对角线方向或所述第六对角线方向的另一个方向上。
  7. 根据权利要求2所述的图像传感器，其特征在于，所述第三颜色子滤光片设置在第七对角线方向和第八对角线方向上，所述第七对角线方向与所述第八对角线方向不同。
  8. 根据权利要求2所述的图像传感器，其特征在于，所述滤光片组中包含2个全色滤光片和2个彩色滤光片，所述最小重复单元为8行8列64个所述子滤光片，排布方式为：
    Figure PCTCN2022114371-appb-100001
    其中，w表示全色子滤光片，a、b和c均表示彩色子滤光片。
  9. 根据权利要求2所述的图像传感器，其特征在于，所述滤光片组中包含2个全色滤光片和2个彩色滤光片，所述最小重复单元为8行8列64个所述子滤光片，排布方式为：
    Figure PCTCN2022114371-appb-100002
    其中，w表示全色子滤光片，a、b和c均表示彩色子滤光片。
  10. 根据权利要求2所述的图像传感器，其特征在于，所述第二滤光片组中所述彩色滤光片还包括第一颜色子滤光片或第二颜色子滤光片。
  11. 根据权利要求10所述的图像传感器，其特征在于，所述第二滤光片组包括两个彩色滤光片，每个彩色滤光片中的同一颜色的子滤光片呈对角排布，同一颜色的子滤光片在两个彩色滤光片中的排布方向相反。
  12. 根据权利要求10所述的图像传感器，其特征在于，所述滤光片组中包含2个全色滤光片和2个彩色滤光片，所述最小重复单元为8行8列64个所述子滤光片，排布方式为：
    Figure PCTCN2022114371-appb-100003
    其中，w表示全色子滤光片，a、b和c均表示彩色子滤光片。
  13. 根据权利要求10所述的图像传感器，其特征在于，所述滤光片组中包含2个全色滤光片和2个彩色滤光片，所述最小重复单元为8行8列64个所述子滤光片，排布方式为：
    Figure PCTCN2022114371-appb-100004
    其中，w表示全色子滤光片，a、b和c均表示彩色子滤光片。
  14. 一种摄像模组，其特征在于，所述摄像模组包括镜头和权利要求1-13中任一项所述的图像传感器；所述图像传感器用于接收穿过所述镜头的光线，所述像素根据所述光线生成电信号。
  15. 一种电子设备，其特征在于，所述电子设备包括权利要求14所述的摄像模组及壳体，所述摄像模组设置在所述壳体上。
  16. 一种图像生成方法，应用于图像传感器，其特征在于，所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，所述最小重复单元包括多个滤光片组，每个滤光片组包括彩色滤光片和全色滤光片，所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量，所述彩色滤光片和所述全色滤光片均包括4个子滤光片，所述多个滤光片组包括第一滤光片组和第二滤光片组，所述第一滤光片组中至少一个所述彩色滤光片包括第一颜色子滤光片和第二颜色子滤光片；所述第二滤光片组中所述彩色滤光片至少包括第三颜色子滤光片；所述像素阵列包括多个像素，所述像素阵列的第一颜色感光像素与所述滤光片阵列的第一颜色子滤光片对应设置，所述像素阵列的第二颜色感光像素与所述滤光片阵列的第二颜色子滤光片对应设置，所述像素阵列的第三颜色感光像素与所述滤光片阵列的第三颜色子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号；
    所述方法包括：
    在第一分辨率模式下，根据所述滤光片组中的所述全色滤光片对应的多个全色像素合并读出的第一像素值，以及所述彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及所述彩色 滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像；
    根据所述彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像；及
    基于所述第一合并图像和所述第二合并图像，得到第一目标图像。
  17. 根据权利要求16所述的方法，其特征在于，所述基于所述第一合并图像和所述第二合并图像，得到第一目标图像，包括：
    遍历所述第一合并图像中的像素值，在所遍历的像素值为第二像素值的情况下，从所述第二合并图像中读取与所述第二像素值位置相同的第四像素值，并将所读取的第四像素值调整至与所述第二像素值相邻，直至所述第一合并图像中的像素值均被遍历后，得到第一目标图像。
  18. 根据权利要求16所述的方法，其特征在于，所述第二滤光片组的彩色滤光片还包括第一颜色子滤光片或第二颜色子滤光片；所述在第一分辨率模式下，根据所述滤光片组中的所述全色滤光片对应的多个全色像素合并读出的第一像素值，以及所述彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及所述彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像，包括：
    在第一分辨率模式下，根据所述滤光片组中的所述全色滤光片对应的多个全色像素合并读出的第一像素值，根据所述第一滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值、根据包含所述第一颜色子滤光片的所述第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值、以及根据所述第二滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，得到第一合并图像；及
    所述根据所述彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像，包括：
    根据所述第一滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，根据所述第二滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值，根据包含所述第二颜色子滤光片的所述第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值，得到第二合并图像。
  19. 根据权利要求16所述的方法，其特征在于，所述方法还包括：
    在第二分辨率模式下，根据所述滤光片组中的所述全色滤光片对应的多个全色像素合并读出的第一像素值，以及所述彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及所述彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像；所述第二分辨率模式对应的分辨率小于所述第一分辨率模式对应的分辨率；
    根据所述彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像；
    基于所述第一合并图像中第九对角线方向上的多个全色像素合并读出的第五像素值，得到全色合并图像；
    基于所述第一合并图像中第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值，以及所述第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值，得到第三合并图像，所述第九对角线方向不同于所述第十对角线方向；
    基于所述第二合并图像中第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，得到第四合并图像；及
    基于所述全色合并图像、所述第三合并图像和所述第四合并图像，得到第二目标图像。
  20. 根据权利要求19所述的方法，其特征在于，所述基于所述全色合并图像、所述第三合并图像和所述第四合并图像，得到第二目标图像，包括：
    遍历所述第三合并图像中的像素；
    在所遍历的像素值为第六像素值或第七像素值的情况下，从所述全色合并图像中读取与所述遍历的像素值对应的第五像素值，并将所读取的第五像素值调整至与所述遍历的像素值相邻；及
    在所遍历的像素值为第六像素值的情况下，从所述第四合并图像中读取与所述第六像素位置相同 的第八像素值，并将所读取的第八像素值调整至与所述第六像素值相邻，直至所述全色合并图像、所述第三合并图像和所述第四合并图像中的像素值均读取后，得到第二目标图像。
  21. 根据权利要求19所述的方法，其特征在于，所述第二滤光片组的彩色滤光片还包括第一颜色子滤光片或第二颜色子滤光片；所述在第二分辨率模式下，根据所述滤光片组中的所述全色滤光片对应的多个全色像素合并读出的第一像素值，以及所述彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及所述彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像，包括：
    在第二分辨率模式下，根据所述滤光片组中的所述全色滤光片对应的多个全色像素合并读出的第一像素值，根据所述第一滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值、根据包含所述第一颜色子滤光片的所述第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值、以及根据所述第二滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，得到第一合并图像；
    所述根据所述彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像，包括：
    根据所述第一滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，根据所述第二滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值，根据包含所述第二颜色子滤光片的所述第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值，得到第二合并图像；
    所述基于所述第一合并图像中第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值，以及在所述第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值，得到第三合并图像，包括：
    基于所述第一合并图像中第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值、在所述第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值、以及在所述第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，第三合并图像；及
    所述基于所述第二合并图像中第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，得到第四合并图像，包括：
    基于所述第二合并图像中，在所述第十对角线方向上的多个第一颜色感光像素合并读出的第六像素值、在所述第十对角线方向上的多个第三颜色感光像素合并读出的第七像素值、以及在所述第十对角线方向上的多个第二颜色感光像素合并读出的第八像素值，得到第四合并图像。
  22. 根据权利要求16所述的方法，其特征在于，所述方法还包括：
    在第二分辨率模式下，根据所述滤光片组对应的多个全色像素合并读出的第一像素值，得到全色图像；
    根据所述滤光片组对应的多个第一颜色感光像素合并读出的第二像素值，以及所述滤光片组对应的多个第三颜色感光像素合并读出的第三像素值，得到第五合并图像；
    根据所述滤光片组对应的多个第二颜色感光像素合并读出的第四像素值，得到第六合并图像；及
    基于所述全色图像、所述第五合并图像和所述第六合并图像，得到第三目标图像。
  23. 根据权利要求22所述的方法，其特征在于，所述第二滤光片组的彩色滤光片还包括第一颜色子滤光片或第二颜色子滤光片；所述根据所述滤光片组对应的多个第一颜色感光像素合并读出的第二像素值，以及所述滤光片组对应的多个第三颜色感光像素合并读出的第三像素值，得到第五合并图像，包括：
    根据所述第一滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值、根据包含所述第一颜色子滤光片的所述第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值、以及根据所述第二滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，得到第五合并图像；及
    所述根据所述滤光片组对应的多个第二颜色感光像素合并读出的第四像素值，得到第六合并图像， 包括：
    根据所述第一滤光片组中多个第二颜色子滤光片对应的第二颜色感光像素合并读出的第四像素值，根据所述第二滤光片组中多个第一颜色子滤光片对应的第一颜色感光像素合并读出的第二像素值，根据包含所述第二颜色子滤光片的所述第二滤光片组中多个第三颜色子滤光片对应的第三颜色感光像素合并读出的第三像素值，得到第六合并图像。
  24. 一种图像生成装置，应用于图像传感器，其特征在于，所述图像传感器包括滤光片阵列和像素阵列，所述滤光片阵列包括最小重复单元，所述最小重复单元包括多个滤光片组，每个滤光片组包括彩色滤光片和全色滤光片，所述全色滤光片透过的进光量大于所述彩色滤光片透过的进光量，所述彩色滤光片和所述全色滤光片均包括4个子滤光片，所述多个滤光片组包括第一滤光片组和第二滤光片组，所述第一滤光片组中至少一个所述彩色滤光片中包括第一颜色子滤光片和第二颜色子滤光片；所述第二滤光片组中所述彩色滤光片至少包括第三颜色子滤光片；所述像素阵列包括多个像素，所述像素阵列的第一颜色感光像素与所述滤光片阵列的第一颜色子滤光片对应设置，所述像素阵列的第二颜色感光像素与所述滤光片阵列的第二颜色子滤光片对应设置，所述像素阵列的第三颜色感光像素与所述滤光片阵列的第三颜色子滤光片对应设置，所述像素阵列被配置成用于接收穿过所述滤光片阵列的光线以生成电信号；
    所述装置包括：
    第一合并模块，用于在第一分辨率模式下，根据所述滤光片组中的所述全色滤光片对应的多个全色像素合并读出的第一像素值，以及所述彩色滤光片对应的多个第一颜色感光像素合并读出的第二像素值，以及所述彩色滤光片对应的多个第三颜色感光像素合并读出的第三像素值，得到第一合并图像；
    第二合并模块，用于根据所述彩色滤光片对应的多个第二颜色感光像素合并读出的第四像素值，得到第二合并图像；及
    图像生成模块，用于基于所述第一合并图像和所述第二合并图像，得到第一目标图像。
  25. 一种电子设备，包括存储器、处理器及图像传感器，所述存储器中储存有计算机程序，其特征在于，所述计算机程序被所述处理器执行时，使得所述处理器执行如权利要求16至23中任一项所述的方法的步骤。
  26. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求16至23中任一项所述的方法的步骤。
  27. 一种计算机程序产品，包括计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求16至23中任一项所述的方法的步骤。
PCT/CN2022/114371 2021-11-12 2022-08-24 图像传感器、摄像模组、电子设备、图像生成方法和装置 WO2023082766A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111339984.2A CN114125318A (zh) 2021-11-12 2021-11-12 图像传感器、摄像模组、电子设备、图像生成方法和装置
CN202111339984.2 2021-11-12

Publications (1)

Publication Number Publication Date
WO2023082766A1 true WO2023082766A1 (zh) 2023-05-19

Family

ID=80379203

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/114371 WO2023082766A1 (zh) 2021-11-12 2022-08-24 图像传感器、摄像模组、电子设备、图像生成方法和装置

Country Status (2)

Country Link
CN (1) CN114125318A (zh)
WO (1) WO2023082766A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114125318A (zh) * 2021-11-12 2022-03-01 Oppo广东移动通信有限公司 图像传感器、摄像模组、电子设备、图像生成方法和装置
CN114554046A (zh) * 2021-12-01 2022-05-27 Oppo广东移动通信有限公司 图像传感器、摄像模组、电子设备、图像生成方法和装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080130991A1 (en) * 2006-11-30 2008-06-05 O'brien Michele Processing images having color and panchromatic pixels
CN102461175A (zh) * 2009-06-09 2012-05-16 全视科技有限公司 用于四通道彩色滤光片阵列的内插
CN111629140A (zh) * 2020-05-15 2020-09-04 深圳市汇顶科技股份有限公司 图像传感器和电子设备
CN112118378A (zh) * 2020-10-09 2020-12-22 Oppo广东移动通信有限公司 图像获取方法及装置、终端和计算机可读存储介质
WO2021174529A1 (zh) * 2020-03-06 2021-09-10 Oppo广东移动通信有限公司 图像传感器、成像装置、电子设备、图像处理系统及信号处理方法
CN113573030A (zh) * 2021-07-01 2021-10-29 Oppo广东移动通信有限公司 图像生成方法、装置、电子设备和计算机可读存储介质
CN114125318A (zh) * 2021-11-12 2022-03-01 Oppo广东移动通信有限公司 图像传感器、摄像模组、电子设备、图像生成方法和装置

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101262507B1 (ko) * 2011-04-11 2013-05-08 엘지이노텍 주식회사 픽셀, 픽셀 어레이, 픽셀 어레이의 제조방법 및 픽셀 어레이를 포함하는 이미지센서
CN213279832U (zh) * 2020-10-09 2021-05-25 Oppo广东移动通信有限公司 图像传感器、相机和终端
CN113556519A (zh) * 2021-07-01 2021-10-26 Oppo广东移动通信有限公司 图像处理方法、电子设备和非易失性计算机可读存储介质

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080130991A1 (en) * 2006-11-30 2008-06-05 O'brien Michele Processing images having color and panchromatic pixels
CN102461175A (zh) * 2009-06-09 2012-05-16 全视科技有限公司 用于四通道彩色滤光片阵列的内插
WO2021174529A1 (zh) * 2020-03-06 2021-09-10 Oppo广东移动通信有限公司 图像传感器、成像装置、电子设备、图像处理系统及信号处理方法
CN111629140A (zh) * 2020-05-15 2020-09-04 深圳市汇顶科技股份有限公司 图像传感器和电子设备
CN111756973A (zh) * 2020-05-15 2020-10-09 深圳市汇顶科技股份有限公司 图像传感器和电子设备
CN112118378A (zh) * 2020-10-09 2020-12-22 Oppo广东移动通信有限公司 图像获取方法及装置、终端和计算机可读存储介质
CN113573030A (zh) * 2021-07-01 2021-10-29 Oppo广东移动通信有限公司 图像生成方法、装置、电子设备和计算机可读存储介质
CN114125318A (zh) * 2021-11-12 2022-03-01 Oppo广东移动通信有限公司 图像传感器、摄像模组、电子设备、图像生成方法和装置

Also Published As

Publication number Publication date
CN114125318A (zh) 2022-03-01

Similar Documents

Publication Publication Date Title
US8355074B2 (en) Exposing pixel groups in producing digital images
US10136107B2 (en) Imaging systems with visible light sensitive pixels and infrared light sensitive pixels
WO2023082766A1 (zh) 图像传感器、摄像模组、电子设备、图像生成方法和装置
EP2087725B1 (en) Improved light sensitivity in image sensors
US7855740B2 (en) Multiple component readout of image sensor
WO2021196553A1 (zh) 高动态范围图像处理系统及方法、电子设备和可读存储介质
CN113840067B (zh) 图像传感器、图像生成方法、装置和电子设备
WO2023098284A1 (zh) 图像传感器、摄像模组、电子设备、图像生成方法和装置
WO2023124607A1 (zh) 图像生成方法、装置、电子设备和计算机可读存储介质
WO2023098282A1 (zh) 图像传感器、摄像模组、电子设备、图像生成方法和装置
WO2023109264A1 (zh) 图像传感器、摄像模组、电子设备、图像生成方法和装置
WO2023109265A1 (zh) 图像传感器、摄像模组、电子设备、图像生成方法和装置
CN113170061B (zh) 图像传感器、成像装置、电子设备、图像处理系统及信号处理方法
CN113573030B (zh) 图像生成方法、装置、电子设备和计算机可读存储介质
CN113676708B (zh) 图像生成方法、装置、电子设备和计算机可读存储介质
EP2502422A1 (en) Sparse color pixel array with pixel substitutes
US20230007191A1 (en) Image sensor, imaging apparatus, electronic device, image processing system, and signal processing method
US8848073B2 (en) Solid-state imaging device and electronic apparatus
WO2023098230A1 (zh) 图像传感器、摄像模组、电子设备、图像生成方法和装置