WO2023098282A1 - Capteur d'image, module de caméra, dispositif électronique et procédé et appareil de génération d'image - Google Patents

Capteur d'image, module de caméra, dispositif électronique et procédé et appareil de génération d'image Download PDF

Info

Publication number
WO2023098282A1
WO2023098282A1 PCT/CN2022/124016 CN2022124016W WO2023098282A1 WO 2023098282 A1 WO2023098282 A1 WO 2023098282A1 CN 2022124016 W CN2022124016 W CN 2022124016W WO 2023098282 A1 WO2023098282 A1 WO 2023098282A1
Authority
WO
WIPO (PCT)
Prior art keywords
color
filter
panchromatic
image
pixel value
Prior art date
Application number
PCT/CN2022/124016
Other languages
English (en)
Chinese (zh)
Inventor
李小涛
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2023098282A1 publication Critical patent/WO2023098282A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/54—Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/57—Mechanical or electrical details of cameras or camera modules specially adapted for being embedded in other devices
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled

Definitions

  • the present application relates to the field of computer technology, in particular to an image sensor, camera module, electronic equipment, image generation method, device, computer-readable storage medium and computer program product.
  • An image sensor is arranged in the camera, and a color image is collected by the image sensor.
  • an optical filter array arranged in the form of a Bayer (Bayer) array is usually arranged in the image sensor, so that multiple pixels in the image sensor can receive light passing through the corresponding optical filter, thereby Generate pixel signals with different color channels to generate an image.
  • Various embodiments according to the present application provide an image sensor, a camera module, an electronic device, an image generating method, an apparatus, a computer-readable storage medium, and a computer program product.
  • An image sensor the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes at least a first filter set and a second filter set; Both the first filter group and the second filter group include a panchromatic filter and a color filter; the color filters in the first filter group include a first color filter sheet and a third color filter, the color filter in the second filter group includes a second color filter; the panchromatic filter and the color filter in the minimum repeating unit are in the Each row and each column of the smallest repeating unit are alternately arranged, and the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter; each of the panchromatic filters includes N Rows and N columns of panchromatic sub-filters, each of the color filters includes N rows and N columns of color sub-filters, and the N rows and N columns of color sub-filters are connected with the color filter The colors are the same, and the N is a positive integer; each pixel in the pixel array
  • a camera module the camera module includes a lens and the above-mentioned image sensor; the image sensor is used to receive light passing through the lens, and the pixels generate electrical signals according to the light.
  • An electronic device comprising:
  • a casing, the camera module is arranged on the casing.
  • the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes at least a first filter set and a second filter set;
  • Both the first filter group and the second filter group include a panchromatic filter and a color filter;
  • the color filters in the first filter group include a first color filter and a third color filter light sheet,
  • the color filter in the second filter group includes a second color filter, the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter, and can pass through the full color when shooting.
  • the color filter can get more light, so that there is no need to adjust the shooting parameters, and it can improve the definition of imaging in dark light without affecting the stability of shooting.
  • both stability and clarity can be taken into account, and the stability and clarity of imaging in dark light are both high.
  • Panchromatic filters and color filters are alternately arranged in each row and column in the smallest repeating unit, and each panchromatic filter includes N rows and N columns of panchromatic sub-filters, and each color filter
  • the chip includes color sub-filters in N rows and N columns, and the color sub-filters in N rows and N columns have the same color as the color filter, and N is a positive integer, and each pixel in the pixel array is the same as the sub-filter of the filter array.
  • Corresponding settings of the optical filters that is, each row and each column in the pixel array includes color pixels of each color, can improve the color resolution of each row and column of imaging, and make the color of imaging richer.
  • panchromatic filters and color filters of various colors are dispersedly arranged, so that the panchromatic pixel values and the color pixel values of various colors are also dispersedly arranged in the imaging, which can improve the color resolution capability and brightness change resolution capabilities.
  • the color filters of various colors are mixed and arranged, so that the color pixel values of various colors in the imaging are also mixed and arranged, which can reduce the risk of false colors.
  • An image generation method applied to an image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes at least a first filter group and a second filter group Two filter groups; the first filter group and the second filter group all include panchromatic filters and color filters; the color filters in the first filter group include the first A color filter and a third color filter, the color filters in the second filter group include a second color filter; the panchromatic filter and the color filter in the minimum repeating unit
  • the slices are alternately arranged on each row and each column of the minimum repeating unit, and the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter; each of the panchromatic filters
  • the light sheet includes N rows and N columns of panchromatic sub-filters, each of the color filters includes N rows and N columns of color sub-filters, and the N rows and N columns of color sub-filters are connected to the The colors of the color filters are the same, and the N is a positive integer greater
  • the methods include:
  • the panchromatic pixel corresponding to each panchromatic sub-filter in the panchromatic filter is read out the full-resolution panchromatic pixel value, and each of the color filters is The color pixel corresponding to the color sub-filter reads out the full-resolution color pixel value;
  • a full resolution target image is generated based on each of said full resolution panchromatic pixel values and each of said full resolution color pixel values.
  • An image generating device applied to an image sensor, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes at least a first filter group and a second filter group Two filter groups; the first filter group and the second filter group include panchromatic filters and color filters; the color filters in the first filter group include the first A color filter and a third color filter, the color filters in the second filter group include a second color filter; the panchromatic filter and the color filter in the minimum repeating unit Alternately arranged on each row and each column of the minimum repeating unit, the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter; each of the panchromatic filters
  • the sheet includes N rows and N columns of panchromatic sub-filters, and each of the color filters includes N rows and N columns of color sub-filters, and the N rows and N columns of color sub-filters are connected to the color sub-filters.
  • each pixel in the pixel array corresponds to the sub-filter of the filter array, and the pixel array is configured for receiving light passing through the filter array to generate an electrical signal;
  • the devices include:
  • the readout module is used to read out the full-resolution panchromatic pixel value from the panchromatic pixel corresponding to each panchromatic sub-filter in the panchromatic filter in the full-resolution mode, and read the full-resolution panchromatic pixel value of the color
  • the color pixel corresponding to each color sub-filter in the filter reads out the full-resolution color pixel value;
  • An image generating module configured to generate a full-resolution target image based on each of the full-resolution panchromatic pixels and each of the full-resolution color pixels.
  • An electronic device includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is made to execute the operation of the above-mentioned image generation method.
  • a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the operation of the above-mentioned method is realized.
  • a computer program product includes a computer program, and when the computer program is executed by a processor, the operations of the above-mentioned method are realized.
  • the panchromatic pixels corresponding to each panchromatic sub-filter in the panchromatic filter are read out in full Resolution panchromatic pixel value, and read out the full resolution color pixel value from the color pixel corresponding to each color sub-filter in the color filter; and the amount of light transmitted by the panchromatic filter is greater than that of the color filter
  • the amount of transmitted light can integrate the panchromatic channel information into the image to increase the overall light input, so that based on each full-resolution panchromatic pixel value and each full-resolution color pixel value, more information and detailed analysis can be generated. Sharper full-resolution target images.
  • panchromatic filters and color filters of various colors are scattered and arranged, so that full-resolution panchromatic pixel values and full-resolution color pixel values of various colors are also scattered and arranged in imaging, It can improve the color resolution capability and brightness change resolution capability.
  • the color filters of various colors are mixed and arranged, so that the full-resolution color pixel values of various colors in the imaging are also mixed and arranged, which can reduce the risk of false colors.
  • each panchromatic filter and the color filter are alternately arranged on each row and each column, and each panchromatic filter includes N rows and N columns of panchromatic sub-filters, and each color filter includes N rows and N columns of color sub-filters, N rows and N columns of color sub-filters have the same color as the color filter, and N is a positive integer; and each pixel in the pixel array and the sub-filter of the filter array
  • the setting corresponding to slices can increase the color resolution of each row and column of the generated full-resolution target image, making the full-resolution target image richer in color.
  • An image generation method applied to an image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes at least a first filter group and a second filter group Two filter groups; the first filter group and the second filter group include panchromatic filters and color filters; the color filters in the first filter group include the first A color filter and a third color filter, the color filters in the second filter group include a second color filter; the panchromatic filter and the color filter in the minimum repeating unit Alternately arranged on each row and each column of the minimum repeating unit, the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter; each pixel in the pixel array and The filters of the filter array are arranged correspondingly, and the pixel array is configured to receive light passing through the filter array to generate electrical signals;
  • the methods include:
  • the full-resolution panchromatic pixel value is read out from the panchromatic pixel corresponding to each of the panchromatic filters, and the full-resolution full-resolution pixel value is read out from the color pixel corresponding to each of the color filters. rate color pixel value;
  • a full resolution target image is generated based on each of said full resolution panchromatic pixel values and each of said full resolution color pixel values.
  • An image generating device applied to an image sensor, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes at least a first filter group and a second filter group Two filter groups; the first filter group and the second filter group all include panchromatic filters and color filters; the color filters in the first filter group include both The first color filter and the third color filter, the color filter in the second filter group includes the second color filter; the panchromatic filter and the color filter in the minimum repeating unit
  • the light sheets are alternately arranged on each row and each column of the minimum repeating unit, and the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter; each of the pixel arrays
  • the pixels are arranged corresponding to the filters of the filter array, and the pixel array is configured to receive light passing through the filter array to generate electrical signals;
  • the devices include:
  • the readout module is used to read out the full-resolution panchromatic pixel value from the panchromatic pixel corresponding to each of the panchromatic filters in the full-resolution mode, and to read the full-resolution panchromatic pixel value corresponding to each of the color filters.
  • An image generating module configured to generate a full-resolution target image based on each of the full-resolution panchromatic pixel values and each of the full-resolution color pixel values.
  • An electronic device includes a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor is made to execute the operation of the above-mentioned image generation method.
  • a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the operation of the above-mentioned method is realized.
  • a computer program product includes a computer program, and when the computer program is executed by a processor, the operations of the above-mentioned method are realized.
  • the panchromatic pixel corresponding to each panchromatic filter is read out the full resolution panchromatic pixel value, And read out the full-resolution color pixel value from the color pixel corresponding to each color filter; and the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter, and the information of the panchromatic channel can be fused Into the image, the overall amount of incoming light is increased, so that based on each full-resolution panchromatic pixel value and each full-resolution color pixel value, a full-resolution target image with more information and clearer detail analysis can be generated.
  • panchromatic filters and color filters of various colors are scattered and arranged, so that full-resolution panchromatic pixel values and full-resolution color pixel values of various colors are also scattered and arranged in imaging, It can improve the color resolution capability and brightness change resolution capability.
  • the color filters of various colors are mixed and arranged, so that the full-resolution color pixel values of various colors in the imaging are also mixed and arranged, which can reduce the risk of false colors.
  • panchromatic filters and color filters are arranged alternately on each row and each column, and each pixel in the pixel array corresponds to the filter of the filter array, which can improve the generated full-resolution target image
  • the color resolution of each row and each column of the image makes the color of the full-resolution target image richer.
  • Fig. 1 is a schematic structural diagram of an electronic device in an embodiment.
  • FIG. 2 is an exploded schematic diagram of an image sensor in one embodiment.
  • Fig. 3 is a schematic diagram of connection between a pixel array and a readout circuit in one embodiment.
  • Fig. 4 is an arrangement of two first filter groups and two second filter groups in one embodiment.
  • FIG. 5 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 1 in an embodiment.
  • FIG. 6 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 1 in another embodiment.
  • FIG. 7 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 1 in another embodiment.
  • FIG. 8 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 1 in another embodiment.
  • FIG. 9 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 2 in an embodiment.
  • FIG. 10 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 2 in another embodiment.
  • FIG. 11 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 2 in another embodiment.
  • FIG. 12 is a schematic diagram of the arrangement of the smallest repeating unit in an optical filter array in which N is 2 in another embodiment.
  • Fig. 13 is a schematic flowchart of an image generation method in an embodiment.
  • Fig. 14 is a schematic flowchart of an image generation method in an embodiment.
  • Fig. 15 is a schematic diagram of a first target image in one embodiment.
  • Fig. 16 is a schematic flowchart of generating a second target image in an embodiment.
  • Figure 17 is a schematic diagram of a first color image and a first panchromatic image in one embodiment.
  • Fig. 18 is a schematic diagram of a second target image in one embodiment.
  • Fig. 19 is a schematic diagram of a second target image in another embodiment.
  • Fig. 20 is a schematic diagram of a second target image in another embodiment.
  • Fig. 21 is a schematic diagram of a second target image in another embodiment.
  • Fig. 22 is a schematic flowchart of generating a third target image in an embodiment.
  • Fig. 23 is a schematic diagram of a second full-color image, a dual-color second color image, and a single-color second color image in one embodiment.
  • Fig. 24 is a schematic diagram of a third target image in one embodiment.
  • Fig. 25 is a schematic diagram of a third target image in another embodiment.
  • Fig. 26 is a schematic flowchart of an image generation method in another embodiment.
  • Fig. 27 is a structural block diagram of an image generating device in an embodiment.
  • Fig. 28 is a schematic diagram of the internal structure of an electronic device in one embodiment.
  • first, second and the like used in this application may be used to describe various elements herein, but these elements are not limited by these terms. These terms are only used to distinguish one element from another element.
  • a first object image could be termed a second object image, and, similarly, a second object image could be termed a first object image, without departing from the scope of the present application.
  • Both the first object image and the second object image are object images, but they are not the same object image.
  • the electronic device 100 includes a mobile phone, a tablet computer, a notebook computer, a teller machine, a gate machine, a smart watch, a head-mounted display device, etc. It can be understood that the electronic device 100 can also be any other device with an image processing function.
  • the electronic device 100 includes a camera module 20 , a processor 30 and a casing 40 .
  • the camera module 20 and the processor 30 are both arranged in the housing 40, and the housing 40 can also be used to install functional modules such as power supply devices and communication devices of the electronic device 100, so that the housing 40 provides dustproof and drop-proof protection for the functional modules. , waterproof and other protection.
  • the camera module 20 may be a front camera module, a rear camera module, a side camera module, an under-screen camera module, etc., which is not limited here.
  • the camera module 20 includes a lens and an image sensor 21. When the camera module 20 captures an image, light passes through the lens and reaches the image sensor 21.
  • the image sensor 21 is used to convert the light signal irradiated on the image sensor 21 into an electrical signal.
  • the image sensor 21 includes a microlens array 22 , a filter array 23 , and a pixel array 24 .
  • the microlens array 22 includes a plurality of microlenses 221, the microlenses 221, the sub-filters in the filter array 23, and the pixels in the pixel array 24 are set in one-to-one correspondence, and the microlenses 221 are used to gather the incident light.
  • the collected light will pass through the corresponding sub-filter, and then projected onto the pixel, and be received by the corresponding pixel, and the pixel converts the received light into an electrical signal.
  • the filter array 23 includes a plurality of minimal repeating units 230 .
  • the minimum repeating unit 230 includes at least a first filter group 231 and a second filter group 232, and both the first filter group 231 and the second filter group 232 include a panchromatic filter 233 and a color filter 234; the color filter 234 in the first filter group 231 includes a first color filter and a third color filter, and the color filter 234 in the second filter group 232 includes a second color filter light sheet; the panchromatic filter 233 and the color filter 234 are arranged alternately on each row and each column, and the amount of light transmitted by the panchromatic filter 233 is greater than the amount of light transmitted by the color filter 234;
  • a panchromatic filter 233 includes N rows and N columns of panchromatic sub-filters, each color filter 234 includes N rows and N columns of color sub-filters, N rows and N columns of color sub-filters and The colors of the color filters are the same, and N is a positive integer.
  • each row and each column of the minimum repeating unit 230 includes a color filter 234 of each color, that is, the color filters of each color are dispersed and arranged, which can improve the color resolution and brightness change resolution.
  • the color filter mix arrangement of colors also reduces the risk of false colors.
  • the panchromatic filter 233 and the color filter 234 are alternately arranged in each row and each column, that is, the panchromatic filter 233 is arranged in the smallest repeating unit, the first filter group or the second filter group.
  • the number of 50% in the filter group can increase the amount of light entering each local area in the imaging.
  • the minimum repeating unit 230 includes 2 first filter groups 231 and 2 second filter groups 232, and the 2 first filter groups 231 are arranged on the diagonal of the minimum repeating unit 230
  • the two second filter groups 232 are arranged on the anti-diagonal lines of the smallest repeating unit 230 .
  • Two first filter sets 231 and two second filter sets 232 are arranged in matrix.
  • the diagonal line may be a connecting line between the upper left corner and the lower right corner, or may be a connecting line between the upper right corner and the lower left corner.
  • Diagonals and anti-diagonals are perpendicular to each other. That is to say, if the diagonal line is the line connecting the upper left corner and the lower right corner, then the anti-diagonal line is the line connecting the upper right corner and the lower left corner; if the diagonal line is the line connecting the upper right corner and the lower left corner, then the anti-diagonal line is is the line connecting the upper left and lower right corners.
  • the color filters 234 of each filter set are arranged on the diagonal of the corresponding filter set and in a direction parallel to the diagonal.
  • the corresponding filter group is the filter group where the color filter 234 is located.
  • the color filters in the second filter group 232 include second color filters, and the second color filters in the second filter group 232 are arranged at the opposite corners of the second filter group 232 Lines and directions parallel to the diagonals can improve the distribution balance of the second color filters on the diagonals.
  • the first color filter or the third color filter is arranged on the diagonal of the first filter set 231 .
  • the first color filters in the two first filter groups 231 are arranged on the diagonal of the first filter group 231, and the third color filter The light sheets are all arranged in a direction parallel to the diagonal of the first filter group 231 .
  • the first color filter in one of the first filter groups of the two first filter groups is arranged at the opposite corner of the first filter group On the line, the third color filter is arranged in the direction parallel to the diagonal line of the first filter group, and the third color filter in the other first filter group is arranged on the first filter group On the diagonal of the group, the first color filter is arranged in a direction parallel to the diagonal of the first filter group.
  • the optical filter array can make the distribution of the first color filter or the third color filter on the diagonal, that is, inclined at 45 degrees, more balanced. For example, imagine some color changing stripes in the column direction, each color line occupies a pixel width, since the filter in this image sensor is arranged on each column, the corresponding color pixels can be obtained, then the column direction can be estimated Colored pixel values for all pixels above.
  • the first color filter or the third color filter is arranged on the diagonal, that is, the first color filter and the third color filter
  • the three color filters are interspersed with each other to increase the arrangement balance of the first color filter and the third color filter on the diagonal.
  • each filter set includes a plurality of subunits, each subunit includes a color filter 234 and a panchromatic filter 233, and the color filters 234 in the subunits are arranged on opposite sides of the subunits. Diagonal line, the panchromatic filter 233 in the sub-unit is arranged on the anti-diagonal line of the sub-unit.
  • the color filter 234 includes a first color filter, a second color filter and a third color filter.
  • the first color filter, the second color filter and the third color filter are filters of three different colors.
  • the colors of the first color filter, the second color filter and the third color filter can all be set as required.
  • the first color filter may be a red filter
  • the second color filter may be a green filter
  • the third color filter may be a blue filter.
  • the width of the wavelength band of the light transmitted by the color filter 234 is smaller than the width of the wavelength band of the light transmitted by the panchromatic filter 233, for example, the wavelength band of the transmitted light of the color filter 234 can correspond to the wavelength band of red light , the wavelength band of green light, or the wavelength band of blue light, the wavelength band of the light transmitted by the panchromatic filter is the wavelength band of all visible light, that is to say, the color filter 234 only allows light of a specific color to transmit light, while the panchromatic filter
  • the light sheet 233 can pass light of all colors.
  • the wavelength band of the light transmitted by the color filter 234 may also correspond to the wavelength band of other colored light, such as magenta light, purple light, cyan light, yellow light, etc., which is not limited here.
  • the pixel array 24 includes a plurality of pixels, and the pixels of the pixel array 24 are arranged corresponding to the sub-filters of the filter array 23 .
  • the pixel array 24 is configured to receive light passing through the filter array 23 to generate electrical signals.
  • the pixel array 24 is configured to receive the light passing through the filter array 23 to generate an electrical signal, which means that the pixel array 24 is used to detect a scene of a given set of subjects passing through the filter array 23
  • the light is photoelectrically converted to generate an electrical signal.
  • the light rays of the scene for a given set of subjects are used to generate image data.
  • the subject is a building
  • the scene of a given set of subjects refers to the scene where the building is located, which may also contain other objects.
  • the pixel array 24 includes a plurality of minimum repeating units 240, and the minimum repeating unit 240 also includes a plurality of panchromatic pixels 241 and a plurality of color pixels 242 of different colors, and the panchromatic pixels 241 and the color pixels 242 are arranged in each row. Alternately arranged on each column, each row and each column include color pixels of each color; each panchromatic pixel 242 is corresponding to a sub-filter in the panchromatic filter 233, and the panchromatic pixel 242 receives the pass through Light that passes through the corresponding sub-filters to generate electrical signals. Each color pixel 242 corresponds to a sub-filter of the color filter 234, and the color pixel 242 receives light passing through the corresponding sub-filter to generate an electrical signal.
  • the readout circuit 25 is electrically connected to the pixel array 24 for controlling the exposure of the pixel array 24 and reading and outputting the pixel values of the pixels.
  • the readout circuit 25 includes a vertical drive unit 251 , a control unit 252 , a column processing unit 253 , and a horizontal drive unit 254 .
  • the vertical driving unit 251 includes a shift register and an address decoder.
  • the vertical driving unit 251 includes readout scanning and reset scanning functions.
  • the control unit 252 configures timing signals according to the operation mode, and uses various timing signals to control the vertical driving unit 251 , the column processing unit 253 and the horizontal driving unit 254 to work together.
  • the column processing unit 253 may have an analog-to-digital (A/D) conversion function for converting an analog pixel signal into a digital format.
  • the horizontal driving unit 254 includes a shift register and an address decoder. The horizontal driving unit 254 sequentially scans the pixel array 24 column by column.
  • the image sensor includes a filter array 23 and a pixel array 24, the filter array 23 includes a minimum repeating unit 230, and the minimum repeating unit 230 includes at least a first filter set 231 and a second filter set 232; Both the first filter group 231 and the second filter group 232 include a panchromatic filter 233 and a color filter 234; the color filter 234 in the first filter group 231 includes a first color filter and the third color filter, the color filter 234 in the second filter group 232 includes a second color filter, the amount of light transmitted by the panchromatic filter 233 is greater than that transmitted by the color filter 234 When shooting, more light can be obtained through the panchromatic filter 233, so that there is no need to adjust the shooting parameters, and the clarity of imaging in dark light can be improved without affecting the stability of shooting. When imaging in dark light, both stability and clarity can be taken into account, and the stability and clarity of imaging in dark light are both high.
  • Panchromatic filters 233 and color filters 234 are alternately arranged in each row and column in the minimum repeating unit 230, and each panchromatic filter 233 includes N rows and N columns of panchromatic sub-filters, each The color filters 234 include color sub-filters in N rows and N columns, and the color sub-filters in N rows and N columns have the same color as the color filter, and N is a positive integer, and each pixel in the pixel array 24 is identical to the color of the color filter.
  • the sub-filters of the filter array 23 are correspondingly arranged, that is, each row and each column in the pixel array 24 includes color pixels of each color, which can improve the color resolution of each row and column of imaging, so that the imaging The colors are richer.
  • the panchromatic filter 233 and the color filter 234 of various colors are dispersedly arranged, make the panchromatic pixel value and the colored pixel value of various colors also dispersedly arrange in imaging, can improve color Resolution capability and brightness change resolution capability.
  • the color filters 234 of various colors are mixed and arranged, so that the color pixel values of various colors are also mixed and arranged in the imaging, which can reduce the risk of false colors.
  • each panchromatic filter 233 includes 1 row and 1 column of panchromatic sub-filters, and each color filter 234 includes 1 row and 1 column of color sub-filters, that is to say, Each panchromatic sub-filter is a panchromatic filter 233 , and each color sub-filter is a color filter 234 .
  • the minimum repeating unit 230 includes 64 filters in 8 rows and 8 columns, and the arrangement is as follows:
  • w represents the panchromatic filter 234
  • a, b and c all represent the color filter 235 .
  • the minimum repeating unit 230 includes 64 optical filters in 8 rows and 8 columns, and the arrangement is as follows:
  • w represents the panchromatic filter 234
  • a, b and c all represent the color filter 235 .
  • the minimum repeating unit 230 includes 64 filters in 8 rows and 8 columns, and the arrangement is as follows:
  • w represents the panchromatic filter 234
  • a, b and c all represent the color filter 235 .
  • the minimum repeating unit 230 includes 64 filters in 8 rows and 8 columns, and the arrangement is as follows:
  • w represents the panchromatic filter 234
  • a, b and c all represent the color filter 235 .
  • w can be a white filter, a a red filter, b a green filter, c a blue filter, or for example a a magenta filter, b a cyan filter, c It is a yellow filter or the like, and is not limited here.
  • the minimum repeating unit 230 includes 256 sub-filters in 16 rows and 16 columns, and the arrangement is as follows:
  • w represents a panchromatic sub-filter
  • a, b, and c all represent color sub-filters.
  • the minimum repeating unit 230 includes 256 sub-filters in 16 rows and 16 columns, and the arrangement is as follows:
  • w represents a panchromatic sub-filter
  • a, b, and c all represent color sub-filters.
  • the minimum repeating unit 230 includes 256 sub-filters in 16 rows and 16 columns, and the arrangement is as follows:
  • w represents a panchromatic sub-filter
  • a, b, and c all represent color sub-filters.
  • the minimum repeating unit 230 includes 256 sub-filters in 16 rows and 16 columns, and the arrangement is as follows:
  • w represents a panchromatic sub-filter
  • a, b, and c all represent color sub-filters.
  • N can also be other positive integers such as 3, 4, or 5, and the arrangement method is similar to that of N being 1 or 2, which will not be repeated here.
  • w can be a white sub-filter, a a red sub-filter, b a green sub-filter, c a blue sub-filter, or for example a a magenta sub-filter and b a cyan Sub-filters, c is a yellow sub-filter, etc., which is not limited here.
  • a camera module is also provided.
  • the camera module includes a lens and the above-mentioned image sensor; the image sensor is used to receive light passing through the lens, and the pixels generate electrical signals according to the light.
  • an electronic device comprising the above-mentioned camera module; and a casing, on which the camera module is arranged.
  • an image generation method is provided, which is applied to an image sensor.
  • the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes at least a first filter group and a pixel array.
  • the image generation method includes:
  • Operation 1302 in the full-resolution mode, read out the full-resolution panchromatic pixel value from the panchromatic pixel corresponding to each panchromatic sub-filter in the panchromatic filter, and read out the full-resolution panchromatic pixel value of each color sub-filter in the color filter The color pixel corresponding to the sub-filter reads out the full-resolution color pixel value.
  • Full resolution mode is a mode in which each sub-filter is read out as one pixel.
  • the color filter 234 has a narrower spectral response than that of the panchromatic filter 233, so the amount of light transmitted by the panchromatic filter 233 is greater than the amount of light transmitted by the color filter 234, that is, the color filter 234
  • the wavelength band width of the transmitted light is smaller than the wavelength band width of the light transmitted by the panchromatic filter 233, and the panchromatic filter 233 transmits more light, and the corresponding panchromatic pixel 241 has With a higher signal-to-noise ratio, the panchromatic pixel 241 contains more information, and can resolve more texture details.
  • the signal-to-noise ratio refers to the ratio between the normal signal and the noise signal. The higher the signal-to-noise ratio of a pixel, the higher the proportion of normal signals contained in the pixel, and the more information can be analyzed from the pixel.
  • the color pixels 242 may be G (Green, green) pixels, R (Red, red) pixels, B (Blue, blue) pixels, etc., but are not limited thereto.
  • the light transmitted by the panchromatic sub-filter in the panchromatic filter 233 is projected onto the corresponding panchromatic pixel 241, and the panchromatic pixel 241 receives the light passing through the panchromatic sub-filter to generate electrical signals.
  • the light transmitted by the color sub-filter in the color filter 234 is projected onto the corresponding color pixel 242 , and the color pixel 242 generates an electrical signal through the light passing through the corresponding color sub-filter.
  • Each panchromatic filter 233 includes N rows and N columns of panchromatic sub-filters, and each panchromatic filter 233 corresponds to N rows and N columns of panchromatic pixels 241 .
  • Each color filter 234 includes N rows and N columns of color sub-filters of the same color, and each color filter 234 corresponds to N rows and N columns of color pixels 242 .
  • N is a positive integer greater than or equal to 2.
  • N may also be 1, that is, each panchromatic filter 233 corresponds to one panchromatic pixel 241 , and each color filter 234 corresponds to one color pixel 242 .
  • a full resolution target image is generated based on the respective full resolution panchromatic pixel values and the respective full resolution color pixel values.
  • the electronic device can read pixel values from each full-resolution panchromatic pixel value and each full-resolution color pixel value according to a preset pixel reading method to generate a full-resolution target image.
  • the preset pixel reading mode is a preset pixel reading mode.
  • the panchromatic pixels corresponding to each panchromatic sub-filter in the panchromatic filter are read out in full Resolution panchromatic pixel value, and read out the full resolution color pixel value from the color pixel corresponding to each color sub-filter in the color filter; and the amount of light transmitted by the panchromatic filter is greater than that of the color filter
  • the amount of transmitted light can integrate the panchromatic channel information into the image to increase the overall light input, so that based on each full-resolution panchromatic pixel value and each full-resolution color pixel value, more information and detailed analysis can be generated. Sharper full-resolution target images.
  • panchromatic filters and color filters of various colors are scattered and arranged, so that full-resolution panchromatic pixel values and full-resolution color pixel values of various colors are also scattered and arranged in imaging, It can improve the color resolution capability and brightness change resolution capability.
  • the color filters of various colors are mixed and arranged, so that the full-resolution color pixel values of various colors in the imaging are also mixed and arranged, which can reduce the risk of false colors.
  • each panchromatic filter and the color filter are alternately arranged on each row and each column, and each panchromatic filter includes N rows and N columns of panchromatic sub-filters, and each color filter includes N rows and N columns of color sub-filters, N rows and N columns of color sub-filters have the same color as the color filter, and N is a positive integer; and each pixel in the pixel array and the sub-filter of the filter array
  • the setting corresponding to slices can increase the color resolution of each row and column of the generated full-resolution target image, making the full-resolution target image richer in color.
  • the above method further includes:
  • Operation 1402 in the first resolution mode, combine the panchromatic pixels corresponding to each panchromatic sub-filter in each of the panchromatic filters to read out the first panchromatic pixel value, and The color pixels corresponding to each color sub-filter in the color filter are combined to read out the first color pixel value; the resolution corresponding to the first resolution mode is smaller than the resolution corresponding to the full resolution mode.
  • the first resolution mode refers to a first-level pixel binning readout mode in which resolution, power consumption, signal-to-noise ratio, and frame rate are relatively balanced.
  • the first resolution mode may be a default mode for shooting images and videos.
  • the first resolution mode is used to respond to the shooting instruction.
  • the light transmitted by the panchromatic sub-filter in the panchromatic filter 233 is projected onto the corresponding panchromatic pixel 241, and the panchromatic pixel 241 receives the light passing through the panchromatic sub-filter. light to generate electrical signals.
  • the light transmitted by the color sub-filter in the color filter 234 is projected onto the corresponding color pixel 242 , and the color pixel 242 generates an electrical signal through the light passing through the corresponding color sub-filter.
  • Combined readout refers to summing the pixel values of multiple pixels, or calculating the average value of the pixel values of multiple pixels.
  • the panchromatic pixels 241 corresponding to each panchromatic sub-filter are averaged, and the average value is read out as the first panchromatic pixel value.
  • sum the panchromatic pixel 241 corresponding to each panchromatic sub-filter and read the sum obtained as the first panchromatic pixel value.
  • the electronic device may also use other methods to combine the panchromatic pixels 241 corresponding to the panchromatic sub-filters to read out the first panchromatic pixel value, which is not limited here.
  • the color pixel 242 corresponding to each color sub-filter is averaged, and the average value is read out as the first color pixel value.
  • the color pixels corresponding to each color sub-filter are summed, and the sum obtained by the addition is read out as the first color pixel value.
  • the electronic device may also combine the color pixels 242 corresponding to the color sub-filters in other ways to read out the first color pixel value, which is not limited here.
  • the way of combining and reading out the first panchromatic pixel values may be the same or different.
  • the way of combining and reading out the first color pixel values may be the same or different.
  • the panchromatic filter 233 and the color filter 234 the way of combining and reading out the first panchromatic pixel value and the first color pixel value may be the same or different.
  • Operation 1404 generating a first target image based on each of the first panchromatic pixel values and each of the first color pixel values.
  • the electronic device can read pixel values from each first panchromatic pixel value and each first color pixel value according to a preset pixel reading method to generate a first target image.
  • the preset pixel reading mode is a preset pixel reading mode. Taking the arrangement of the smallest repeating unit of the filter array 23 as shown in FIG. 9 as an example, the generated first target image is shown in FIG. 15 .
  • the panchromatic pixels 241 corresponding to each panchromatic sub-filter in each panchromatic filter 233 are combined to read out the first panchromatic pixel value, and each The color pixels 242 corresponding to each color sub-filter in each color filter 234 are combined to read the first color pixel value, and the amount of light transmitted by the panchromatic filter 233 is greater than the amount of light transmitted by the color filter 234 , can integrate the panchromatic channel information into the image to increase the overall light input, so that based on each first panchromatic pixel value and each first color pixel value, the first target image with more information and clearer detail analysis can be generated .
  • panchromatic filter 233 and the color filter 234 are alternately arranged on each row and each column, and each panchromatic filter 233 includes N rows and N columns of panchromatic sub-filters, and each color filter
  • the light sheet 234 includes N rows and N columns of color sub-filters, the color sub-filters of N rows and N columns are the same as the color of the color filter, and N is a positive integer; and each pixel in the pixel array 24 and the color filter
  • the sub-filters of the array 23 are correspondingly set, that is, each row and each column in the pixel array 24 includes color pixels of each color, which can improve the color resolution of each row and each column of the generated first target image, so that The color of the first target image is richer.
  • each filter group includes a plurality of subunits, each subunit includes panchromatic filters and color filters, and the color filters in the subunits are arranged on the diagonal of the subunits, The panchromatic filters in the subunits are arranged on anti-diagonal lines of the subunits.
  • the sub-unit includes 2 rows and 2 columns of filters, the color filters in the sub-unit are arranged on the diagonal of the sub-unit, and the panchromatic filters in the sub-unit are arranged on the anti-diagonal of the sub-unit . That is to say, the filters in 2 rows and 2 columns are arranged in a matrix.
  • Operation 1602 in the second resolution mode, combine the multiple first panchromatic pixel values corresponding to each subunit in the first target image to read out the second panchromatic pixel value, and based on each second panchromatic pixel value value produces the first panchromatic image; the second resolution mode corresponds to a smaller resolution than the first resolution mode.
  • the second resolution mode refers to the mode used in the scene where the resolution requirement is lower than that of the first resolution mode. out mode.
  • the resolution and power consumption corresponding to the second resolution mode are smaller than the resolution and power consumption corresponding to the first resolution mode.
  • the signal-to-noise ratio and frame rate corresponding to the second resolution mode are greater than the signal-to-noise ratio and frame rate corresponding to the first resolution mode.
  • the second resolution mode may specifically be a preview mode during image capture, a preview mode during video capture, or a scene with lower resolution requirements such as image capture and video capture under night scenes, but is not limited thereto.
  • the preview modes of video shooting include 1080p video preview, application video preview, etc.
  • the shooting instruction is preview shooting.
  • the second resolution mode is triggered.
  • the electronic device detects whether the current environment is a night scene, and if the current environment is a night scene, triggers the second resolution mode.
  • the readout mode corresponding to the second resolution mode is triggered.
  • the electronic device combines the multiple first panchromatic pixel values corresponding to each subunit in the first target image to read out the second panchromatic pixel value, and according to the preset pixel reading method, from each second panchromatic pixel value Color Pixel Values Pixel values are read to generate a first full-color image.
  • each pixel value in the first target image is obtained by combining the pixels corresponding to the sub-filters in each filter in the filter array in the first resolution mode, then the first target
  • Each pixel value in the image corresponds to each filter in the filter array, and also corresponds to multiple sub-filters in each filter.
  • the electronic device determines a plurality of pixel values of each subunit in the first target image, acquires a plurality of first panchromatic pixels from the plurality of pixel values and reads out a second panchromatic pixel value, And acquiring a plurality of first color pixel values of the same color from the plurality of pixel values and reading out a second color pixel value.
  • combining readout may include one of methods such as averaging, summing, or weighted averaging, which is not limited here.
  • Operation 1604 combining multiple first color pixel values corresponding to the same color in each subunit in the first target image to read out second color pixel values, and generating a first color image based on each second color pixel value.
  • the electronic device combines multiple first color pixel values corresponding to each subunit in the first target image to read out the second color pixel value, and calculates the second color pixel value according to the preset pixel value.
  • pixel values are read from respective second color pixel values to generate a first color image.
  • combining readout may include one of methods such as averaging, summing, or weighted averaging, which is not limited here.
  • the generated first color image is shown as 1702 in FIG. 17
  • the generated first full-color image is shown as 1704 .
  • Operation 1606 based on the first panchromatic image and the first color image, generate a second target image.
  • the electronic device may generate a second target image based on the first full-color image and the first color image, and transmit the second target image.
  • the electronic device arranges each row of second panchromatic pixel values in the first full-color image alternately with each row of second color pixel values in the first color image to generate a second target image; or the first full-color image
  • Each column of second panchromatic pixel values in the first color image is alternately arranged with each column of second color pixel values in the first color image to generate a second target image.
  • FIG. 18 and 19 are schematic diagrams of the second target image obtained by alternately arranging each row of second panchromatic pixel values in the first full-color image and each row of second color pixel values in the first color image.
  • FIG. 20 and FIG. 21 are the second target image obtained by alternately arranging each column of second panchromatic pixel values in the first full-color image and each column of second color pixel values in the first color image.
  • the electronic device may also combine pixel values at the same position in the first full-color image and the first color image to obtain combined pixel values at corresponding positions, and form the second target image based on the combined pixel values.
  • the merging may adopt one of methods such as averaging, weighted averaging, or adding and summing.
  • the electronic device may also use other methods to generate the second target image, which is not limited here.
  • each first panchromatic pixel values corresponding to each subunit in the first target image are combined to read out the second panchromatic pixel value
  • the first target In the image multiple first color pixel values of the same color corresponding to each subunit are combined to read out the second color pixel value, and each different color pixel 242 can be mixed and arranged so that the generated second target image
  • the distribution of each second color pixel such as RGB pixel is more uniform, and the image quality is higher.
  • the resolution and image size of the obtained second target image are further reduced, and the panchromatic pixel 241 has a higher signal-to-noise ratio, and the frame rate of the image is high, so that the power consumption of the secondary pixel combination output is lower , Better signal-to-noise ratio image processing effect.
  • the panchromatic pixel 241 has a higher signal-to-noise ratio, and the frame rate of the image is high, so that the power consumption of the secondary pixel combination output is lower , Better signal-to-noise ratio image processing effect.
  • the second resolution mode there are full-arranged panchromatic pixels without interpolation, which improves the overall resolution.
  • the color pixels of each color such as the pixels of the first color and the pixels of the third color, are more dispersed and balanced in the diagonal direction or anti-diagonal direction.
  • each filter group includes a plurality of subunits, each subunit includes panchromatic filters and color filters, and the color filters in the subunits are arranged on the diagonal of the subunits, The panchromatic filters in the subunits are arranged on anti-diagonal lines of the subunits; the above method also includes: in the second resolution mode, each panchromatic sub-filter of the plurality of panchromatic filters in each subunit The panchromatic pixels corresponding to the light sheet are combined to read out the fourth panchromatic pixel value, and a third panchromatic image is generated based on each fourth panchromatic pixel value; The color pixels corresponding to the sub-filters are combined to read fourth color pixel values, and a third color image is generated based on each fourth color pixel value; a fourth target image is generated based on the third panchromatic image and the third color image.
  • the combined readout method may be one of averaging, weighted averaging, or addition.
  • the electronic device may generate a fourth target image based on the third panchromatic image and the third color image, and then transmit the fourth target image.
  • generating the fourth target image includes: combining the fourth panchromatic pixel value of each row in the third panchromatic image with the fourth panchromatic pixel value of each row in the third color image The four color pixel values are arranged alternately to generate the fourth target image; or the fourth panchromatic pixel values of each column in the third panchromatic image and the fourth color pixel values of each column in the third color image are arranged alternately to generate the fourth target image. target image.
  • the electronic device may also combine pixel values at the same position in the third panchromatic image and the third color image to obtain combined pixel values at corresponding positions, and form a fourth target image based on the combined pixel values.
  • the combined readout may adopt one of methods such as averaging, weighted averaging, or adding and summing.
  • the electronic device may also use other methods to generate the fourth target image, which is not limited here.
  • the panchromatic pixels corresponding to the panchromatic sub-filters of the plurality of panchromatic filters in each subunit are combined to read out the fourth panchromatic pixel value, and Combining the color pixels corresponding to the color sub-filters of multiple color filters of the same color in each subunit to read out the fourth color pixel value can generate the third panchromatic image and the third color image more quickly, thereby Generate the fourth target image more quickly.
  • the above-mentioned embodiment can mix and arrange different color pixels, so that the distribution of each fourth color pixel value, such as RGB pixels, in the generated fourth target image is more uniform, and the image quality is higher. Moreover, the resolution and image size of the obtained fourth target image are further reduced, and the panchromatic pixel 241 has a higher signal-to-noise ratio, and the frame rate of the image is high, so that the power consumption of the secondary pixel combination output is lower , Better signal-to-noise ratio image processing effect.
  • each fourth color pixel value such as RGB pixels
  • the above method further includes:
  • Operation 2202 in the third resolution mode, combine the multiple second panchromatic pixel values corresponding to the same filter set in the first panchromatic image to read out the third panchromatic pixel value, and based on each third panchromatic pixel value
  • the panchromatic pixel values generate a second panchromatic image; the resolution corresponding to the third resolution mode is smaller than the resolution corresponding to the second resolution mode.
  • the third resolution mode refers to the mode used in the scene where the resolution requirement is lower than that of the second resolution mode. It is a three-level pixel binning read with low resolution, low power consumption, high SNR and high frame rate out mode.
  • the resolution and power consumption corresponding to the third resolution mode are smaller than the resolution and power consumption corresponding to the second resolution mode.
  • the signal-to-noise ratio and frame rate corresponding to the third resolution mode are greater than the signal-to-noise ratio and frame rate corresponding to the second resolution mode.
  • the third resolution mode may specifically be a preview mode during image capture, a preview mode during video capture, or a night scene mode for image capture and video capture under night scenes, etc., but is not limited thereto.
  • the preview modes of video shooting include 720p video preview, application video preview, etc.
  • the electronic device determines each filter set from the filter array, acquires multiple second panchromatic pixel values in the first panchromatic image obtained by each filter set, and converts the multiple second panchromatic pixel values The values are combined to read out the third panchromatic pixel value.
  • the electronic device reads pixel values from each third panchromatic pixel value to form a second panchromatic image according to a preset pixel reading method.
  • Operation 2204 combine and read out the third color pixel value of the first color and the third color pixel value of the second color corresponding to a plurality of second color pixel values of the same color in the same filter group in the first color image value and the third color pixel value of the third color, and based on the third color pixel value of the first color, the third color pixel value of the second color and the third color pixel value of the third color, generate the second color pixel value of the double color a color image and a single-color second-color image; the dual-color second-color image includes third-color pixel values of the second color, and third-color pixel values of the first color and third-color pixel values of the third color In one of them, the single-color second color image includes the other one of the third color pixel value of the first color and the third color pixel value of the third color.
  • the third color pixel value of the first color is the pixel value read from the pixel corresponding to the first color filter
  • the third color pixel value of the second color is the pixel value read from the pixel corresponding to the second color filter
  • the third color pixel value of the third color is the pixel value read out from the pixel corresponding to the third color filter.
  • the electronic device determines each filter set from the filter array, and obtains multiple second color pixel values of the same color in the first color image obtained by each filter set, and second color pixel values of the same color
  • the color pixel value includes a second color pixel value of the first color, a second color pixel value of the second color, and a second color pixel value of the third color, and a plurality of second color pixel values of the first color are combined to read out the first color pixel value.
  • For the third panchromatic pixel value of one color combine multiple second color pixel values of the second color to read out the third panchromatic pixel value of the second color, combine and read multiple second color pixel values for the third color
  • the bicolor second color image refers to an image including pixel values of two colors, and the bicolor second color image includes third color pixel values of the second color.
  • the second color image of a single color includes an image of pixel values of one color.
  • the electronic device uses the third color pixel value of the first color and the third color pixel value of the second color to form a two-color second color image, and forms the third color pixel value of the third color to A second color image of a single color.
  • the electronic device uses the third color pixel value of the third color and the third color pixel value of the second color to form a two-color second color image, and combines the third color pixel value of the first color Constitutes a second color image of a single color.
  • the electronic device converts the first panchromatic image 1704 into multiple second panchromatic images corresponding to the same filter set. Combine the pixel values to read out the third panchromatic pixel value, and generate the second panchromatic image 2302 in FIG.
  • the resolution corresponding to the third resolution mode is smaller than that corresponding to the second resolution mode Resolution: combine the second color pixel values corresponding to multiple same colors in the same filter group in the first color image 1702 to read out the third color pixel value of the first color and the third color of the second color pixel value and the third color pixel value of the third color, and based on the third color pixel value of the first color, the third color pixel value of the second color and the third color pixel value of the third color, generate the A second color image 2304 of two colors and a second color image 2306 of one color. It should be noted that, in FIG.
  • the single-color second color image 2306 only includes the third color pixel value of the third color, and the third color pixel value of the third color is the same as that in the dual-color second color image 2304.
  • the third color pixel value of the first color corresponds, and in the single-color second color image 2306, the position corresponding to the second color third color pixel value of the double-color second color image 2304 is an empty pixel . Empty pixels are pixels without any information.
  • a third target image is generated based on the second panchromatic image, the two-color second color image, and the single-color second color image.
  • the electronic device can The second color image is used to generate a third target image, and then the third target image is transmitted.
  • the electronic device sets the third panchromatic pixel value in the second panchromatic image with the same coordinates, the third color pixel value in the double-color second color image, and the third color pixel value in the single-color second color image Arranged alternately in the row direction to generate the third target image; or the third panchromatic pixel value in the second panchromatic image with the same coordinates, the third color pixel value in the double-color second color image and the third color pixel value in the single color
  • the pixel values of the third color in the two-color image are arranged alternately in the column direction to generate the third target image.
  • FIG. 23 Three full-color pixel values, the third color pixel value in the second color image of two colors and the third color pixel value in the second color image of single color are arranged alternately in the row direction to generate the third target image;
  • Figure 25 is In another embodiment, the third panchromatic pixel value in the second panchromatic image with the same coordinates, the third color pixel value in the double-color second color image, and the third color pixel value in the single-color second color image are The generated third target images are arranged alternately in the row direction.
  • the third panchromatic pixel value in the second panchromatic image with the same coordinates, the third color pixel value in the double-color second color image, and the third color pixel value in the single-color second color image are arranged in The order is not limited.
  • the electronic device may also use other methods to generate the third target image, which is not limited here.
  • a plurality of second panchromatic pixel values corresponding to the same filter set in the first panchromatic image are combined to read out the third panchromatic pixel value, and the second panchromatic pixel value is read out.
  • the third color pixel value of the first color, the third color pixel value of the second color and the third color pixel value are combined and read out.
  • the value of the third color pixel can mix and arrange different color pixels, so that the distribution of the third color pixels in the generated third target image, such as RGB pixels, is more uniform and the image quality is higher.
  • the resolution and image size of the obtained third target image are further reduced, and the panchromatic pixels have a higher signal-to-noise ratio, and the frame rate of the image is high, so that the power consumption of the three-level pixel combination output is lower, Image processing effect with better signal-to-noise ratio.
  • the third target image includes fully arranged panchromatic pixels, which can improve the overall resolution. At the same time, in the third resolution mode, there is no need to merge pixels of the same color across cycles, and no interpolation is required, which improves the overall resolution.
  • the color pixels of each color such as the pixel values of the first color and the pixel values of the third color, are more dispersed and balanced on the diagonal or anti-diagonal.
  • the full arrangement means that each coordinate has the pixel, and interpolation estimation is not required.
  • the color filters of each filter group are arranged on the diagonal of the corresponding filter group and in a direction parallel to the diagonal, and the panchromatic filter of each filter group
  • the slices are arranged on the anti-diagonal line of the corresponding filter set and the direction parallel to the anti-diagonal line; the above method also includes: in the third resolution mode, a plurality of panchromatic filters in each filter set
  • the panchromatic pixels corresponding to each panchromatic sub-filter of the light sheet are combined to read out the fifth panchromatic pixel value, and a fourth panchromatic image is generated based on each fifth panchromatic pixel value;
  • the color pixels corresponding to each color sub-filter of a color filter of the same color are combined to read out the fifth color pixel value of the first color, the fifth color pixel value of the second color and the fifth color pixel value of the third color.
  • a fourth color image of two colors and a fourth color image of single color are generated Image;
  • the fourth color image of two colors includes the fifth color pixel value of the second color, and one of the fifth color pixel value of the first color and the fifth color pixel value of the third color, and the fourth color pixel value of the single color
  • the color image includes another one of the fifth color pixel value of the first color and the fifth color pixel value of the third color; based on the fourth full-color image, the fourth color image of two colors and the fourth color image of single color, A fifth target image is generated.
  • the combined readout method may be one of averaging, weighted averaging, or addition.
  • the electronic device can The fourth color image is used to generate a fifth target image, and then the fifth target image is transmitted.
  • generating the fifth target image includes: converting the fifth panchromatic pixel value in the fourth panchromatic image with the same coordinates
  • the fifth color pixel value in the fourth color image of two colors and the fifth color pixel value in the fourth color image of single color are arranged alternately in the row direction to generate the fifth target image; or the fourth full color image with the same coordinates
  • the fifth full-color pixel value in the color image, the fifth color pixel value in the double-color fourth color image, and the fifth color pixel value in the single-color fourth color image are arranged alternately in the column direction to generate a fifth target image.
  • the electronic device may also use other methods to generate the fifth target image, which is not limited here.
  • the panchromatic pixels corresponding to the panchromatic sub-filters of the plurality of panchromatic filters in each filter set are combined to read out the fifth panchromatic Pixel values, so that the fourth panchromatic image can be generated more quickly;
  • the color pixels corresponding to the color sub-filters of multiple color filters of the same color in each filter group are combined to read out the color pixels of the first color
  • the fifth color pixel value, the fifth color pixel value of the second color, and the fifth color pixel value of the third color so that a fourth color image of two colors and a fourth color image of single color can be generated more quickly.
  • the above-mentioned embodiment can mix and arrange different color pixels, so that the distribution of each fifth color pixel value, such as RGB pixels, in the generated fifth target image is more uniform, and the image quality is higher. Moreover, the resolution and image size of the obtained fifth target image are further reduced, and the panchromatic pixel 241 has a higher signal-to-noise ratio, and the frame rate of the image is high, so that the power consumption of the three-level pixel combination output is lower , Better signal-to-noise ratio image processing effect.
  • each fifth color pixel value such as RGB pixels
  • another image generation method is provided, which is applied to an image sensor, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes at least a first filter
  • the first filter group and the second filter group include panchromatic filters and color filters; the color filters in the first filter group include the first The color filter and the third color filter, the color filter in the second filter group includes the second color filter; the panchromatic filter and the color filter in the smallest repeating unit are in the smallest repeating unit Alternately arranged on each row and each column, the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter; each pixel in the pixel array is set correspondingly to the filter of the filter array, and the pixel the array is configured to receive light passing through the filter array to generate an electrical signal;
  • the image generation method includes:
  • Operation 2602 in the full resolution mode, read out the full resolution panchromatic pixel value from the panchromatic pixel corresponding to each panchromatic filter, and read out the full resolution full resolution color pixel from the color pixel corresponding to each color filter Color pixel value.
  • a full resolution target image is generated based on the respective full resolution panchromatic pixel values and the respective full resolution color pixel values.
  • the principle of generating the full-resolution target image in this embodiment is similar to the principle of generating the full-resolution target image in the embodiment of FIG. 13 , and will not be repeated here.
  • each filter group includes a plurality of subunits, each subunit includes panchromatic filters and color filters, and the color filters in the subunits are arranged on the diagonal of the subunits, The panchromatic filters in the subunits are arranged on anti-diagonal lines of the subunits; the above method also includes: in the first resolution mode, combining the panchromatic pixels corresponding to the multiple panchromatic filters in each subunit Read out the sixth panchromatic pixel value, and generate a fifth panchromatic image based on each sixth panchromatic pixel value; the resolution corresponding to the first resolution mode is smaller than the resolution corresponding to the full resolution mode; The color pixels corresponding to each color sub-filter of a color filter of the same color are combined to read out the sixth color pixel value, and a fifth color image is generated based on each sixth color pixel value; based on the fifth panchromatic image and the sixth color image. five color images to generate a sixth target image.
  • the electronic device may generate a sixth target image based on the fifth panchromatic image and the fifth color image, and then transmit the sixth target image.
  • generating the sixth target image includes: combining the sixth panchromatic pixel value of each row in the fifth panchromatic image with the sixth color pixel value of each row in the fifth color image Arranging alternately to generate the sixth target image; or alternately arranging each column of sixth panchromatic pixel values in the fifth panchromatic image and each column of sixth color pixel values in the fifth color image to generate the sixth target image.
  • the above method further includes: in the second resolution mode, combining multiple sixth panchromatic pixel values corresponding to the same filter set in the fifth panchromatic image to read out the seventh panchromatic pixel value pixel value, and generate a sixth panchromatic image based on each seventh panchromatic pixel value; the resolution corresponding to the second resolution mode is smaller than the resolution corresponding to the first resolution mode; the fifth color image corresponds to the same filter Multiple sixth color pixel values of the same color in the slice group are combined to read out the seventh color pixel value of the first color, the seventh color pixel value of the second color, and the seventh color pixel value of the third color, and based on The seventh color pixel value of the first color, the seventh color pixel value of the second color and the seventh color pixel value of the third color generate the sixth color image of two colors and the sixth color image of single color; The sixth color image includes the seventh color pixel value of the second color, and one of the seventh color pixel value of the first color and the seventh color pixel value of the third color,
  • the electronic device can The sixth color image is used to generate a seventh target image, and then the seventh target image is transmitted.
  • the seventh target image is generated, including: the seventh panchromatic pixel value in the sixth panchromatic image with the same coordinates , the pixel values of the seventh color in the sixth color image of two colors and the pixel values of the seventh color in the sixth color image of single color are arranged alternately in the row direction to generate the seventh target image; or the sixth full color image with the same coordinates
  • the seventh panchromatic pixel value in the full-color image, the seventh color pixel value in the sixth dual-color image, and the seventh color pixel value in the sixth single-color image are arranged alternately in the column direction to generate a seventh target image.
  • the color filters of each filter group are arranged on the diagonal of the corresponding filter group and in a direction parallel to the diagonal, and the panchromatic filter of each filter group
  • the slices are arranged on the anti-diagonal line of the corresponding filter set and the direction parallel to the anti-diagonal line; the above method also includes: in the second resolution mode, a plurality of panchromatic filters in each filter set The panchromatic pixels corresponding to the light sheet are combined to read out the eighth panchromatic pixel value, and the seventh panchromatic image is generated based on each eighth panchromatic pixel value; multiple color filters of the same color in each filter group
  • the color pixels corresponding to the slice are combined to read out the eighth color pixel value of the first color, the eighth color pixel value of the second color, and the eighth color pixel value of the third color, and based on the eighth color pixel value of the first color,
  • the eighth color pixel value of the second color and the eighth color pixel value of the third color generate a seventh color image of two colors and a seventh color
  • the electronic device can The seventh color image is used to generate an eighth target image, and then the eighth target image is transmitted.
  • generating the eighth target image includes: the eighth panchromatic pixel value in the seventh panchromatic image with the same coordinates
  • the eighth color pixel value in the seventh color image of two colors and the eighth color pixel value in the seventh color image of single color are arranged alternately in the row direction to generate the eighth target image; or the seventh full color image with the same coordinates
  • the eighth panchromatic pixel value in the full-color image, the eighth color pixel value in the seventh dual-color image, and the eighth color pixel value in the seventh single-color image are arranged alternately in the column direction to generate an eighth target image.
  • the principle of generating the sixth target image, the seventh target image, and the eighth target image is similar to the principle of generating the second target image, third target image, and fifth target image, and will not be repeated here.
  • 26 may include multiple sub-operations or multiple stages, and these sub-operations or stages are not necessarily performed at the same time, but may be completed at the same time Executed at different times, the execution order of these sub-operations or stages is not necessarily sequential, but may be performed in turn or alternately with other operations or at least a part of sub-operations or stages of other operations.
  • Fig. 27 is a structural block diagram of an image generating device of an embodiment.
  • an image generating device is provided, which is applied to an image sensor, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes at least a first filter group and the second filter group; the first filter group and the second filter group include panchromatic filters and color filters; the color filters in the first filter group include the first color filter light sheet and the third color filter, the color filter in the second filter group includes the second color filter; the panchromatic filter and the color filter in the minimum repeating unit are in each minimum repeating unit Arranged alternately in one row and each column, the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter; each panchromatic filter includes N rows and N columns of panchromatic sub-filters, Each color filter includes N rows and N columns of color sub-filters, N rows and N columns of color sub-filters have the same color as the color filter, and
  • the readout module 2702 is configured to read out the full-resolution panchromatic pixel value from the panchromatic pixel corresponding to each panchromatic sub-filter in the panchromatic filter in the full-resolution mode, and read out the full-resolution panchromatic pixel value of the color filter The full-resolution color pixel value is read out from the color pixel corresponding to each color sub-filter.
  • An image generating module 2704 configured to generate a full-resolution target image based on each full-resolution panchromatic pixel value and each full-resolution color pixel value.
  • the above image generation device reads out the full-resolution panchromatic pixel value from the panchromatic pixel corresponding to each panchromatic sub-filter in the panchromatic filter in the full-resolution mode, and reads out the full-resolution panchromatic pixel value of each panchromatic sub-filter in the color filter.
  • the color pixel corresponding to each color sub-filter reads out the full-resolution color pixel value; and the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter, and the panchromatic channel information can be integrated into the image.
  • the overall light input amount is increased, so that based on each full-resolution panchromatic pixel value and each full-resolution color pixel value, a full-resolution target image with more information and clearer detail analysis can be generated.
  • panchromatic filters and color filters of various colors are scattered and arranged, so that full-resolution panchromatic pixel values and full-resolution color pixel values of various colors are also scattered and arranged in imaging, It can improve the color resolution capability and brightness change resolution capability.
  • the color filters of various colors are mixed and arranged, so that the full-resolution color pixel values of various colors in the imaging are also mixed and arranged, which can reduce the risk of false colors.
  • each panchromatic filter and the color filter are alternately arranged on each row and each column, and each panchromatic filter includes N rows and N columns of panchromatic sub-filters, and each color filter includes N rows and N columns of color sub-filters, N rows and N columns of color sub-filters have the same color as the color filter, and N is a positive integer; and each pixel in the pixel array and the sub-filter of the filter array
  • the setting corresponding to slices can increase the color resolution of each row and column of the generated full-resolution target image, making the full-resolution target image richer in color.
  • the above-mentioned readout module 2702 is also used to combine the panchromatic pixels corresponding to each panchromatic sub-filter in each panchromatic filter to read out the first panchromatic pixel in the first resolution mode.
  • the image generating module 2704 is further configured to generate a first target image based on each first panchromatic pixel value and each first color pixel value.
  • each filter group includes a plurality of subunits, each subunit includes panchromatic filters and color filters, and the color filters in the subunits are arranged on the diagonal of the subunits, The panchromatic filters in the subunits are arranged on the anti-diagonal lines of the subunits;
  • the above-mentioned readout module 2702 is also used to convert the first target image corresponding to multiple The first panchromatic pixel values are combined to read out the second panchromatic pixel values, and the above-mentioned image generation module 2704 is also configured to generate the first panchromatic image based on each second panchromatic pixel value; the resolution corresponding to the second resolution mode is smaller than that of the first panchromatic pixel value A resolution corresponding to a resolution mode; the above-mentioned readout module 2702 is also used to combine and read out the second color pixel values corresponding to a plurality of first color pixel values of the same color in each subunit in the first target image, and the above-mentioned The image generation
  • the above-mentioned image generation module 2704 is further configured to alternately arrange the second panchromatic pixel values of each row in the first panchromatic image and the second color pixel values of each row in the first color image to generate the second target image; or alternately arrange each column of second panchromatic pixel values in the first panchromatic image with each column of second color pixel values in the first color image to generate a second target image.
  • the above-mentioned readout module 2702 is also used to combine and read out the second panchromatic pixel values corresponding to the same filter set in the first panchromatic image in the third resolution mode.
  • the image generating module 2704 is further configured to generate a second panchromatic image based on each third panchromatic pixel value; the resolution corresponding to the third resolution mode is smaller than the resolution corresponding to the second resolution mode; the above The readout module 2702 is also used to combine and read out the third color pixel value of the first color, the The third color pixel value and the third color pixel value of the third color, the above-mentioned image generation module 2704 is further configured to be based on the third color pixel value of the first color, the third color pixel value of the second color and the third color pixel value of the third color Three-color pixel values to generate a two-color second color image and a single-color second color image; the two-color second color image includes the third color pixel value of the second color
  • the image generating module 2704 is further configured to convert the third panchromatic pixel value in the second panchromatic image with the same coordinates, the third color pixel value in the dual-color second color image, and the third color pixel value in the single-color
  • the third color pixel values in the two-color image are arranged alternately in the row direction to generate the third target image;
  • the three-color pixel values and the third-color pixel values in the single-color second-color image are arranged alternately in a column direction to generate a third target image.
  • each filter group includes a plurality of subunits, each subunit includes panchromatic filters and color filters, and the color filters in the subunits are arranged on the diagonal of the subunits, The panchromatic filters in the subunits are arranged on the anti-diagonal lines of the subunits; the above-mentioned readout module 2702 is also used for, in the second resolution mode, each panchromatic filter in each subunit The panchromatic pixels corresponding to the color sub-filters are combined to read out the fourth panchromatic pixel value, and the above-mentioned image generation module 2704 is also used to generate a third panchromatic image based on each fourth panchromatic pixel value; the above-mentioned readout module 2702 is also used to In order to combine the color pixels corresponding to the color sub-filters of multiple color filters of the same color in each subunit to read out the fourth color pixel value, the above-mentioned image generation module 2704 is also used to Generate a third color image; the image generating
  • the above-mentioned image generation module 2704 is further configured to alternately arrange the fourth panchromatic pixel values in each row in the third panchromatic image and the fourth color pixel values in each row in the third color image to generate the fourth target image; or alternately arrange each column of fourth panchromatic pixel values in the third panchromatic image and each column of fourth color pixel values in the third color image to generate a fourth target image.
  • the color filters of each filter group are arranged on the diagonal of the corresponding filter group and in a direction parallel to the diagonal, and the panchromatic filter of each filter group
  • the slices are arranged on the anti-diagonal line of the corresponding filter set and the direction parallel to the anti-diagonal line; the above-mentioned readout module 2702 is also used in the third resolution mode, multiple The panchromatic pixels corresponding to each panchromatic sub-filter of the panchromatic filter are combined to read out the fifth panchromatic pixel value, and the above-mentioned image generation module 2704 is also used to generate a fourth panchromatic image based on each fifth panchromatic pixel value
  • the readout module 2702 is also used to combine the color pixels corresponding to the color sub-filters of multiple color filters of the same color in each filter set to read out the fifth color pixel value of the first color , the fifth color pixel value of the second color and the fifth color pixel value of the third color, the above-mentioned image generation module 2704 is also used for
  • the image generating module 2704 is further configured to convert the fifth panchromatic pixel value in the fourth panchromatic image with the same coordinates, the fifth color pixel value in the double-color fourth color image, and the fifth color pixel value in the single color
  • the fifth color pixel values in the four-color image are arranged alternately in the row direction to generate the fifth target image;
  • the five-color pixel values and the fifth-color pixel values in the single-color fourth-color image are arranged alternately in a column direction to generate a third target image.
  • another image generating device is provided, which is applied to an image sensor, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes at least a first filter
  • the first filter group and the second filter group include panchromatic filters and color filters; the color filters in the first filter group include the first The color filter and the third color filter, the color filter in the second filter group includes the second color filter; the panchromatic filter and the color filter in the smallest repeating unit are in the smallest repeating unit Alternately arranged on each row and each column, the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter; each pixel in the pixel array is set correspondingly to the filter of the filter array, and the pixel
  • the array is configured to receive light passing through the filter array to generate electrical signals;
  • the image generation device includes: a readout module and an image generation module, wherein:
  • the readout module is used to read out the full-resolution panchromatic pixel value from the panchromatic pixel corresponding to each panchromatic filter in the full-resolution mode, and read out the color pixel corresponding to each color filter Full resolution color pixel values.
  • An image generating module configured to generate a full-resolution target image based on each full-resolution panchromatic pixel value and each full-resolution color pixel value.
  • the above image generation device reads out the full resolution panchromatic pixel value from the panchromatic pixel corresponding to each panchromatic filter, and reads out the full resolution panchromatic pixel value from the color pixel corresponding to each color filter.
  • the resolution color pixel value; and the amount of light transmitted by the panchromatic filter is greater than the amount of light transmitted by the color filter, which can integrate the information of the panchromatic channel into the image and improve the overall light input, so that based on each full resolution Full-resolution full-color pixel values and individual full-resolution color pixel values can generate full-resolution target images with more information and clearer detail resolution.
  • panchromatic filters and color filters of various colors are scattered and arranged, so that full-resolution panchromatic pixel values and full-resolution color pixel values of various colors are also scattered and arranged in imaging, It can improve the color resolution capability and brightness change resolution capability.
  • the color filters of various colors are mixed and arranged, so that the full-resolution color pixel values of various colors in the imaging are also mixed and arranged, which can reduce the risk of false colors.
  • panchromatic filters and color filters are arranged alternately on each row and each column, and each pixel in the pixel array corresponds to the filter of the filter array, which can improve the generated full-resolution target image
  • the color resolution of each row and each column of the image makes the color of the full-resolution target image richer.
  • each filter group includes a plurality of subunits, each subunit includes panchromatic filters and color filters, and the color filters in the subunits are arranged on the diagonal of the subunits, The panchromatic filters in the subunits are arranged on the anti-diagonal lines of the subunits; the above-mentioned readout module is also used to convert the panchromatic colors corresponding to the multiple panchromatic filters in each subunit in the first resolution mode.
  • the image generation module is also used to generate a fifth panchromatic image based on each sixth panchromatic pixel value; the resolution corresponding to the first resolution mode is smaller than the resolution corresponding to the full resolution mode rate; the above-mentioned readout module is also used to combine and read out the sixth color pixel value corresponding to the color pixels corresponding to each color sub-filter of a plurality of color filters of the same color in each subunit, and the above-mentioned image generation module is also used for Generate a fifth color image based on each sixth color pixel value; the image generation module is further configured to generate a sixth target image based on the fifth panchromatic image and the fifth color image.
  • the above image generation module is further configured to alternately arrange the sixth panchromatic pixel values in each row in the fifth panchromatic image and the sixth color pixel values in each row in the fifth color image to generate the sixth target image ; or alternately arrange the sixth panchromatic pixel values in each column of the fifth panchromatic image and the sixth color pixel values in each column of the fifth color image to generate a sixth target image.
  • the above-mentioned readout module is also used to combine and read out the seventh panchromatic pixel values corresponding to the same filter set in the fifth panchromatic image in the second resolution mode.
  • the image generating module is further configured to generate a sixth panchromatic image based on each seventh panchromatic pixel value; the resolution corresponding to the second resolution mode is smaller than the resolution corresponding to the first resolution mode; the readout The module is also used to combine and read out the seventh color pixel value of the first color and the seventh color pixel value of the second color corresponding to multiple sixth color pixel values of the same color in the same filter group in the fifth color image.
  • the pixel value and the seventh color pixel value of the third color is also used for the seventh color pixel value based on the first color, the seventh color pixel value of the second color and the seventh color pixel value of the third color , generate the sixth color image of two colors and the sixth color image of single color;
  • the sixth color image of two colors includes the seventh color pixel value of the second color, and the seventh color pixel value of the first color and the third color
  • One of the seventh color pixel values of the single color, the sixth color image of the single color includes the other one of the seventh color pixel value of the first color and the seventh color pixel value of the third color;
  • the image generation module is also used for A seventh target image is generated based on the sixth panchromatic image, the sixth color image of two colors and the sixth color image of single color.
  • the above-mentioned image generation module is also used to convert the seventh panchromatic pixel value in the sixth panchromatic image with the same coordinates, the seventh color pixel value in the sixth color image of two colors, and the sixth color pixel value in the single color
  • the seventh color pixel values in the color image are arranged alternately in the row direction to generate the seventh target image;
  • the color pixel values and the seventh color pixel values in the single-color sixth color image are arranged alternately in the column direction to generate a seventh target image.
  • the color filters of each filter group are arranged on the diagonal of the corresponding filter group and in a direction parallel to the diagonal, and the panchromatic filter of each filter group
  • the slices are arranged on the anti-diagonal line of the corresponding filter set and the direction parallel to the anti-diagonal line;
  • the above-mentioned readout module is also used in the second resolution mode, multiple full The panchromatic pixels corresponding to the color filter are combined to read out the eighth panchromatic pixel value, and the above image generation module is also used to generate a seventh panchromatic image based on each eighth panchromatic pixel value;
  • the above readout module is also used to combine each The color pixels corresponding to a plurality of color filters of the same color in a filter group are combined to read out the eighth color pixel value of the first color, the eighth color pixel value of the second color and the eighth color pixel value of the third color.
  • the above-mentioned image generation module is also used to generate the seventh color image of two colors and the eighth color pixel value of the third color based on the eighth color pixel value of the first color, the eighth color pixel value of the second color and the third color
  • the seventh color image of single color includes the eighth color pixel value of the second color, and one of the eighth color pixel value of the first color and the eighth color pixel value of the third color
  • the seventh color image of a single color includes another one of the eighth color pixel value of the first color and the eighth color pixel value of the third color
  • the seventh color image and the single-color seventh color image generate an eighth target image.
  • the above-mentioned image generation module is also used to convert the eighth panchromatic pixel value in the seventh panchromatic image with the same coordinates, the eighth color pixel value in the seventh color image of two colors, and the seventh color pixel value in the single color
  • the eighth color pixel value in the color image is arranged alternately in the row direction to generate the eighth target image;
  • the color pixel values and the eighth color pixel values in the single-color seventh color image are arranged alternately in the column direction to generate an eighth target image.
  • each module in the above image generating device is only for illustration. In other embodiments, the image generating device can be divided into different modules according to needs, so as to complete all or part of the functions of the above image generating device.
  • Each module in the above-mentioned image generating device can be fully or partially realized by software, hardware and a combination thereof.
  • the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that the processor can invoke and execute the corresponding operations of the above-mentioned modules.
  • Fig. 28 is a schematic diagram of the internal structure of an electronic device in one embodiment.
  • the electronic device can be any terminal device such as mobile phone, tablet computer, notebook computer, desktop computer, PDA (Personal Digital Assistant, personal digital assistant), POS (Point of Sales, sales terminal), vehicle-mounted computer, wearable device, etc.
  • the electronic device includes a processor and memory connected by a system bus.
  • the processor may include one or more processing units.
  • the processor can be a CPU (Central Processing Unit, central processing unit) or a DSP (Digital Signal Processing, digital signal processor), etc.
  • the memory may include non-volatile storage media and internal memory. Nonvolatile storage media store operating systems and computer programs.
  • the computer program can be executed by a processor to implement an image generation method provided in the following embodiments.
  • the internal memory provides a high-speed running environment for the operating system computer program in the non-volatile storage medium.
  • each module in the image generation device provided in the embodiment of the present application may be in the form of a computer program.
  • the computer program can run on a terminal or a server.
  • the program modules constituted by the computer program can be stored in the memory of the electronic device.
  • the operations of the methods described in the embodiments of the present application are realized.
  • the embodiment of the present application also provides a computer-readable storage medium.
  • One or more non-transitory computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform the operations of the image generation method.
  • the embodiment of the present application also provides a computer program product containing instructions, which, when run on a computer, causes the computer to execute the image generation method.
  • Non-volatile memory can include ROM (Read-Only Memory, read-only memory), PROM (Programmable Read-only Memory, programmable read-only memory), EPROM (Erasable Programmable Read-Only Memory, erasable programmable read-only memory) Memory), EEPROM (Electrically Erasable Programmable Read-only Memory, Electrically Erasable Programmable Read-only Memory) or flash memory.
  • Volatile memory can include RAM (Random Access Memory, Random Access Memory), which is used as external cache memory.
  • RAM is available in various forms, such as SRAM (Static Random Access Memory, static random access memory), DRAM (Dynamic Random Access Memory, dynamic random access memory), SDRAM (Synchronous Dynamic Random Access Memory , synchronous dynamic random access memory), double data rate DDR SDRAM (Double Data Rate Synchronous Dynamic Random Access memory, double data rate synchronous dynamic random access memory), ESDRAM (Enhanced Synchronous Dynamic Random Access memory, enhanced synchronous dynamic random access memory access memory), SLDRAM (Sync Link Dynamic Random Access Memory, synchronous link dynamic random access memory), RDRAM (Rambus Dynamic Random Access Memory, bus dynamic random access memory), DRDRAM (Direct Rambus Dynamic Random Access Memory, interface dynamic random access memory) memory).
  • SRAM Static Random Access Memory, static random access memory
  • DRAM Dynanamic Random Access Memory, dynamic random access memory
  • SDRAM Synchronous Dynamic Random Access Memory , synchronous dynamic random access memory
  • double data rate DDR SDRAM Double Data Rate Synchronous Dynamic Random Access memory, double

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Color Television Image Signal Generators (AREA)

Abstract

L'invention concerne un capteur d'image (21), un module de caméra (20), un dispositif électronique (100), un procédé et un appareil de génération d'images, un support de stockage lisible par ordinateur et un produit de programme informatique. Le capteur d'image (21) comprend une matrice de filtres (23) et une matrice de pixels (24), la matrice de filtres (23) comprenant une unité de répétition minimale (230), et l'unité de répétition minimale (230) comprenant au moins un premier groupe de filtres (231) et un second groupe de filtres (232) ; chaque groupe de filtres comprend des filtres panchromatiques (233) et des filtres de couleur (234) ; les filtres de couleur (234) dans le premier groupe de filtres (231) comprennent un premier filtre de couleur et un troisième filtre de couleur, et les filtres de couleur (234) dans le second groupe de filtres (232) comprennent un second filtre de couleur ; les filtres panchromatiques (233) et les filtres de couleur (234) de l'unité de répétition minimale (230) sont disposés alternativement sur les lignes et les colonnes de l'unité de répétition minimale (230), et la quantité de lumière entrant à travers les filtres panchromatiques (233) est supérieure à la quantité de lumière entrant à travers les filtres de couleur (234) ; et chaque filtre comprend N lignes et N colonnes de sous-filtres de la même couleur que celle du filtre, et N est un nombre entier positif.
PCT/CN2022/124016 2021-12-01 2022-10-09 Capteur d'image, module de caméra, dispositif électronique et procédé et appareil de génération d'image WO2023098282A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111456002.8 2021-12-01
CN202111456002.8A CN114040084A (zh) 2021-12-01 2021-12-01 图像传感器、摄像模组、电子设备、图像生成方法和装置

Publications (1)

Publication Number Publication Date
WO2023098282A1 true WO2023098282A1 (fr) 2023-06-08

Family

ID=80139579

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/124016 WO2023098282A1 (fr) 2021-12-01 2022-10-09 Capteur d'image, module de caméra, dispositif électronique et procédé et appareil de génération d'image

Country Status (2)

Country Link
CN (1) CN114040084A (fr)
WO (1) WO2023098282A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114554046A (zh) * 2021-12-01 2022-05-27 Oppo广东移动通信有限公司 图像传感器、摄像模组、电子设备、图像生成方法和装置
CN114040084A (zh) * 2021-12-01 2022-02-11 Oppo广东移动通信有限公司 图像传感器、摄像模组、电子设备、图像生成方法和装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009237321A (ja) * 2008-03-27 2009-10-15 Fujifilm Corp 画像露光装置
CN104280803A (zh) * 2013-07-01 2015-01-14 全视科技有限公司 彩色滤光片阵列、彩色滤光片阵列设备及图像传感器
CN111818314A (zh) * 2020-06-23 2020-10-23 北京迈格威科技有限公司 一种滤波器阵列及图像传感器
CN112118378A (zh) * 2020-10-09 2020-12-22 Oppo广东移动通信有限公司 图像获取方法及装置、终端和计算机可读存储介质
CN213279832U (zh) * 2020-10-09 2021-05-25 Oppo广东移动通信有限公司 图像传感器、相机和终端
CN114040084A (zh) * 2021-12-01 2022-02-11 Oppo广东移动通信有限公司 图像传感器、摄像模组、电子设备、图像生成方法和装置

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113676675B (zh) * 2021-08-16 2023-08-15 Oppo广东移动通信有限公司 图像生成方法、装置、电子设备和计算机可读存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2009237321A (ja) * 2008-03-27 2009-10-15 Fujifilm Corp 画像露光装置
CN104280803A (zh) * 2013-07-01 2015-01-14 全视科技有限公司 彩色滤光片阵列、彩色滤光片阵列设备及图像传感器
CN111818314A (zh) * 2020-06-23 2020-10-23 北京迈格威科技有限公司 一种滤波器阵列及图像传感器
CN112118378A (zh) * 2020-10-09 2020-12-22 Oppo广东移动通信有限公司 图像获取方法及装置、终端和计算机可读存储介质
CN213279832U (zh) * 2020-10-09 2021-05-25 Oppo广东移动通信有限公司 图像传感器、相机和终端
CN114040084A (zh) * 2021-12-01 2022-02-11 Oppo广东移动通信有限公司 图像传感器、摄像模组、电子设备、图像生成方法和装置

Also Published As

Publication number Publication date
CN114040084A (zh) 2022-02-11

Similar Documents

Publication Publication Date Title
WO2023098284A1 (fr) Capteur d'image, module de caméra, dispositif électronique, et procédé et appareil de génération d'image
WO2023098282A1 (fr) Capteur d'image, module de caméra, dispositif électronique et procédé et appareil de génération d'image
WO2023109265A1 (fr) Capteur d'image, module de prise de vue, dispositif électronique, procédé et appareil de génération d'images
WO2023109264A1 (fr) Capteur d'image, module de caméra, dispositif électronique et procédé et appareil de génération d'image
US10136107B2 (en) Imaging systems with visible light sensitive pixels and infrared light sensitive pixels
US7999870B2 (en) Sampling and readout of an image sensor having a sparse color filter array pattern
WO2021223590A1 (fr) Capteur d'image, procédé de commande, ensemble caméra et terminal mobile
US8922683B2 (en) Color imaging element and imaging apparatus
WO2023082766A1 (fr) Capteur d'image, module de caméra, dispositif électronique et procédé et appareil de génération d'image
WO2023124607A1 (fr) Procédé et appareil de génération d'image, dispositif électronique et support de stockage lisible par ordinateur
CN113840067B (zh) 图像传感器、图像生成方法、装置和电子设备
CN113170061B (zh) 图像传感器、成像装置、电子设备、图像处理系统及信号处理方法
CN113676708B (zh) 图像生成方法、装置、电子设备和计算机可读存储介质
US8111298B2 (en) Imaging circuit and image pickup device
CN113573030B (zh) 图像生成方法、装置、电子设备和计算机可读存储介质
US20230007191A1 (en) Image sensor, imaging apparatus, electronic device, image processing system, and signal processing method
WO2023098230A1 (fr) Capteur d'image, module de caméra, dispositif électronique et procédé et appareil de génération d'image
US20220150450A1 (en) Image capturing method, camera assembly, and mobile terminal
WO2014091706A1 (fr) Dispositif de capture d'image
WO2021248379A1 (fr) Capteur d'image et dispositif électronique
CN113132692B (zh) 图像传感器、成像装置、电子设备、图像处理系统及信号处理方法
KR20230067491A (ko) 이미지 센서, 어플리케이션 프로세서 및 이미지 센싱 장치