WO2023035900A1 - Capteur d'images, procédé et appareil de génération d'images, et dispositif électronique - Google Patents

Capteur d'images, procédé et appareil de génération d'images, et dispositif électronique Download PDF

Info

Publication number
WO2023035900A1
WO2023035900A1 PCT/CN2022/113450 CN2022113450W WO2023035900A1 WO 2023035900 A1 WO2023035900 A1 WO 2023035900A1 CN 2022113450 W CN2022113450 W CN 2022113450W WO 2023035900 A1 WO2023035900 A1 WO 2023035900A1
Authority
WO
WIPO (PCT)
Prior art keywords
color
pixel
filter
panchromatic
pixels
Prior art date
Application number
PCT/CN2022/113450
Other languages
English (en)
Chinese (zh)
Inventor
杨鑫
Original Assignee
Oppo广东移动通信有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Oppo广东移动通信有限公司 filed Critical Oppo广东移动通信有限公司
Publication of WO2023035900A1 publication Critical patent/WO2023035900A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50—Constructional details
    • H04N23/55—Optical parts specially adapted for electronic image sensors; Mounting thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/40—Extracting pixel data from image sensors by controlling scanning circuits, e.g. by modifying the number of pixels sampled or to be sampled
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/71—Charge-coupled device [CCD] sensors; Charge-transfer registers specially adapted for CCD sensors
    • H04N25/75—Circuitry for providing, modifying or processing image signals from the pixel array
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/268—Signal distribution or switching

Definitions

  • the present application relates to the field of image technology, and in particular to an image sensor, an image generation method, device, electronic equipment, and computer-readable storage medium.
  • An image sensor the image sensor includes a filter array and a pixel array
  • the filter array includes a minimum repeating unit, the minimum repeating unit includes a plurality of filter groups, and the filter groups include color a color filter and a panchromatic filter, the color filter has a narrower spectral response than that of the panchromatic filter, and each of the color filter and the panchromatic filter includes 9 sub- Optical filter
  • the pixel array includes a plurality of pixels, the pixels of the pixel array correspond to the sub-filters of the optical filter array, and the pixel array is configured to receive light passing through the filter light from an array of chips to generate electrical signals.
  • the above-mentioned image sensor includes a filter array and a pixel array, and the filter array includes a minimum repeating unit, and the minimum repeating unit includes a plurality of filter groups, and the filter groups include color filters and panchromatic
  • the color filter has a narrower spectral response than the panchromatic filter, and can obtain more light when shooting, so that there is no need to adjust the shooting parameters, without affecting the stability of the shooting In the case of low light, the imaging quality in dark light can be improved. When imaging in dark light, both stability and quality can be taken into account. The stability and quality of imaging in dark light are both high.
  • the pixel array includes a plurality of pixels, and the pixels of the pixel array are arranged corresponding to the sub-filters of the filter array, and the pixel array is configured to receive light passing through the filter array to generate an electrical signal.
  • the pixels corresponding to the 9 sub-filters can be combined and output to obtain an image with a high signal-to-noise ratio.
  • the pixels corresponding to each sub-filter can be output separately to obtain Images with high resolution and high signal-to-noise ratio.
  • the image sensor can output images with various resolutions, so as to adapt to different application scenarios and improve the imaging quality in each application scenario.
  • An image generation method applied to an image sensor, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, the minimum repeating unit includes a plurality of filter groups, the The filter set includes a color filter and a panchromatic filter, the color filter having a narrower spectral response than the panchromatic filter, the color filter and the panchromatic filter
  • the light sheets each include 9 sub-filters; the pixel array includes a plurality of pixels, the pixels of the pixel array correspond to the sub-filters of the filter array, and the pixel array is configured to receive light passing through the filter array to generate an electrical signal;
  • the method also includes:
  • the full-size full-color pixels are panchromatic pixels
  • a first target image is generated based on the full-size panchromatic channel image and the original image.
  • An image generating device applied to an image sensor, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, the minimum repeating unit includes a plurality of filter groups, the The filter set includes a color filter and a panchromatic filter, the color filter having a narrower spectral response than the panchromatic filter, the color filter and the panchromatic filter
  • the light sheets each include 9 sub-filters; the pixel array includes a plurality of pixels, the pixels of the pixel array correspond to the sub-filters of the filter array, and the pixel array is configured to receive light passing through the filter array to generate an electrical signal;
  • the devices include:
  • An interpolation module configured to use the texture information of the color pixels in the original image in the first resolution mode to interpolate all the color pixels in the original image into panchromatic pixels to obtain a full-size panchromatic channel image;
  • the pixels in the full-size panchromatic channel map are panchromatic pixels;
  • a generating module configured to generate a first target image based on the full-size panchromatic channel image and the original image.
  • An electronic device including a memory, a processor, and an image sensor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor performs the following operations:
  • the full-size full-color pixels are panchromatic pixels
  • a first target image is generated based on the full-size panchromatic channel image and the original image.
  • a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the following operations are realized:
  • the full-size full-color pixels are panchromatic pixels
  • a first target image is generated based on the full-size panchromatic channel image and the original image.
  • a computer program product including a computer program, the computer program is executed by a processor to achieve the following operations:
  • the full-size full-color pixels are panchromatic pixels
  • a first target image is generated based on the full-size panchromatic channel image and the original image.
  • the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes a plurality of filter groups , the filter set includes a color filter and a panchromatic filter, the color filter has a narrower spectral response than the panchromatic filter, and both the color filter and the panchromatic filter include 9 sub-filters light sheet;
  • the pixel array includes a plurality of pixels, the pixels of the pixel array are arranged corresponding to the sub-filters of the filter array, and the pixel array is configured to receive light passing through the filter array to generate an electrical signal, and to
  • the first resolution mode is used in scenes with high resolution requirements, and the texture information of color pixels in the original image is used to interpolate the color pixels in the original image into panchromatic pixels to obtain a full-size full-color image with the same size as the original image.
  • the pixels in the full-size panchromatic channel map are all panchromatic pixels
  • the first target image is generated based on the full-size panchromatic channel map and the original image, and the panchromatic channel information can be fused into the original image, so that more information can be generated, Detailed analysis of the clearer first target image, so as to achieve the image processing effect of high-resolution full-size full-resolution output, which can meet the high-quality requirements of users for images.
  • Fig. 1 is a schematic structural diagram of an electronic device in an embodiment.
  • FIG. 2 is an exploded schematic diagram of an image sensor in one embodiment.
  • Fig. 3 is a schematic diagram of connection between a pixel array and a readout circuit in one embodiment.
  • Figure 4 is a schematic diagram of the arrangement of the smallest repeating units in an embodiment.
  • Fig. 5 is a schematic diagram of the arrangement of the smallest repeating units in another embodiment.
  • Figure 6 is a schematic diagram of the arrangement of the smallest repeating units in one of the embodiments.
  • Fig. 7 is a schematic diagram of the arrangement of the smallest repeating units in another embodiment.
  • Fig. 8 is a flowchart of an image generation method in one embodiment.
  • Fig. 9 is a schematic diagram of generating a first target image using a full-resolution output mode under the first resolution mode in an embodiment.
  • Fig. 10a is a schematic diagram of associated pixels for each texture direction in one embodiment.
  • Fig. 10b is a schematic diagram of associated pixels of each texture direction in another embodiment.
  • Fig. 11 is a schematic diagram of calculating interpolation weights corresponding to color pixels in an embodiment.
  • Fig. 12 is a schematic diagram of generating a full-size panchromatic channel image in an embodiment.
  • Fig. 13a is a schematic diagram of obtaining a second target image in a second resolution mode in an embodiment.
  • Fig. 13b is a schematic flowchart of obtaining a third target image in a third resolution mode in an embodiment.
  • Fig. 13c is a schematic flowchart of generating a fourth target image in a fourth resolution mode in an embodiment.
  • Fig. 14 is a structural block diagram of an image generating device in an embodiment.
  • Fig. 15 is a block diagram of the internal structure of an electronic device in one embodiment.
  • first and second may be used to describe various elements herein, but these elements are not limited by these terms. These terms are only used to distinguish one element from another element.
  • a first object image could be termed a second object image, and, similarly, a second object image could be termed a first object image, without departing from the scope of the present application.
  • Both the first and the second are target images, but they are not the same target image.
  • the electronic device 100 includes a mobile phone, a tablet computer, a notebook computer, an ATM, a gate machine, a smart watch, a head-mounted display device, etc. It can be understood that the electronic device 100 can also be any other device with image processing functions.
  • the electronic device 100 includes a camera 20 , a processor 30 and a housing 40 . Both the camera 20 and the processor 30 are disposed in the casing 40, and the casing 40 can also be used to install functional modules such as a power supply device and a communication device of the terminal 100, so that the casing 40 provides dustproof, dropproof, waterproof, etc. for the functional modules. Protect.
  • the camera 20 may be a front camera, a rear camera, a side camera, an under-screen camera, etc., which is not limited here.
  • the camera 20 includes a lens and an image sensor 21. When the camera 20 captures an image, light passes through the lens and reaches the image sensor 21.
  • the image sensor 21 is used to convert the light signal irradiated on the image sensor 21 into an electrical signal.
  • the image sensor 21 includes a microlens array 22 , a filter array 23 , and a pixel array 24 .
  • the microlens array 22 includes a plurality of microlenses 221, the microlenses 221, the sub-filters in the filter array 23, and the pixels in the pixel array 24 are set in one-to-one correspondence, and the microlenses 221 are used to gather the incident light.
  • the collected light will pass through the corresponding sub-filter, and then projected onto the pixel, and be received by the corresponding pixel, and the pixel converts the received light into an electrical signal.
  • the filter array 23 includes a plurality of minimal repeating units 231 .
  • the minimum repeating unit 231 may include a plurality of filter sets 232 .
  • Each filter set 232 includes a panchromatic filter 233 and a color filter 234 having a narrower spectral response than the panchromatic filter 233 .
  • Each panchromatic filter 233 includes 9 sub-filters 2331
  • each color filter 234 includes 9 sub-filters 2341 .
  • Different color filters 234 are also included in different filter sets.
  • the colors corresponding to the wavelength bands of the transmitted light of the color filters 234 of the filter sets 232 in the minimum repeating unit 231 include color a, color b and/or color c.
  • the color corresponding to the wavelength band of the transmitted light of the color filter 234 of the filter group 232 includes color a, color b and color c, or color a, color b or color c, or color a and color b, or color b and color c, or color a and color c.
  • the color a is red
  • the color b is green
  • the color c is blue, or for example, the color a is magenta, the color b is cyan, and the color c is yellow, etc., which are not limited here.
  • the width of the wavelength band of the light transmitted by the color filter 234 is smaller than the width of the wavelength band of the light transmitted by the panchromatic filter 233, for example, the wavelength band of the light transmitted by the color filter 234 It can correspond to the wavelength band of red light, the wavelength band of green light, or the wavelength band of blue light.
  • the wavelength band of the light transmitted by the panchromatic filter 233 is the wavelength band of all visible light, that is to say, the color filter 234 only allows specific color light
  • the panchromatic filter 233 can pass light of all colors.
  • the wavelength band of the light transmitted by the color filter 234 may also correspond to the wavelength band of other colored light, such as magenta light, violet light, cyan light, yellow light, etc., which is not limited here.
  • the ratio of the number of color filters 234 to the number of panchromatic filters 233 in the filter set 232 may be 1:3, 1:1 or 3:1.
  • the ratio of the number of color filters 234 to the number of panchromatic filters 233 is 1:3, then the number of color filters 234 is 1, and the number of panchromatic filters 233 is 3.
  • Color filter 233 quantity is more, and the imaging quality under dark light is better;
  • the ratio of the quantity of color filter 234 and the quantity of panchromatic filter 233 is 1:1, then color filter 234
  • the number of color filters 234 is 2, and the number of panchromatic filters 233 is 2.
  • the imaging quality under dark light is also better; or, the number of color filters 234 and the full color filter
  • the ratio of the number of color filters 233 is 3:1, then the number of color filters 234 is 3, and the number of panchromatic filters 233 is 1. At this time, better color performance can be obtained, and dark Image quality under light.
  • the pixel array 24 includes a plurality of pixels, and the pixels of the pixel array 24 are arranged corresponding to the sub-filters of the filter array 23 .
  • the pixel array 24 is configured to receive light passing through the filter array 23 to generate electrical signals.
  • the pixel array 24 is configured to receive the light passing through the filter array 23 to generate an electrical signal, which means that the pixel array 24 is used to detect a scene of a given set of subjects passing through the filter array 23
  • the light is photoelectrically converted to generate an electrical signal.
  • the light rays of the scene for a given set of subjects are used to generate image data.
  • the subject is a building
  • the scene of a given set of subjects refers to the scene where the building is located, which may also contain other objects.
  • the pixel array 24 includes a plurality of minimal repeating units 241 , the smallest repeating unit 241 includes a plurality of pixel groups 242 , and the plurality of pixel groups 242 includes a panchromatic pixel group 243 and a color pixel group 244 .
  • Each panchromatic pixel group 243 includes 9 panchromatic pixels 2431
  • each color pixel group 244 includes 9 color pixels 2441 .
  • Each panchromatic pixel 2431 corresponds to a sub-filter 2331 in the panchromatic filter 233, and the panchromatic pixel 2431 receives light passing through the corresponding sub-filter 2331 to generate an electrical signal.
  • Each color pixel 2441 corresponds to a sub-filter 2341 of the color filter 234, and the color pixel 2441 receives light passing through the corresponding sub-filter 2341 to generate an electrical signal.
  • the image sensor 21 in this embodiment includes a filter array 23 and a pixel array 24, the filter array 23 includes a minimum repeating unit 231, the minimum repeating unit 231 includes a plurality of filter groups 232, and the filter group includes panchromatic filters
  • the light sheet 233 and the color filter 234, the color filter 234 has a narrower spectral response than the panchromatic filter 233, and more light can be obtained when shooting, so there is no need to adjust the shooting parameters without affecting
  • the imaging quality in dark light can be improved.
  • both stability and quality can be taken into account.
  • the stability and quality of imaging in dark light are both high.
  • each panchromatic filter 233 includes 9 sub-filters 2331
  • each color filter 234 includes 9 sub-filters 2341
  • the pixel array 24 includes a plurality of panchromatic pixels 2431 and a plurality of color pixels 2441
  • each panchromatic pixel 2431 corresponds to a sub-filter 2331 of the panchromatic filter 233
  • each color pixel 2441 corresponds to a sub-filter 2341 of the color filter 234
  • the panchromatic pixel 2431 and the color pixel 2441 It is used to receive the light passing through the corresponding sub-filter to generate an electrical signal.
  • the pixels corresponding to the 9 sub-filters can be combined and output to obtain an image with a high signal-to-noise ratio.
  • the pixels corresponding to each sub-filter can be output separately, so as to obtain images with high resolution and signal-to-noise ratio, which can adapt to different application scenarios and improve the image quality in each scene. image quality.
  • the smallest repeating unit 231 in the filter array 23 includes 4 filter groups 232 , and the 4 filter groups 232 are arranged in a matrix.
  • Each filter group 232 comprises a panchromatic filter 233 and a color filter 234, each panchromatic filter 233 and each color filter 234 have 9 sub-filters, then the filter Group 232 includes a total of 36 sub-filters.
  • the pixel array 24 includes a plurality of minimum repeating units 241 corresponding to the plurality of minimum repeating units 231 .
  • Each minimum repeating unit 241 includes 4 pixel groups 242 , and the 4 pixel groups 242 are arranged in a matrix.
  • Each pixel group 242 corresponds to a filter group 232 .
  • the readout circuit 25 is electrically connected to the pixel array 24 for controlling the exposure of the pixel array 24 and reading and outputting the pixel values of the pixel points.
  • the readout circuit 25 includes a vertical drive unit 251 , a control unit 252 , a column processing unit 253 , and a horizontal drive unit 254 .
  • the vertical driving unit 251 includes a shift register and an address decoder.
  • the vertical driving unit 251 includes readout scanning and reset scanning functions.
  • the control unit 252 configures timing signals according to the operation mode, and uses various timing signals to control the vertical driving unit 251 , the column processing unit 253 and the horizontal driving unit 254 to work together.
  • the column processing unit 253 may have an analog-to-digital (A/D) conversion function for converting an analog pixel signal into a digital format.
  • the horizontal driving unit 254 includes a shift register and an address decoder. The horizontal driving unit 254 sequentially scans the pixel array 24 column by column.
  • each filter group 232 includes a color filter 234 and a panchromatic filter 233, and each panchromatic filter 233 in the filter group 232 is arranged on In the first diagonal direction D1, each color filter 234 in the filter set 232 is arranged in the second diagonal direction.
  • the direction of the first diagonal line D1 and the direction of the second diagonal line D2 are different, which can give consideration to both color performance and low-light imaging quality.
  • the direction of the first diagonal line D1 is different from the direction of the second diagonal line D2. Specifically, the direction of the first diagonal line D1 is not parallel to the direction of the second diagonal line D2, or the direction of the first diagonal line D1 is not parallel to the direction of the second diagonal line.
  • the direction of the diagonal line D2 is vertical, etc.
  • one color filter 234 and one panchromatic filter 233 can be located on the first diagonal line D1, and the other color filter 234 and another panchromatic filter 233 can be located on the second pair of diagonals. Corner line D2.
  • the smallest repeating unit 231 in the filter array 23 includes 4 filter groups 232 , and the 4 filter groups 232 are arranged in a matrix.
  • Each filter set 232 includes two panchromatic filters 233 and two color filters 234 .
  • the panchromatic filter 233 includes 9 sub-filters 2331, and the color filter 234 includes 9 sub-filters 2341, then the minimum repeating unit 231 is 12 rows and 12 columns with 144 sub-filters, and the arrangement is as follows:
  • w represents the panchromatic sub-filter 2331
  • a, b and c all represent the color sub-filter 2341 .
  • the panchromatic sub-filter 2331 refers to a sub-filter that can filter out all light rays other than the visible light band
  • the color sub-filter 2341 includes a red sub-filter, a green sub-filter, and a blue sub-filter. filter, magenta sub-filter, cyan sub-filter, and yellow sub-filter.
  • the red sub-filter is a sub-filter for filtering all light except red light
  • the green sub-filter is a sub-filter for filtering all light except green light
  • the blue sub-filter is a sub-filter for filtering A sub-filter for all light except blue
  • a magenta sub-filter for all light except magenta and a cyan sub-filter for all light except cyan A sub-filter for all light rays
  • the yellow sub-filter is a sub-filter for filtering out all light rays except yellow light.
  • a can be red sub-filter, green sub-filter, blue sub-filter, magenta sub-filter, cyan sub-filter or yellow sub-filter
  • b can be red sub-filter, Green sub-filter, blue sub-filter, magenta sub-filter, cyan sub-filter or yellow sub-filter
  • c can be red sub-filter, green sub-filter, blue sub-filter filter, magenta sub-filter, cyan sub-filter, or yellow sub-filter.
  • b is the red sub-filter, a is the green sub-filter, c is the blue sub-filter; or, c is the red sub-filter, a is the green sub-filter, b is the blue sub-filter Filter; another example, c is a red sub-filter, a is a green sub-filter, b is a blue sub-filter; or, a is a red sub-filter, b is a blue sub-filter , c is a green sub-filter, etc., which are not limited here; for another example, b is a magenta sub-filter, a is a cyan sub-filter, b is a yellow sub-filter, etc.
  • the color filter may further include sub-filters of other colors, such as an orange sub-filter, a purple sub-filter, etc., which are not limited here.
  • the smallest repeating unit 231 in the filter array 23 includes 4 filter groups 232 , and the 4 filter groups 232 are arranged in a matrix.
  • Each filter set 232 includes a color filter 234 and a panchromatic filter 233, and each color filter 234 in the filter set 232 is arranged in the direction of the first diagonal line D1, and the filter set 232
  • Each panchromatic filter 233 in is arranged in the direction of the second diagonal line D2.
  • each filter set 232 includes 2 panchromatic filters 233 and 2 color filters 234, the panchromatic filters 233 include 9 sub-filters 2331, and the color filters Nine sub-filters 2341 are included in the sheet 234, and the minimum repeating unit 231 is 12 rows and 12 columns with 144 sub-filters, as shown in Figure 5, the arrangement is:
  • w represents a panchromatic sub-filter
  • a, b, and c all represent color sub-filters.
  • the advantage of quad is that it can locally combine pixels by 2 by 2 and binning by 3 by 3 to obtain images of different resolutions, and has a high signal-to-noise ratio.
  • the quad full-size output has high pixels, and a full-size full-resolution image is obtained with higher definition.
  • the advantage of RGBW is that it uses W pixels to increase the overall light intake of the image, thereby improving the signal-to-noise ratio of the image quality.
  • the smallest repeating unit 231 in the filter array 23 includes 4 filter groups 232 , and the 4 filter groups 232 are arranged in a matrix.
  • Each filter set 232 includes a panchromatic filter 233 and a color filter 234 .
  • the panchromatic filter 233 is arranged on the third diagonal and the fourth diagonal
  • the color filter 234 is arranged on the third diagonal D3 direction or the fourth diagonal
  • the direction of the line D4 the direction of the third diagonal line D3 is different from the direction of the fourth diagonal line D4.
  • the color filter 234 may be disposed on a line parallel to the third diagonal line D3, or the color filter 234 may be disposed on a line parallel to the fourth diagonal line D4.
  • the direction of the third diagonal is different from the direction of the fourth diagonal. Specifically, the direction of the third diagonal is not parallel to the direction of the fourth diagonal, or the direction of the third diagonal is perpendicular to the direction of the fourth diagonal. wait.
  • the panchromatic filters 233 and the color filters 234 in the filter set 232 are arranged in a matrix, and each filter set 232 contains 5 panchromatic filters. 233 and four color filters 234, the number of panchromatic filters 233 is more than the number of color filters 234, so that the imaging quality in dark light is better.
  • the panchromatic filter 233 includes 9 sub-filters 2331, and the color filter 234 includes 9 sub-filters 2341, so the minimum repeating unit is 18 rows and 18 columns with 324 sub-filters. for:
  • w represents a panchromatic sub-filter
  • a, b, and c all represent color sub-filters.
  • the color filter 234 included in the filter set 232 of FIG. 234 is in the direction of the fourth diagonal D4.
  • the minimum repeating unit 231 in the filter array 23 includes 4 filter groups 232 , and the 4 filter groups 232 are arranged in a matrix.
  • Each filter set 232 includes a panchromatic filter 233 and a color filter 234 .
  • the color filter 234 is arranged on the fifth diagonal line D5 and the sixth diagonal line D6, and the panchromatic filter 233 is arranged on the fifth diagonal line D5 or the sixth diagonal line D5.
  • the direction of the diagonal line D6, the direction of the fifth diagonal line D5 is different from the direction of the sixth diagonal line D6.
  • the panchromatic filter 233 may be disposed on a line parallel to the fifth diagonal line D5, or the panchromatic filter 233 may be disposed on a line parallel to the sixth diagonal line D6. .
  • the fifth diagonal direction is different from the sixth diagonal direction, specifically, the fifth diagonal direction is not parallel to the sixth diagonal direction, or the fifth diagonal direction is perpendicular to the sixth diagonal direction wait.
  • the panchromatic filters 233 and the color filters 234 in the filter sets 232 are arranged in a matrix, and each filter set 232 contains 4 panchromatic filters. 233 and five color filters 234, the number of color filters 234 is more than the number of panchromatic filters 233, better color performance can be obtained, and the imaging quality in dark light can be improved.
  • the panchromatic filter 233 includes 9 sub-filters 2331, and the color filter 234 includes 9 sub-filters 2341, so the minimum repeating unit is 18 rows and 18 columns with 324 sub-filters. for:
  • w represents a panchromatic sub-filter
  • a, b, and c all represent color sub-filters.
  • panchromatic filter 233 contained in the filter set 232 of FIG. 233 is in the direction of the sixth diagonal line D6.
  • an image generation method is provided, which is applied to an image sensor 21 as shown in FIG.
  • the minimum repeating unit includes a plurality of filter groups 232, the filter group 232 includes a color filter 234 and a panchromatic filter 233, and the color filter 234 has a narrower spectrum than the panchromatic filter 233 Response, color filter and panchromatic filter 233 all comprise 9 sub-filters;
  • Pixel array 24 comprises a plurality of pixels, and the pixel of pixel array 24 is set correspondingly with the sub-filter of filter array 23, and pixel array 24 is configured to receive light passing through the filter array 23 to generate electrical signals;
  • the pixel array 24 includes a plurality of panchromatic pixels 2431 and a plurality of color pixels 2441, each panchromatic pixel 2431 corresponds to a sub-filter 2331 of the panchromatic filter 233, and each color pixel 2441 corresponds to A sub-filter 2341 of the color filter 234;
  • the image generation method includes:
  • Operation 802 in the first resolution mode, use the texture information of the color pixels in the original image to interpolate all the color pixels in the original image into panchromatic pixels to obtain a full-size panchromatic channel image; the full-size panchromatic channel
  • the pixels in the picture are panchromatic pixels.
  • a first target image is generated based on the full-size panchromatic channel image and the original image.
  • the first resolution mode refers to a full-resolution (fullsize) output mode with high resolution, low signal-to-noise ratio, and low frame rate.
  • the first resolution mode may be Blu-ray 1080P, ultra-clear 720P mode, but not limited thereto.
  • the color filter 234 has a narrower spectral response than the panchromatic filter 233, so the amount of light transmitted by the panchromatic filter is greater than that of the color filter, that is, the amount of light transmitted by the color filter.
  • the band width of the light is smaller than the band width of the light transmitted by the panchromatic filter, and the panchromatic filter transmits more light, and the corresponding panchromatic pixel obtained through the panchromatic filter has a higher signal-to-noise ratio
  • the panchromatic pixels contain more information and can resolve more texture details.
  • the signal-to-noise ratio refers to the ratio between the normal signal and the noise signal. The higher the signal-to-noise ratio of a pixel, the higher the proportion of normal signals contained in the pixel, and the more information can be analyzed from the pixel.
  • the color pixels may be G (Green, green) pixels, R (Red, red) pixels, B (Blue, blue) pixels, etc., but not limited thereto.
  • the sub-filter 2331 in the panchromatic filter 233 transmits the The filtered light is projected onto the corresponding panchromatic pixel 2431, and the panchromatic pixel 2431 receives the light passing through the sub-filter 2331 to generate an electrical signal.
  • the light transmitted by the sub-filter 2341 in the color filter 234 is projected onto the corresponding color pixel 2441, and the color pixel 2441 receives the light passing through the corresponding sub-filter 2341 to generate an electrical signal. Based on electrical signals corresponding to each panchromatic pixel 2431 and each color pixel 2441, an original image is obtained.
  • the first resolution mode uses the first resolution mode to capture images on a sunny day or when the lighting environment is relatively good.
  • the texture information includes at least one of texture direction, texture position, and texture intensity.
  • the electronic device determines the pixel position of the color pixel in the original image, and uses the texture information of the color pixel in the original image to interpolate each color pixel in the original image into a corresponding panchromatic pixel to obtain a full-size panchromatic channel image,
  • the pixels in the full-size panchromatic channel map are panchromatic pixels, and a full-size first target image is generated based on the full-size panchromatic channel map and the original image.
  • the original image 902 is obtained, and the texture information of the color pixels in the original image 902 is used to interpolate the color pixels in the original image 902 into panchromatic pixels to obtain full-size full-color pixels.
  • Color channel image the pixels in this full-size panchromatic channel image are panchromatic pixels.
  • the resolution of the first target image 904 is the same as the resolution of the original image 902, realizing full-size and full-resolution output image processing effect.
  • the texture information of color pixels in the original image is used to interpolate all the color pixels in the original image into panchromatic pixels, and obtain a full-size panchromatic channel with the same size as the original image image.
  • the pixels in the full-size panchromatic channel map are all panchromatic pixels, and the first target image is generated based on the full-size panchromatic channel map and the original image, and the panchromatic channel information can be fused into the original image, so that more information can be generated, Detailed analysis of the clearer first target image, so as to achieve the image processing effect of full-size and full-resolution output with high definition, low signal-to-noise ratio and low frame rate, which can meet the high-quality requirements of users for images.
  • the texture information of the color pixels in the original image is used to interpolate the color pixels in the original image into panchromatic pixels to obtain a full-size panchromatic channel image, including: traversing each color pixel in the original image pixel; in the case of determining that the current pixel of the original image is a color pixel, determine the texture information of the color pixel based on each pixel in the preset range containing the color pixel; based on the texture information of the color pixel, obtain the interpolation weight corresponding to the color pixel, and The color pixels are interpolated into panchromatic pixels according to the interpolation weights of the color pixels, until the full-size panchromatic channel image is obtained when the traversal is completed.
  • Preset ranges containing colored pixels can be set as desired.
  • the preset range may be the range of a 10*10 rectangular window centered on the color pixel.
  • the preset range may be a range of an 8*8 rectangular window centered on a color pixel.
  • the preset range may not be centered on the color pixel, for example, the color pixel may be at least one of the upper area, lower area, left area, and right area of the preset range.
  • a sliding window is used to traverse whether each pixel is a color pixel. Since the position of the color filter 224 in the filter array is periodically changed, it can be determined according to the law of the periodic change whether the current pixel is passed through the color filter. Slice 224 to get the color pixels.
  • each traversal when the electronic device determines that the current pixel of the original image is a color pixel, based on each pixel in the preset range containing the color pixel, not only the information of the color pixel itself can be obtained, but also the information of the color pixel adjacent to it can be obtained.
  • the information of the pixels in the area can more accurately determine the texture information of the color pixels.
  • the interpolation weight corresponding to the color pixel at the color pixel position in the original image is calculated.
  • the color pixel can be accurately interpolated as the corresponding panchromatic pixel until each pixel in the original image is The traversal is completed when the color pixels are all interpolated to the corresponding panchromatic pixels, so that a full-size panchromatic channel image can be accurately obtained.
  • determining the texture information of the color pixel based on each pixel in the preset range including the color pixel includes: determining the degree of dispersion among the pixels in the preset range including the color pixel; if the degree of dispersion is less than a discrete threshold, Then the color pixel is in the flat area; if the degree of dispersion is greater than or equal to the discrete threshold, the color pixel is in the texture area.
  • the discrete threshold can be set as required.
  • the electronic device can represent the degree of dispersion by determining the variance of each pixel within a preset range containing colored pixels; the electronic device can also determine the standard deviation of each pixel within a preset range containing The difference indicates the degree of dispersion; the degree of dispersion can also be expressed in other ways, which are not limited here.
  • Variance (var) is a measure of the degree of dispersion of a random variable or a set of data in probability theory and statistical variance. Standard Deviation can reflect the degree of dispersion of a data set.
  • determining the texture information of the color pixel based on each pixel in the preset range including the color pixel includes: determining the variance of each pixel in the preset range including the color pixel; if the variance is less than a preset threshold, the color pixel In the flat area; if the variance is greater than or equal to the preset threshold, the color pixel is in the texture area.
  • Preset thresholds can be set as desired. Flat areas are areas where there is weak or no texture. Textured areas are areas where strong textures exist.
  • the variance is smaller than the preset threshold, it means that the discreteness of each pixel in the preset range is small, and it can be considered that the texture of the preset range where the color pixel is located is weak or has no texture, and the color pixel is in a flat area. If the variance is greater than or equal to the preset threshold, it means that the discreteness of each pixel in the preset range is relatively large, and it can be considered that the texture of the preset range where the color pixel is located is strong, and the color pixel is in the texture area.
  • the variance of each panchromatic pixel within a preset range including the colored pixels may be determined. It can determine each panchromatic pixel within the preset range including the color pixel, calculate the pixel mean value of the color pixel and each panchromatic pixel, calculate the square value of the difference between the pixel value of the color pixel and the pixel mean value, and calculate respectively The square of the difference between the pixel value of each panchromatic pixel and the pixel mean. Determine the first pixel quantity corresponding to the color pixel and each panchromatic pixel, and use the ratio of the sum of the square values to the first pixel quantity as the variance. The first pixel quantity is the sum of the color pixel and the quantity of each panchromatic pixel within a preset range.
  • the variance of each panchromatic pixel and each color pixel within a preset range including the color pixels may be determined.
  • Each panchromatic pixel and each color pixel within the preset range including the color pixel can be determined, the pixel mean value is calculated for each color pixel and each panchromatic pixel, and the difference between the pixel value of each color pixel and the pixel mean value is calculated respectively , and calculate the square of the difference between the pixel value of each panchromatic pixel and the pixel mean.
  • the second pixel quantity corresponding to each color pixel and each panchromatic pixel is determined, and the ratio of the sum of each square value to the second pixel quantity is used as a variance.
  • the second number of pixels is the sum of the numbers of each color pixel and each panchromatic pixel within a preset range.
  • the variance can be calculated as follows:
  • x 1 , x 2 , and x n are pixel values, which may be the pixel values of panchromatic pixels or color pixels, M is the mean value of pixels, n is the number of pixels, and s 2 is the variance.
  • the texture information of the colored pixels can be accurately determined.
  • the interpolation weight corresponding to the color pixel is obtained based on the texture information of the color pixel, including: in the case that the color pixel is in a flat area, determining the first pixel of each panchromatic pixel in a preset range including the color pixel The mean value, and the second pixel mean value of each color pixel within the preset range; based on the proportional relationship between the first pixel mean value and the second pixel mean value, the interpolation weight corresponding to the color pixel is obtained.
  • the first pixel mean value is the pixel mean value of all panchromatic pixels within a preset range including color pixels.
  • the second pixel mean value is the pixel mean value of each color pixel within a preset range including the color pixels.
  • the electronic device multiplies the ratio value between the first pixel mean value and the second pixel mean value by the pixel value of the panchromatic pixel to obtain the interpolation weight corresponding to the color pixel .
  • the first pixel mean value of each panchromatic pixel in the preset range including the color pixel, and the second pixel mean value of each color pixel in the preset range are determined, based on The proportional relationship between the first pixel mean value and the second pixel mean value can accurately calculate the interpolation weight corresponding to the color pixel at the color pixel position in the original image.
  • the interpolation weight corresponding to the color pixel is obtained based on the texture information of the color pixel, including: determining the target texture direction of the color pixel when the color pixel is in the texture area; For each associated pixel, the interpolation weight corresponding to the color pixel is obtained.
  • the associated pixels may include panchromatic associated pixels and color associated pixels.
  • a panchromatic associated pixel is a panchromatic pixel that has an associated relationship with the colored pixel.
  • a color associated pixel is a color pixel associated with the color pixel.
  • the electronic device may preset a plurality of texture directions, and select a target texture direction of the color pixel from the plurality of texture directions when the panchromatic pixel is in the texture area.
  • the texture direction is symmetrical or asymmetrical, and the number of texture directions can also be set as required.
  • the number of texture directions can be 4, 8, or 12, etc.
  • the texture directions can be horizontal, vertical, diagonal and anti-diagonal.
  • Determining the target texture direction of the color pixel includes: determining the gradient value of the color pixel in each texture direction; and determining the texture direction of the color pixel based on the gradient value in each texture direction.
  • the electronic device may determine the texture direction with the smallest gradient value as the texture direction of the color pixel.
  • the electronic device may determine the texture direction with the second smallest gradient value as the texture direction of the color pixel.
  • the electronic device may also use other methods to determine the texture direction of the color pixel.
  • Associated pixels are pixels associated with color pixels.
  • the association relationship may be that the associated pixel is in the texture direction of the color pixel, or the association relationship may be that the associated pixel is within a preset area of the color pixel, and so on.
  • the associated pixel is located in at least one of the upper region, lower region, left region and right region of the color pixel.
  • the electronic device determines the associated pixels of the color pixel in each texture direction, and determines the target texture direction of the color pixel according to the associated pixels of the color pixel in each texture direction. Based on each associated pixel of the color pixel in the target texture direction, the interpolation weight corresponding to the color pixel is calculated. According to the same processing method, the interpolation weight corresponding to each color pixel in the original image can be calculated.
  • the target texture direction of the color pixel is determined, and the interpolation weight corresponding to each color pixel is accurately calculated based on each associated pixel of the color pixel in the target texture direction.
  • determining the target texture direction of the color pixel includes: when the color pixel is in the texture area, determining the pan-color associated pixels of the color pixel in each texture direction ; Based on the pan-color associated pixels associated with each texture direction, determine the first associated value corresponding to the color pixel in each texture direction; the texture direction corresponding to the first associated value that satisfies the first associated condition among the first associated values , as the target texture direction for colored pixels.
  • the electronic device determines panchromatic associated pixels that the color pixel is associated with in each texture direction. For example, the electronic device determines that the color pixel is associated with each pan-color associated pixel in the horizontal direction, each pan-color associated pixel associated with the vertical direction, each pan-color associated pixel associated with the diagonal direction, and the opposite angle Each pan-color associated pixel associated in the line direction.
  • the sum of the absolute values of the differences of the panchromatic associated pixels is calculated to obtain the first associated value corresponding to each texture direction. If there is a first correlation value that satisfies the first correlation condition among the first correlation values, the texture direction corresponding to the first correlation value that satisfies the first correlation condition is used as the target texture direction of the color pixel.
  • the first association condition may be that the difference between the first association values is greater than a preset difference, or the difference between the smallest first association value and the next smallest first association value is greater than a preset difference.
  • the electronic device determines whether the difference between the first correlation values is greater than a preset difference, and if the difference between the first correlation values is greater than the preset difference, the texture corresponding to the smallest first correlation value Direction, as the target texture direction for colored pixels.
  • the electronic device determines the smallest first associated value and the second smallest first associated value among the first associated values, and determines whether the difference between the smallest first associated value and the second smallest first associated value is greater than a preset If the difference value is greater than the preset difference value, the texture direction corresponding to the smallest first correlation value is used as the target texture direction of the color pixel.
  • the pan-color associated pixels associated with the color pixel in each texture direction are determined, so as to determine the target texture direction of the color pixel through the pan-color associated pixels associated with the color pixel. Based on the pan-color associated pixels associated with each texture direction, determine the first associated value of the color pixel corresponding to each texture direction, and determine the degree of association between each pan-color associated pixel and the color pixel, so that based on the pan-color association The degree of correlation between the pixel and the color pixel accurately determines the target texture direction of the color pixel.
  • the method further includes: in the case that the first associated values corresponding to the color pixels in each texture direction do not satisfy the first association condition, determining the panchromatic associated pixels and Color associated pixels; based on the panchromatic associated pixels and color associated pixels associated with each texture direction, determine the second associated value of the color pixel corresponding to each texture direction; the second associated value that satisfies the second associated condition in each second associated value
  • the texture direction corresponding to the associated value is used as the target texture direction of the color pixel.
  • the electronic device determines the panchromatic associated pixels and color associated pixels respectively associated with the color pixel in each texture direction.
  • the electronic device calculates the absolute value of the difference of each panchromatic associated pixel, and the absolute value of the difference of each color associated pixel, for each absolute value of the same texture direction
  • the values are summed. Determine the sum of the pixel numbers of each panchromatic associated pixel and color associated pixel, and divide the sum of the absolute values by the sum of the pixel numbers to obtain the second associated value corresponding to the texture direction, thereby obtaining the second associated value corresponding to each texture direction. associated value. If there is a second correlation value satisfying the second correlation condition among the second correlation values, the texture direction corresponding to the second correlation value satisfying the second correlation condition is used as the target texture direction of the color pixel.
  • the second association condition may be that the difference between the second association values is greater than a preset difference, or the difference between the smallest second association value and the next smallest second association value is greater than a preset difference. It can be understood that the preset difference in the first association condition may be the same as or different from the preset difference in the second association condition.
  • the electronic device determines whether the difference between the second correlation values is greater than a preset difference, and if the difference between the second correlation values is greater than the preset difference, the texture corresponding to the smallest second correlation value Direction, as the target texture direction for colored pixels.
  • the electronic device determines the smallest second associated value and the second smallest second associated value among the second associated values, and determines whether the difference between the smallest second associated value and the second smallest second associated value is greater than a preset If the difference is greater than the preset difference, the texture direction corresponding to the smallest second correlation value is used as the target texture direction of the color pixel.
  • the first correlation values corresponding to the color pixels in each texture direction do not satisfy the first correlation condition, it means that the target texture direction of the color pixel cannot be accurately determined only by using panchromatic correlation pixels, then determine the color A pan-color associated pixel and a color-associated pixel associated with the pixel in each texture direction, so that the target texture direction of the color pixel is jointly determined by the pan-color-associated pixel and the color-associated pixel associated with the color pixel.
  • determine the second associated value of the color pixel corresponding to each texture direction which uses a large amount of information, and the calculated associated value covers more information.
  • Fig. 10a is a schematic diagram of associated pixels for each texture direction in one embodiment. As shown in Figure 10a, taking the 10*10 pixel window as an example, it shows the color pixels (pixels at the position of the black dot in the figure) in the horizontal direction, vertical direction, diagonal direction and anti-diagonal direction respectively. Associated pixels. The associated pixel is the panchromatic associated pixel pointed by the arrow in Fig. 10a.
  • the absolute value of the difference between the two panchromatic associated pixels pointed by the same arrow is calculated to obtain two absolute values.
  • the two absolute values in the horizontal direction are summed to obtain the first correlation value corresponding to the horizontal direction.
  • the first correlation values respectively corresponding to the vertical direction, the diagonal direction and the anti-diagonal direction can be obtained.
  • the texture direction corresponding to the smallest first correlation value is used as the target texture direction of the color pixel.
  • the associated pixels as shown in FIG. 10b are used to determine the target texture direction of the color pixel.
  • the associated pixels of the color pixel in FIG. 10b include panchromatic associated pixels and color associated pixels.
  • panchromatic associated pixels and color associated pixels in the horizontal direction calculate the absolute value of the difference between the two panchromatic associated pixels pointed by each same arrow, and the absolute value of the difference between the two color associated pixels pointed by each identical arrow value, multiple absolute values can be obtained. Sum the multiple absolute values in the horizontal direction, divide the sum by the sum of the pixel numbers of the panchromatic associated pixels and the color associated pixels, and obtain the second associated value corresponding to the horizontal direction. According to the same processing method, the second joint values respectively corresponding to the vertical direction, the diagonal direction and the anti-diagonal direction can be obtained.
  • the texture direction corresponding to the smallest second correlation value is used as the target texture direction of the color pixel.
  • the interpolation weight W_C1 corresponding to the color pixel C1 is calculated through each pixel as shown in FIG. 11 .
  • Flat area: W_C1 0.5*C1*(W1+W2+W3+W4+W5+W6+W7+W8)/(C1+C2+C3+C4), when the color pixel C1 is in the flat area, find The average value of W1 to W8 in Figure 9 and the average value of C1 to C4, and then multiply the ratio of the two average values by C1 to obtain the interpolation weight W_C1.
  • the interpolation weight W_C1 corresponding to the color pixel C1 can be calculated as follows:
  • W_C1 0.5*W2+0.5*W7
  • the interpolation weight corresponding to each color pixel can be obtained, thereby obtaining the interpolation weight map.
  • the interpolation weight map and the original image are fused to obtain a full-size panchromatic channel image.
  • w2', w3', and w4' can be calculated to obtain a partial panchromatic image as shown in Figure 10.
  • this partial panchromatic channel image the color pixels C1, C2, C3, and C4 have been interpolated for full-color pixels.
  • the color pixels in the original image can be interpolated into corresponding panchromatic pixels, thereby obtaining a full-size panchromatic channel image, that is, a W channel image, and the size of the full-size panchromatic channel image is the same as that of the original image are the same size.
  • the interpolation weights corresponding to the color pixels are obtained based on the associated pixels of the color pixels in the target texture direction, including: obtaining the color The interpolation weight corresponding to the pixel.
  • the electronic device After determining the target texture direction of the color pixel, calculates the interpolation weight corresponding to the color pixel according to the corresponding pan-color associated pixels of the color pixel in the target texture direction and according to the proportional relationship between the pan-color associated pixels.
  • Each pixel in the original image corresponding to the color pixel is traversed in the same processing manner, and the interpolation weight corresponding to each color pixel in the original image can be obtained when the traverse is completed.
  • the interpolation weight corresponding to the color pixel is calculated according to the proportional relationship between the panchromatic associated pixels associated with the color pixel in the target texture direction, not only using the information of the color pixel itself, but also using the adjacent area of the color pixel
  • the information of the associated panchromatic pixels can calculate the interpolation weights corresponding to the color pixels more accurately.
  • the method also includes:
  • the adjacent preset number of panchromatic pixels are combined to read out one panchromatic pixel, and the panchromatic color corresponding to the panchromatic filter is obtained.
  • the second number of panchromatic pixels is less than the first number; for the first number of color pixels corresponding to the color filter, the adjacent preset number of color pixels are combined to read out one color pixel to obtain a color
  • a second number of color pixels corresponding to the filter based on a second number of panchromatic pixels corresponding to each panchromatic filter and a second number of color pixels corresponding to each color filter, a second target image is obtained ;
  • the resolution corresponding to the second resolution mode is smaller than the resolution corresponding to the first resolution mode.
  • the second resolution mode refers to the pixel binning readout mode with medium resolution, medium power consumption, medium signal-to-noise ratio, and medium frame rate.
  • the resolution and power consumption corresponding to the second resolution mode are lower than those corresponding to the first resolution mode. resolution and power consumption.
  • the signal-to-noise ratio and frame rate corresponding to the second resolution mode are greater than the signal-to-noise ratio and frame rate corresponding to the first resolution mode.
  • the shooting instruction it is detected whether the user selects the required resolution mode, whether to use preview shooting, and detects the current environment.
  • the light transmitted by the sub-filter 2331 in the panchromatic filter 233 is projected onto the corresponding panchromatic pixel 2431, and the panchromatic pixel 2431 receives the light passing through the sub-filter Light from the sheet 2331 to generate an electrical signal.
  • the light transmitted by the sub-filter 2341 in the color filter 234 is projected onto the corresponding color pixel 2441, and the color pixel 2441 receives the light passing through the corresponding sub-filter 2341 to generate an electrical signal.
  • Each panchromatic filter 233 corresponds to the first number of panchromatic pixels 2431 respectively.
  • the electronic device For the first number of panchromatic pixels 2431 respectively corresponding to the same panchromatic filter 233, the electronic device combines the adjacent preset number of panchromatic pixels to read out one panchromatic pixel, and obtains the panchromatic pixel corresponding to the panchromatic filter.
  • a second number of panchromatic pixels the second number being less than the first number.
  • Each color filter 234 corresponds to the first number of color pixels 2441 respectively.
  • the electronic device For the first number of color pixels 2441 respectively corresponding to the same color filter 234, the electronic device combines the adjacent preset number of color pixels and reads out one color pixel to obtain the second number of color pixels corresponding to the color filter 234. colored pixels.
  • the second target image can be obtained after obtaining the second number of panchromatic pixels corresponding to each panchromatic filter 233 and the second number of color pixels corresponding to each color filter 234 .
  • the corresponding nine panchromatic pixels of the panchromatic filter are arranged in 3 ⁇ 3, and the adjacent four panchromatic pixels among the nine panchromatic pixels are combined to read out one
  • four panchromatic pixels in a 2 ⁇ 2 arrangement corresponding to the panchromatic filter can be obtained.
  • the 9 color pixels corresponding to the color filter are arranged in 3 ⁇ 3, and after combining the 4 adjacent color pixels among the 9 color pixels to read out one color pixel, the 2 color pixels corresponding to the color filter can be obtained. 4 color pixels arranged in ⁇ 2. After all panchromatic pixels and color pixels are combined and read out in the above manner, the second target image is obtained.
  • the image sensor can first output the full-resolution image, and then realize the combination of four adjacent pixels into one pixel by means of digital synthesis at the back end, or The output of combining four adjacent pixels into one pixel is realized through the circuit inside the image sensor.
  • the adjacent preset number of panchromatic pixels are combined to read one panchromatic pixel to obtain a full color
  • the second number of panchromatic pixels corresponding to the color filter the second number is smaller than the first number
  • the adjacent preset number of color pixels are combined and read out one Color pixels, obtaining a second number of color pixels corresponding to the color filter, so that based on the second number of panchromatic pixels corresponding to each panchromatic filter and the second number of color pixels corresponding to each color filter.
  • the size of the obtained second target image is smaller than the image obtained in the first resolution mode, the power consumption is less than the power consumption in the first resolution mode, and the frame rate is higher than the frame rate in the first resolution mode, which can Adapt to different application scenarios.
  • the method further includes: in the third resolution mode, according to the plurality of panchromatic pixels corresponding to the same panchromatic filter in the filter group, combining the first pixel value read out and the same color
  • the plurality of color pixels corresponding to the filter combine the read-out second pixel values to obtain a third target image; the resolution corresponding to the third resolution mode is smaller than the resolution corresponding to the first resolution mode.
  • the third resolution mode refers to a first-level pixel binning readout mode with medium resolution, medium power consumption, medium signal-to-noise ratio, and medium frame rate, which is different from the pixel binning readout mode corresponding to the second resolution mode.
  • the resolution and power consumption corresponding to the third resolution mode are smaller than the resolution and power consumption corresponding to the first resolution mode.
  • the signal-to-noise ratio and frame rate corresponding to the third resolution mode are greater than the signal-to-noise ratio and frame rate corresponding to the first resolution mode.
  • the resolution and power consumption corresponding to the third resolution mode are smaller than the resolution and power consumption corresponding to the second resolution mode, and the signal-to-noise ratio and frame rate corresponding to the third resolution mode are greater than the signal-to-noise ratio corresponding to the second resolution mode Ratio, frame rate.
  • the third resolution mode may be a default mode for shooting images and videos.
  • the shooting instruction it is detected whether the user selects the required resolution mode, whether to use preview shooting, and detects the current environment.
  • the user has not selected the resolution mode to be used, the preview shooting is not used, and the current environment is not the night scene mode, the third resolution mode is used to respond to the shooting instruction.
  • the light transmitted by the sub-filter 2331 in the panchromatic filter 233 is projected onto the corresponding panchromatic pixel 2431, and the panchromatic pixel 2431 receives the light passing through the sub-filter 2331 to Generate electrical signals.
  • the light transmitted by the sub-filter 2341 in the color filter 234 is projected onto the corresponding color pixel 2441, and the color pixel 2441 receives the light passing through the corresponding sub-filter 2341 to generate an electrical signal.
  • the electronic device combines and reads out the first pixel value of multiple panchromatic pixels 2431 corresponding to the same panchromatic filter 233 and reads out the second pixel value of multiple color pixels 2441 corresponding to the same color filter 234, Get the third target image.
  • the 9 panchromatic pixels corresponding to the panchromatic filter are arranged in 3 ⁇ 3, and the 9 color pixels corresponding to the color filter are arranged in 3 ⁇ 3.
  • the first pixel values read out are combined according to the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, and the multiple corresponding to the color filters
  • the color pixels combine the read-out second pixel values, so that the size of the generated third target image is reduced, and the power consumption required to generate the image is low.
  • the method further includes: in the fourth resolution mode, according to the plurality of panchromatic pixels corresponding to the panchromatic filters in the filter set, combining the first pixel values read out, and the color filter A plurality of color pixels corresponding to the light sheet are combined with the second pixel values read out to obtain a combined image; a plurality of panchromatic pixels in the direction of the seventh diagonal in the combined image are combined, and the pixels in the direction of the eighth diagonal are combined. A plurality of color pixels on the above are merged to obtain a fourth target image; the seventh diagonal direction is different from the eighth diagonal direction, and the resolution corresponding to the fourth resolution mode is smaller than the resolution corresponding to the first resolution mode.
  • the fourth resolution mode refers to the mode used in scenes with relatively low resolution requirements, which is a secondary pixel binning readout mode with low resolution, low power consumption, high signal-to-noise ratio, and high frame rate.
  • the resolution and power consumption corresponding to the fourth resolution mode are smaller than the resolution and power consumption corresponding to the first resolution mode.
  • the signal-to-noise ratio and frame rate corresponding to the fourth resolution mode are greater than the signal-to-noise ratio and frame rate corresponding to the first resolution mode.
  • the resolution and power consumption corresponding to the fourth resolution mode are smaller than the resolution and power consumption corresponding to the second resolution mode, and the signal-to-noise ratio and frame rate corresponding to the fourth resolution mode are greater than the signal-to-noise ratio corresponding to the second resolution mode Ratio, frame rate.
  • the resolution and power consumption corresponding to the fourth resolution mode are smaller than the resolution and power consumption corresponding to the third resolution mode, and the signal-to-noise ratio and frame rate corresponding to the fourth resolution mode are greater than the signal-to-noise ratio corresponding to the third resolution mode Ratio, frame rate.
  • the fourth resolution mode may specifically be a preview mode during image capture, a preview mode during video capture, or a night scene mode for image capture or video capture under night scenes, but is not limited to scenes with lower resolution requirements.
  • the preview modes of video shooting include 1080p video preview, WeChat video preview, etc.
  • a shooting instruction When a shooting instruction is received, it is determined whether the shooting instruction is preview shooting. In the case that the shooting instruction is preview shooting, a fourth resolution mode is triggered. Alternatively, the electronic device detects whether the current environment is a night scene, and if the current environment is a night scene, triggers the fourth resolution mode. Alternatively, when the user selects the fourth resolution mode, trigger the readout mode corresponding to the fourth resolution mode.
  • the light transmitted by the sub-filter 2331 in the panchromatic filter 233 is projected onto the corresponding panchromatic pixel 2431, and the panchromatic pixel 2431 receives the light passing through the sub-filter 2331 to Generate electrical signals.
  • the light transmitted by the sub-filter 2341 in the color filter 234 is projected onto the corresponding color pixel 2441, and the color pixel 2441 receives the light passing through the corresponding sub-filter 2341 to generate an electrical signal.
  • the electronic device combines and reads out the first pixel value of multiple panchromatic pixels 2431 corresponding to the same panchromatic filter 233 and reads out the second pixel value of multiple color pixels 2441 corresponding to the same color filter 234, Get the merged image.
  • the electronic device for each panchromatic filter 233, the electronic device combines the 9 panchromatic pixels corresponding to the 9 sub-filters included in the panchromatic filter 233 to read out the first pixel value, thereby Each first pixel value is obtained.
  • the 9 color pixels corresponding to the 9 sub-filters included in the color filter 234 are combined to read out the second pixel value, so as to obtain each second pixel value.
  • the electronic device combines the plurality of panchromatic pixels in the direction of the seventh diagonal in the combined image, and combines the plurality of color pixels in the direction of the eighth diagonal in the combined image to obtain a fourth target image.
  • the seventh diagonal direction is different from the eighth diagonal direction. Specifically, the seventh diagonal direction is not parallel to the eighth diagonal direction, or the seventh diagonal direction is not parallel to the eighth diagonal direction. vertical etc.
  • the seventh diagonal direction is perpendicular to the eighth diagonal direction.
  • Each of the plurality of panchromatic pixels and the plurality of color pixels may be at least two. For example, merging two panchromatic pixels in the direction of the seventh diagonal line D7, and merging two color pixels of the same color in the direction of the eighth diagonal line D8 in the combined image to obtain the fourth target image, As shown in Figure 13c.
  • the first pixel values read out are combined according to the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, and the multiple corresponding to the color filters Combining the second pixel values read out by color pixels to obtain a combined image, combining a plurality of panchromatic pixels in the direction of the seventh diagonal in the combined image, and combining the eighth pair of pixels in the direction different from the seventh diagonal Multiple color pixels in the direction of the corner line are merged, and the all-in-one pixel readout method makes the generated image noise less, while the light input of the panchromatic channel is larger, and the panchromatic pixels have a higher signal-to-noise ratio, so that the obtained
  • the resolution of the fourth target image is further reduced, and the panchromatic pixels have a higher signal-to-noise ratio, and the frame rate of the image is high, thereby achieving an image with lower power consumption and better signal-to-noise ratio output by secondary pixel combination processing effect.
  • CMOS Image Sensor Complementary Metal Oxide Semiconductor Image Sensor
  • 3 ⁇ 3RGBW pattern 4 resolution modes can be provided, and each resolution mode corresponds to a respective resolution output.
  • the first resolution mode (fullsize mode) corresponding to a resolution of 108M (megabytes) as an example
  • the first resolution mode (108M) has the highest resolution, the lowest signal-to-noise ratio, and the lowest frame rate, usually in sunny days or Use when the lighting environment is relatively good.
  • the second resolution mode that is, 3 ⁇ 3 to 2 ⁇ 2 mode
  • the corresponding resolution of the second resolution mode is 48M
  • the resolution in the second resolution mode (48M) is the second highest
  • the signal-to-noise ratio is the second lowest
  • the second-lowest rate is usually used when shooting and previewing 8K videos, or when the resolution is required to be between fullsize and binning.
  • the third resolution mode that is, the first-level binning mode, the corresponding resolution of the third resolution mode is 12M, the resolution of the third resolution mode (12M) is the second lowest, the signal-to-noise ratio is the second highest, and the frame rate is the second highest. It is usually used for taking pictures in normal mode, 4K video shooting and previewing.
  • the fourth resolution mode that is, the second-level binning mode, the corresponding resolution of the fourth resolution mode is 3M, the resolution of the fourth resolution mode (3M) is the lowest, the signal-to-noise ratio is the highest, and the frame rate is the highest, usually at 1080p It can be used to take pictures in video scenes or night scenes.
  • the 3 ⁇ 3 RGBW pattern can directly output a fullsize image, or convert the 3 ⁇ 3 structure into a 2 ⁇ 2 RGBW pattern
  • the output can also support 9-in-1 primary binning output and 18-in-1 secondary binning output, so that the 3 ⁇ 3 RGBW pattern can output more resolutions to adapt to different application scenarios.
  • it can improve the signal-to-noise ratio and clarity of the image, and improve the effect of taking photos and videos at night.
  • an image generation method is provided, which is applied to an image sensor of an electronic device, wherein the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, and the minimum repeating unit includes Multiple filter sets, filter sets include color filters and panchromatic filters, color filters have a narrower spectral response than panchromatic filters, color filters and panchromatic filters
  • Each chip includes 9 sub-filters; the pixel array includes multiple panchromatic pixels and multiple color pixels, each panchromatic pixel corresponds to a sub-filter of the panchromatic filter, and each color pixel corresponds to a color filter A sub-filter of
  • the image generation method includes:
  • the first resolution mode traverse each pixel in the original image corresponding to the color pixel; in the case of determining that the current pixel of the original image is a color pixel, determine the variance of each pixel within the preset range containing the color pixel; if the variance is less than If the preset threshold is set, the color pixel is in the flat area; if the variance is greater than or equal to the preset threshold, the color pixel is in the texture area.
  • the color pixel is in the flat area, determine the first pixel mean value of each panchromatic pixel in the preset range including the color pixel, and the second pixel mean value of each color pixel in the preset range; based on the first pixel mean value and the proportional relationship between the mean value of the second pixel to obtain the interpolation weight corresponding to the color pixel.
  • the color pixel is in the texture area, determine the pan-color associated pixels of the color pixels in each texture direction; based on the pan-color associated pixels associated with each texture direction, determine the corresponding The first correlation value: taking the texture direction corresponding to the first correlation value satisfying the first correlation condition among the first correlation values as the target texture direction of the color pixel.
  • the panchromatic associated pixels and color associated pixels respectively associated with the color pixels in each texture direction; For the associated panchromatic associated pixels and color associated pixels, determine the second associated values corresponding to the color pixels in each texture direction; use the texture direction corresponding to the second associated value that satisfies the second associated condition among the second associated values as Destination texture direction for colored pixels.
  • the interpolation weights corresponding to the color pixels are obtained according to the proportional relationship between the panchromatic associated pixels associated with the color pixels in the target texture direction.
  • the color pixels are interpolated into panchromatic pixels according to the interpolation weights of the color pixels, until the full-size panchromatic channel image is obtained when the traversal is completed; the pixels in the full-size panchromatic channel image are all panchromatic pixels; based on the full-size panchromatic The channel map and the original image generate the first target image.
  • the adjacent preset number of panchromatic pixels are combined to read out one panchromatic pixel to obtain a panchromatic
  • the second number of panchromatic pixels corresponding to the color filter the second number is less than the first number
  • the adjacent preset number of color pixels are combined to read a color Pixels to obtain the second number of color pixels corresponding to the color filter; based on the second number of panchromatic pixels corresponding to each panchromatic filter and the second number of color pixels corresponding to each color filter, obtain The second target image; the resolution corresponding to the second resolution mode is smaller than the resolution corresponding to the first resolution mode.
  • the first pixel value read out according to the multiple panchromatic pixels corresponding to the same panchromatic filter in the filter set and the multiple corresponding to the same color filter The color pixels combine the read-out second pixel values to obtain a third target image; the resolution corresponding to the third resolution mode is smaller than the resolution corresponding to the second resolution mode.
  • the first pixel values read out are combined according to the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, and the multiple color pixels corresponding to the color filters Combining the read-out second pixel values to obtain a combined image; combining a plurality of panchromatic pixels in the direction of the seventh diagonal in the combined image, and combining a plurality of color pixels in the direction of the eighth diagonal , to obtain the fourth target image; the seventh diagonal direction is different from the eighth diagonal direction, and the resolution corresponding to the fourth resolution mode is smaller than the resolution corresponding to the third resolution mode.
  • an image sensor structure supporting output of multiple resolutions is provided, thereby providing output modes of four resolutions, which can adapt to different application scenarios.
  • the first resolution mode in scenes with high resolution requirements use the texture information of color pixels in the original image, interpolate the color pixels in the original image into panchromatic pixels, and obtain the full color image with the same size as the original image Dimensional panchromatic channel image.
  • the pixels in the full-size panchromatic channel map are all panchromatic pixels, and the first target image is generated based on the full-size panchromatic channel map and the original image, which can integrate the panchromatic channel information into the original image, making the generated information more detailed Analyze the clearer first target image, so as to realize the image processing effect of full-size and full-resolution output with high resolution, low signal-to-noise ratio and low frame rate, which can meet the high-quality requirements of users for images.
  • the adjacent preset number of panchromatic pixels are combined to read out one panchromatic pixel, and the panchromatic color corresponding to the panchromatic filter is obtained.
  • the second number of panchromatic pixels the second number is less than the first number
  • the adjacent preset number of color pixels are combined to read out one color pixel to obtain a color
  • the second number of color pixels corresponding to the filter so that the second number of color pixels corresponding to each color filter and the second number of color pixels corresponding to each color filter is obtained.
  • the size of the target image is smaller than the image obtained in the first resolution mode, the power consumption is less than that in the first resolution mode, and the frame rate is higher than the frame rate in the first resolution mode, which can be adapted to different applications Scenes.
  • the third resolution mode In the general shooting or preview scene, use the third resolution mode, combine and read out the first pixel value according to the multiple panchromatic pixels corresponding to the panchromatic filter in the filter set, and the multiple corresponding to the color filter Combining the read-out second pixel values with the color pixels, so that the size of the generated third target image is reduced, and the power consumption required to generate the image is low.
  • the fourth resolution mode is used in scenes with lower resolution requirements such as night scene shooting, and the first pixel value read out by combining multiple panchromatic pixels corresponding to the panchromatic filter in the filter set, and the color filter A plurality of color pixels corresponding to the light sheet are combined to read out the second pixel values to obtain a combined image, and a plurality of panchromatic pixels in the direction of the seventh diagonal in the combined image are combined, and the values at different angles from the seventh diagonal are combined.
  • the all-in-one pixel readout method makes the generated image noise less, while the light input of the panchromatic channel is larger, and the panchromatic pixels have higher signal noise ratio, so that the resolution of the obtained fourth target image is further reduced, and the panchromatic pixels have a higher signal-to-noise ratio, and the frame rate of the image is high, so that the power consumption of the second-level pixel combination output is lower, and the signal
  • the image processing effect with better noise ratio can further improve the night photo and video effects.
  • FIGS. 2-13 may include multiple sub-operations or multiple stages. These sub-operations or stages are not necessarily performed at the same time, but may be performed at different times. These sub-operations or The execution order of the phases is not necessarily performed sequentially, but may be performed in turn or alternately with other operations or sub-operations of other operations or at least a part of phases.
  • Fig. 14 is a structural block diagram of an image generating device of an embodiment.
  • the image generating device 1400 is applied to an image sensor, the image sensor includes a filter array and a pixel array, the filter array includes a minimum repeating unit, the minimum repeating unit includes a plurality of filter groups, and the filter group Including color filter and panchromatic filter, the color filter has a narrower spectral response than the panchromatic filter, both the color filter and the panchromatic filter include 9 sub-filters;
  • the pixel array It includes a plurality of pixels, the pixels of the pixel array correspond to the sub-filters of the filter array, and the pixel array is configured to receive light passing through the filter array to generate electrical signals;
  • the image generating device 1400 includes:
  • the interpolation module 1402 is used to interpolate all the color pixels in the original image into full-color pixels by using the texture information of the color pixels in the original image in the first resolution mode to obtain a full-size full-color channel image;
  • the pixels in the color channel map are panchromatic pixels;
  • a generating module 1404 configured to generate a first target image based on the full-size panchromatic channel image and the original image.
  • the texture information of color pixels in the original image is used to interpolate all the color pixels in the original image into panchromatic pixels, and obtain a full-size panchromatic channel with the same size as the original image image.
  • the pixels in the full-size panchromatic channel map are all panchromatic pixels, and the first target image is generated based on the full-size panchromatic channel map and the original image, which can integrate the panchromatic channel information into the original image, making the generated information more detailed Analyze the clearer first target image, so as to realize the image processing effect of full-size and full-resolution output with high definition, low signal-to-noise ratio and low frame rate, which can meet the high-quality requirements of users for images.
  • the generation module 1404 is also used to traverse each pixel in the original image corresponding to the color pixel; when it is determined that the current pixel of the original image is a color pixel, determine based on each pixel in the preset range containing the color pixel Texture information of color pixels; based on the texture information of color pixels, the interpolation weight corresponding to the color pixel is obtained, and the color pixel is interpolated into a panchromatic pixel according to the interpolation weight of the color pixel, until the full-size panchromatic channel image is obtained when the traversal is completed.
  • the texture information of color pixels can be determined more accurately. Based on the texture information of the color pixel, the interpolation weight corresponding to the color pixel at the color pixel position in the original image is calculated.
  • the color pixel can be accurately interpolated as the corresponding panchromatic pixel until each pixel in the original image is The traversal is completed when the color pixels are all interpolated to the corresponding panchromatic pixels, so that a full-size panchromatic channel image can be accurately obtained.
  • the generation module 1404 is also used to determine the variance of each pixel within a preset range including color pixels; if the variance is less than a preset threshold, the color pixel is in a flat area; if the variance is greater than or equal to a preset threshold, Then the color pixel is in the texture area.
  • the texture information of the colored pixels can be accurately determined.
  • the generation module 1404 is also used to determine the first pixel mean value of each panchromatic pixel within the preset range including the color pixel, and the first pixel mean value of each color pixel within the preset range when the color pixel is in a flat area.
  • the second pixel mean value based on the proportional relationship between the first pixel mean value and the second pixel mean value, the interpolation weight corresponding to the color pixel is obtained.
  • the first pixel mean value of each panchromatic pixel in the preset range including the color pixel, and the second pixel mean value of each color pixel in the preset range are determined, based on The proportional relationship between the first pixel mean value and the second pixel mean value can accurately calculate the interpolation weight corresponding to the color pixel at the color pixel position in the original image.
  • the generation module 1404 is also used to determine the target texture direction of the color pixel when the color pixel is in the texture area; based on the associated pixels of the color pixel in the target texture direction, the corresponding interpolation value of the color pixel is obtained Weights.
  • the target texture direction of the color pixel is determined, and the interpolation weight corresponding to each color pixel is accurately calculated based on each associated pixel of the color pixel in the target texture direction.
  • the generation module 1404 is further configured to determine the panchromatic associated pixels of the colored pixels in each texture direction when the colored pixel is in the texture area; based on the panchromatic associated pixels associated with each texture direction, Determining the first correlation values corresponding to the color pixels in each texture direction; taking the texture direction corresponding to the first correlation value satisfying the first correlation condition among the first correlation values as the target texture direction of the color pixel.
  • the pan-color associated pixels associated with the color pixel in each texture direction are determined, so as to determine the target texture direction of the color pixel through the pan-color associated pixels associated with the color pixel. Based on the pan-color associated pixels associated with each texture direction, determine the first associated value of the color pixel corresponding to each texture direction, and determine the degree of association between each pan-color associated pixel and the color pixel, so that based on the pan-color association The degree of correlation between the pixel and the color pixel accurately determines the target texture direction of the color pixel.
  • the generating module 1404 is further configured to determine the panchromatic associations of the color pixels in each texture direction when the first association values corresponding to the color pixels in each texture direction do not satisfy the first association condition Pixels and color associated pixels; based on the panchromatic associated pixels and color associated pixels associated with each texture direction, determine the second associated value of the color pixel corresponding to each texture direction; the second associated value that satisfies the second associated condition The texture direction corresponding to the second associated value is used as the target texture direction of the color pixel.
  • the first correlation values corresponding to the color pixels in each texture direction do not satisfy the first correlation condition, it means that the target texture direction of the color pixel cannot be accurately determined only by using panchromatic correlation pixels, then determine the color A pan-color associated pixel and a color-associated pixel associated with the pixel in each texture direction, so that the target texture direction of the color pixel is jointly determined by the pan-color-associated pixel and the color-associated pixel associated with the color pixel.
  • determine the second associated value of the color pixel corresponding to each texture direction which uses a large amount of information, and the calculated associated value covers more information.
  • the generation module 1404 is further configured to obtain the interpolation weight corresponding to the color pixel according to the proportional relationship between the color pixel and the pan-color associated pixels associated with the target texture direction.
  • the interpolation weight corresponding to the color pixel is calculated according to the proportional relationship between the panchromatic associated pixels associated with the color pixel in the target texture direction, not only using the information of the color pixel itself, but also using the adjacent area of the color pixel
  • the information of the associated panchromatic pixels can calculate the interpolation weights corresponding to the color pixels more accurately.
  • the device also includes:
  • the first merging module is configured to combine the adjacent preset number of panchromatic pixels to read out one panchromatic pixel for the first number of panchromatic pixels corresponding to the panchromatic filter in the second resolution mode, Obtaining the second number of panchromatic pixels corresponding to the panchromatic filter; the second number is less than the first number; for the first number of color pixels corresponding to the color filter, combining and reading the adjacent preset number of color pixels A color pixel is generated to obtain a second number of color pixels corresponding to the color filter.
  • An obtaining module configured to obtain a second target image based on a second number of panchromatic pixels corresponding to each panchromatic filter and a second number of color pixels corresponding to each color filter; the second resolution mode corresponds to The resolution of is smaller than the resolution corresponding to the first resolution mode.
  • the adjacent preset number of panchromatic pixels are combined to read one panchromatic pixel to obtain a full color
  • the second number of panchromatic pixels corresponding to the color filter the second number is smaller than the first number
  • the adjacent preset number of color pixels are combined and read out one Color pixels, obtaining a second number of color pixels corresponding to the color filter, so that based on the second number of panchromatic pixels corresponding to each panchromatic filter and the second number of color pixels corresponding to each color filter.
  • the size of the obtained second target image is smaller than the image obtained in the first resolution mode, the power consumption is less than the power consumption in the first resolution mode, and the frame rate is higher than the frame rate in the first resolution mode, which can Adapt to different application scenarios.
  • the device also includes:
  • the second merging module is used to merge and read out the first pixel value corresponding to the same color filter according to the plurality of panchromatic pixels corresponding to the same panchromatic filter in the filter set in the third resolution mode
  • the second pixel values read out are combined to obtain a third target image; the resolution corresponding to the third resolution mode is smaller than the resolution corresponding to the first resolution mode.
  • the first pixel values read out are combined according to the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set, and the multiple corresponding to the color filters
  • the color pixels combine the read-out second pixel values, so that the size of the generated third target image is reduced, and the power consumption required to generate the image is low.
  • the device also includes:
  • the third combining module is used to combine and read out the first pixel values according to the multiple panchromatic pixels corresponding to the panchromatic filters in the filter set in the fourth resolution mode, and the values corresponding to the color filters Combining the second pixel values read out by a plurality of color pixels to obtain a combined image; combining a plurality of panchromatic pixels in the seventh diagonal direction in the combined image, and combining a plurality of panchromatic pixels in the eighth diagonal direction Color pixels are combined to obtain a fourth target image; the seventh diagonal direction is different from the eighth diagonal direction, and the resolution corresponding to the fourth resolution mode is smaller than the resolution corresponding to the first resolution mode.
  • the all-in-one pixel readout method makes the generated image less noisy, and the panchromatic channel has a larger light input amount, and the panchromatic pixel has a higher signal-to-noise ratio, which further reduces the resolution of the obtained fourth target image , and the panchromatic pixels have a higher signal-to-noise ratio, and the frame rate of the image is high, so as to achieve the image processing effect of the secondary pixel combination output with lower power consumption and better signal-to-noise ratio.
  • each module in the above image generating device is only for illustration. In other embodiments, the image generating device can be divided into different modules as required to complete all or part of the functions of the above image generating device.
  • Fig. 15 is a schematic diagram of the internal structure of an electronic device in one embodiment.
  • the electronic device includes a processor and a memory connected through a system bus.
  • the processor is used to provide computing and control capabilities to support the operation of the entire electronic device.
  • the memory may include non-volatile storage media and internal memory. Nonvolatile storage media store operating systems and computer programs.
  • the computer program can be executed by a processor to implement an image generation method provided in the following embodiments.
  • the internal memory provides a high-speed running environment for the operating system computer program in the non-volatile storage medium.
  • the electronic device may be a mobile phone, a tablet computer, a personal digital assistant, or a wearable device.
  • each module in the image generation device provided in the embodiment of the present application may be in the form of a computer program.
  • the computer program can run on a terminal or a server.
  • the program modules constituted by the computer program can be stored in the memory of the terminal or server.
  • the computer program is executed by the processor, the operations of the methods described in the embodiments of the present application are realized.
  • the embodiment of the present application also provides a computer-readable storage medium.
  • One or more non-transitory computer-readable storage media containing computer-executable instructions that, when executed by one or more processors, cause the processors to perform the operations of the image generation method.
  • a computer program product comprising instructions which, when run on a computer, cause the computer to perform an image generation method.
  • Nonvolatile memory may include read only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory.
  • Volatile memory can include random access memory (RAM), which acts as external cache memory.
  • RAM is available in many forms such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced SDRAM (ESDRAM), Synchronous Synchlink DRAM (SLDRAM), Memory Bus (Rambus) Direct RAM (RDRAM), Direct Memory Bus Dynamic RAM (DRDRAM), and Memory Bus Dynamic RAM (RDRAM).
  • SRAM Static RAM
  • DRAM Dynamic RAM
  • SDRAM Synchronous DRAM
  • DDR SDRAM Double Data Rate SDRAM
  • ESDRAM Enhanced SDRAM
  • SLDRAM Synchronous Synchlink DRAM
  • SLDRAM Synchronous Synchlink DRAM
  • Memory Bus Radbus
  • RDRAM Direct RAM
  • DRAM Direct Memory Bus Dynamic RAM
  • RDRAM Memory Bus Dynamic RAM

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Color Television Image Signal Generators (AREA)

Abstract

L'invention concerne un capteur d'image, un appareil de génération d'image, un dispositif électronique, un support de stockage lisible par ordinateur, et un produit de programme informatique. Le capteur d'images (21) comprend un réseau de filtres (23) et un réseau de pixels (24). Le réseau de filtres (23) comprend une unité de répétition minimale (231). L'unité de répétition minimale (231) comprend une pluralité d'ensembles de filtres (232). Chaque ensemble de filtres (232) comprend un filtre panchromatique (233) et un filtre de couleur (234). Le filtre coloré (234) a une réponse spectrale plus étroite que le filtre panchromatique (233). Le filtre couleur (234) et le filtre panchromatique (233) comprennent tous deux neuf sous-filtres. Le réseau de pixels (24) comprend une pluralité de pixels, et les pixels du réseau de pixels (24) sont disposés en correspondance avec les sous-filtres du réseau de filtres (23). Le réseau de pixels (24) est configuré pour recevoir la lumière passant à travers le réseau de filtres (23) pour générer un signal électrique.
PCT/CN2022/113450 2021-09-10 2022-08-19 Capteur d'images, procédé et appareil de génération d'images, et dispositif électronique WO2023035900A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202111062952.2A CN113840067B (zh) 2021-09-10 2021-09-10 图像传感器、图像生成方法、装置和电子设备
CN202111062952.2 2021-09-10

Publications (1)

Publication Number Publication Date
WO2023035900A1 true WO2023035900A1 (fr) 2023-03-16

Family

ID=78958915

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/113450 WO2023035900A1 (fr) 2021-09-10 2022-08-19 Capteur d'images, procédé et appareil de génération d'images, et dispositif électronique

Country Status (2)

Country Link
CN (1) CN113840067B (fr)
WO (1) WO2023035900A1 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113840067B (zh) * 2021-09-10 2023-08-18 Oppo广东移动通信有限公司 图像传感器、图像生成方法、装置和电子设备
CN114554046B (zh) * 2021-12-01 2024-08-13 Oppo广东移动通信有限公司 图像传感器、摄像模组、电子设备、图像生成方法和装置
CN114338988B (zh) * 2021-12-29 2024-08-02 Oppo广东移动通信有限公司 图像生成方法、装置、电子设备和计算机可读存储介质

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111225135A (zh) * 2019-03-07 2020-06-02 Oppo广东移动通信有限公司 图像传感器、成像装置、电子设备、图像处理系统及信号处理方法
US20210021790A1 (en) * 2019-07-16 2021-01-21 Mritunjay Singh Image Sensors with Color, Panchromatic and Infrared Pixels
CN112508783A (zh) * 2020-11-19 2021-03-16 西安全志科技有限公司 基于方向插值的图像处理方法、计算机装置及计算机可读存储介质
CN213279832U (zh) * 2020-10-09 2021-05-25 Oppo广东移动通信有限公司 图像传感器、相机和终端
CN113840067A (zh) * 2021-09-10 2021-12-24 Oppo广东移动通信有限公司 图像传感器、图像生成方法、装置和电子设备
CN114338988A (zh) * 2021-12-29 2022-04-12 Oppo广东移动通信有限公司 图像生成方法、装置、电子设备和计算机可读存储介质

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8274715B2 (en) * 2005-07-28 2012-09-25 Omnivision Technologies, Inc. Processing color and panchromatic pixels
CN106454288B (zh) * 2016-11-29 2018-01-19 广东欧珀移动通信有限公司 控制方法、控制装置、成像装置及电子装置
CN111405204B (zh) * 2020-03-11 2022-07-26 Oppo广东移动通信有限公司 图像获取方法、成像装置、电子设备及可读存储介质
CN111314592B (zh) * 2020-03-17 2021-08-27 Oppo广东移动通信有限公司 图像处理方法、摄像头组件及移动终端
CN113766231A (zh) * 2020-07-08 2021-12-07 Oppo广东移动通信有限公司 图像获取方法、摄像头组件及移动终端
CN112118378B (zh) * 2020-10-09 2024-08-02 Oppo广东移动通信有限公司 图像获取方法及装置、终端和计算机可读存储介质

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111225135A (zh) * 2019-03-07 2020-06-02 Oppo广东移动通信有限公司 图像传感器、成像装置、电子设备、图像处理系统及信号处理方法
US20210021790A1 (en) * 2019-07-16 2021-01-21 Mritunjay Singh Image Sensors with Color, Panchromatic and Infrared Pixels
CN213279832U (zh) * 2020-10-09 2021-05-25 Oppo广东移动通信有限公司 图像传感器、相机和终端
CN112508783A (zh) * 2020-11-19 2021-03-16 西安全志科技有限公司 基于方向插值的图像处理方法、计算机装置及计算机可读存储介质
CN113840067A (zh) * 2021-09-10 2021-12-24 Oppo广东移动通信有限公司 图像传感器、图像生成方法、装置和电子设备
CN114338988A (zh) * 2021-12-29 2022-04-12 Oppo广东移动通信有限公司 图像生成方法、装置、电子设备和计算机可读存储介质

Also Published As

Publication number Publication date
CN113840067B (zh) 2023-08-18
CN113840067A (zh) 2021-12-24

Similar Documents

Publication Publication Date Title
WO2023035900A1 (fr) Capteur d'images, procédé et appareil de génération d'images, et dispositif électronique
CN102365861B (zh) 在产生数字图像时曝光像素组
US10136107B2 (en) Imaging systems with visible light sensitive pixels and infrared light sensitive pixels
US7636393B2 (en) Image processing method, image recording method, image processing device and image file format
CN113676708B (zh) 图像生成方法、装置、电子设备和计算机可读存储介质
CN113170061B (zh) 图像传感器、成像装置、电子设备、图像处理系统及信号处理方法
WO2023082766A1 (fr) Capteur d'image, module de caméra, dispositif électronique et procédé et appareil de génération d'image
WO2023098284A1 (fr) Capteur d'image, module de caméra, dispositif électronique, et procédé et appareil de génération d'image
WO2023124607A1 (fr) Procédé et appareil de génération d'image, dispositif électronique et support de stockage lisible par ordinateur
WO2023109264A1 (fr) Capteur d'image, module de caméra, dispositif électronique et procédé et appareil de génération d'image
WO2023109265A1 (fr) Capteur d'image, module de prise de vue, dispositif électronique, procédé et appareil de génération d'images
WO2023098282A1 (fr) Capteur d'image, module de caméra, dispositif électronique et procédé et appareil de génération d'image
CN113573030B (zh) 图像生成方法、装置、电子设备和计算机可读存储介质
EP4202822A1 (fr) Procédé de traitement d'image, système de traitement d'image, dispositif électronique et support de stockage lisible
WO2023087908A1 (fr) Procédé et appareil de commande de mise au point, capteur d'image, dispositif électronique et support de stockage lisible par ordinateur
WO2023098230A1 (fr) Capteur d'image, module de caméra, dispositif électronique et procédé et appareil de génération d'image
Lukac Single-sensor imaging in consumer digital cameras: a survey of recent advances and future directions
CN115280766A (zh) 图像传感器、成像装置、电子设备、图像处理系统及信号处理方法
WO2022073364A1 (fr) Procédé et appareil d'obtention d'images, terminal, et support de stockage lisible par ordinateur

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22866373

Country of ref document: EP

Kind code of ref document: A1