WO2022222634A1 - Image processing method, image processing apparatus, electronic device, and storage medium - Google Patents

Abstract

An image processing method, an image processing apparatus (100), an electronic device (200), and a storage medium. The image processing method comprises: converting an original image according to a current scene so as to obtain a Bayer image, the pixels of the Bayer image being arranged in a Bayer array; and outputting a target image according to the Bayer image. An original image outputted by a polarization sensor can be processed by an electronic device such as a mobile phone

Description

Image processing method, image processing device, electronic device, and storage medium

priority information

This application claims the priority and rights of patent application No. 202110442129.8 filed with the State Intellectual Property Office of China on April 23, 2021, and is hereby incorporated by reference in its entirety.

technical field

The present application relates to the technical field of image processing, and in particular, to an image processing method, an image processing apparatus, an electronic device, and a storage medium.

Background technique

At present, polarization sensors are mainly used in industrial cameras. The image data output by the polarization sensor is in a non-Bayer array, while the image data processed by electronic devices such as mobile phones is in a Bayer array. That is to say, the image data output by the polarization sensor in the related art is not Can be processed by electronic devices such as mobile phones.

SUMMARY OF THE INVENTION

Embodiments of the present application provide an image processing method, an image processing apparatus, an electronic device, and a storage medium.

The image processing method of the embodiment of the present application is used for a polarization sensor. The polarization sensor includes a polarizer array and a pixel array. The polarizer array includes a plurality of polarizer modules, and each of the polarizer modules includes four polarizer groups. The four polarizer groups in each polarizer module are arranged in a Bayer array. Each of the polarizer groups includes a plurality of polarizers. The polarization directions of the polarizers in each polarizer group are different from each other, and the color components of the polarizers in each polarizer group are the same. The pixel array includes a plurality of pixel units, and the plurality of the pixel units and the plurality of the polarizers are arranged in a one-to-one correspondence. The pixel unit is used for receiving light transmitted through the polarizer to generate electrical signals. The polarization sensor is used to generate a raw image from the electrical signal. The pixels of the original image are arranged in a non-Bayer array. The image processing method includes: converting the original image according to the current scene to obtain a Bayer image, the pixels of the Bayer image are arranged in a Bayer array; and outputting a target image according to the Bayer image.

The image processing apparatus of the embodiment of the present application is used for a polarization sensor. The polarization sensor includes a polarizer array and a pixel array. The polarizer array includes a plurality of polarizer modules, and each of the polarizer modules includes four polarizer groups. The four polarizer groups in each polarizer module are arranged in a Bayer array. Each of the polarizer groups includes a plurality of polarizers. The polarization directions of the polarizers in each polarizer group are different from each other, and the color components of the polarizers in each polarizer group are the same. The pixel array includes a plurality of pixel units, and the plurality of the pixel units and the plurality of the polarizers are arranged in a one-to-one correspondence. The pixel unit is used for receiving light transmitted through the polarizer to generate electrical signals. The polarization sensor is used to generate a raw image from the electrical signal. The pixels of the original image are arranged in a non-Bayer array. The image processing device includes a conversion module and an output module. The conversion module is configured to convert the original image according to the current scene to obtain a Bayer image, and the pixels of the Bayer image are arranged in a Bayer array. The output module is used for outputting the target image according to the Bayer image.

The electronic device of the embodiments of the present application includes one or more processors and a memory. The memory stores a computer program. When the computer program is executed by the processor, the steps of the image processing method described in the above embodiments are implemented.

The computer-readable storage medium of the embodiments of the present application stores a computer program thereon, and when the program is executed by the processor, implements the steps of the image processing method described in the above-mentioned embodiments.

Additional aspects and advantages of the present application will be set forth, in part, from the following description, and in part will become apparent from the following description, or may be learned by practice of the present application.

Description of drawings

The above and/or additional aspects and advantages of the present application will become apparent and readily understood from the following description of embodiments taken in conjunction with the accompanying drawings, wherein:

1 is a schematic flowchart of an image processing method according to an embodiment of the present application;

2 is a schematic three-dimensional structure diagram of a polarization sensor of an image processing method according to an embodiment of the present application;

3 is a schematic diagram of an image processing apparatus according to an embodiment of the present application;

4 is a schematic diagram of an electronic device according to an embodiment of the present application;

5 is a schematic structural diagram of a polarizer module of a polarization sensor of an image processing method according to an embodiment of the present application;

6 is a schematic flowchart of an image processing method according to an embodiment of the present application;

7 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application;

8 is a schematic flowchart of an image processing method according to an embodiment of the present application;

9 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application;

10 is a schematic flowchart of an image processing method according to an embodiment of the present application;

11 is a schematic diagram of an image processing apparatus according to an embodiment of the present application;

12 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application;

13 is a schematic flowchart of an image processing method according to an embodiment of the present application;

14 is a schematic diagram of an image processing apparatus according to an embodiment of the present application;

15 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application;

16 is a schematic flowchart of an image processing method according to an embodiment of the present application;

17 is a schematic flowchart of an image processing method according to an embodiment of the present application;

18 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application;

19 is a schematic flowchart of an image processing method according to an embodiment of the present application;

20 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application;

21 is a schematic flowchart of an image processing method according to an embodiment of the present application;

22 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application;

FIG. 23 is a schematic diagram of a scene of an image processing method according to an embodiment of the present application.

Detailed ways

The following describes in detail the embodiments of the present application, the embodiments of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals refer to the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary, only used to explain the present application, and should not be construed as a limitation on the present application.

In the description of the embodiments of the present application, the terms "first" and "second" are only used for description purposes, and cannot be understood as indicating or implying relative importance or implying the number of indicated technical features. Thus, features defined as "first", "second" may expressly or implicitly include one or more of said features. In the description of the embodiments of the present application, "plurality" means two or more, unless otherwise expressly and specifically defined.

Embodiments of the present application provide an image processing method for a polarization sensor. The polarization sensor includes a polarizer array and a pixel array, the polarizer array includes a plurality of polarizer modules, each polarizer module includes four polarizer groups, and the four polarizer groups in each polarizer module are arranged in a Bayer array, Each polarizer group includes a plurality of polarizers, the polarization directions of each polarizer in each polarizer group are different from each other, the color components of each polarizer in each polarizer group are the same, and the pixel array includes a plurality of pixel units , a plurality of pixel units and a plurality of polarizers are arranged in one-to-one correspondence, the pixel unit is used to receive the light passing through the polarizer to generate an electrical signal, the polarization sensor is used to generate an original image according to the electrical signal, and the pixels of the original image are in a non-Bayer array Arrangement, the image processing method includes: converting the original image according to the current scene to obtain a Bayer image, the pixels of the Bayer image are arranged in a Bayer array; and outputting a target image according to the Bayer image.

In some embodiments, the current scene includes a bright scene, and according to the current scene, converting the original image to obtain a Bayer image includes: in the case that the current scene is a bright scene, extracting the same polarizers in each polarizer group The polarizer of the polarization direction corresponds to the pixels of the original image and the extracted pixels are combined to obtain a Bayer image.

In some embodiments, the current scene includes a common scene, and according to the current scene, converting the original image to obtain a Bayer image includes: in the case that the current scene is a common scene, fusing the corresponding polarizers in each polarizer group pixels of the original image and combine the fused pixels to obtain a Bayer image.

In some embodiments, the current scene includes an overall dark and partially bright scene, and according to the current scene, converting the original image to obtain a Bayer image includes: when the current scene is an overall dark and partially bright scene, extracting The pixels of the original image corresponding to the polarizers with the same polarization direction in each polarizer group and the extracted pixels are combined to obtain a single-polarization image; the pixels of the original image corresponding to each polarizer in each polarizer group are fused and The fused pixels are combined to obtain an unpolarized image; the single-polarized and unpolarized images are fused to obtain a Bayer image.

In certain embodiments, fusing the single polarization image and the unpolarized image to obtain the Bayer image includes: filtering the single polarization image and the unpolarized image; and fusing the filtered single polarization image and the unpolarized image to obtain the Bayer image.

In some embodiments, outputting the target image according to the Bayer image includes: sequentially processing the Bayer image through a dead pixel compensation module, a demosaicing module, a color correction module, a gamma correction module, and a color conversion module to output the target image.

In some embodiments, the current scene includes a bright scene, and according to the current scene, converting the original image to obtain a Bayer image includes: in the case that the current scene is a bright scene, using a Bayer image regeneration algorithm to generate a Bayer image for the output of the polarization sensor. The original image is converted to obtain a Bayer image.

In some embodiments, outputting the target image according to the Bayer image includes: sequentially passing the Bayer image through a camera serial interface decoder, a black level module, a lens shading correction module, a dead pixel compensation module, a demosaicing module, and a color correction module , gamma correction module, color conversion module to output the target image after processing.

Embodiments of the present application provide an image processing apparatus for a polarization sensor. The polarization sensor includes a polarizer array and a pixel array, the polarizer array includes a plurality of polarizer modules, each polarizer module includes four polarizer groups, and the four polarizer groups in each polarizer module are arranged in a Bayer array, Each polarizer group includes a plurality of polarizers, the polarization directions of each polarizer in each polarizer group are different from each other, the color components of each polarizer in each polarizer group are the same, and the pixel array includes a plurality of pixel units , a plurality of pixel units and a plurality of polarizers are arranged in one-to-one correspondence, the pixel unit is used to receive the light passing through the polarizer to generate an electrical signal, the polarization sensor is used to generate an original image according to the electrical signal, and the pixels of the original image are in a non-Bayer array Arranged, the image processing device includes a conversion module and an output module. The conversion module is used to convert the original image according to the current scene to obtain a Bayer image, and the pixels of the Bayer image are arranged in a Bayer array; the output module is used to output the target image according to the Bayer image.

In some embodiments, the current scene includes a bright scene, and the conversion module is further configured to extract the pixels of the original image corresponding to the polarizers with the same polarization direction in each polarizer group when the current scene is a bright scene and combine the extracted pixels to obtain a Bayer image.

In some embodiments, the current scene includes an ordinary scene, and the conversion module is further configured to: in the case that the current scene is an ordinary scene, fuse the pixels of the original image corresponding to each polarizer in each polarizer group Pixels are combined to obtain a Bayer image.

In some embodiments, the current scene includes an overall dark scene and a partially bright scene, and the conversion module includes an extraction unit, a first fusion unit, and a second fusion unit. The extraction unit is used to extract the pixels of the original image corresponding to the polarizers with the same polarization direction in each polarizer group and combine the extracted pixels to obtain a single polarized image; the first fusion unit is used to fuse the pixels of the original image corresponding to each polarizer in each polarizer group and combine the fused pixels to obtain an unpolarized image; the second fusion unit is used to fuse the single-polarized image and unpolarized images to obtain Bayer images.

In some embodiments, the second fusion unit includes a filtering subunit and a fusion subunit. The filtering subunit is used to filter the single polarization image and the unpolarized image; the fusion subunit is used to fuse the filtered single polarization image and the unpolarized image to obtain the Bayer image.

In some embodiments, the output module is further configured to sequentially process the Bayer image through a dead pixel compensation module, a demosaic module, a color correction module, a gamma correction module, and a color conversion module to output the target image.

In some embodiments, the current scene includes a bright scene, and the conversion module is further configured to convert the original image output by the polarization sensor using a Bayer image regeneration algorithm to obtain a Bayer image when the current scene is a bright scene.

In some embodiments, the output module is further used for: passing the Bayer image through the camera serial interface decoder, black level module, lens shading correction module, dead pixel compensation module, demosaicing module, color correction module, gamma module in sequence The correction module and the color conversion module output the target image after processing.

Embodiments of the present application provide an electronic device. The polarization sensor includes a polarizer array and a pixel array, the polarizer array includes a plurality of polarizer modules, each polarizer module includes four polarizer groups, and the four polarizer groups in each polarizer module are arranged in a Bayer array, Each polarizer group includes a plurality of polarizers, the polarization directions of each polarizer in each polarizer group are different from each other, the color components of each polarizer in each polarizer group are the same, and the pixel array includes a plurality of pixel units , a plurality of pixel units and a plurality of polarizers are arranged in one-to-one correspondence, the pixel unit is used to receive the light passing through the polarizer to generate an electrical signal, the polarization sensor is used to generate an original image according to the electrical signal, and the pixels of the original image are in a non-Bayer array Arrangement, the electronic device includes one or more processors and a memory, the memory stores a computer program, and when the computer program is executed by the processor, the processor is used for: according to the current scene, convert the original image to obtain the Bayer image, the Bayer image The pixels of the image are arranged in a Bayer array; the target image is output according to the Bayer image.

In some embodiments, the current scene includes a bright scene, and the processor is further configured to extract the pixels of the original image corresponding to the polarizers with the same polarization direction in each polarizer group when the current scene is a bright scene and combine the extracted pixels to obtain a Bayer image.

In some embodiments, the current scene includes an ordinary scene, and the processor is further configured to fuse the pixels of the original image corresponding to each polarizer in each polarizer group and merge the pixels after fusion when the current scene is an ordinary scene. combined to obtain a Bayer image.

In some embodiments, the current scene includes an overall dark and partially bright scene, and the processor is further configured to: in the case that the current scene is an overall dark and partially bright scene, extract each polarizer group with the same polarization direction The pixels of the original image corresponding to the polarizers of each polarizer group are combined and the extracted pixels are combined to obtain a single-polarization image; the pixels of the original image corresponding to each polarizer in each polarizer group are fused and the fused pixels are combined to obtain a Unpolarized images; fusing single and unpolarized images to obtain Bayer images.

In certain embodiments, the processor is further configured to: filter the single polarization image and the unpolarized image; and fuse the filtered single polarization image and the unpolarized image to obtain a Bayer image.

In some embodiments, the processor is further configured to: output the target image after processing the Bayer image sequentially through a dead pixel compensation module, a demosaicing module, a color correction module, a gamma correction module, and a color conversion module.

In some embodiments, the processor is further configured to: convert the original image output by the polarization sensor using a Bayer image regeneration algorithm to obtain a Bayer image when the current scene is a bright scene.

In certain embodiments, the processor is also used to:

The Bayer image is sequentially processed by the camera serial interface decoder, black level module, lens shading correction module, dead pixel compensation module, demosaicing module, color correction module, gamma correction module, and color conversion module to output the target image.

Embodiments of the present application also provide a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and when the program is executed by the processor, the steps of the image processing method in any of the above-mentioned embodiments are implemented.

Referring to FIGS. 1-4 , the image processing method of the embodiment of the present application is used for the polarization sensor 300 . Polarization sensor 300 includes polarizer array 310 and pixel array 320 . The polarizer array 310 includes a plurality of polarizer modules 311 , and each polarizer module 311 includes four polarizer groups 312 . The four polarizer groups 312 in each polarizer module 311 are arranged in a Bayer array. Each polarizer group 312 includes a plurality of polarizers 314 . The polarization directions of each polarizer 314 in each polarizer group 312 are different from each other, and the color components of each polarizer 314 in each polarizer group 312 are the same. The pixel array 320 includes a plurality of pixel units 321 . The plurality of pixel units 321 and the plurality of polarizers 314 are arranged in a one-to-one correspondence. The pixel unit 321 is used for receiving light transmitted through the polarizer 314 to generate electrical signals. The polarization sensor 300 is used to generate a raw image from the electrical signal. The pixels of the original image are arranged in a non-Bayer array. Image processing methods include:

01: According to the current scene, convert the original image to obtain a Bayer image, and the pixels of the Bayer image are arranged in a Bayer array;

03: Output the target image according to the Bayer image.

The image processing method of the embodiment of the present application can be implemented by the image processing apparatus 100 of the embodiment of the present application. Specifically, an image processing method is used for the polarization sensor 300 . Polarization sensor 300 includes polarizer array 310 and pixel array 320 . The polarizer array 310 includes a plurality of polarizer modules 311 , and each polarizer module 311 includes four polarizer groups 312 . The four polarizer groups 312 in each polarizer module 311 are arranged in a Bayer array. Each polarizer group 312 includes a plurality of polarizers 314 . The polarization directions of each polarizer 314 in each polarizer group 312 are different from each other, and the color components of each polarizer 314 in each polarizer group 312 are the same. The pixel array 320 includes a plurality of pixel units 321 . The plurality of pixel units 321 and the plurality of polarizers 314 are arranged in a one-to-one correspondence. The pixel unit 321 is used for receiving light transmitted through the polarizer 314 to generate electrical signals. The polarization sensor 300 is used to generate a raw image from the electrical signal. The pixels of the original image are arranged in a non-Bayer array. The image processing apparatus 100 includes a conversion module 10 and an output module 30 . The conversion module 10 is configured to convert the original image according to the current scene to obtain a Bayer image, and the pixels of the Bayer image are arranged in a Bayer array. The output module 30 is used for outputting the target image according to the Bayer image.

The image processing method of the embodiment of the present application may be implemented by the electronic device 200 of the embodiment of the present application. Specifically, the electronic device 200 includes one or more processors 201 and a memory 202 . The memory 202 stores computer programs. When the computer program is executed by the processor 201, the above steps 01 and 03 are implemented.

The above-mentioned image processing method, image processing apparatus 100 and electronic device 200 can convert the original image arranged in a non-Bayer array output by the polarization sensor 300 into a Bayer image arranged in a Bayer array, so that the original image output by the polarization sensor 300 can be It is processed by the electronic device 200 such as a mobile phone.

Specifically, the polarization sensor 300 can receive external light, and generate an original image according to the received light. The pixels of the original image are arranged in a non-Bayer array, and the electronic device 200 such as a mobile phone cannot process the original image whose pixels are arranged in a non-Bayer array, and the pixels of the original image need to be converted into a format arranged in a Bayer array. The polarization sensor 300 further includes a microlens array 330 disposed directly above the polarizer array 310. The microlens array 330 includes a plurality of microlenses 331, and the microlenses 331, the polarizers 314 and the pixel units 321 are in one-to-one correspondence. Please refer to FIG. 5 . FIG. 5 is a schematic diagram of the polarizer module 311 of the polarization sensor 300 . In some embodiments, the polarizer module 311 may include two first polarizer groups 3122 , one second polarizer group 3124 and one The third polarizer group 3126, the four polarizer groups 312 are arranged in a Bayer array, wherein the color components of all the polarizers 314 in the first polarizer group 3122 are green (Green, G), and the second polarizer group 312 The color components of all polarizers 314 in 3124 are blue (Blue, B), and the color components of all polarizers 314 in the third polarizer group 3126 are red (Red, R). The color components of the polarizers 314 are the same, that is, the polarizers 314 can transmit visible light with the same wavelength range, for example, both can transmit red light with a wavelength range of 625-740 nm, or both can transmit green light with a wavelength range of 492-577 nm , or both can transmit blue light in the wavelength range of 440 to 475 nm. Two first polarizer groups 3122, one second polarizer group 3124 and one third polarizer group 3126 are arranged in a 2*2 form, and the two first polarizer groups 3122 are arranged along the first diagonal direction D1 cloth, a second polarizer group 3124 and a third polarizer group 3126 are arranged along the second diagonal direction D2, and the first diagonal direction D1 and the second diagonal direction D2 are perpendicular to each other. In an example, the arrangement of the polarizer modules 311 is GRBG, and in other examples, the arrangement of the polarizer modules 311 can also be RGGB, BGGR or GBRG. In the embodiment of FIG. 5 , the polarizers 314 in each polarizer group 312 are arranged in the form of 2*2, that is, each polarizer group 312 includes four polarizers 314 , and the polarizers of the four polarizers 314 The directions are different from each other and are respectively a first polarization direction, a second polarization direction, a third polarization direction and a fourth polarization direction. A pixel unit 321 is disposed under each polarizer 314 , and when external light passes through the polarizer 314 and irradiates the pixel unit 321 , the pixel unit 321 can generate a corresponding electrical signal. In one example, the first polarization direction, the second polarization direction, the third polarization direction and the fourth polarization direction are 0°, 45°, 90° and 135°, respectively. It should be noted that, in other embodiments, the polarizers 314 in each polarizer group 312 may also be arranged in the form of 3*3 or 4*4. When the polarizers 314 are arranged in the form of 3*3, Each polarizer group 312 includes 9 polarizers 314 with different polarization directions; when the polarizers 314 are arranged in a 4*4 form, each polarizer group 312 includes 16 polarizers 314 with different polarization directions.

In step 01, the current scene may include a bright scene, a normal scene, a partial dark scene and an overall bright scene, and a bright scene. In some embodiments, the normal scene is a scene where the light intensity is between the first intensity value and the second intensity value, the bright scene is a scene where the light intensity is between the third intensity value and the fourth intensity value, and the bright scene In the scene where the light intensity is greater than the fifth intensity value, the first intensity value is smaller than the second intensity value, the second intensity value is not greater than the third intensity value, the third intensity value is smaller than the fourth intensity value, and the fourth intensity value is not greater than the fifth intensity value intensity value. In some embodiments, the bright scene may include a cloudy outdoor, and when the light intensity of the current scene is between 2000 lux-3000 lux (eg, 2500 lux), it may be determined that the current scene is a cloudy outdoor. In some embodiments, the common scene may include an indoor shopping mall, and when the light intensity of the current scene is between 950 lux-1050 lux (eg, 1000 lux), the current scene may be determined to be an indoor shopping mall. In some embodiments, the partially dark and overall bright scene may include a night scene of street lights or a scene with lights at night. When the light intensity of the current scene is between 300lux and 500lux (for example, 400lux), it may be determined that the current scene is a scene with lights at night. Lighting scene. In some embodiments, the bright and bright scene may include a sunny outdoor, and when the light intensity of the current scene is greater than 8000 lux (eg, 9000 lux), it may be determined that the current scene is a sunny outdoor. In some embodiments, before step 01, the image processing method further includes: determining the current scene. In some embodiments, the electronic device 200 includes a light sensor, and the current scene can be determined according to the measurement value of the light sensor when shooting. In some embodiments, machine learning may be performed on different shooting environments in advance based on the training model, so that the current scene is automatically determined according to the shooting environment during shooting. In other embodiments, the current scene can also be set by the user. After the current scene is determined, the original image is converted according to the current scene to obtain a Bayer image corresponding to the current scene. The pixels of the Bayer image are arranged in a Bayer array, and the Bayer image can be further processed by electronic devices 200 such as mobile phones.

In step 03, the electronic device 200 such as a mobile phone may include an image pipeline, and the image pipeline processes the Bayer image, thereby outputting the target image. In the embodiment shown in FIG. 4 , the electronic device 200 is a mobile phone. In other embodiments, the electronic device 200 may include a tablet computer, a smart watch, or other mobile terminals configured with the polarization sensor 300 .

Please refer to FIG. 6 and FIG. 7, in some embodiments, the current scene includes a bright scene, and step 01 includes:

011: When the current scene is a bright scene, extract the pixels of the original image corresponding to the polarizers 314 having the same polarization direction in each polarizer group 312 and combine the extracted pixels to obtain a Bayer image.

The image processing methods of the above embodiments can be implemented by the image processing apparatus 100 of the embodiments of the present application. Specifically, the conversion module 10 is configured to extract the pixels of the original image corresponding to the polarizers 314 having the same polarization direction in each polarizer group 312 and combine the extracted pixels when the current scene is a bright scene. Obtain a Bayer image.

The image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application. Specifically, the processor is configured to extract the pixels of the original image corresponding to the polarizers 314 having the same polarization direction in each polarizer group 312 and combine the extracted pixels to obtain Bayer when the current scene is a bright scene image.

In this way, in a bright scene, a Bayer image with a single polarization direction can be output without using a complex algorithm to process the original image.

Specifically, the brighter scene may include a cloudy outdoor. In an example, each polarizer group 312 includes polarizers 314 with four polarization directions of 0°, 45°, 90°, and 135°, and when the current scene is a bright scene, extract each polarizer group The pixels of the original image corresponding to the polarizers 314 in the 0° polarization direction in 312, and the extracted pixels are combined according to the combination between the corresponding polarizer groups 312 to generate a 0° polarized image; each polarizer group is extracted. The pixels of the original image corresponding to the polarizers 314 in the 45° polarization direction in 312, and the extracted pixels are combined according to the combination between the corresponding polarizer groups 312 to generate a 45° polarized image; each polarizer group is extracted The pixels of the original image corresponding to the polarizers 314 in the 90° polarization direction in 312, and the extracted pixels are combined according to the combination between the corresponding polarizer groups 312 to generate a 90° polarized image; each polarizer group is extracted The pixels of the original image corresponding to the polarizers 314 in the 135° polarization direction in 312, and the extracted pixels are combined according to the combination between the corresponding polarizer groups 312 to generate a 135° polarized image. Further, 0 can be selected. At least one of a ° polarization image, a 45° polarization image, a 90° polarization image, or a 135° polarization image serves as a Bayer image. When the current scene is a bright scene, each frame of the obtained Bayer image is a single-polarization image, and the number of pixels of each frame of the obtained Bayer image is one-fourth of the number of pixels of the original image.

Please refer to FIG. 8 and FIG. 9. In some embodiments, the current scene includes a common scene, and step 01 includes:

012: When the current scene is an ordinary scene, fuse the pixels of the original image corresponding to each polarizer 314 in each polarizer group 312 and combine the fused pixels to obtain a Bayer image.

The image processing methods of the above embodiments can be implemented by the image processing apparatus 100 of the embodiments of the present application. Specifically, the conversion module 10 is configured to fuse the pixels of the original image corresponding to each polarizer 314 in each polarizer group 312 and combine the fused pixels to obtain a Bayer image when the current scene is an ordinary scene.

The image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application. Specifically, the processor is configured to fuse the pixels of the original image corresponding to each polarizer 314 in each polarizer group 312 and combine the fused pixels to obtain a Bayer image when the current scene is an ordinary scene.

In this way, in a common scenario, an unpolarized Bayer image with a high signal-to-noise ratio can be output without using a complex algorithm to process the original image.

Specifically, common scenarios include indoor shopping malls. In some embodiments, when the current scene is an ordinary scene, the pixel values of the original images corresponding to all the polarizers 314 in one polarizer group 312 are added or averaged to obtain the corresponding polarizer in the Bayer image. The pixel values at the positions of the groups 312 are determined by performing the above operations on all the polarizer groups 312 to determine the pixel values corresponding to the positions of each polarizer group 312 in the Bayer image. The Bayer image thus obtained has a higher signal-to-noise ratio. When the current scene is an ordinary scene, the obtained Bayer image is an unpolarized image, and the number of pixels of the obtained Bayer image is a quarter of the number of pixels of the original image.

Please refer to FIGS. 10-12. In some embodiments, the current scene includes an overall dark scene and a partially bright scene. Step 01 includes:

013: In the case where the current scene is an overall dark and partially bright scene, extract the pixels of the original image corresponding to the polarizers 314 having the same polarization direction in each polarizer group 312 and combine the extracted pixels to obtain a single polarized image;

014: fuse the pixels of the original image corresponding to each polarizer 314 in each polarizer group 312 and combine the fused pixels to obtain an unpolarized image;

015: Fuse single-polarized and unpolarized images to obtain a Bayer image.

The image processing methods of the above embodiments can be implemented by the image processing apparatus 100 of the embodiments of the present application. Specifically, the conversion module 10 includes an extraction unit 13 , a first fusion unit 14 , and a second fusion unit 15 . The extraction unit 13 is used for extracting the pixels of the original image corresponding to the polarizers 314 having the same polarization direction in each polarizer group 312 and combining the extracted pixels when the current scene is an overall darker and partially brighter scene, to obtain single-polarization images. The first fusion unit 14 is configured to fuse the pixels of the original image corresponding to each polarizer 314 in each polarizer group 312 and combine the fused pixels to obtain an unpolarized image. The second fusion unit 15 is used to fuse the single-polarization image and the non-polarization image to obtain a Bayer image.

The image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application. Specifically, the processor is configured to extract the pixels of the original image corresponding to the polarizers 314 having the same polarization direction in each polarizer group 312 and combine the extracted pixels when the current scene is an overall darker and partially brighter scene , to obtain a single-polarization image, and for fusing the pixels of the original image corresponding to each polarizer 314 in each polarizer group 312 and combining the fused pixels to obtain an unpolarized image, and for fusing the single-polarization image and unpolarized images to obtain Bayer images.

In this way, when the current scene is an overall dark scene and a partially bright scene, a single-polarization image and a non-polarization image are fused to obtain a Bayer image with polarization information.

Specifically, the overall dark and partially bright scene may include a street lamp night scene or a scene with lights at night. Each polarizer group 312 includes a plurality of polarizers 314 with different polarization directions, and extracting the original images corresponding to the polarizers 314 with the same polarization direction in each polarizer group 312 can obtain multiple frames of single-polarization images. The polarization information is different. For example, when the polarizer group 312 includes polarizers 314 with four polarization directions of 0°, 45°, 90° and 135°, 4 frames of single-polarization images can be obtained in step 013, which are the 0° polarized image, the 45° polarized image, and the 45° polarized image. ° polarized image, 90 ° polarized image and 135 ° polarized image, and the number of pixels of each frame of single polarized image is one-fourth of that of the original image.

In step 014, the pixel values of the original images corresponding to all the polarizers 314 in one polarizer group 312 are added or averaged to obtain the pixel values corresponding to the position of the polarizer group 312 in the unpolarized image, and for all polarizers 312 The plate group 312 performs the above operations to determine the pixel value corresponding to the position of each polarizer plate group 312 in the unpolarized image. The unpolarized image has one quarter the number of pixels of the original image.

In step 015, when the current scene is an overall dark and partially bright scene, select a frame of images from multiple frames of single-polarization images, so as to fuse a frame of single-polarization images and a frame of unpolarized images to obtain a frame with Bayer image of polarization information. Since the number of pixels of each frame of single-polarization image is one-fourth of the number of pixels of the original image, and the number of pixels of each frame of unpolarized image is also one-fourth of the number of pixels of the original image, in the current scene, the overall darkening and partial polarization In the case of a bright scene, the number of pixels in each frame of the Bayer image obtained by fusing the single-polarization image and the non-polarization image is a quarter of the pixel number of the original image, and the Bayer image is a fused polarization image.

Referring to Figures 13-15, in some embodiments, step 015 includes:

0151: Filter the single-polarization image and the non-polarization image;

0153: Fuse the filtered single-polarization image and the unpolarized image to obtain a Bayer image.

The image processing methods of the above embodiments can be implemented by the image processing apparatus 100 of the embodiments of the present application. Specifically, the second fusion unit 15 includes a filtering subunit 151 and a fusion subunit 153 . The filtering subunit 151 is used for filtering the single-polarization image and the non-polarization image. The fusion subunit 153 is used to fuse the filtered single-polarization image and the non-polarization image to obtain a Bayer image.

The image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application. Specifically, the processor is used for filtering the single-polarization image and the non-polarization image, and for fusing the filtered single-polarization image and the non-polarization image to obtain a Bayer image.

In this way, a Bayer image with better visual effect can be obtained.

Specifically, in step 0151, in some embodiments, the single-polarization image and the non-polarization image are respectively subjected to bilateral filtering with the unpolarized image as the guide map, so as to perform a smoothing process on the flat area. The processing can be represented by the following formula:

Among them, k _p =∑ _q∈Ω f(||pq||)g(||I _p ′-I _q ′||), J _p is the output pixel value, k _p is the weight sum, Ω is the filter window ( For example, a 3*3 window), p is the coordinate of the pixel to be filtered in the phase map, q is the coordinate of the pixel in the filter window in the phase map, I _q is the pixel value corresponding to point q, and I _p ′ is The pixel value corresponding to the pixel to be filtered in the guide map, and I _q ′ is the pixel value corresponding to point q in the guide map. f represents the weight of each coordinate point in the filtering window, the weight of each coordinate point in the filtering window is fixed, and the closer to the center, the greater the weight. g represents the weight of the difference between the pixels in other positions and the center pixel, the larger the difference, the smaller the weight. In some embodiments, f and g can be any weight distribution function, and they can be the same or different. For example, f and g are both Gaussian functions, and the Gaussian function is, for example,

a, b and c are adjustment coefficients.

In step 0153, a Fusion algorithm is used to fuse the filtered single-polarization image and the non-polarization image to obtain a Bayer image. Please refer to Figure 15, taking the green pixel unit as an example. G_smooth represents the green pixel unit in the filtered single-polarization image, including five pixel points G1, G2, G3, G4, and G5. G'_smooth represents the green pixel unit in the filtered unpolarized image, including five pixel points G'1, G'2, G'3, G'4, and G'5. In the process of fusing G_smooth and G'_smooth, the pixel value of the center point after fusion can be represented by the following formula: G5_output=G_mean*(G' ₅ /G'_mean), where G5_output represents the pixel value of the center point,

Σα _n =1, G _n represents the pixel value corresponding to the Gn point, and G′ _n represents the pixel value corresponding to the G′n point. The fusion process of the red pixel unit and the blue pixel unit of the filtered single-polarization image and the non-polarization image is basically the same as the fusion process of the green pixel unit. In order to avoid redundancy, details are not repeated here.

16 and 17, in some embodiments, step 03 includes:

031: Pass the Bayer image through a bad pixel correction (BPC) module, a demosaic (DM) module, a color correction (CC) module, a gamma (Gamma) correction module, and a color conversion (color) module in turn. conversion, CV) module to output the target image after processing.

The image processing methods of the above embodiments can be implemented by the image processing apparatus 100 of the embodiments of the present application. Specifically, the output module 30 is configured to sequentially process the Bayer image through a dead pixel compensation module, a demosaic module, a color correction module, a gamma correction module, and a color conversion module to output the target image.

The image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application. Specifically, the processor is configured to sequentially process the Bayer image through a dead pixel compensation module, a demosaic module, a color correction module, a gamma correction module, and a color conversion module to output the target image.

In this way, the Bayer image enters the image pipeline, and after being processed by the image pipeline, it is converted into the target image and output.

Specifically, the target image includes a YUV image. In some embodiments, the original image (Quadbayer polarized RAW) generated by the polarization sensor (Sensor) 300 is sequentially passed through a camera serial interface decoder (Camera Serial Interface Decoder, CSID), a black level (Optical black, OB) module, After the lens shading correction (LSC) module, it enters the double-rate synchronous dynamic random access memory (Double Data Rate, DDR), and then converts it into a Bayer image in the Binning Algorithm module, and then passes through the dead pixel compensation module in turn. The mosaic module, color correction module, gamma correction module, and color conversion module process and output the target image.

Further, the camera serial interface decoder is used to decode and identify the original image. The circuit of the polarization sensor 300 itself will have dark current, so when there is no light irradiation, the pixel unit 321 also has a certain output voltage. Therefore, the influence of the dark current needs to be subtracted, that is, black level correction is performed. Lens shading correction is to solve the problem of shadows appearing around the lens due to the optical characteristics of the lens, that is, the lens has uneven optical refraction. Since the lens itself is a convex lens, due to the principle of convex lens, the center must receive more light than the periphery, and the light intensity received by the edge area of the polarization sensor 300 image area is smaller than that of the center, resulting in inconsistent brightness between the center and the four corners. The double-rate synchronous dynamic random access memory is used to store the original image after successively passing through the camera serial interface decoder, black level module and lens shading correction module. Please refer to Figure 18, the Binning Algorithm module can retrieve the original image from the DDP and execute steps 011, 012, 013, 014, 015, 0151, and 0153 to convert the original image into a Bayer image. Bad pixel compensation is bad pixel compensation, which refers to bad pixel correction. When a pixel unit 321 in the polarization sensor 300 cannot work normally, the pixel value corresponding to the pixel unit 321 can be obtained by using bad pixel correction. The demosaicing module can interpolate the Bayer image of the Bayer array into an RGB image, wherein the RGB image can be three frames, namely, the R image, the G image and the B image obtained after interpolation. The color saturation of the image after color correction is more obvious, which is more in line with the perception of the human eye. Gamma correction can make the image look more in line with the characteristics of the human eye. For example, the gamma correction formula is Out=Ingamma or Out=In1/gamma, where In represents the input image and Out represents the output image. The color conversion module can convert RGB images to YUV images.

It should be noted that, in the embodiment of FIG. 16 , the schematic flowchart includes step 011 and step 031; in some embodiments, the schematic flowchart may include step 012 and step 031; in some embodiments, the schematic flowchart may Including step 013, step 014, step 015 and step 031; in some embodiments, the schematic flowchart may include step 013, step 014, step 0151, step 0153 and step 031, which is not limited herein.

Please refer to FIG. 19 and FIG. 20. In some embodiments, the current scene includes a bright scene, and step 01 includes:

016: In the case that the current scene is a bright scene, use a Bayer image regeneration (Remosaic) algorithm to convert the original image output by the polarization sensor 300 to obtain the Bayer image.

The image processing methods of the above embodiments can be implemented by the image processing apparatus 100 of the embodiments of the present application. Specifically, the conversion module 10 is configured to convert the original image output by the polarization sensor 300 by using a Bayer image regeneration algorithm to obtain the Bayer image when the current scene is a bright scene.

The image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application. Specifically, the processor is configured to convert the original image output by the polarization sensor 300 by using a Bayer image regeneration algorithm to obtain the Bayer image when the current scene is a bright scene.

In this way, when the current scene is a bright scene, a Bayer image with a higher resolution can be obtained by using the Remosaic algorithm.

Specifically, the bright scene may include a sunny day outdoors. In some embodiments, when the current scene is a bright scene, the Remosaic algorithm can be used to convert the original image into a Bayer image with a higher resolution with a specific polarization direction. In some embodiments, when the current scene is a bright scene, the Remosaic algorithm can be used to convert the original image into a Bayer image with higher resolution without polarization information. When the current scene is a bright scene, the obtained Bayer image is a specific polarized image or a full-size unpolarized image, and the obtained Bayer image has the same number of pixels as the original image, so the Bayer image has a higher resolution.

Referring to Figures 21-23, in some embodiments, step 03 includes:

033: The Bayer image is sequentially processed by the camera serial interface decoder, black level module, lens shading correction module, dead pixel compensation module, demosaicing module, color correction module, gamma correction module, and color conversion module, and then output the target image .

The image processing methods of the above embodiments can be implemented by the image processing apparatus 100 of the embodiments of the present application. Specifically, the output module 30 is used to sequentially pass the Bayer image through the camera serial interface decoder, the black level module, the lens shading correction module, the dead pixel compensation module, the demosaicing module, the color correction module, the gamma correction module, and the color conversion module. The module outputs the target image after processing.

The image processing method of the above embodiment can be implemented by the electronic device 200 of the embodiment of the present application. Specifically, the processor is configured to sequentially process the Bayer image through a camera serial interface decoder, a black level module, a lens shading correction module, a dead pixel compensation module, a demosaicing module, a color correction module, a gamma correction module, and a color conversion module Then output the target image.

In this way, the Bayer image enters the image pipeline, and after being processed by the image pipeline, it is converted into the target image and output.

Specifically, referring to FIG. 22 , in some embodiments, the original image (Quadbayer polarized RAW) generated by the polarization sensor (Sensor) 300 is converted into a Bayer image of a Bayer array after passing through the Fullsize Algorithm module, and the Bayer image sequentially passes through the camera string. Line interface decoder, black level module, lens shading correction module, dead pixel compensation module, demosaicing module, color correction module, gamma correction module, and color conversion module output the target image after processing. The Fullsize Algorithm module is used to perform step 016 to convert the original image to a Bayer image. Referring to FIG. 23, in some embodiments, the original image (Quadbayer polarized RAW) generated by the polarization sensor (Sensor) 300 is converted into a Bayer image of the Bayer array after passing through the Binning Algorithm module, and the Bayer image is sequentially decoded by the camera serial interface The output target image is processed by the camera, black level module, lens shading correction module, dead pixel compensation module, demosaicing module, color correction module, gamma correction module, and color conversion module. For the explanation of the camera serial interface decoder, black level module, lens shading correction module, dead pixel compensation module, demosaicing module, color correction module, gamma correction module, and color conversion module, reference may be made to the above embodiments.

It should be noted that, in the embodiment of FIG. 21 , the schematic flowchart includes steps 016 and 033; in some embodiments, the schematic flowchart may include steps 011 and 033; in some embodiments, the schematic flowchart may Including step 012 and step 033; in some embodiments, the schematic flow diagram may include step 013, step 014, step 015 and step 033; Step 0153 and Step 033 are not limited here.

It should be pointed out that the specific numerical values mentioned above are only used as examples to describe the implementation of the present application in detail, and should not be construed as a limitation on the present application. In other examples or implementations or embodiments, other numerical values may be selected according to the present application, which are not specifically limited herein.

The computer-readable storage medium of the embodiments of the present application stores a computer program thereon, and when the program is executed by a processor, the steps of the image processing method of any of the above-mentioned embodiments are implemented.

For example, when the program is executed by the processor, the steps of the following image processing method are implemented:

01: According to the current scene, convert the original image to obtain a Bayer image, and the pixels of the Bayer image are arranged in a Bayer array;

03: Output the target image according to the Bayer image.

It is understood that a computer program includes computer program code. The computer program code may be in source code form, object code form, an executable file or some intermediate form, or the like. Computer-readable storage media may include: any entity or device capable of carrying computer program codes, recording media, U disk, removable hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random storage Access memory (RAM, Random Access Memory), and software distribution media, etc. The processor can be a central processing unit, or other general-purpose processors, digital signal processors (Digital Signal Processors, DSPs), application specific integrated circuits (Application Specific Integrated Circuits, ASICs), ready-made programmable gate arrays (Field-Programmable Gate arrays) Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

In the description of this specification, description with reference to the terms "one embodiment," "some embodiments," "example," "specific example," or "some examples", etc., mean specific features described in connection with the embodiment or example , structure, material or feature is included in at least one embodiment or example of the present application. In this specification, schematic representations of the above terms are not necessarily directed to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

Any description of a process or method in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing a specified logical function or step of the process , and the scope of the preferred embodiments of the present application includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions substantially concurrently or in the reverse order depending upon the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present application belong.

Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limitations to the present application. Embodiments are subject to variations, modifications, substitutions and variations.

Claims (25)

1. An image processing method for a polarization sensor, wherein the polarization sensor includes a polarizer array and a pixel array, the polarizer array includes a plurality of polarizer modules, each of the polarizer modules includes four polarizer groups , the four polarizer groups in each of the polarizer modules are arranged in a Bayer array, each of the polarizer groups includes a plurality of polarizers, and each of the polarizers in each of the polarizer groups The polarization directions are different from each other, the color components of the polarizers in each polarizer group are the same, and the pixel array includes a plurality of pixel units, a plurality of the pixel units and a plurality of the polarizers. In a corresponding arrangement, the pixel unit is configured to receive the light passing through the polarizer to generate an electrical signal, the polarization sensor is configured to generate an original image according to the electrical signal, and the pixels of the original image are arranged in a non-Bayer array. cloth, the image processing method includes:

   According to the current scene, the original image is converted to obtain a Bayer image, and the pixels of the Bayer image are arranged in a Bayer array;

   A target image is output based on the Bayer image.
2. The image processing method according to claim 1, wherein the current scene includes a bright scene, and converting the original image to obtain a Bayer image according to the current scene comprises:

   In the case where the current scene is the bright scene, extracting the pixels of the original image corresponding to the polarizers having the same polarization direction in each of the polarizer groups, and combining the extracted pixels to obtain Obtain the Bayer image.
3. The image processing method according to claim 1, wherein the current scene includes a common scene, and converting the original image according to the current scene to obtain a Bayer image comprises:

   In the case that the current scene is the common scene, the pixels of the original image corresponding to each of the polarizers in each polarizer group are fused, and the fused pixels are combined to obtain the Bayer image.
4. The image processing method according to claim 1, wherein the current scene includes an overall dark scene and a partially bright scene, and converting the original image to obtain a Bayer image according to the current scene includes:

   In the case that the current scene is the overall dark and partially bright scene, extract the pixels of the original image corresponding to the polarizers with the same polarization direction in each polarizer group, and analyze the extracted pixels combined to obtain a single polarization image;

   fusing the pixels of the original image corresponding to each of the polarizers in each of the polarizer groups and combining the fused pixels to obtain an unpolarized image;

   The single polarization image and the unpolarized image are fused to obtain the Bayer image.
5. The image processing method according to claim 4, wherein the fusing the single-polarization image and the non-polarization image to obtain the Bayer image comprises:

   filtering the single polarization image and the unpolarized image;

   The filtered single polarization image and the unpolarized image are fused to obtain the Bayer image.
6. The image processing method according to any one of claims 2-5, wherein the outputting the target image according to the Bayer image comprises:

   The Bayer image is sequentially processed by a dead pixel compensation module, a demosaic module, a color correction module, a gamma correction module, and a color conversion module to output the target image.
7. The image processing method according to claim 1, wherein the current scene includes a bright scene, and converting the original image according to the current scene to obtain a Bayer image comprises:

   When the current scene is the bright scene, a Bayer image regeneration algorithm is used to convert the original image output by the polarization sensor to obtain the Bayer image.
8. The image processing method according to any one of claims 2-5 and 7, wherein the outputting the target image according to the Bayer image comprises:

   The Bayer image is sequentially processed by a camera serial interface decoder, a black level module, a lens shading correction module, a dead pixel compensation module, a demosaicing module, a color correction module, a gamma correction module, and a color conversion module to output the described Bayer image. target image.
9. An image processing device for a polarization sensor, wherein the polarization sensor includes a polarizer array and a pixel array, the polarizer array includes a plurality of polarizer modules, each of the polarizer modules includes four polarizer groups , the four polarizer groups in each of the polarizer modules are arranged in a Bayer array, each of the polarizer groups includes a plurality of polarizers, and each of the polarizers in each of the polarizer groups The polarization directions are different from each other, the color components of the polarizers in each polarizer group are the same, and the pixel array includes a plurality of pixel units, a plurality of the pixel units and a plurality of the polarizers. In a corresponding arrangement, the pixel unit is configured to receive the light passing through the polarizer to generate an electrical signal, the polarization sensor is configured to generate an original image according to the electrical signal, and the pixels of the original image are arranged in a non-Bayer array. cloth, the image processing device includes:

   a conversion module, configured to convert the original image according to the current scene to obtain a Bayer image, the pixels of the Bayer image are arranged in a Bayer array;

   An output module, configured to output a target image according to the Bayer image.
10. The image processing apparatus according to claim 9, wherein the current scene includes a bright scene, and the conversion module is further configured to:

    In the case where the current scene is the bright scene, extracting the pixels of the original image corresponding to the polarizers having the same polarization direction in each of the polarizer groups, and combining the extracted pixels to obtain Obtain the Bayer image.
11. The image processing apparatus according to claim 9, wherein the current scene includes a common scene, and the conversion module is further configured to:

    In the case that the current scene is the common scene, the pixels of the original image corresponding to each of the polarizers in each polarizer group are fused, and the fused pixels are combined to obtain the Bayer image.
12. The image processing apparatus according to claim 9, wherein the current scene comprises an overall dark and partially bright scene, and the conversion module comprises:

    The extraction unit is configured to extract the pixels of the original image corresponding to the polarizers having the same polarization direction in each of the polarizer groups in the case that the current scene is the overall darker and partially brighter scene, and analyze the The extracted pixels are combined to obtain a single polarization image;

    a first fusion unit, configured to fuse the pixels of the original image corresponding to each of the polarizers in each of the polarizer groups and combine the fused pixels to obtain an unpolarized image;

    A second fusion unit, configured to fuse the single-polarization image and the non-polarization image to obtain the Bayer image.
13. The image processing apparatus according to claim 12, wherein the second fusion unit comprises:

    a filtering subunit, configured to filter the single-polarization image and the non-polarization image;

    A fusion subunit, configured to fuse the filtered single-polarization image and the non-polarization image to obtain the Bayer image.
14. The image processing device according to claims 10-13, wherein the output module is further configured to:

    The Bayer image is sequentially processed by a dead pixel compensation module, a demosaic module, a color correction module, a gamma correction module, and a color conversion module to output the target image.
15. The image processing apparatus according to claim 9, wherein the current scene includes a bright scene, and the conversion module is further configured to:

    When the current scene is the bright scene, a Bayer image regeneration algorithm is used to convert the original image output by the polarization sensor to obtain the Bayer image.
16. The image processing apparatus according to claims 10-13 and 15, wherein the output module is further configured to:

    The Bayer image is sequentially processed by a camera serial interface decoder, a black level module, a lens shading correction module, a dead pixel compensation module, a demosaicing module, a color correction module, a gamma correction module, and a color conversion module to output the described Bayer image. target image.
17. An electronic device, wherein a polarization sensor includes a polarizer array and a pixel array, the polarizer array includes a plurality of polarizer modules, each of the polarizer modules includes four polarizer groups, each of the polarizer modules Four of the polarizer groups are arranged in a Bayer array, each of the polarizer groups includes a plurality of polarizers, and the polarization directions of the polarizers in each of the polarizer groups are different from each other, and each polarizer group has a different polarization direction. The color components of each of the polarizers in each of the polarizer groups are the same, the pixel array includes a plurality of pixel units, and a plurality of the pixel units and a plurality of the polarizers are arranged in a one-to-one correspondence, and the pixel units is used for receiving light transmitted through the polarizer to generate an electrical signal, the polarization sensor is used for generating an original image according to the electrical signal, the pixels of the original image are arranged in a non-Bayer array, and the electronic device includes a or a plurality of processors and a memory, the memory storing a computer program, where the computer program is executed by the processor, the processor is used to:

    According to the current scene, the original image is converted to obtain a Bayer image, and the pixels of the Bayer image are arranged in a Bayer array;

    A target image is output based on the Bayer image.
18. The electronic device according to claim 17, wherein the current scene includes a bright scene, and the processor is further configured to:

    In the case where the current scene is the bright scene, extracting the pixels of the original image corresponding to the polarizers having the same polarization direction in each of the polarizer groups, and combining the extracted pixels to obtain Obtain the Bayer image.
19. The image processing method according to claim 17, wherein the current scene includes a common scene, and the processor is further configured to:

    In the case that the current scene is the common scene, the pixels of the original image corresponding to each of the polarizers in each polarizer group are fused, and the fused pixels are combined to obtain the Bayer image.
20. The image processing method according to claim 17, wherein the current scene comprises an overall dark scene and a partially bright scene, and the processor is further configured to:

    In the case that the current scene is the overall dark and partially bright scene, extract the pixels of the original image corresponding to the polarizers with the same polarization direction in each polarizer group, and analyze the extracted pixels combined to obtain a single polarization image;

    fusing the pixels of the original image corresponding to each of the polarizers in each of the polarizer groups and combining the fused pixels to obtain an unpolarized image;

    The single polarization image and the unpolarized image are fused to obtain the Bayer image.
21. The image processing method according to claim 20, wherein the processor is further used for:

    filtering the single polarization image and the unpolarized image;

    The filtered single polarization image and the unpolarized image are fused to obtain the Bayer image.
22. The image processing method according to any one of claims 18-21, wherein the processor is further configured to:

    The Bayer image is sequentially processed by a dead pixel compensation module, a demosaic module, a color correction module, a gamma correction module, and a color conversion module to output the target image.
23. The image processing method according to claim 17, wherein the processor is further configured to:

    When the current scene is the bright scene, a Bayer image regeneration algorithm is used to convert the original image output by the polarization sensor to obtain the Bayer image.
24. The image processing method according to any one of claims 18-21 and 23, wherein the processor is further configured to:

    The Bayer image is sequentially processed by a camera serial interface decoder, a black level module, a lens shading correction module, a dead pixel compensation module, a demosaicing module, a color correction module, a gamma correction module, and a color conversion module to output the described Bayer image. target image.
25. A computer-readable storage medium on which a computer program is stored, wherein, when the program is executed by a processor, the steps of the image processing method according to any one of claims 1-8 are implemented.

Images