CN115118859A - Electronic device and processing method - Google Patents
Electronic device and processing method Download PDFInfo
- Publication number
- CN115118859A CN115118859A CN202210736133.XA CN202210736133A CN115118859A CN 115118859 A CN115118859 A CN 115118859A CN 202210736133 A CN202210736133 A CN 202210736133A CN 115118859 A CN115118859 A CN 115118859A
- Authority
- CN
- China
- Prior art keywords
- image
- images
- pixel
- same
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 36
- 230000010287 polarization Effects 0.000 claims abstract description 137
- 239000003086 colorant Substances 0.000 claims abstract description 6
- 238000000034 method Methods 0.000 claims description 25
- 230000004927 fusion Effects 0.000 claims description 17
- 230000006698 induction Effects 0.000 claims description 5
- 238000012545 processing Methods 0.000 claims description 3
- 230000004044 response Effects 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 24
- 230000008569 process Effects 0.000 description 6
- 238000003384 imaging method Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000000694 effects Effects 0.000 description 3
- 238000013459 approach Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 238000007499 fusion processing Methods 0.000 description 2
- 239000011521 glass Substances 0.000 description 2
- 230000000750 progressive effect Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 1
- 238000009432 framing Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 239000000758 substrate Substances 0.000 description 1
- 230000001960 triggered effect Effects 0.000 description 1
Images
Landscapes
- Studio Devices (AREA)
Abstract
The application discloses an electronic device and a processing method, wherein the electronic device comprises: a first camera module; wherein, first camera module includes: an image sensor, a photosensitive array of the image sensor comprising a plurality of pixel groups, the pixel groups comprising a plurality of pixel cells; the pixel units of each pixel group sense the light with the same color; the pixel units of the adjacent pixel groups sense light with different colors; the polarizing device is positioned on a photosensitive light path of the image sensor and comprises a plurality of polarizing groups, and each polarizing group is provided with a plurality of polarizing units with different polarization directions; the plurality of pixel groups correspond to the plurality of polarized light groups one to one; the number of the pixel units is the same as that of the polarizing units, and the pixel units are arranged in a one-to-one correspondence manner.
Description
Technical Field
The present application relates to the field of electronic devices, and more particularly, to an electronic device and a processing method.
Background
With the continuous development of scientific technology, more and more electronic devices with image acquisition functions are widely applied to daily life and work of people, bring great convenience to the daily life and work of people, and become an indispensable important tool for people at present.
In the existing electronic equipment, due to the influence of the reflected polarized light, the image quality of the obtained image in the corresponding specular reflection area (such as the surface of a smooth object like a glass window, a spectacle lens, a desktop and the like) is poor.
Disclosure of Invention
In view of this, the present application provides an electronic device and a processing method, and the scheme is as follows:
an electronic device, comprising:
a first camera module;
wherein, first camera module includes:
an image sensor, a photosensitive array of the image sensor comprising a plurality of pixel groups, the pixel groups comprising a plurality of pixel cells; the pixel units of each pixel group sense the light with the same color; the pixel units of the adjacent pixel groups sense light with different colors;
the polarizing device is positioned on a photosensitive light path of the image sensor and comprises a plurality of polarizing groups, and each polarizing group is provided with a plurality of polarizing units with different polarization directions; the plurality of pixel groups correspond to the plurality of polarized light groups one to one; the number of the pixel units is the same as that of the polarizing units, and the pixel units are arranged in a one-to-one correspondence manner.
Preferably, in the electronic device, the first camera module further includes:
a lens assembly;
wherein the micro lens is arranged opposite to at least one pixel unit; wherein the polarization unit is located between the corresponding pixel unit and the microlens.
Preferably, in the above electronic device, the electronic device further includes:
the processor is used for obtaining the induction parameters of each pixel unit in the image sensor;
obtaining a plurality of images based on the number of the polarization units of the polarization group, wherein the same image corresponds to the polarization units with the same polarization direction;
and determining a target image from the plurality of images, wherein the polarization directions of the polarization units corresponding to the target image are the same.
Preferably, in the electronic device, the processor is further configured to respond to a photographing instruction to obtain at least one fused image based on the target image; the polarizing unit corresponding to each fused image in the at least one fused image and the polarizing unit corresponding to the target image are the same polarizing unit;
fusing the target image and the at least one fused image to generate a photographed image;
saving the photographed image;
or;
the processor is further used for responding to a photographing instruction and obtaining a plurality of fusion images based on the target image; the polarization unit corresponding to each fused image in the multiple fused images and the polarization unit corresponding to the target image are the same polarization unit;
fusing the plurality of fused images to generate a photographed image;
and saving the photographed image.
Preferably, in the above electronic device, the electronic device further includes:
a second camera module;
the processor is also used for responding to a photographing instruction, and obtaining a fused image and a standard image obtained through the second camera module based on the target image; the polarizing unit corresponding to the fused image and the polarizing unit corresponding to the target image are the same polarizing unit;
fusing the fused image and the standard image to generate a photographed image;
and saving the photographed image.
The application also provides a processing method, which comprises the following steps:
caching at least one group of image groups, wherein the image groups comprise a plurality of images formed on the basis of induction parameters obtained by an image sensor of a first camera module at the same moment, the same image is formed on the basis of polarized light in the same polarization direction, and different images are formed on the basis of polarized light in different polarization directions;
and in the same image group, determining a target image in the plurality of images.
Preferably, in the above processing method, the method further includes:
responding to a photographing instruction, and obtaining at least one fused image based on the target image; the polarizing unit corresponding to each fused image in the at least one fused image and the polarizing unit corresponding to the target image are the same polarizing unit;
fusing the target image and the at least one fused image to generate a photographed image;
and saving the photographed image.
Or;
responding to a photographing instruction, and obtaining a plurality of fusion images based on the target image; the polarization unit corresponding to each fused image in the multiple fused images and the polarization unit corresponding to the target image are the same polarization unit;
fusing the plurality of fused images to generate a photographed image;
saving the photographed image;
or;
responding to a photographing instruction, and obtaining a fused image and a standard image obtained through a second camera module based on the target image; the polarizing unit corresponding to the fused image and the polarizing unit corresponding to the target image are the same polarizing unit;
fusing the fused image and the standard image to generate a photographed image;
and saving the photographed image.
Preferably, in the above processing method, the method of obtaining a fusion image based on the target image includes:
determining the target image based on a plurality of images in the same image group;
and determining an image with the same mark as the target image from the images of the other image groups as a fused image based on the mark of the target image.
Preferably, in the above processing method, the determining the target image from the plurality of images includes:
determining a target region for each of the plurality of images; the target area of each image is the same area;
and determining a target image from the plurality of images, wherein the brightness of the target area of the target image is lower than the brightness of the target areas of other images in the plurality of images.
Preferably, in the processing method, the determining the target region of each of the plurality of images for the same image group includes:
acquiring the brightness standard deviation of each pixel point based on the plurality of images;
determining the target region in the image based on the brightness standard deviation.
As can be seen from the above description, in the electronic device and the processing method provided in the technical solution of the present application, the electronic device includes: a first camera module; wherein, first camera module includes: an image sensor, a photosensitive array of the image sensor comprising a plurality of pixel groups, the pixel groups comprising a plurality of pixel cells; the pixel units of each pixel group sense the light with the same color; the pixel units of the adjacent pixel groups sense light with different colors; the polarizing device is positioned on a photosensitive light path of the image sensor and comprises a plurality of polarizing groups, and each polarizing group is provided with a plurality of polarizing units with different polarization directions; the plurality of pixel groups correspond to the plurality of polarized light groups one to one; the number of the pixel units is the same as that of the polarizing units, and the pixel units are arranged in a one-to-one correspondence manner.
Drawings
In order to more clearly illustrate the embodiments of the present application or technical solutions in related arts, the drawings used in the description of the embodiments or prior arts will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
The structures, proportions, and dimensions shown in the drawings and described in the specification are for illustrative purposes only and are not intended to limit the scope of the present disclosure, which is defined by the claims, but rather by the claims, it is understood that these drawings and their equivalents are merely illustrative and not intended to limit the scope of the present disclosure.
Fig. 1 is a schematic structural diagram of a first camera module according to an embodiment of the present disclosure;
fig. 2 is a schematic structural diagram of an image sensor in the first camera module shown in fig. 1;
FIG. 3 is a schematic structural diagram of a polarizer in the first camera module shown in FIG. 1;
FIG. 4 is a schematic diagram of the working principle of a polarizer;
fig. 5 is a schematic structural diagram of another first camera module according to an embodiment of the present disclosure;
FIG. 6 is a schematic structural diagram of a lens assembly in the first camera shown in FIG. 5;
fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application;
fig. 8 is a schematic structural diagram of another electronic device provided in the embodiment of the present application;
fig. 9 is a schematic flow chart of a processing method according to an embodiment of the present application;
FIG. 10 is a schematic flow chart of another processing method provided in the embodiments of the present application;
FIG. 11 is a schematic flow chart diagram illustrating another exemplary processing method according to an embodiment of the present disclosure;
FIG. 12 is a schematic flow chart diagram illustrating yet another exemplary processing method according to an embodiment of the present disclosure;
fig. 13 is a flowchart of a method for obtaining a fused image based on the target image according to an embodiment of the present disclosure;
FIG. 14 is a flowchart of a method for determining a target image in a plurality of images in an image group according to an embodiment of the present disclosure;
FIG. 15 is a flowchart of a method for determining a target region of each of a plurality of images in the same image group according to an embodiment of the present disclosure;
fig. 16 is a schematic diagram illustrating a photographing effect of four images in an image group according to an embodiment of the present application;
fig. 17 is a schematic diagram illustrating a principle of obtaining a photographed image based on the processing method according to the embodiment of the present application.
Detailed Description
Embodiments of the present application will now be described more fully hereinafter with reference to the accompanying drawings, in which embodiments of the application are shown, and in which it is to be understood that the embodiments described are merely illustrative of some, but not all, of the embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
As described in the background art, in the existing electronic device, due to the influence of the reflected polarized light, the image quality of the obtained image in the corresponding specular reflection area (e.g. the surface of a smooth object such as a glass window, a spectacle lens, and a desktop) is poor.
In order to solve this problem, one way is to mount an external manual polarizer on the electronic device. The manual polarizer is arranged at the front end of the camera module, and the manual polarizer is manually and carefully rotated, so that the reflected light is minimized or even disappears, and a photo without specular reflection light can be shot. If a blue sky is photographed, the sky will appear bluer and darker. The manual polarizer not only filters out polarized light, but also filters out the unpolarized light in the same direction as the vibration direction of the polarized light. Therefore, after using the manual polarizer, the exposure amount is generally increased by more than one level. The disadvantages of this approach include: the manual polarizer is inconvenient to carry, reflected polarized light can be successfully offset only by adjusting the angle of the manual polarizer during use, high shooting skill is required, and operation is inconvenient.
Another way is to integrate the programmed polarizer inside the electronic device. The program-controlled polarizer can automatically adjust the angle in the shooting process to obtain the best reflection pressing effect and the best picture. When a shooting command is sent, the program-controlled polarizer continuously shoots a plurality of pictures of the same scene through the program-controlled rotating polarizer, and the brightness of the plurality of pictures is analyzed through an algorithm, so that the best picture with the minimum specular reflection light is obtained. The disadvantages of this approach include: the program-controlled polarizer can not photograph at the same time in different polarization states, has long photographing time and large time delay, is only suitable for photographing static scenes, and can hardly be used for dynamic scenes.
In order to solve the above problem, an embodiment of the present application provides an electronic device and a processing method, in which multiple images can be simultaneously obtained at the same time, each image corresponds to a polarization unit with the same polarization direction, different images correspond to polarization units with different polarization directions, and a photographed image with minimum specular reflection light is obtained based on the multiple images.
In order to make the aforementioned objects, features and advantages of the present application more comprehensible, the present application is described in further detail with reference to the accompanying drawings and the detailed description.
Referring to fig. 1 to fig. 3, fig. 1 is a schematic structural diagram of a first camera module according to an embodiment of the present disclosure, fig. 2 is a schematic structural diagram of an image sensor in the first camera module shown in fig. 1, and fig. 3 is a schematic structural diagram of a polarizer in the first camera module shown in fig. 1.
The electronic device according to the embodiment of the present application includes a first camera module 10 shown in fig. 1. The first camera module 10 includes: an image sensor 11 and a polarizer 12. The polarization device 12 is located on a photosensitive path of the image sensor 11.
As shown in fig. 2, the photosensitive array of the image sensor 11 includes a plurality of pixel groups 21, and the pixel groups 21 include a plurality of pixel units 22; the pixel cells 22 of each pixel group 21 sense the same color of light; the pixel cells of the adjacent pixel group 21 sense different colors of light.
For convenience of image data processing, the number of pixel units 22 in each of the pixel groups 21 is set to be the same. In the mode shown in fig. 2, only four pixel groups 21 are shown, and it is obvious that the number of pixel groups 21 and the number of pixel units 22 in a pixel group 21 can be set based on requirements, and is not limited to the mode shown in fig. 2, and any number of pixel groups 21 can be set based on requirements, and any number of pixel units 22 in the pixel group 21 can be set based on requirements.
The image sensor 11 includes three pixel units 22 for sensing different colors of light respectively. The three kinds of pixel units 22 include: a red pixel unit R for sensing red light; a green pixel cell G for sensing green light; and the blue pixel unit B is used for sensing blue light B.
A plurality of pixel groups 21 are arranged in an array, and the image sensor 11 includes a plurality of repeating units as shown in fig. 2.
As shown in fig. 3, the polarization device 12 includes a plurality of polarization groups 31, each polarization group 31 having a plurality of polarization units 32 with different polarization directions; the plurality of pixel groups 21 correspond to the plurality of polarization groups 31 one to one; the number of the pixel units 22 is the same as that of the polarization units 32, and the pixel units are arranged in a one-to-one correspondence.
Referring to fig. 4, fig. 4 is a schematic diagram illustrating the operation principle of the polarizer. As shown in the left drawing of fig. 4, when incident light having a plurality of polarization directions is incident to the polarizer, polarized light having the same polarization direction as that of the polarizer can pass through the polarizer.
As shown in the middle diagram of fig. 4, when the polarization directions of the two polarizers are perpendicular, when the incident light having a plurality of polarization directions passes through the first polarizer, the polarized light having the same polarization direction as the first polarizer can pass through the first polarizer, and at this time, the polarized light passing through the first polarizer is perpendicular to the polarization direction of the second polarizer, and will be blocked by the second polarizer.
As shown in the right diagram of fig. 4, when the polarization directions of the two polarizers are parallel, the polarized light perpendicular to the polarization direction of the polarizers in the incident light with multiple polarization directions can pass through the two polarizers, and the polarizations of other polarization directions cannot pass through the polarizers.
If the pixel group 21 has N pixel units 22, the corresponding polarization group 31 has N polarization units 32. For any one of the pixel groups 21, the N pixel units 22 in the pixel group 21 are sequentially set to be the 1 st pixel unit to the nth pixel unit, and each pixel unit corresponds to the polarization unit 32 with different polarization directions.
When all the pixel units 22 are subjected to photosensitive imaging at the same time, the ith pixel unit in each pixel group 21 can form an image sensing light rays in the same polarization direction, and all the pixel units 22 can form N images corresponding to different polarization directions. In the embodiment of the present application, each polarization unit 32 is equivalent to a micro polarizer. The image with the minimum brightness of the reflection area (i.e. the target area in the following) can be determined in the N images based on the working principle of the polarizer, so that the problem of poor image quality of the reflection area due to the reflection polarized light is solved.
Referring to fig. 5 and fig. 6, fig. 5 is a schematic structural diagram of another first camera module provided in this embodiment of the present application, and fig. 6 is a schematic structural diagram of a lens assembly in the first camera shown in fig. 5, based on the above embodiment, the first camera module shown in fig. 5 further includes: a lens assembly 13; wherein the lens assembly 13 comprises a plurality of micro lenses 41, and the micro lenses 41 are arranged opposite to at least one pixel unit 22; wherein the polarization unit 32 is located between the corresponding pixel unit 22 and the microlens 41. By providing the microlens 41, the light incident on the pixel unit 22 can be adjusted, thereby improving the imaging quality.
As shown in fig. 6, all the pixel units 22 in the same pixel group 21 may be arranged to correspond to the same microlens 41, and different pixel groups 21 may correspond to different microlenses 41, in this case, the microlenses 41 and the pixel groups 21 are arranged in a one-to-one correspondence, so that different images correspond to the same microlens to perform the same light modulation function.
In other embodiments, the microlenses 41 and the pixel units 22 may be provided in a one-to-one correspondence, or the microlenses 41 may be provided in correspondence with a plurality of arbitrary pixel units 22.
As shown in fig. 7, fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application, where the electronic device further includes a processor 14, and the processor 14 is electrically connected to the first camera module 10. The processor 14 is configured to obtain a sensing parameter of each pixel unit 22 in the image sensor 11, obtain a plurality of images based on the number of the polarization units 32 of the polarization group 31, and determine a target image from the plurality of images. Wherein the same image corresponds to the polarization unit with the same polarization direction; and the polarization directions of the polarization units corresponding to the target images are the same.
As described above, when the pixel group 21 is set to have N pixel units 22, each polarization group 31 corresponds to N different polarization units 32, and the processor 14 can obtain N images based on the number of polarization units 32 of the polarization group 31, wherein the same image corresponds to the polarization unit 32 with the same polarization direction, and different images correspond to the polarization units 32 with different polarization directions.
The electronic device can obtain the sensing parameters of all the pixel units 22 in all the image sensors 11 at the same time at one time, so as to obtain a plurality of images corresponding to the time.
In the embodiment of the present application, the electronic device can obtain multiple images through the sensing parameter of each pixel unit 22 in the image sensor 11. The processor 14 is capable of determining a target image and at least one fused image from the plurality of images, determining a photographed image based on the target image and/or the fused image, and storing the photographed image. Specifically, the processor 14 can take the target image as the photographed image; or, fusing the target image and at least one fused image to obtain the photographed image; or fusing a plurality of fused images to obtain the photographed image. The fused image corresponds to the target image at different time points, and the fused image corresponds to the same polarization unit 32 as the target image. And the brightness of the photographed image can be improved by fusing and processing a plurality of images to obtain the photographed image.
The target image may be determined by a plurality of images obtained at a time, specifically, the processor 14 obtains a plurality of images corresponding to different polarization directions corresponding to the time based on the sensing parameters of all the pixel units 22 in the image sensor 11 at the time, for example, if the pixel group 21 has N pixel units 22, N images can be obtained based on the sensing parameters at the time, and the processor 14 can determine one image as the target image in the N images. At this time, after the photographing program is started and the photographing instruction is obtained, the photographing instruction is executed to obtain a plurality of images.
The target image may also be determined by a plurality of images corresponding to a plurality of different time instants, specifically, at each of the plurality of different time instants, the processor 14 obtains a plurality of images based on all the pixel units 22 in the image sensor 11, and determines the target image based on the plurality of images corresponding to each of the plurality of time instants, for example, if the pixel group 21 has N pixel units 22, each time instant can obtain N images, and the processor 14 can determine one image as the target image in all the images. At this time, a plurality of images can be obtained based on all the pixel cells 22 in the image sensor 11 at a plurality of different times after the photographing process is started.
When the target image and the at least one fused image are fused to obtain the photographed image, the processor 14 is further configured to respond to a photographing instruction, obtain at least one fused image based on the target image, and fuse and process the target image and the at least one fused image to generate the photographed image; saving the photographed image; the polarization unit 32 corresponding to each of the at least one fused image and the polarization unit 32 corresponding to the target image are the same polarization unit 32, for example, the pixel group 21 has N pixel units 22, the same pixel group 21 corresponds to N polarization units 32 with different polarization directions, the N polarization units 32 are the 1 st polarization unit to the nth polarization unit in sequence, if the target image corresponds to the ith polarization unit, the fused image also corresponds to the ith polarization unit, and i is a positive integer not greater than N.
When a plurality of fused images are fused to obtain the photographed image, the processor 14 is further configured to respond to a photographing instruction to obtain a plurality of fused images based on the target image; fusing the plurality of fused images to generate a photographed image; and saving the photographed image. The polarization unit 32 corresponding to each of the multiple fused images and the polarization unit 32 corresponding to the target image are the same polarization unit 32.
In the embodiment of the present application, the polarizing unit 32 serves as a pixel-level micro polarizer. Theoretically, most of the specular reflection light that may exist during the photographing process can be successfully removed by the first camera module 10. But since the polarization unit 32 blocks a portion of the light, a longer exposure time is required to obtain the same brightness under the same conditions. In order to solve the problem, as shown in fig. 8, a second camera module 15 with non-polarized light filtering may be integrated into the electronic device at the same time, and if the second camera module 15 is used for complete framing, a target area and a target image with minimum brightness specular reflection light are determined by the first camera module 10, and a shot image without specular reflection light is determined by fusion of two camera modules.
Referring to fig. 8, fig. 8 is a schematic structural diagram of another electronic device provided in an embodiment of the present application, and based on any one of the foregoing manners, the electronic device shown in fig. 8 further includes: and a second camera module 15. The second camera module 15 is a camera module with an unbiased filtering function, and cannot filter out reflected polarized light. The processor 14 is further configured to obtain a fused image based on the target image and a standard image obtained by the second camera module 15 in response to a photographing instruction, fuse the fused image and the standard image to generate a photographed image, and store the photographed image. The polarization unit 32 corresponding to the fusion image and the polarization unit 32 corresponding to the target image are the same polarization unit 32. And fusing the fused image and the standard image to generate a photographed image, so that the brightness of the photographed image can be improved.
It should be noted that, when multiple images are obtained based on the sensing parameters of each pixel unit 22 in the image sensor 11, the exposure time or other exposure parameters for obtaining all the sensing parameters of the multiple images may be set based on requirements, which is not limited in the embodiment of the present application.
In this embodiment, the polarization unit 32 is equivalent to a micro polarizer corresponding to a pixel unit, and the electronic device detects the specular reflection light in real time through the processor 14 based on the pixel-level micro polarizer, that is, detects the target area in the target image in real time through the processor 14, and the polarization direction of the polarization unit corresponding to the target image and the polarization direction of the specular reflection light in the target area satisfy a vertical condition, so as to determine the polarization direction of the currently required optimal polarization unit.
Based on the foregoing embodiment, another embodiment of the present application further provides a processing method, which is used in the foregoing electronic device and can be executed by a processor in the electronic device, and the processing method is shown in fig. 9.
Referring to fig. 9, fig. 9 is a schematic flowchart of a processing method according to an embodiment of the present application, where the processing method includes:
step S11: at least one group of images is cached.
The image group comprises a plurality of images formed on the basis of induction parameters obtained by an image sensor of a first camera module at the same moment, wherein in the images, the same image is formed on the basis of polarized light in the same polarization direction, and different images are formed on the basis of polarized light in different polarization directions; different image groups correspond to different time instants.
In step S11, based on the light sensing parameters of all the pixel cells 22 in the image sensor 11 at the same time, a plurality of images in the next image group at that time are obtained. If the pixel group 21 has N pixel units 22, then there are N images in each image group.
Step S12: in the same image group, a target image is determined in the plurality of images.
In the embodiment of the application, the target image can be determined through a preview process, or the target image can be determined by the electronic device automatically in a photographing process, or the target image can be determined through a detection program hiding a background.
Referring to fig. 10, fig. 10 is a schematic flow chart of another processing method provided in the embodiment of the present application, and based on the manner shown in fig. 9, the processing method shown in fig. 10 further includes:
step S131: and responding to a photographing instruction, and obtaining at least one fused image based on the target image.
And the fused image corresponds to a target image at different moments, and the polarizing unit corresponding to each fused image in the at least one fused image is the same as the polarizing unit corresponding to the target image.
Step S141: and fusing the target image and the at least one fused image to generate a photographed image.
Step S151: and saving the photographed image.
Referring to fig. 11, fig. 11 is a schematic flowchart of another processing method provided in an embodiment of the present application, and based on the manner shown in fig. 9, the processing method shown in fig. 11 further includes:
step S132: and responding to a photographing instruction, and obtaining a plurality of fusion images based on the target image.
The fusion image and the target image correspond to different moments, and the polarization unit corresponding to each fusion image in the multiple fusion images and the polarization unit corresponding to the target image are the same.
Step S142: and fusing the plurality of fused images to generate a photographed image.
Step S152: and saving the photographed image.
Referring to fig. 12, fig. 12 is a schematic flowchart of another processing method provided in an embodiment of the present application, and based on the manner shown in fig. 9, the processing method shown in fig. 12 further includes:
step S133: and responding to a photographing instruction, and obtaining a fused image and a standard image obtained through a second camera module based on the target image.
The fusion image and the target image correspond to different moments, and the polarization unit corresponding to the fusion image and the polarization unit corresponding to the target image are the same.
Step S143: and fusing the fused image and the standard image to generate a photographed image.
Step S153: and saving the photographed image.
In the processing method according to the embodiment of the present application, a method for obtaining a fusion image based on the target image is shown in fig. 13.
Referring to fig. 13, fig. 13 is a flowchart of a method for obtaining a fused image based on the target image according to an embodiment of the present application, where the method includes:
step S21: determining the target image based on a plurality of images in the same image group.
Step S22: and determining an image with the same mark as the target image from the images of the other image groups as a fused image based on the mark of the target image.
In the mode shown in fig. 13, if the pixel group 21 is set to have N pixel units 22, the image group corresponding to each time has N images. And setting N images of the same image group as a 1 st image to an Nth image in sequence, and if the ith image is the target image, taking the ith image in the other image groups as a fusion image.
In the above step S12, a method of determining the target image among the plurality of images may be as shown in fig. 14.
Referring to fig. 14, fig. 14 is a flowchart of a method for determining a target image in a plurality of images in an image group according to an embodiment of the present application, where the method includes:
step S31: determining a target region for each of the plurality of images; the target area of each image is the same area.
The images in the same image group are obtained at the same time, and the images correspond to the polarizing units with different polarization directions. Each image corresponds to the same target area of the light reflection area in the environment.
Step S32: and determining a target image from the plurality of images, wherein the brightness of the target area of the target image is lower than the brightness of the target areas of other images in the plurality of images.
One of the images in the same image group is a target image, and the difference between the included angle between the polarization direction of the polarization unit corresponding to the target image and the polarization direction of the incident light of the corresponding target area and 90 degrees is the minimum, so that the brightness of the target area in the target image is the minimum.
The method for determining the target area of each of the plurality of images in the same image group is shown in fig. 15.
Referring to fig. 15, fig. 15 is a flowchart of a method for determining a target area of each image in multiple images of the same image group according to an embodiment of the present application, where the method includes:
step S41: and acquiring the brightness standard deviation of each pixel point based on the plurality of images.
The image includes a plurality of pixel points. In the image, each pixel point corresponds to one pixel unit, and different pixel points correspond to different pixel units.
For the same image group, the brightness of the same pixel point in different images is related to the polarization direction of the polarization unit corresponding to the images, and can be determined based on the photosensitive parameters of the photosensitive units.
Any pixel group is set to have N pixel units, and the N pixel units are the 1 st pixel unit to the Nth pixel unit in sequence. For the same image group, there are N images. For the same pixel point, the brightness of the pixel point corresponding to the ith pixel unit in each image is x in sequence 1 To x N Because the brightness of the same pixel point in different images is related to the polarization direction of the polarization unit corresponding to the images and can be determined based on the photosensitive parameters of the photosensitive unit, x 1 To x N Is a known constant. Based on x 1 To x N To determine the standard deviation of the brightness of the pixel point.
Step S42: determining the target region in the image based on the brightness standard deviation.
When polarized light is reflected due to no mirror reflection, the standard deviation of the brightness of each pixel point is zero or approximately 0. If the standard deviation of the brightness of the pixel point is larger than the set threshold, the pixel point can be determined to be located in the target area, namely the area with the mirror reflection is correspondingly located. The set threshold is related to a photosensitive parameter of the first camera, and the application is not limited in particular.
In the embodiment of the application, the target area can be determined based on a plurality of images in one image group, the target image can be determined based on the target area, and the fused image can be determined in a plurality of images in other respective image groups based on the mark of the target image. For example, if any pixel group is set to have N pixel cells corresponding to the polarization units having different polarization directions, there are N images for each image group. In the same pixel group, the N images are the 1 st image to the Nth image in sequence. When the target image is determined by using N images in one pixel group, the ith image in the pixel group is added as the target image, and the ith image in the other image group can be determined as the image corresponding to the same polarization unit as the target image based on the target image, namely the ith image in the other image group is the fused image.
The processing method described in the embodiments of the present application is further described below with an example of a 4-in-one pixel arrangement. The 4-in-one pixel arrangement means that N is 4, and the same pixel group has 4 pixel units. At any time, the pixel units corresponding to each group of polarizing units with the same polarization direction form an image correspondingly. When the indoor scene shown in fig. 16 is photographed, the polarization directions of the four groups of polarization units are different from the polarization direction of the reflected light from the mirror reflection area in the current photographed scene, such as the lower left corner in fig. 16
Referring to fig. 16, fig. 16 is a schematic diagram illustrating the photographing effect of four images in an image group according to an embodiment of the present application, where the lower circles of the four images in fig. 16 correspond to the polarization directions of the four polarization units 32 in fig. 3, respectively. Because the polarization directions of the four groups of polarization units are different, when the specific indoor scene is photographed, the included angle between the polarization direction of the reflected light corresponding to the mirror reflection area in the photographing scene corresponding to the target area shown by the dotted line square frame and the polarization direction of the mirror reflection light and the polarization direction of the polarization unit corresponding to the lower left corner image in fig. 16 is equal to or approximately equal to 90 degrees, namely, the vertical condition is met, so that the polarized light reflected by the mirror can be well eliminated, and the image is determined to be the target image and the target area.
When the electronic equipment is provided with the double camera modules, the mirror reflection area in the current shooting scene can be detected in real time through the first camera module. When photographing is carried out, when the area corresponding to the specular reflection area exceeds the area threshold, the specular reflection light can be automatically triggered and eliminated based on the processing method, and the method comprises the following two modes:
the first mode is that a target area and a target image are determined based on at least one image group acquired by the first camera module, and the target image is taken as a final photographed image. The method has the advantages of high photographing speed and no time delay. The disadvantage is low picture resolution. The resolution problem can be solved through a hyper-resolution algorithm, for example, the target image can be fused with at least one fused image, and at this time, the image group needs to be acquired through the first camera module at different times respectively so as to acquire the target image and the fused image. And the final photographed image is obtained through the fusion processing of the icon image and the fusion image or the fusion processing of a plurality of fusion images, so that the resolution of the final photographed image is improved.
The second way is to use the AI algorithm to fuse the target image determined in the first camera module and the standard image obtained by the second camera module, i.e. the mirror surface emission light can be removed without losing the image resolution. At this time, the principle of obtaining the final photographed image is shown in fig. 17.
Referring to fig. 17, fig. 17 is a schematic diagram illustrating a principle of obtaining a photographed image based on the processing method according to the embodiment of the present application, when a photographing program is started, at a plurality of different times of a time axis t, a first camera module sequentially obtains a plurality of image groups, and a second camera module sequentially obtains a plurality of standard images corresponding to the image groups one to one. Each image group has 4 images, and each image in the same image group respectively corresponds to a group of polarization units with different polarization directions.
And setting the moment for triggering the photographing key, wherein the first camera module obtains a third image group, and the second camera module obtains a third standard image. And the moment of triggering the photographing key corresponds to the moment of executing the photographing instruction. If the non-mirror surface light reflection area, namely the non-target area, is determined based on the image group, at this time, the brightness standard deviation of each pixel point is zero for the same image group. At this time, it is only necessary to take the third standard image as the final photographed image. If the target area exists, the target area and the target image are determined based on the 3 rd image group, and the 3 rd image in the 3 rd image group is assumed to be the target image. A fused image corresponding to the same polarization unit may be determined in the other image group based on the target image. And fusing the standard image obtained at the moment of executing the photographing instruction with the target image and/or the fused image to obtain a final photographed image.
It should be noted that the present application is not limited to 4-in-one, and may also be a 9-in-one pixel array or a 16-in-one pixel array, that is, N is 9 or 16, and the value of N may be set according to requirements, which is not specifically limited in the embodiments of the present application.
The processing method in the embodiment of the application performs photographing imaging based on the electronic device in the embodiment, and can realize real-time HDR (high dynamic range imaging).
The embodiments in the present description are described in a progressive manner, or in a parallel manner, or in a combination of a progressive manner and a parallel manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments can be referred to each other.
It is to be understood that in the description of the present application, the drawings and the description of the embodiments are to be regarded as illustrative in nature and not as restrictive. Like numerals refer to like structures throughout the description of the embodiments. Additionally, the figures may exaggerate the thicknesses of some layers, films, panels, regions, etc. for ease of understanding and ease of description. It will also be understood that when an element such as a layer, film, region, or substrate is referred to as being "on" another element, it can be directly on the other element or intervening elements may also be present. In addition, "on …" means that an element is positioned on or under another element, but does not essentially mean that it is positioned on the upper side of another element according to the direction of gravity.
The terms "upper," "lower," "top," "bottom," "inner," "outer," and the like refer to an orientation or positional relationship relative to an orientation or positional relationship shown in the drawings for ease of description and simplicity of description, but do not indicate or imply that the referenced device or element must have a particular orientation, be constructed and operated in a particular orientation, and thus should not be construed as limiting the present application. When a component is referred to as being "connected" to another component, it can be directly connected to the other component or intervening components may also be present.
It is further noted that, herein, relational terms such as first and second, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that an article or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such article or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in an article or device that comprises the element.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. An electronic device, comprising:
a first camera module;
wherein, first camera module includes:
an image sensor, a photosensitive array of the image sensor comprising a plurality of pixel groups, the pixel groups comprising a plurality of pixel cells; the pixel units of each pixel group sense the light with the same color; the pixel units of the adjacent pixel groups sense light with different colors;
the polarizing device is positioned on a photosensitive light path of the image sensor and comprises a plurality of polarizing groups, and each polarizing group is provided with a plurality of polarizing units with different polarization directions; the plurality of pixel groups correspond to the plurality of polarized light groups one to one; the number of the pixel units is the same as that of the polarizing units, and the pixel units are arranged in a one-to-one correspondence manner.
2. The electronic device of claim 1, the first camera module further comprising:
a lens assembly;
wherein the lens assembly comprises a plurality of microlenses, the microlenses being disposed opposite to at least one of the pixel cells; wherein the polarization unit is located between the corresponding pixel unit and the microlens.
3. The electronic device of claim 1, further comprising:
the processor is used for obtaining the induction parameters of each pixel unit in the image sensor;
obtaining a plurality of images based on the number of the polarization units of the polarization group, wherein the same image corresponds to the polarization units with the same polarization direction;
and determining a target image from the plurality of images, wherein the polarization directions of the polarization units corresponding to the target image are the same.
4. The electronic device of claim 3, the processor further configured to obtain at least one fused image based on the target image in response to a photographing instruction; the polarizing unit corresponding to each fused image in the at least one fused image and the polarizing unit corresponding to the target image are the same polarizing unit;
fusing the target image and the at least one fused image to generate a photographed image;
saving the photographed image;
or;
the processor is further used for responding to a photographing instruction and obtaining a plurality of fusion images based on the target image; the polarization unit corresponding to each fused image in the multiple fused images and the polarization unit corresponding to the target image are the same polarization unit;
fusing the plurality of fused images to generate a photographed image;
and saving the photographed image.
5. The electronic device of claim 3, further comprising:
a second camera module;
the processor is also used for responding to a photographing instruction, and obtaining a fused image and a standard image obtained through the second camera module based on the target image; the polarizing unit corresponding to the fused image and the polarizing unit corresponding to the target image are the same polarizing unit;
fusing the fused image and the standard image to generate a photographed image;
and saving the photographed image.
6. A method of processing, the method comprising:
caching at least one group of image groups, wherein the image groups comprise a plurality of images formed on the basis of induction parameters obtained by an image sensor of a first camera module at the same moment, the same image is formed on the basis of polarized light in the same polarization direction, and different images are formed on the basis of polarized light in different polarization directions;
and in the same image group, determining a target image in the plurality of images.
7. The processing method of claim 6, further comprising:
responding to a photographing instruction, and obtaining at least one fused image based on the target image; the polarizing unit corresponding to each fused image in the at least one fused image and the polarizing unit corresponding to the target image are the same polarizing unit;
fusing the target image and the at least one fused image to generate a photographed image;
saving the photographed image;
or;
responding to a photographing instruction, and obtaining a plurality of fusion images based on the target image; the polarization unit corresponding to each fused image in the multiple fused images and the polarization unit corresponding to the target image are the same polarization unit;
fusing the plurality of fused images to generate a photographed image;
saving the photographed image;
or;
responding to a photographing instruction, and obtaining a fused image and a standard image obtained through a second camera module based on the target image; the polarizing unit corresponding to the fused image and the polarizing unit corresponding to the target image are the same polarizing unit;
fusing the fused image and the standard image to generate a photographed image;
and saving the photographed image.
8. The processing method according to claim 7, the method of obtaining a fused image based on the target image comprising:
determining the target image based on a plurality of images in the same image group;
and determining an image with the same mark as the target image from the images of the other image groups as a fused image based on the mark of the target image.
9. The processing method of claim 6, determining a target image among the plurality of images, comprising:
determining a target region for each of the plurality of images; the target area of each image is the same area;
and determining a target image from the plurality of images, wherein the brightness of the target area of the target image is lower than that of the target areas of other images in the plurality of images.
10. The processing method according to claim 9, wherein determining the target region for each of the plurality of images for the same group of images comprises:
acquiring the standard brightness difference of each pixel point based on the plurality of images;
determining the target region in the image based on the brightness standard deviation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210736133.XA CN115118859A (en) | 2022-06-27 | 2022-06-27 | Electronic device and processing method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210736133.XA CN115118859A (en) | 2022-06-27 | 2022-06-27 | Electronic device and processing method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115118859A true CN115118859A (en) | 2022-09-27 |
Family
ID=83329305
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210736133.XA Pending CN115118859A (en) | 2022-06-27 | 2022-06-27 | Electronic device and processing method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115118859A (en) |
Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104836947A (en) * | 2015-05-06 | 2015-08-12 | 广东欧珀移动通信有限公司 | Image shooting method and apparatus |
| CN104869309A (en) * | 2015-05-15 | 2015-08-26 | 广东欧珀移动通信有限公司 | Shooting method and shooting apparatus |
| CN107507160A (en) * | 2017-08-22 | 2017-12-22 | 努比亚技术有限公司 | A kind of image interfusion method, terminal and computer-readable recording medium |
| CN109164624A (en) * | 2018-09-30 | 2019-01-08 | 京东方科技集团股份有限公司 | Display panel, display device and its image acquiring method |
| CN110248098A (en) * | 2019-06-28 | 2019-09-17 | Oppo广东移动通信有限公司 | Image processing method, device, storage medium and electronic equipment |
| CN110784633A (en) * | 2019-11-12 | 2020-02-11 | Oppo广东移动通信有限公司 | Image sensor, camera module, terminal and imaging method |
| CN110995968A (en) * | 2019-11-25 | 2020-04-10 | Oppo广东移动通信有限公司 | Image sensor, image pickup apparatus, electronic device, and imaging method |
| CN112785535A (en) * | 2020-12-30 | 2021-05-11 | 北京迈格威科技有限公司 | Method and device for acquiring night scene light rail image and handheld terminal |
| CN113516607A (en) * | 2021-04-23 | 2021-10-19 | Oppo广东移动通信有限公司 | Image processing method, image processing apparatus, electronic device, and storage medium |
| CN113992861A (en) * | 2020-07-27 | 2022-01-28 | 虹软科技股份有限公司 | Image processing method and image processing device |
-
2022
- 2022-06-27 CN CN202210736133.XA patent/CN115118859A/en active Pending
Patent Citations (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104836947A (en) * | 2015-05-06 | 2015-08-12 | 广东欧珀移动通信有限公司 | Image shooting method and apparatus |
| CN104869309A (en) * | 2015-05-15 | 2015-08-26 | 广东欧珀移动通信有限公司 | Shooting method and shooting apparatus |
| CN107507160A (en) * | 2017-08-22 | 2017-12-22 | 努比亚技术有限公司 | A kind of image interfusion method, terminal and computer-readable recording medium |
| CN109164624A (en) * | 2018-09-30 | 2019-01-08 | 京东方科技集团股份有限公司 | Display panel, display device and its image acquiring method |
| CN110248098A (en) * | 2019-06-28 | 2019-09-17 | Oppo广东移动通信有限公司 | Image processing method, device, storage medium and electronic equipment |
| CN110784633A (en) * | 2019-11-12 | 2020-02-11 | Oppo广东移动通信有限公司 | Image sensor, camera module, terminal and imaging method |
| CN110995968A (en) * | 2019-11-25 | 2020-04-10 | Oppo广东移动通信有限公司 | Image sensor, image pickup apparatus, electronic device, and imaging method |
| CN113992861A (en) * | 2020-07-27 | 2022-01-28 | 虹软科技股份有限公司 | Image processing method and image processing device |
| CN112785535A (en) * | 2020-12-30 | 2021-05-11 | 北京迈格威科技有限公司 | Method and device for acquiring night scene light rail image and handheld terminal |
| CN113516607A (en) * | 2021-04-23 | 2021-10-19 | Oppo广东移动通信有限公司 | Image processing method, image processing apparatus, electronic device, and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11245825B2 (en) | Dual-camera module, electronic device, and image acquisition method | |
| CN109164624B (en) | Display panel, display device and image acquisition method thereof | |
| WO2018196698A1 (en) | Photography processing method and device for camera module, and terminal apparatus | |
| US9848118B2 (en) | Phase detection autofocus using opposing filter masks | |
| WO2021000592A1 (en) | Image capturing device and method | |
| US8411195B2 (en) | Focus direction detection confidence system and method | |
| CN104717482A (en) | Multi-spectral multi-depth-of-field array shooting method and shooting camera | |
| WO2011095026A1 (en) | Method and system for photography | |
| JP2022552430A (en) | Imaging system with rotatable reflector | |
| US20060104627A1 (en) | Apparatus and method for excluding vignetting in a digital camera | |
| CN107995396A (en) | A kind of dual camera module and terminal | |
| Ben-Ezra | High resolution large format tile-scan camera: Design, calibration, and extended depth of field | |
| JP2017228983A (en) | Image processing apparatus, imaging device, image processing method | |
| CN111123386A (en) | Polarization imaging detection device and detection method based on dynamic vision sensor | |
| US9560279B2 (en) | Camera device and projector device having protective lens | |
| US8885086B2 (en) | Camera device and projector device having protective lens | |
| CN115118859A (en) | Electronic device and processing method | |
| US20180006066A1 (en) | Control apparatus, image pickup apparatus, and image pickup control method | |
| KR20150134019A (en) | Picture quality enhancement apparatus, digital photographing apparatus with the same and picture quality enhancement method thereof | |
| KR101592407B1 (en) | Picture quality enhancement apparatus, digital photographing apparatus with the same and picture quality enhancement method thereof | |
| JP6672049B2 (en) | Imaging device, control device, control method, control program, and recording medium | |
| CN103312965A (en) | photography apparatus and a photography method | |
| CN112104796B (en) | Image processing method and device, electronic equipment and computer readable storage medium | |
| KR101613433B1 (en) | Apparatus and method for improving picture quality of digital photographing device | |
| CN107370963B (en) | Image processing method, image processing device and electronic equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |