CN114866699A - Image processing method and device, computer readable storage medium and electronic device - Google Patents
Image processing method and device, computer readable storage medium and electronic device Download PDFInfo
- Publication number
- CN114866699A CN114866699A CN202210562945.7A CN202210562945A CN114866699A CN 114866699 A CN114866699 A CN 114866699A CN 202210562945 A CN202210562945 A CN 202210562945A CN 114866699 A CN114866699 A CN 114866699A
- Authority
- CN
- China
- Prior art keywords
- image
- lens
- processed
- cover plate
- processing method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 37
- 230000001681 protective effect Effects 0.000 claims abstract description 80
- 230000005856 abnormality Effects 0.000 claims description 20
- 238000005457 optimization Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 3
- 238000000034 method Methods 0.000 abstract description 32
- 238000010586 diagram Methods 0.000 description 21
- 230000008569 process Effects 0.000 description 21
- 230000002159 abnormal effect Effects 0.000 description 10
- 230000006870 function Effects 0.000 description 6
- 230000004044 response Effects 0.000 description 4
- 239000000463 material Substances 0.000 description 3
- 239000000428 dust Substances 0.000 description 2
- 230000000670 limiting effect Effects 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000001133 acceleration Effects 0.000 description 1
- 230000009471 action Effects 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 210000000988 bone and bone Anatomy 0.000 description 1
- 230000015556 catabolic process Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 238000006731 degradation reaction Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000036961 partial effect Effects 0.000 description 1
- 230000002829 reductive effect Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 239000011347 resin Substances 0.000 description 1
- 229920005989 resin Polymers 0.000 description 1
- 230000002441 reversible effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 210000004243 sweat Anatomy 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/67—Focus control based on electronic image sensor signals
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
- G06V10/761—Proximity, similarity or dissimilarity measures
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/62—Control of parameters via user interfaces
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/80—Camera processing pipelines; Components thereof
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20212—Image combination
- G06T2207/20221—Image fusion; Image merging
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- Artificial Intelligence (AREA)
- Computing Systems (AREA)
- Databases & Information Systems (AREA)
- Evolutionary Computation (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Software Systems (AREA)
- Studio Devices (AREA)
Abstract
The disclosure provides an image processing method, an image processing device, a computer readable storage medium and an electronic device, and relates to the technical field of image processing. The image processing method comprises the following steps: acquiring a reference image, wherein the reference image is obtained by shooting based on a first focusing distance, and the first focusing distance is determined according to the distance between a protective cover plate of a first lens and the first lens; acquiring an image to be processed shot by the first lens based on a second focusing distance, wherein the second focusing distance is different from the first focusing distance; and optimizing the image to be processed by using the reference image so as to eliminate or reduce the influence of the abnormity of the protective cover plate on the image shot by the first lens. The method and the device can eliminate or reduce the influence of the abnormity of the protective cover plate of the lens on the image, thereby improving the image quality.
Description
Technical Field
The present disclosure relates to the field of image processing technologies, and in particular, to an image processing method, an image processing apparatus, a computer-readable storage medium, and an electronic device.
Background
As an important carrier of information transmission, images are applied in a wider range of scenes and users have higher requirements on image quality. With the popularization of terminal equipment such as mobile phones and tablet computers, people can shoot at any time and any place.
For a terminal device, in order to protect a lens, a protective cover is generally provided. The abnormality of the protective cover may affect the captured image, for example, if the protective cover is soiled, the image quality may be degraded.
Disclosure of Invention
The present disclosure provides an image processing method, an image processing apparatus, a computer-readable storage medium, and an electronic device, thereby overcoming, at least to some extent, a problem of image quality degradation due to abnormality of a protective cover.
According to a first aspect of the present disclosure, there is provided an image processing method including: acquiring a reference image, wherein the reference image is obtained by shooting based on a first focusing distance, and the first focusing distance is determined according to the distance between a protective cover plate of a first lens and the first lens; acquiring an image to be processed shot by the first lens based on a second focusing distance, wherein the second focusing distance is different from the first focusing distance; and optimizing the image to be processed by using the reference image so as to eliminate or reduce the influence of the abnormity of the protective cover plate on the image shot by the first lens.
According to a second aspect of the present disclosure, there is provided an image processing apparatus comprising: the first image acquisition module is used for acquiring a reference image, the reference image is obtained by shooting based on a first focusing distance, and the first focusing distance is determined according to the distance between a protective cover plate of the first lens and the first lens; the second image acquisition module is used for acquiring an image to be processed shot by the first lens based on a second focusing distance, and the second focusing distance is different from the first focusing distance; and the image optimization module is used for optimizing the image to be processed by utilizing the reference image so as to eliminate or reduce the influence of the abnormity of the protective cover plate on the image shot by the first lens.
According to a third aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the image processing method described above.
According to a fourth aspect of the present disclosure, there is provided an electronic device comprising a processor; a memory for storing one or more programs which, when executed by the processor, cause the processor to implement the image processing method described above.
In some embodiments of the present disclosure, first, on one hand, a reference image captured based on a first focus distance determined according to a distance from a protective cover of a first lens to the first lens is obtained, that is, the reference image is an image obtained by performing focus capture on the protective cover, and on the other hand, an image to be processed captured by the first lens based on a second focus distance is obtained. And then, optimizing the image to be processed by using the reference image so as to eliminate or reduce the influence of the abnormity of the protective cover plate on the image shot by the first lens and further improve the quality of the image output by the first lens. According to the scheme, the effect of improving the image quality can be achieved without adding extra hardware, the universality of the scheme is high, and the scheme can be applied to various types of terminal equipment.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings in the following description are merely exemplary of the disclosure, and that other drawings may be derived from those drawings by one of ordinary skill in the art without the exercise of inventive faculty. In the drawings:
fig. 1 shows a schematic diagram of an exemplary architecture of a terminal device of some embodiments of the present disclosure;
FIG. 2 shows a schematic diagram of an exemplary architecture of a terminal device of further embodiments of the present disclosure;
FIG. 3 illustrates a schematic diagram of the distinction of a normal protective cover plate from an abnormal protective cover plate of an embodiment of the present disclosure;
FIG. 4 schematically illustrates a flow chart of an image processing method according to an exemplary embodiment of the present disclosure;
FIG. 5 illustrates a schematic diagram of acquiring a reference image with a first lens according to some embodiments of the present disclosure;
FIG. 6 is a schematic diagram illustrating the acquisition of a reference image using a second lens according to further embodiments of the present disclosure;
FIG. 7 is a schematic diagram illustrating an optimized mode of opening a protective cover exception in response to a user operation according to an embodiment of the disclosure;
fig. 8 is a schematic diagram showing a terminal device of the embodiment of the present disclosure determining whether to execute an image processing procedure of the present disclosure using a history image;
FIG. 9 schematically illustrates a flow chart of a process of an image processing method of some embodiments of the present disclosure;
FIG. 10 schematically illustrates a flow chart of a process of an image processing method of further embodiments of the disclosure;
FIG. 11 is a comparison diagram before and after an image to be processed is optimized by applying the image processing method according to the embodiment of the disclosure;
fig. 12 schematically shows a block diagram of an image processing apparatus according to an exemplary embodiment of the present disclosure;
fig. 13 schematically shows a block diagram of an image processing apparatus according to another exemplary embodiment of the present disclosure;
fig. 14 schematically shows a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.
Detailed Description
Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and the like. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.
Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.
The flow charts shown in the drawings are merely illustrative and do not necessarily include all of the steps. For example, some steps may be decomposed, and some steps may be combined or partially combined, so that the actual execution sequence may be changed according to the actual situation. In addition, all of the following terms "first" and "second" are used for distinguishing purposes only and should not be construed as limiting the present disclosure.
The lens of the terminal device is usually equipped with a protective cover plate to protect and prevent dust. The material of the protective cover plate in the embodiments of the present disclosure is not limited, and may be glass or another material such as resin. In addition, the protective cover is often transparent, however, the protective cover of the present disclosure may also be a cover that is transparent at the time of shooting and is consistent with the color of the terminal device housing at other times, and may be made of a material that becomes transparent after being powered on, which is not limited by the present disclosure.
On one hand, because the protective cover plate is exposed outside, the surface of the protective cover plate exposed outside is easy to be polluted, and the pollution can be caused by sweat stain of hands, or can be caused by foreign matters which are infected when the protective cover plate is contacted with other objects, or can be dust, rainwater and the like. Due to the existence of dirt, when the lens is used for shooting, marks corresponding to the dirt appear in an image, and the image quality is low.
On the other hand, the terminal equipment is inevitably bumped in the using process, so that scratches can be formed on the protective cover plate. Similarly, when the lens is used for shooting, marks corresponding to scratches appear in the image, and the quality of the image is also affected.
In the exemplary embodiment of the present disclosure, the condition of dirt, partial shielding, scratch, even breakage, etc. can be considered as the protection cover plate abnormality. It is understood that the abnormality of the protective cover is relative to the normality of the protective cover that does not affect the captured image.
In order to at least solve the influence of the abnormal protection cover plate on the shot image, the embodiment of the disclosure provides a new image processing scheme.
The image processing scheme of the embodiments of the present disclosure may be implemented by a terminal device, that is, the terminal device may perform the respective steps of the image processing method of the present disclosure, and the image processing apparatus described below may be configured in the terminal device. The terminal device may include, but is not limited to, a smart phone, a tablet computer, a dedicated camera, a notebook computer, a recording instrument, a personal computer, a projector, etc., and the present disclosure does not limit the type of the terminal device.
However, the image processing scheme of the present disclosure may also be implemented by a server, such as a terminal device sending the acquired image to the server, and the server performing the process of image optimization. A server may also be referred to as a server, a service platform, a processing platform, etc.
The following description will be given taking an example in which the terminal device implements the image processing scheme according to the embodiment of the present disclosure.
Fig. 1 shows a schematic diagram of an exemplary architecture of a terminal device of some embodiments of the present disclosure. Referring to fig. 1, the terminal device 1 includes at least a processor 10, a first lens 11, and a protective cover 12 corresponding to the first lens 11. In the example where the terminal device 1 is a smartphone, the first lens 11 may be a front camera or a rear camera of the smartphone.
Firstly, on the one hand, a reference image can be obtained by the first lens 11 based on a first focus distance determined by the distance of the first lens 11 from the protective cover 12; on the other hand, the image to be processed may also be obtained by shooting through the first lens 11 based on a second focus distance, which is different from the first focus distance. In this case, the first lens 11 may be a zoom camera.
It should be noted that the embodiment of the present disclosure does not limit the order in which the reference image and the image to be processed are captured by the first lens 11. That is, the first lens 11 may capture a reference image, transmit the reference image to the processor 10, capture an image to be processed, and transmit the image to be processed to the processor 10. Alternatively, the first lens 11 may capture an image to be processed first, transmit the image to be processed to the processor 10, capture a reference image, and transmit the reference image to the processor 10.
Next, the processor 10 may optimize the image to be processed using the reference image to eliminate or reduce the influence of the abnormality of the protective cover on the image captured by the first lens.
Fig. 2 shows a schematic diagram of an exemplary architecture of a terminal device of further embodiments of the present disclosure. Referring to fig. 2, the terminal device 2 includes at least a processor 20, a first lens 21, a second lens 22, and a protective cover 23. The protective covers of the first lens 21 and the second lens 22 are both protective covers 23, that is, the first lens 21 and the second lens 23 share the same protective cover 23. In the example where the terminal device 2 is a smartphone, the first lens 21 and the second lens 22 may both be front cameras of the smartphone or both be rear cameras of the smartphone.
Firstly, on the one hand, a reference image can be obtained by shooting through the second lens 22 based on a first focus distance, the first focus distance is determined according to the distance from the first lens 21 to the protective cover 23, and the distance from the first lens 21 to the protective cover 23 is the same as the distance from the second lens 22 to the protective cover 23; on the other hand, the image to be processed may also be obtained by shooting through the first lens 21 based on a second focus distance, which is different from the first focus distance. In this case, the second lens 22 may always be focused on the protective cover 23.
It should be noted that the present embodiment does not limit the sequence of the reference image captured by the second lens 22 and the to-be-processed image captured by the first lens 21, and may capture the reference image and the to-be-processed image simultaneously. The first lens 21 may transmit a photographed image to be processed to the processor 20, and the second lens 22 may transmit a photographed reference image to the processor 20.
Next, the processor 20 may optimize the image to be processed using the reference image to eliminate or reduce the influence of the abnormality of the protective cover on the image captured by the first lens.
Fig. 3 illustrates the difference between normal and abnormal protective covers, and by comparing the differences illustrated in fig. 3, it can be known that for an abnormal protective cover, an image shot by a corresponding lens may have a mark inconsistent with a real scene.
The following image processing method can be adopted to solve the problem of the influence of the abnormality of the protective cover plate on the quality of the shot image.
Fig. 4 schematically shows a flowchart of an image processing method of an exemplary embodiment of the present disclosure. Referring to fig. 4, the image processing method may include the steps of:
and S42, acquiring a reference image, wherein the reference image is obtained by shooting based on a first focusing distance, and the first focusing distance is determined according to the distance between a protective cover plate of the first lens and the first lens.
In an exemplary embodiment of the present disclosure, the reference image is photographed based on a first focus distance determined according to a distance of the protective cover plate of the first lens from the first lens, that is, the reference image is an image obtained by photographing the protective cover plate in focus.
Since stains or scratches are generally present on a surface (referred to as an outer surface) of the protective cover plate on a side away from the lens, more specifically, the reference image is an image obtained by taking a picture of the outer surface of the protective cover plate in focus.
According to some embodiments of the present disclosure, the reference image may be captured by the first lens based on the first focus distance. Referring to fig. 5, the first lens 51 may perform focus photographing on the protective cover 50 corresponding thereto to obtain a reference image.
In view of the fact that the first lens needs to capture an image to be processed in addition to the reference image, in this case, a problem that the camera is turned on slowly in normal shooting may result.
To address the problem of the user experience being affected by the camera turn-on slowing that may result, according to other embodiments of the present disclosure, the reference image may be captured by the second lens based on the first focus distance. The protection cover plate of the first lens and the protection cover plate of the second lens are the same.
Referring to fig. 6, the first lens 61 and the second lens 62 may share the same protective cover 60. Since the protective cover 60 is provided in an integrated form with respect to the first and second lenses 61 and 62, and the position of dirt or scratch is generally close to that of the two lenses, a reference image can be acquired using the second lens 62. That is, the second lens 62 may perform focus shooting on the protective cover 60 to obtain a reference image.
And S44, acquiring an image to be processed shot by the first lens based on a second focusing distance, wherein the second focusing distance is different from the first focusing distance.
The first lens may capture an image to be processed based on the second focus distance. In view of the different types of the terminal devices and the application scenarios, the second focus distance may be greater than the first focus distance, and the second focus distance may also be smaller than the first focus distance, which is not limited by the present disclosure.
In some embodiments, the image to be processed is an image obtained by normal shooting, for example, the image to be processed is an image obtained by normal shooting of a human image by a smartphone. For another example, the image to be processed is a current frame image when a video is captured. In addition, the image to be processed may also be a preview image.
It should be understood that the present disclosure does not limit the execution sequence of steps S42 and S44, that is, when the image processing method according to the embodiment of the present disclosure is implemented, the reference image may be acquired first and then the image to be processed is acquired, or the reference image may be acquired first and then the image to be processed is acquired.
And S46, optimizing the image to be processed by using the reference image so as to eliminate or reduce the influence of the abnormity of the protective cover plate on the image shot by the first lens.
First, the terminal device may determine an image area corresponding to the abnormality of the protective cover from the image to be processed using the reference image. Specifically, the terminal device may compare the similarity between the reference image and the image to be processed to determine the image area. For example, the image to be processed may be divided into a plurality of image blocks, feature point information (e.g., corner point information) of each image block may be extracted, similarity comparison may be performed between the feature point information and the feature point information of the reference image, and an image area corresponding to the abnormality of the protective cover plate may be determined from the image to be processed according to a result of the similarity comparison.
Next, the terminal device may optimize the image region, and fuse the optimized image region with the image to be processed to obtain a processed image.
Specifically, the terminal device may determine pixel information around the image area in the image to be processed, and estimate the pixel information of the image area by using the surrounding pixel information to obtain the optimized image area. For example, the pixel information of the image area may be reassigned by means of interpolation. And replacing the original image area in the image to be processed by the optimized image area to obtain the processed image.
In the embodiment where the image area corresponding to the protective cover plate in the image to be processed is the area after the image is blurred, for example, when a portrait is collected, the image area other than the portrait in the image to be processed is blurred. In this case, the image region can be optimized by the degree of blurring of the image region. Specifically, the blurring degree may be characterized by a circle of confusion diameter, which may be determined based on object distance, focal length, and aperture value. In the case of obtaining the degree of blurring, a pixel-level luminance subtraction operation may be performed based on the degree of blurring to obtain a processed image.
Furthermore, it can be understood that, in the case that the protective cover is normal, in order to save resources of the terminal device, the image processing method described above in the present disclosure need not be executed. That is, before the image processing method of the above-described steps S42 to S46 is performed, it may be determined whether the protective cover is normal or not.
According to some embodiments of the present disclosure, whether the protective cover is abnormal or not can be determined by a user. If the human eyes find that the protective cover plate is abnormal, the process of the method can be started in a manual setting mode.
Specifically, the terminal device may respond to a start operation of the optimization mode for the abnormality of the protective cover, and start the optimization mode for the abnormality of the protective cover.
As shown in fig. 7, a camera setting interface 72 can be entered in the shooting preview interface 71 to check the start button for the protection cover abnormality optimization. In addition, in addition to the scenario shown in fig. 7, a shortcut button for an optimization mode for protecting the cover abnormality may be provided in the photographing preview interface 71 to directly open the optimization mode in the photographing preview interface 71.
In the case that the optimization mode for protecting the cover plate from abnormality is turned on, the terminal device may acquire the reference image and the image to be processed in response to a photographing operation for the first lens, for example, in response to an operation of clicking a photographing button by a user, and optimize the image to be processed using the reference image, that is, perform the processes of the above-described steps S42 to S46.
According to other embodiments of the present disclosure, the terminal device may determine whether to execute the image processing method of the present disclosure from a line in combination with a plurality of history images.
Specifically, the terminal device may acquire a plurality of history images, for example, 3 history images, 5 history images, and the like, captured by the first lens in response to the capturing operation for the first lens. In the case that it is determined that the protective cover of the first lens is abnormal according to the plurality of history images, the terminal device may acquire the reference image and the image to be processed, and optimize the image to be processed by using the reference image, that is, perform the processes of step S42 to step S46.
For example, the terminal device may compare the similarity of a plurality of historical images, and if the shooting scenes of the historical images are different but image areas with similar positions and contents exist, the protective cover plate is considered to be abnormal.
Referring to fig. 8, the terminal device may determine whether the protective cover is normal or abnormal using a plurality of history images, and in the case where the protective cover is normal, the terminal device may perform a general photographing process; in case of an abnormal protection cover, the terminal device may apply the above-described image processing procedure of the present disclosure.
Referring to fig. 9, a process of an image processing method according to some embodiments of the present disclosure will be described.
In step S902, the terminal device performs focus shooting on the protective cover plate by using the first lens to obtain a reference image;
in step S904, the terminal device responds to the shooting operation of the user and normally shoots with the first lens to obtain an image to be processed;
in step S906, the terminal device optimizes the image to be processed by using the reference image to obtain an optimized image;
in step S908, the terminal device may output the optimized image. The step of outputting the optimized image may refer to directly saving the optimized image, or uploading the optimized image to a cloud or other devices, or taking the optimized image as an input to further execute a subsequent image processing process, which is not limited by the present disclosure.
Referring to fig. 10, a process of an image processing method according to further embodiments of the present disclosure will be described.
In step S1002, the terminal device responds to a shooting operation of a user and normally shoots with a first lens to obtain an image to be processed;
in step S1004, the terminal device performs focus shooting on the protective cover plate by using the second lens to obtain a reference image;
in step S1006, the terminal device optimizes the image to be processed by using the reference image to obtain an optimized image;
in step S1008, the terminal device may output the optimized image. The step of outputting the optimized image may refer to directly saving the optimized image, or uploading the optimized image to a cloud or other devices, or taking the optimized image as an input to further execute a subsequent image processing process, which is not limited in this disclosure.
Fig. 11 shows a comparison diagram before and after optimization of an image to be processed by applying the image processing method according to the embodiment of the present disclosure. Referring to fig. 11, after the image to be processed is subjected to the image processing process according to the exemplary embodiment of the present disclosure, the influence of the abnormality of the protective cover on the captured image may be eliminated, and the quality of the image may be improved.
It should be understood that, even if the above-mentioned image processing method cannot completely eliminate the influence of the abnormality of the protective cover on the shooting result, the influence of the abnormality of the protective cover on the shot image can be reduced, and the image quality can be improved to some extent.
It should be noted that although the various steps of the methods of the present disclosure are depicted in the drawings in a particular order, this does not require or imply that these steps must be performed in this particular order, or that all of the depicted steps must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions, etc.
Further, an image processing apparatus is also provided in the present exemplary embodiment.
Fig. 12 schematically shows a block diagram of an image processing apparatus of an exemplary embodiment of the present disclosure. Referring to fig. 12, the image processing apparatus 12 according to an exemplary embodiment of the present disclosure may include a first image acquisition module 121, a second image acquisition module 123, and an image optimization module 125.
Specifically, the first image obtaining module 121 may be configured to obtain a reference image, where the reference image is obtained by shooting based on a first focusing distance, and the first focusing distance is determined according to a distance from a protective cover plate of the first lens to the first lens; the second image obtaining module 123 may be configured to obtain an image to be processed, which is captured by the first lens based on a second focus distance, where the second focus distance is different from the first focus distance; the image optimization module 125 can be used to optimize the image to be processed by using the reference image to eliminate or reduce the influence of the abnormality of the protective cover plate on the image captured by the first lens.
According to an exemplary embodiment of the present disclosure, the first image acquisition module 121 may be configured to perform: a reference image captured by the first lens based on the first focus distance is acquired.
According to an exemplary embodiment of the present disclosure, the first image acquisition module 121 may be further configured to perform: acquiring a reference image shot by a second lens based on a first focusing distance; the protection cover plate of the first lens and the protection cover plate of the second lens are the same.
According to an exemplary embodiment of the present disclosure, referring to fig. 13, the image processing apparatus 13 may further include a process starting module 131, compared to the image processing apparatus 12.
In particular, the process initiation module 131 may be configured to perform: responding to the starting operation of the optimization mode aiming at the abnormity of the protection cover plate, and starting the optimization mode aiming at the abnormity of the protection cover plate; and responding to the shooting operation aiming at the first lens, acquiring a reference image and an image to be processed, and optimizing the image to be processed by using the reference image.
According to an exemplary embodiment of the present disclosure, the process starting module 131 may be configured to perform: responding to shooting operation aiming at the first lens, and acquiring a plurality of historical images shot by the first lens; and under the condition that the abnormality of the protective cover plate of the first lens is determined according to the plurality of historical images, acquiring a reference image and an image to be processed, and optimizing the image to be processed by using the reference image.
According to an example embodiment of the present disclosure, the image optimization module 125 may be configured to perform: determining an image area corresponding to the abnormity of the protective cover plate from the image to be processed by utilizing the reference image; and optimizing the image area, and fusing the optimized image area with the image to be processed to obtain a processed image.
According to an exemplary embodiment of the present disclosure, the process of optimizing the image region by the image optimization module 125 may be configured to perform: determining pixel information around an image area; and estimating the pixel information of the image area by using the pixel information around the image area to obtain the optimized image area.
Since each functional module of the image processing apparatus according to the embodiment of the present disclosure is the same as that in the embodiment of the method described above, it is not described herein again.
FIG. 14 shows a schematic diagram of an electronic device suitable for use in implementing exemplary embodiments of the present disclosure. The terminal device of the exemplary embodiment of the present disclosure may be configured as in fig. 14. It should be noted that the electronic device shown in fig. 14 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
The electronic device of the present disclosure includes at least a processor and a memory for storing one or more programs, which when executed by the processor, cause the processor to implement the image processing method of the exemplary embodiments of the present disclosure.
Specifically, as shown in fig. 14, the electronic device 140 may include: the mobile phone includes a processor 1410, an internal memory 1421, an external memory interface 1422, a Universal Serial Bus (USB) interface 1430, a charging management Module 1440, a power management Module 1441, a battery 1442, an antenna 1, an antenna 2, a mobile communication Module 1450, a wireless communication Module 1460, an audio Module 1470, a sensor Module 1480, a display 1490, a camera Module 1491, an indicator 1492, a motor 1493, a button 1494, a Subscriber Identity Module (SIM) card interface 1495, and the like. The sensor module 1480 may include a depth sensor, a pressure sensor, a gyroscope sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.
It is to be understood that the illustrated structure of the embodiments of the present disclosure does not constitute a specific limitation to the electronic device 140. In other embodiments of the present disclosure, the electronic device 140 may include more or fewer components than illustrated, or combine certain components, or split certain components, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.
Processor 1410 may include one or more processing units, such as: the Processor 1410 may include an Application Processor (AP), a modem Processor, a Graphics Processing Unit (GPU), an Image Signal Processor (ISP), a controller, a video codec, a Digital Signal Processor (DSP), a baseband Processor, and/or a Neural-Network Processor (NPU), and the like. The different processing units may be separate devices or may be integrated into one or more processors. Additionally, a memory may be disposed within processor 1410 for storing instructions and data.
The electronic device 140 may implement a shooting function through the ISP, the camera module 1491, the video codec, the GPU, the display 1490, the application processor, and the like. In some embodiments, the electronic device 140 may include 1 or N camera modules 1491, where N is a positive integer greater than 1, and if the electronic device 140 includes N lenses, one of the N lenses is a main lens.
The internal memory 1421 may be used to store computer-executable program code, which includes instructions. The internal memory 1421 may include a program storage area and a data storage area. The external memory interface 1422 may be used to connect an external memory card, such as a Micro SD card, to extend the memory capability of the electronic device 140.
The present disclosure also provides a computer-readable storage medium, which may be contained in the electronic device described in the above embodiments; or may exist separately without being assembled into the electronic device.
A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device.
A computer readable storage medium may transmit, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable storage medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
The computer readable storage medium carries one or more programs which, when executed by one of the electronic devices, cause the electronic device to implement the method as described in the embodiments of the present disclosure.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in the embodiments of the present disclosure may be implemented by software, or may be implemented by hardware, and the described units may also be disposed in a processor. Wherein the names of the elements do not in some way constitute a limitation on the elements themselves.
Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein may be implemented by software, or by software in combination with necessary hardware. Therefore, the technical solution according to the embodiments of the present disclosure may be embodied in the form of a software product, which may be stored in a non-volatile storage medium (which may be a CD-ROM, a usb disk, a removable hard disk, etc.) or on a network, and includes several instructions to enable a computing device (which may be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present disclosure.
Furthermore, the above-described drawings are merely schematic illustrations of processes involved in methods according to exemplary embodiments of the present disclosure, and are not intended to be limiting. It will be readily understood that the processes shown in the above figures are not intended to indicate or limit the chronological order of the processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, e.g., in multiple modules.
It should be noted that although in the above detailed description several modules or units of the device for action execution are mentioned, such a division is not mandatory. Indeed, the features and functionality of two or more modules or units described above may be embodied in one module or unit, according to embodiments of the present disclosure. Conversely, the features and functions of one module or unit described above may be further divided into embodiments by a plurality of modules or units.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is to be limited only by the terms of the appended claims.
Claims (10)
1. An image processing method, comprising:
acquiring a reference image, wherein the reference image is obtained by shooting based on a first focusing distance, and the first focusing distance is determined according to the distance between a protective cover plate of a first lens and the first lens;
acquiring an image to be processed shot by the first lens based on a second focusing distance, wherein the second focusing distance is different from the first focusing distance;
and optimizing the image to be processed by utilizing the reference image so as to eliminate or reduce the influence of the abnormity of the protective cover plate on the image shot by the first lens.
2. The image processing method according to claim 1, wherein acquiring the reference image comprises:
and acquiring a reference image shot by the first lens based on the first focusing distance.
3. The image processing method according to claim 1, wherein acquiring the reference image comprises:
acquiring a reference image shot by a second lens based on the first focusing distance;
the protection cover plate of the first lens and the protection cover plate of the second lens are the same.
4. The image processing method according to any one of claims 1 to 3, characterized in that the image processing method further comprises:
responding to the starting operation of the optimization mode aiming at the abnormity of the protection cover plate, and starting the optimization mode aiming at the abnormity of the protection cover plate;
and responding to shooting operation aiming at the first lens, acquiring the reference image and the image to be processed, and optimizing the image to be processed by utilizing the reference image.
5. The image processing method according to any one of claims 1 to 3, characterized in that the image processing method further comprises:
responding to shooting operation aiming at a first lens, and acquiring a plurality of historical images shot by the first lens;
and under the condition that the abnormality of the protective cover plate of the first lens is determined according to the plurality of historical images, acquiring the reference image and the image to be processed, and optimizing the image to be processed by using the reference image.
6. The image processing method according to claim 1, wherein optimizing the image to be processed using the reference image comprises:
determining an image area corresponding to the abnormity of the protective cover plate from the image to be processed by utilizing the reference image;
and optimizing the image area, and fusing the optimized image area and the image to be processed to obtain a processed image.
7. The image processing method of claim 6, wherein optimizing the image region comprises:
determining pixel information around the image area;
and estimating the pixel information of the image area by using the pixel information around the image area to obtain the optimized image area.
8. An image processing apparatus characterized by comprising:
the first image acquisition module is used for acquiring a reference image, wherein the reference image is obtained by shooting based on a first focusing distance, and the first focusing distance is determined according to the distance between a protective cover plate of a first lens and the first lens;
the second image acquisition module is used for acquiring an image to be processed shot by the first lens based on a second focusing distance, and the second focusing distance is different from the first focusing distance;
and the image optimization module is used for optimizing the image to be processed by utilizing the reference image so as to eliminate or reduce the influence of the abnormity of the protective cover plate on the image shot by the first lens.
9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out an image processing method according to any one of claims 1 to 7.
10. An electronic device, comprising:
a processor;
a memory for storing one or more programs which, when executed by the processor, cause the processor to implement the image processing method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210562945.7A CN114866699A (en) | 2022-05-23 | 2022-05-23 | Image processing method and device, computer readable storage medium and electronic device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210562945.7A CN114866699A (en) | 2022-05-23 | 2022-05-23 | Image processing method and device, computer readable storage medium and electronic device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN114866699A true CN114866699A (en) | 2022-08-05 |
Family
ID=82638915
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210562945.7A Pending CN114866699A (en) | 2022-05-23 | 2022-05-23 | Image processing method and device, computer readable storage medium and electronic device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114866699A (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104751500A (en) * | 2013-12-31 | 2015-07-01 | 厦门美图网科技有限公司 | Quick image inpainting method |
| CN109639896A (en) * | 2018-12-19 | 2019-04-16 | Oppo广东移动通信有限公司 | Block object detecting method, device, storage medium and mobile terminal |
| CN110876000A (en) * | 2018-08-31 | 2020-03-10 | 北京小米移动软件有限公司 | Camera module, image correction method and device, electronic equipment and storage medium |
| CN110971825A (en) * | 2019-12-06 | 2020-04-07 | 深圳前海达闼云端智能科技有限公司 | Image correction method, electronic device and storage medium |
| CN111405177A (en) * | 2020-03-09 | 2020-07-10 | Oppo广东移动通信有限公司 | Image processing method, terminal and computer readable storage medium |
| CN113132626A (en) * | 2021-03-26 | 2021-07-16 | 联想(北京)有限公司 | Image processing method and electronic equipment |
| CN114390206A (en) * | 2022-02-10 | 2022-04-22 | 维沃移动通信有限公司 | Shooting method and device and electronic equipment |
| CN114445315A (en) * | 2022-01-29 | 2022-05-06 | 维沃移动通信有限公司 | Image quality enhancement method and electronic device |
-
2022
- 2022-05-23 CN CN202210562945.7A patent/CN114866699A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104751500A (en) * | 2013-12-31 | 2015-07-01 | 厦门美图网科技有限公司 | Quick image inpainting method |
| CN110876000A (en) * | 2018-08-31 | 2020-03-10 | 北京小米移动软件有限公司 | Camera module, image correction method and device, electronic equipment and storage medium |
| CN109639896A (en) * | 2018-12-19 | 2019-04-16 | Oppo广东移动通信有限公司 | Block object detecting method, device, storage medium and mobile terminal |
| CN110971825A (en) * | 2019-12-06 | 2020-04-07 | 深圳前海达闼云端智能科技有限公司 | Image correction method, electronic device and storage medium |
| CN111405177A (en) * | 2020-03-09 | 2020-07-10 | Oppo广东移动通信有限公司 | Image processing method, terminal and computer readable storage medium |
| CN113132626A (en) * | 2021-03-26 | 2021-07-16 | 联想(北京)有限公司 | Image processing method and electronic equipment |
| CN114445315A (en) * | 2022-01-29 | 2022-05-06 | 维沃移动通信有限公司 | Image quality enhancement method and electronic device |
| CN114390206A (en) * | 2022-02-10 | 2022-04-22 | 维沃移动通信有限公司 | Shooting method and device and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| EP3079010B1 (en) | Photographing method for dual-camera device and dual-camera device | |
| US10178293B2 (en) | Controlling a camera using a voice command and image recognition | |
| US11233948B2 (en) | Exposure control method and device, and electronic device | |
| CN109040594B (en) | Photographing method and device | |
| WO2019015571A1 (en) | Method for zooming by switching between dual cameras, mobile terminal and storage device | |
| WO2011148212A1 (en) | Camera system and method for taking photographs that correspond to user preferences | |
| CN107690804B (en) | Image processing method and user terminal | |
| CN107704798B (en) | Image blurring method and device, computer readable storage medium and computer device | |
| CN111654637B (en) | Focusing method, focusing device and terminal equipment | |
| CN113810604B (en) | Document shooting method, electronic device and storage medium | |
| CN111756989A (en) | Method and device for controlling focusing of lens | |
| CN111970437B (en) | Text shooting method, wearable device and storage medium | |
| CN103402058A (en) | Shot image processing method and device | |
| CN114554096A (en) | Processing method and device and electronic equipment | |
| CN106982327B (en) | Image processing method and device | |
| CN107295261B (en) | Image defogging method and device, storage medium and mobile terminal | |
| CN112437235B (en) | Night scene picture generation method and device and mobile terminal | |
| CN113205011A (en) | Image mask determining method and device, storage medium and electronic equipment | |
| CN110855957B (en) | Image processing method and device, storage medium and electronic equipment | |
| CN114866699A (en) | Image processing method and device, computer readable storage medium and electronic device | |
| CN114143471B (en) | Image processing method, system, mobile terminal and computer readable storage medium | |
| JP2012235257A (en) | Photographing device | |
| CN111669572A (en) | Camera module detection method and device, medium and electronic equipment | |
| CN115134532A (en) | Image processing method, image processing device, storage medium and electronic equipment | |
| CN114565962A (en) | Face image processing method and device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |