CN116757940A - Method, device, equipment and medium for eliminating specular reflection area - Google Patents
Method, device, equipment and medium for eliminating specular reflection area Download PDFInfo
- Publication number
- CN116757940A CN116757940A CN202310512900.3A CN202310512900A CN116757940A CN 116757940 A CN116757940 A CN 116757940A CN 202310512900 A CN202310512900 A CN 202310512900A CN 116757940 A CN116757940 A CN 116757940A
- Authority
- CN
- China
- Prior art keywords
- reflection area
- specular reflection
- picture
- ambient light
- pictures
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 67
- 238000012549 training Methods 0.000 claims description 51
- 238000001514 detection method Methods 0.000 claims description 27
- 238000002372 labelling Methods 0.000 claims description 20
- 238000003860 storage Methods 0.000 claims description 19
- 238000004590 computer program Methods 0.000 claims description 18
- 230000004044 response Effects 0.000 claims description 4
- 230000008030 elimination Effects 0.000 abstract description 7
- 238000003379 elimination reaction Methods 0.000 abstract description 7
- 238000005516 engineering process Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 15
- 230000006870 function Effects 0.000 description 13
- 238000012545 processing Methods 0.000 description 11
- 238000004891 communication Methods 0.000 description 6
- 230000004927 fusion Effects 0.000 description 6
- 230000003287 optical effect Effects 0.000 description 6
- 230000008439 repair process Effects 0.000 description 6
- 230000000694 effects Effects 0.000 description 4
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 3
- 230000004075 alteration Effects 0.000 description 2
- 238000013459 approach Methods 0.000 description 2
- 239000011521 glass Substances 0.000 description 2
- 239000013307 optical fiber Substances 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 239000004065 semiconductor Substances 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012805 post-processing Methods 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 229920006395 saturated elastomer Polymers 0.000 description 1
- 230000001502 supplementing effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T5/00—Image enhancement or restoration
- G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Studio Devices (AREA)
Abstract
The invention provides a method, a device, equipment and a medium for eliminating a specular reflection area, which are used for solving the technical problems that the photo highlight reflection elimination quality is low and the requirements of certificate photographing cannot be completely met in the related technology, and the method comprises the following steps: acquiring at least two pictures containing target objects, wherein at least one picture contains a specular reflection area; identifying at least one specular reflection area corresponding to the picture; and in at least one other picture, obtaining a diffuse reflection area at a position corresponding to the specular reflection area, and repairing the specular reflection area by using the diffuse reflection area to obtain at least one picture which does not contain the specular reflection area.
Description
Technical Field
The invention relates to the technical fields of photo processing field, deep learning field and the like, in particular to a method, a device, equipment and a medium for eliminating a specular reflection area.
Background
In recent years, with popularization and digitalization of certificates, the quality requirements of photographs of certificates are gradually improved. However, when taking a photograph of a document, it is often found that there is a reflection problem in the photograph, resulting in the photograph being blurred or unrecognizable.
In the related art, one approach to solve the problem of light reflection is to use a "polarizer" and equip the camera and the flash with a polarizer filter, which can effectively reduce the problem of light reflection. The other is the function of eliminating reflection provided by the application program, and the essence is that the image is processed by a software processing algorithm to achieve the effect of eliminating reflection, such as the function of beautifying in photographing.
However, the above solutions have some drawbacks. First, the use of polarizers can reduce the reflection problem, but can also create other problems, such as non-uniform refraction, etc., that affect the quality of the photograph. Second, software processing algorithms to eliminate glints require extensive computation and processing of the image, and thus may reduce the speed of shooting and image quality. Meanwhile, because the requirements of the certificate photo are very accurate, the software processing result is unstable, errors are easy to occur, and the requirements of the certificate photo cannot be completely met.
Therefore, there is a need for further research and innovation in eliminating the problem of light reflection in credentials, providing a more reliable and efficient solution to meet the increasing demands of people.
Disclosure of Invention
The invention provides a method, a device, equipment and a medium for eliminating a specular reflection area, which are used for solving the technical problems that the high-light reflection elimination quality of a photo in the related technology is low and the requirements of photographing credentials cannot be completely met.
In a first aspect, an embodiment of the present invention provides a method for eliminating a specular reflection area, where the method includes:
acquiring at least two pictures containing a target object, wherein at least one picture contains a specular reflection area;
identifying a specular reflection area corresponding to at least one picture;
and in the other at least one picture, obtaining a diffuse reflection area at a position corresponding to the specular reflection area, and repairing the specular reflection area by using the diffuse reflection area to obtain at least one picture which does not contain the specular reflection area.
In a possible implementation manner, in the method provided by the embodiment of the present invention, at least two pictures including a target object are obtained, including:
and instructing the user to shoot at least two pictures with different shooting angles in a mode of adjusting the shooting angles.
In a possible implementation manner, in the method provided by the embodiment of the present invention, at least two pictures including a target object are obtained, including:
instructing a user to adjust the state of the flash lamp, and taking at least two pictures in different flash lamp states; or automatically adjusting the flash state and continuously and automatically taking at least two pictures in different flash states.
In a possible implementation manner, in the method provided by the embodiment of the present invention, before taking at least two pictures, the method further includes:
Determining a current ambient light state according to a current shooting environment;
and according to the ambient light state, instructing the user to take at least two pictures containing the target object.
In a possible implementation manner, in the method provided by the embodiment of the present invention, the ambient light state includes a strong ambient light state and a weak ambient light state, and determining the current ambient light state according to the current shooting environment includes:
when the light intensity of the ambient light is lower than a preset threshold value, determining that the ambient light state is a weak ambient light state;
and when the light intensity of the ambient light is higher than or equal to a preset threshold value, determining that the ambient light state is a strong ambient light state.
In a possible implementation manner, in the method provided by the embodiment of the present invention, according to an ambient light state, a user is instructed to take at least two pictures including a target object, including:
when the ambient light state is a weak ambient light state, at least two pictures with different shooting angles are shot in a mode of indicating a user to adjust the shooting angles; and/or
And when the ambient light state is a strong ambient light state, at least two pictures in different flash lamp states are shot.
In a possible implementation manner, in a method provided by an embodiment of the present invention, identifying a specular reflection area of at least one picture includes:
Selecting at least one picture as a target picture;
and inputting the target picture into a specular reflection area detection model to obtain a specular reflection area.
In a possible implementation manner, in the method provided by the embodiment of the present invention, at least one picture is selected as a target picture, including:
selecting at least one picture meeting preset conditions as a target picture; /or
And displaying the picture to the user, and determining the target picture according to the selection operation of the user.
In one possible implementation manner, in the method provided by the embodiment of the present invention, the specular reflection area detection model is generated by training in the following manner:
acquiring a plurality of training samples containing pictures and labeling results;
and comparing the specular reflection area output by each training sample with the labeling result of the training sample by taking each picture in the plurality of training samples as input, and generating a specular reflection area detection model based on the difference training of the specular reflection area output by each training sample and the labeling result of the training sample, wherein the labeling result is the specular reflection area labeled in the training sample in advance.
In one possible implementation manner, in the method provided by the embodiment of the present invention, the method further includes:
Displaying the specular reflection area to a user;
in response to a user operation, the specular reflection area is corrected.
In a possible implementation manner, in the method provided by the embodiment of the present invention, at least two pictures including a target object are obtained, including:
and identifying whether the target object is a card, and acquiring at least two pictures containing the target object when the target object is the card.
In a second aspect, an embodiment of the present invention provides an apparatus for eliminating a specular reflection area, including:
an acquisition unit, configured to acquire at least two pictures including a target object, where at least one picture includes a specular reflection area;
the identifying unit is used for identifying a specular reflection area corresponding to at least one picture;
and the restoration unit is used for acquiring the diffuse reflection area at the position corresponding to the specular reflection area in at least one other picture, and restoring the specular reflection area by utilizing the diffuse reflection area to obtain at least one picture which does not contain the specular reflection area.
In a possible implementation manner, in the device provided by the embodiment of the present invention, the obtaining unit is specifically configured to:
and instructing the user to shoot at least two pictures with different shooting angles in a mode of adjusting the shooting angles.
In a possible implementation manner, in the device provided by the embodiment of the present invention, the obtaining unit is specifically configured to:
instructing a user to adjust the state of the flash lamp, and taking at least two pictures in different flash lamp states; or automatically adjusting the flash state and continuously and automatically taking at least two pictures in different flash states.
In a possible implementation manner, in the device provided by the embodiment of the present invention, the obtaining unit is further configured to:
determining a current ambient light state according to a current shooting environment;
and according to the ambient light state, instructing the user to take at least two pictures containing the target object.
In a possible implementation manner, in the device provided by the embodiment of the present invention, the ambient light state includes a strong ambient light state and a weak ambient light state, and the obtaining unit is further configured to:
when the light intensity of the ambient light is lower than a preset threshold value, determining that the ambient light state is a weak ambient light state;
and when the light intensity of the ambient light is higher than or equal to a preset threshold value, determining that the ambient light state is a strong ambient light state.
In a possible implementation manner, in the device provided by the embodiment of the present invention, the obtaining unit is specifically configured to:
when the ambient light state is a weak ambient light state, at least two pictures with different shooting angles are shot in a mode of indicating a user to adjust the shooting angles; and/or
And when the ambient light state is a strong ambient light state, at least two pictures in different flash lamp states are shot.
In a possible implementation manner, in the device provided by the embodiment of the present invention, the identifying unit is specifically configured to:
selecting at least one picture as a target picture;
and inputting the target picture into a specular reflection area detection model to obtain a specular reflection area.
In a possible implementation manner, in the device provided by the embodiment of the present invention, the identifying unit is specifically configured to:
selecting at least one picture meeting preset conditions as a target picture; /or
And displaying the picture to the user, and determining the target picture according to the selection operation of the user.
In a possible implementation manner, in the device provided by the embodiment of the present invention, the identifying unit is specifically configured to train and generate the specular reflection area detection model by:
acquiring a plurality of training samples containing pictures and labeling results;
and comparing the specular reflection area output by each training sample with the labeling result of the training sample by taking each picture in the plurality of training samples as input, and generating a specular reflection area detection model based on the difference training of the specular reflection area output by each training sample and the labeling result of the training sample, wherein the labeling result is the specular reflection area labeled in the training sample in advance.
In a possible implementation manner, in the device provided by the embodiment of the present invention, the identifying unit is further configured to:
displaying the specular reflection area to a user;
in response to a user operation, the specular reflection area is corrected.
In a possible implementation manner, in the device provided by the embodiment of the present invention, the obtaining unit is further configured to:
and identifying whether the target object is a card, and acquiring at least two pictures containing the target object when the target object is the card.
In a third aspect, an embodiment of the present invention provides an electronic device, including: at least one processor, at least one memory and computer program instructions stored in the memory, which when executed by the processor implement the method as provided by the first aspect of the embodiments of the invention.
In a fourth aspect, embodiments of the present invention provide a computer readable storage medium having stored thereon computer program instructions which, when executed by a processor, implement a method as provided by the first aspect of embodiments of the present invention.
In the embodiment of the invention, at least two pictures containing a target object are firstly obtained, then a specular reflection area corresponding to at least one picture is identified, finally a diffuse reflection area corresponding to the specular reflection area is obtained in other at least one picture, and the diffuse reflection area is utilized to repair the specular reflection area, so that at least one picture which does not contain the specular reflection area is obtained. Compared with the prior art, the method solves the problems that the photo highlight reflection elimination quality is low and the requirement of photographing of credentials cannot be completely met, and can eliminate the specular reflection area by utilizing a plurality of pictures, so that the finally obtained pictures are clearer and the information is not lost, the requirement of photographing of credentials is met, and the user experience is improved.
Drawings
Fig. 1 is a schematic flow chart of a method for eliminating a specular reflection area according to an embodiment of the present invention;
fig. 2 is a schematic flow chart of a method for eliminating a specular reflection area according to an embodiment of the present invention;
FIG. 3 is a flowchart of a training method of a specular reflection area detection model according to an embodiment of the present invention;
fig. 4 is a schematic structural diagram of a specular reflection area detection model according to an embodiment of the present invention;
FIG. 5 is a schematic flow chart of another method for eliminating specular reflection area according to an embodiment of the present invention;
fig. 6 is a schematic view of photographing in a weak ambient light state according to an embodiment of the present invention;
fig. 7 is an example of a picture in a shooting manner according to an embodiment of the present invention;
fig. 8 is a schematic view of photographing in a strong ambient light state according to an embodiment of the present invention;
fig. 9 is a schematic illustration of another photographing mode according to an embodiment of the present invention;
fig. 10 is an interface schematic diagram of a picture preview according to an embodiment of the present invention;
FIG. 11 is a schematic diagram of an interface for mobile phone operation according to an embodiment of the present invention;
fig. 12 is an example of a picture after mirror area repair according to an embodiment of the present invention;
FIG. 13 is a schematic structural diagram of a device for eliminating specular reflection areas according to an embodiment of the present invention;
fig. 14 is a schematic structural diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention will be described in further detail below with reference to the accompanying drawings, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Some words appearing hereinafter are explained:
1. in the embodiment of the invention, the term "and/or" describes the association relation of the association objects, which means that three relations can exist, for example, a and/or B can be expressed as follows: a exists alone, A and B exist together, and B exists alone. The character "/" generally indicates that the context-dependent object is an "or" relationship.
2. The term "specular reflection" in embodiments of the present invention refers to the bright reflection of light at the surface of an object, typically occurring on a flat, smooth surface, such as a water surface, a glass surface, or the like. During shooting, the camera lens typically captures a specularly reflected portion of the object surface, which typically exhibits relatively strong brightness and contrast. In order to maintain the sense of reality and texture of a scene when a camera shoots, specular reflection in a photo needs to be processed, a light source and environmental conditions during shooting can be controlled by adjusting parameters such as exposure time, aperture size, iso and the like, and the intensity of the specular reflection can be enhanced or weakened through post-processing to achieve an ideal effect.
In recent years, with popularization and digitalization of certificates, the quality requirements of photographs of certificates are gradually improved. However, when taking a photograph of a document, it is often found that there is a reflection problem in the photograph, resulting in the photograph being blurred or unrecognizable.
In the related art, one approach to solve the problem of light reflection is to use a "polarizer" and equip the camera and the flash with a polarizer or filter, which can effectively reduce the problem of light reflection. The other is the function of eliminating reflection provided by the application program, and the essence is that the image is processed by a software processing algorithm to achieve the effect of eliminating reflection, such as the function of beautifying in photographing.
However, the above solutions have some drawbacks. First, the use of polarizers can reduce the reflection problem, but can also create other problems, such as non-uniform refraction, etc., that affect the quality of the photograph. Second, software processing algorithms to eliminate glints require extensive computation and processing of the image, and thus may reduce the speed of shooting and image quality. Meanwhile, because the requirements of the certificate photo are very accurate, the software processing result is unstable, errors are easy to occur, and the requirements of the certificate photo cannot be completely met.
Therefore, there is a need for further research and innovation in eliminating the problem of light reflection in credentials, providing a more reliable and efficient solution to meet the increasing demands of people.
The present invention provides a method for eliminating a specular reflection area, and the method, apparatus, device and medium for eliminating a specular reflection area provided by the present invention are described in more detail below with reference to the accompanying drawings and examples.
An embodiment of the present invention provides a method for eliminating a specular reflection area, as shown in fig. 1, including:
step S101, at least two pictures including the target object are acquired.
In specific implementation, at least two pictures are obtained, each picture contains a target object, and at least one picture contains a specular reflection area. Specular reflection refers to the bright reflection of light at the surface of an object, typically on a flat, smooth surface, such as a water surface, glass surface, or the like. During shooting, a camera lens typically captures a specularly reflected portion of the object surface, which typically exhibits relatively strong brightness and contrast, and portions other than the specularly reflected portion exhibit relatively weak brightness, which may be considered diffuse reflection regions. The target object may be any object that needs to be photographed and has a specular reflection portion on the surface, for example, credentials photo, portrait, etc., and when specifically obtained, the target object may instruct a user to photograph a plurality of pictures with different photographing angles by adjusting the photographing angles, so as to make the specular reflection area of each picture inconsistent, for example, obtain the pictures with different photographing angles by horizontal overturn, vertical overturn, or other forms of adjustment; the method can also instruct a user to adjust the state of the flash lamp or automatically adjust the state of the flash lamp to obtain various pictures in different states of the flash lamp, such as pictures in various states of the flash lamp, such as a flash lamp on state, a flash lamp off state, a flash lamp brightness value setting state and the like. Of course, the shooting of photographs at different angles or photographs of different flash lamp states can be determined by judging the state of the ambient light at the current shooting time, and specific shooting rules can be set according to requirements. Before the picture is obtained, whether the target object is a card or not can be detected, when the target object is the card, two pictures containing the target object are obtained, and the situation that objects with reflecting surfaces, such as the lamps of a car, the sun and the like, enter multiple photo shooting interfaces unnecessarily to cause complex operation is prevented. Through the identification card, the method for eliminating the specular reflection area can be only executed when the card is taken, such as a card object such as a certificate photograph, a business card, a bank card, an identity card and the like, so that the operation of a user is simple, and the experience is better.
Step S102, identifying a specular reflection area corresponding to at least one picture.
When the method is implemented, at least one picture is selected as a target picture from the obtained pictures when the specular reflection area is determined, then the target picture is input into a specular reflection area detection model, the specular reflection area is determined through the model, when the target picture is selected, the picture can be selected according to the active selection of a user, the picture meeting the preset condition can also be selected, such as the picture parameters (such as exposure, brightness, picture resolution and the like) are within a certain range, or the clear visible degree and the like of a target object in the picture, and the specific condition can be set according to the requirement.
The specular reflection area detection model is generated by training in the following way:
firstly, a plurality of training samples containing pictures and labeling results are obtained, then each picture in the plurality of training samples is used as input, the specular reflection area output by each training sample is compared with the labeling results of the training samples, a specular reflection area detection model is generated based on the difference training of the specular reflection area output by each training sample and the labeling results of the training samples, and the labeling results are specular reflection areas labeled in the training samples in advance.
After the specular reflection area is determined, the specular reflection area can be displayed to the user, the user can adjust the size and the position of the specular reflection area according to the interface observed by the user, and other positions which are displayed in the picture and are not clear or have strong light reflection can be set as the specular reflection area, so that the judgment of the specular reflection area is more accurate, and the requirements of the user are met.
Step S103, obtaining a diffuse reflection area at a position corresponding to the specular reflection area in at least one other picture, and repairing the specular reflection area by using the diffuse reflection area to obtain at least one picture which does not contain the specular reflection area.
In the specific implementation, a diffuse reflection area corresponding to the position of the specular reflection area in the target picture is obtained in at least one other picture, namely, a non-target picture, and then the diffuse reflection area is utilized to repair the specular reflection area, so that at least one picture which does not contain the specular reflection area is obtained.
The following describes in detail the method for eliminating the specular reflection area in the embodiment of the present invention with reference to fig. 2, and as shown in fig. 2, the method for eliminating the specular reflection area provided in the embodiment of the present invention may include the following specific steps:
Step S201, determining a current ambient light state according to a current shooting environment.
In specific implementation, the ambient light state includes a strong ambient light state and a weak ambient light state, when the light intensity of the ambient light is lower than a preset threshold value, the ambient light state is determined to be the weak ambient light state, and when the light intensity of the ambient light is higher than or equal to the preset threshold value, the ambient light state is determined to be the strong ambient light state. The preset threshold value can be set according to the requirement, the ambient light state can be determined through a sensor arranged on the shooting equipment of the user, the brightness of the preview picture can be judged, and the user can select the ambient light state.
Step S202, according to the ambient light state, the user is instructed to take at least two pictures containing the target object.
In specific implementation, when the ambient light state is a weak ambient light state, the user is instructed to adjust the shooting angle to shoot at least two pictures with different shooting angles, for example, pictures with different shooting angles are obtained through horizontal overturn or vertical overturn or other forms of adjustment. When the ambient light state is a strong ambient light state, at least two pictures in different flash lamp states are shot, namely at least two of the following types are obtained: pictures taken when the flash is on, pictures taken when the flash is off, and pictures taken when the flash is at different brightness.
Step S203, selecting at least one picture as a target picture.
In the implementation, the selection can be performed according to the active selection of the picture by the user, or the picture meeting the preset conditions can be selected, for example, the picture parameters (such as exposure, brightness, picture resolution, etc.) are within a certain range, or the clear visible degree of the target object in the picture can be set according to the requirement.
Step S204, inputting the target picture into a specular reflection area detection model to obtain a specular reflection area.
In the implementation, the target picture is input into the specular reflection area detection model, the specular reflection area corresponding to the target picture is determined, and in the step, the specular reflection area can be corrected through user operation, so that the area division is more accurate.
In step S205, in at least one other picture, a diffuse reflection area corresponding to the specular reflection area is obtained, and the diffuse reflection area is used to repair the specular reflection area, so as to obtain at least one picture that does not include the specular reflection area.
In the specific implementation, a diffuse reflection area corresponding to the position of the specular reflection area in the target picture is obtained in at least one other picture, namely, a non-target picture, and then the diffuse reflection area is utilized to repair the specular reflection area, so that at least one picture which does not contain the specular reflection area is obtained. During restoration, pictures of the same object at different angles are shot for multiple times by means of homography transformation to be registered, and then splicing and fusion of a specular reflection area in a target picture and diffuse reflection areas corresponding to other pictures are realized by means of a poisson fusion algorithm, so that highlight elimination is completed, and natural smoothness of images of fusion parts is ensured as much as possible.
Next, in conjunction with fig. 3, the training of the specular reflection area detection model in step S204 will be described in detail, and the training procedure of the specific specular reflection area detection model is as follows:
step S301, collecting training data.
In the specific implementation, the real pictures are collected by means of network searching and the like, so that the surface portraits, background textures and text information of objects in the pictures are required to be clearly visible, and the surfaces of the objects are not provided with specular reflection areas.
Step S302, training data is processed.
In specific implementation, a highlight effect is added to the acquired picture in a synthetic mode. Specifically, the original picture and the round highlight mask are directly added by adopting pixel-level addition to form a highlight image, the saturated pixel points after addition are required to be cut off, and the intensity of the highlight mask is gradually reduced from the center to the outside.
Step S303, training a specular reflection area detection model by using the training data.
In specific implementation, the training data is used to train a specular reflection area detection model, and the trained specular reflection area detection model is shown in fig. 4, is of a U-shaped network structure, and detects high light by means of structural similarity.
The following describes in detail a method for eliminating a specular reflection area in an embodiment of the present invention by taking two pictures as an example and taking an operation of a user to take a certificate with a mobile phone, and as shown in fig. 5, the method for eliminating a specular reflection area provided in the embodiment of the present invention may include the following specific steps:
Step S501 determines a current ambient light state according to the current shooting environment, and if the ambient light state is a weak ambient light state, step S502 is executed, and if the ambient light state is a strong ambient light state, step S503 is executed.
Step S502 instructs the user to take two pictures with different shooting angles in a manner of adjusting the shooting angle, and then step S504 is performed.
In specific implementation, during shooting, whether the target object is an object with a reflective film such as a card or not can be detected first, so that unnecessary two pictures are prevented from being shot, when the ambient light state is a weak ambient light state, namely, the ambient light is weak, the mobile phone flash is required to be always on for light supplementing, and high light is generated, the shooting mode is as shown in fig. 6, two card pictures with different angles are shot for the horizontal overturning device, in one example, the shot pictures are as shown in fig. 7, and the specular reflection areas on the card in the two pictures are not overlapped.
Step S503, two pictures in different flash states are taken, and then step S504 is performed.
In specific implementation, during shooting, whether the target object is an object with a reflective film such as a card or the like can be detected first, so that unnecessary two photos are avoided from being shot, when the ambient light state is a strong ambient light state, that is, when the ambient light is strong and a slice or point highlight is generated by the ambient light, the shooting mode is as shown in fig. 8, in order to automatically and continuously shoot two photos in different flash states by pressing a shutter next time, in one example, the shot pictures are as shown in fig. 9, wherein the flash is not started when the first photo is shot and is started when the second photo is shot, and the position of the mobile phone is kept the same as possible when the two shots are shot.
Step S504, selecting a picture as a target picture.
In the implementation, the selection can be performed according to the active selection of the picture by the user, or the picture meeting the preset conditions can be selected, for example, the picture parameters (such as exposure, brightness, picture resolution, etc.) are within a certain range, or the clear visible degree of the target object in the picture can be set according to the requirement.
Before selecting the target picture, the two pictures shot in step S502 or step S503 may be displayed to the user, as shown in fig. 10, where the two pictures shot are displayed in a preview area 1002 in a mobile phone 1001 of the user, and when displaying, a prompt may be output to the picture which does not meet the requirement of the shooting method, and prompt to re-shoot, where two pictures are displayed in a vertical arrangement as shown in fig. 10, and of course, may be displayed in a horizontal display form or any other form, which is not limited by the embodiment of the present invention.
In step S505, the target picture is input to the specular reflection area detection model, and the specular reflection area of the target picture is determined.
In the specific implementation, the target picture is input into the specular reflection area detection model to determine the specular reflection area of the target picture, and in this step, the specular reflection area may be corrected by a user operation, so that the area division is more accurate, as shown in fig. 11, in a picture 1102 displayed on a mobile phone 1101 of the user, the specular reflection area 1103 of the picture is detected by the specular reflection area detection model, and the user may modify the area by himself.
Step S506, repairing the specular reflection area of the target picture by using the diffuse reflection area corresponding to the position of the specular reflection area in the other picture.
In the specific implementation, the two pictures are registered by means of homography transformation, the splicing fusion of the specular reflection area in the target picture and the corresponding area of the other picture is realized by poisson fusion, the highlight elimination is completed, the natural smoothness of the image of the fusion part is ensured as much as possible, and finally, the card picture without the highlight is output. The pictures of fig. 7 and 9 taken in the above example finally result in a repaired picture as shown in fig. 12.
As shown in fig. 13, the present invention also provides an apparatus for eliminating a specular reflection area based on the same inventive concept as the elimination method of the specular reflection area, including:
an obtaining unit 1301, configured to obtain at least two pictures including a target object, where at least one picture includes a specular reflection area;
An identifying unit 1302, configured to identify a specular reflection area corresponding to at least one picture;
and the restoration unit 1303 is configured to obtain a diffuse reflection area corresponding to the specular reflection area in at least one other picture, and restore the specular reflection area by using the diffuse reflection area, so as to obtain at least one picture that does not include the specular reflection area.
In a possible implementation manner, in the apparatus provided by the embodiment of the present invention, the obtaining unit 1301 is specifically configured to:
and instructing the user to shoot at least two pictures with different shooting angles in a mode of adjusting the shooting angles.
In a possible implementation manner, in the apparatus provided by the embodiment of the present invention, the obtaining unit 1301 is specifically configured to:
instructing a user to adjust the state of the flash lamp, and taking at least two pictures in different flash lamp states; or automatically adjusting the flash state and continuously and automatically taking at least two pictures in different flash states.
In a possible implementation manner, in the apparatus provided by the embodiment of the present invention, the obtaining unit 1301 is further configured to:
determining a current ambient light state according to a current shooting environment;
and according to the ambient light state, instructing the user to take at least two pictures containing the target object.
In a possible implementation manner, in the apparatus provided by the embodiment of the present invention, the ambient light state includes a strong ambient light state and a weak ambient light state, and the obtaining unit 1301 is further configured to:
when the light intensity of the ambient light is lower than a preset threshold value, determining that the ambient light state is a weak ambient light state;
and when the light intensity of the ambient light is higher than or equal to a preset threshold value, determining that the ambient light state is a strong ambient light state.
In a possible implementation manner, in the apparatus provided by the embodiment of the present invention, the obtaining unit 1301 is specifically configured to:
when the ambient light state is a weak ambient light state, at least two pictures with different shooting angles are shot in a mode of indicating a user to adjust the shooting angles; and/or
And when the ambient light state is a strong ambient light state, at least two pictures in different flash lamp states are shot.
In one possible implementation manner, in the device provided by the embodiment of the present invention, the identifying unit 1302 is specifically configured to:
selecting at least one picture as a target picture;
and inputting the target picture into a specular reflection area detection model to obtain a specular reflection area.
In one possible implementation manner, in the device provided by the embodiment of the present invention, the identifying unit 1302 is specifically configured to:
Selecting at least one picture meeting preset conditions as a target picture; /or
And displaying the picture to the user, and determining the target picture according to the selection operation of the user.
In one possible implementation manner, in the device provided by the embodiment of the present invention, the identifying unit 1302 is specifically configured to train and generate the specular reflection area detection model by:
acquiring a plurality of training samples containing pictures and labeling results;
and comparing the specular reflection area output by each training sample with the labeling result of the training sample by taking each picture in the plurality of training samples as input, and generating a specular reflection area detection model based on the difference training of the specular reflection area output by each training sample and the labeling result of the training sample, wherein the labeling result is the specular reflection area labeled in the training sample in advance.
In one possible implementation manner, in the apparatus provided by the embodiment of the present invention, the identifying unit 1302 is further configured to:
displaying the specular reflection area to a user;
in response to a user operation, the specular reflection area is corrected.
In a possible implementation manner, in the apparatus provided by the embodiment of the present invention, the obtaining unit 1301 is further configured to:
And identifying whether the target object is a card, and acquiring at least two pictures containing the target object when the target object is the card.
In addition, the method and apparatus for eliminating the specular reflection area according to the embodiments of the present application described in connection with fig. 1 to 13 may be implemented by an electronic device. Fig. 14 shows a schematic hardware structure of an electronic device according to an embodiment of the present application.
As shown in fig. 14, the electronic apparatus 1400 may include a processing device (e.g., a central processing unit, a graphics processor, etc.) 1401, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 1402 or a program loaded from a storage device 1408 into a Random Access Memory (RAM) 1403 to implement the specular reflection area removal method according to the embodiments described in the present disclosure. In the RAM 1403, various programs and data required for the operation of the electronic device 1400 are also stored. The processing device 1401, the ROM 1402, and the RAM 1403 are connected to each other through a bus 1404. An input/output (I/O) interface 1405 is also connected to the bus 1404.
In general, the following devices may be connected to the I/O interface 1405: input devices 1406 including, for example, a touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, and the like; an output device 1407 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 1408 including, for example, magnetic tape, hard disk, etc.; and communication means 1409. The communication means 1409 may allow the electronic device 1400 to communicate wirelessly or by wire with other devices to exchange data. While fig. 14 shows an electronic device 1400 having various means, it is to be understood that not all illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a non-transitory computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts, thereby implementing the method as mentioned above. In such an embodiment, the computer program may be downloaded and installed from a network via the communication means 1409, or installed from the storage means 1408, or installed from the ROM 1402. The above-described functions defined in the methods of the embodiments of the present disclosure are performed when the computer program is executed by the processing device 1401.
It should be noted that the computer readable medium described in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device.
The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to:
acquiring at least two pictures containing a target object, wherein at least one picture contains a specular reflection area;
identifying a specular reflection area corresponding to at least one picture;
and in the other at least one picture, obtaining a diffuse reflection area at a position corresponding to the specular reflection area, and repairing the specular reflection area by using the diffuse reflection area to obtain at least one picture which does not contain the specular reflection area.
Alternatively, the electronic device may perform other steps mentioned in the above embodiments when one or more of the above-described programs are executed by the electronic device.
Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including, but not limited to, an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. Wherein the names of the units do not constitute a limitation of the units themselves in some cases.
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
In the context of this disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
In the embodiment of the application, at least two pictures containing a target object are firstly obtained, then a specular reflection area corresponding to at least one picture is identified, finally a diffuse reflection area corresponding to the specular reflection area is obtained in other at least one picture, and the diffuse reflection area is utilized to repair the specular reflection area, so that at least one picture which does not contain the specular reflection area is obtained. Compared with the prior art, the method solves the problems that the photo highlight reflection elimination quality is low and the requirement of photographing of credentials cannot be completely met, and can eliminate the specular reflection area by utilizing a plurality of pictures, so that the finally obtained pictures are clearer and the information is not lost, the requirement of photographing of credentials is met, and the user experience is improved.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
While preferred embodiments of the present application have been described, additional variations and modifications in those embodiments may occur to those skilled in the art once they learn of the basic inventive concepts. It is therefore intended that the following claims be interpreted as including the preferred embodiments and all such alterations and modifications as fall within the scope of the application.
It will be apparent to those skilled in the art that various modifications and variations can be made to the present application without departing from the spirit or scope of the application. Thus, it is intended that the present application also include such modifications and alterations insofar as they come within the scope of the appended claims or the equivalents thereof.
Claims (14)
1. A method of eliminating a specular reflection area, comprising:
Acquiring at least two pictures containing target objects, wherein at least one picture contains a specular reflection area;
identifying at least one specular reflection area corresponding to the picture;
and in at least one other picture, obtaining a diffuse reflection area at a position corresponding to the specular reflection area, and repairing the specular reflection area by using the diffuse reflection area to obtain at least one picture which does not contain the specular reflection area.
2. The method of claim 1, wherein the obtaining at least two pictures containing the target object comprises:
and instructing the user to take at least two pictures with different shooting angles by adjusting the shooting angles.
3. The method of claim 1, wherein the obtaining at least two pictures containing the target object comprises:
indicating a user to adjust the state of the flash lamp, and taking at least two pictures in different flash lamp states; /or
Automatically adjusting the state of the flash lamp and continuously and automatically taking at least two pictures in different flash lamp states.
4. A method according to claim 2 or 3, wherein prior to taking the at least two of the pictures, the method further comprises:
Determining a current ambient light state according to a current shooting environment;
and according to the ambient light state, indicating the user to shoot at least two pictures containing the target object.
5. The method of claim 4, wherein the ambient light state comprises a strong ambient light state and a weak ambient light state, the determining the current ambient light state from the current shooting environment comprising:
when the light intensity of the ambient light is lower than a preset threshold value, determining that the ambient light state is a weak ambient light state;
and when the light intensity of the ambient light is higher than or equal to a preset threshold value, determining that the ambient light state is a strong ambient light state.
6. The method of claim 5, wherein the instructing the user to take at least two pictures containing the target object based on the ambient light condition comprises:
when the ambient light state is a weak ambient light state, the user is instructed to shoot at least two pictures with different shooting angles in a mode of adjusting the shooting angle; and/or
And when the ambient light state is a strong ambient light state, at least two pictures in different flash lamp states are shot.
7. The method of claim 1, wherein identifying the specular reflection area of at least one of the pictures comprises:
Selecting at least one picture as a target picture;
and inputting the target picture into the specular reflection area detection model to obtain the specular reflection area.
8. The method of claim 7, wherein selecting at least one of the pictures as a target picture comprises:
selecting at least one picture meeting preset conditions as the target picture; /or
And displaying the picture to a user, and determining the target picture according to the selection operation of the user.
9. The method of claim 7, wherein the specular reflection area detection model is generated by training:
acquiring a plurality of training samples containing pictures and labeling results;
and comparing the specular reflection area output by each training sample with the labeling result of the training sample by taking each picture in a plurality of training samples as input, and generating the specular reflection area detection model based on the difference training of the specular reflection area output by each training sample and the labeling result of the training sample, wherein the labeling result is the specular reflection area labeled in the training sample in advance.
10. The method of claim 7, wherein the method further comprises:
displaying the specular reflection area to a user;
the specular reflection area is corrected in response to an operation by the user.
11. The method of claim 1, wherein the obtaining at least two pictures containing the target object comprises:
and identifying whether the target object is a card, and acquiring at least two pictures containing the target object when the target object is the card.
12. A specular reflection area canceling apparatus, comprising:
an acquisition unit, configured to acquire at least two pictures including a target object, where at least one of the pictures includes a specular reflection area;
the identification unit is used for identifying at least one specular reflection area corresponding to the picture;
and the restoration unit is used for acquiring a diffuse reflection area at a position corresponding to the specular reflection area in at least one other picture, and restoring the specular reflection area by using the diffuse reflection area to obtain at least one picture which does not contain the specular reflection area.
13. An electronic device, comprising: at least one processor, at least one memory, and computer program instructions stored in the memory, which when executed by the processor, implement the method of any one of claims 1-11.
14. A computer readable storage medium having stored thereon computer program instructions, which when executed by a processor, implement the method of any of claims 1-11.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310512900.3A CN116757940A (en) | 2023-05-08 | 2023-05-08 | Method, device, equipment and medium for eliminating specular reflection area |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310512900.3A CN116757940A (en) | 2023-05-08 | 2023-05-08 | Method, device, equipment and medium for eliminating specular reflection area |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116757940A true CN116757940A (en) | 2023-09-15 |
Family
ID=87954072
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310512900.3A Pending CN116757940A (en) | 2023-05-08 | 2023-05-08 | Method, device, equipment and medium for eliminating specular reflection area |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116757940A (en) |
-
2023
- 2023-05-08 CN CN202310512900.3A patent/CN116757940A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102126300B1 (en) | Method and apparatus for generating an all-in-focus image | |
| US9600741B1 (en) | Enhanced image generation based on multiple images | |
| US9591237B2 (en) | Automated generation of panning shots | |
| US9838616B2 (en) | Image processing method and electronic apparatus | |
| US11016563B2 (en) | Image changes based on voice | |
| US10165201B2 (en) | Image processing method and apparatus and terminal device to obtain a group photo including photographer | |
| CN108337505B (en) | Information acquisition method and device | |
| WO2017114399A1 (en) | Backlight photographing method and device | |
| CN111553362B (en) | Video processing method, electronic device and computer readable storage medium | |
| CN111163265A (en) | Image processing method, image processing device, mobile terminal and computer storage medium | |
| US20170351932A1 (en) | Method, apparatus and computer program product for blur estimation | |
| CN111325698A (en) | Image processing method, device and system and electronic equipment | |
| CN114022662A (en) | Image recognition method, device, equipment and medium | |
| CN108289176B (en) | Photographing question searching method, question searching device and terminal equipment | |
| CN112508959B (en) | Video object segmentation method and device, electronic equipment and storage medium | |
| CN110349108B (en) | Method, apparatus, electronic device, and storage medium for processing image | |
| WO2023071933A1 (en) | Camera photographing parameter adjustment method and apparatus and electronic device | |
| US20220245920A1 (en) | Object display method and apparatus, electronic device, and computer readable storage medium | |
| CN111385460A (en) | Image processing method and device | |
| CN116757940A (en) | Method, device, equipment and medium for eliminating specular reflection area | |
| CN111654623B (en) | Photographing method and device and electronic equipment | |
| KR20140134844A (en) | Method and device for photographing based on objects | |
| CN113379624A (en) | Image generation method, training method, device and equipment of image generation model | |
| CN110971813B (en) | Focusing method and device, electronic equipment and storage medium | |
| CN111325148A (en) | Method, device and equipment for processing remote sensing image and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |