WO2021129806A1

WO2021129806A1 - Image processing method, apparatus, electronic device, and readable storage medium

Info

Publication number: WO2021129806A1
Application number: PCT/CN2020/139403
Authority: WO
Inventors: 常文涛
Original assignee: 深圳市万普拉斯科技有限公司
Priority date: 2019-12-26
Filing date: 2020-12-25
Publication date: 2021-07-01
Also published as: CN113055579A; CN113055579B

Abstract

The present application relates to the technical field of image processing and provides an image processing method, an apparatus, an electronic device, and a readable storage medium. Detection is performed on an obtained image to be processed, a specular reflection region within the image to be processed is determined, and substitution or modification is subsequently performed on a pattern within the specular reflection region according to a reference image, so as to alter the pattern within the specular reflection region.

Description

Image processing method, device, electronic equipment and readable storage medium

cross reference

The present invention claims the priority of a Chinese patent application filed with the Chinese Patent Office, the application number is 201911363993.8, and the invention title is "Image processing methods, devices, electronic equipment and readable storage media" on December 26, 2019. All of the above applications The content is incorporated in the present invention by reference.

Technical field

This application relates to the field of image processing technology, and in particular to an image processing method, device, electronic equipment, and readable storage medium.

Background technique

With the continuous advancement of technology, more and more devices have camera functions. When the user is shooting, if there is an object with specular reflection in the shooting range of the camera, since the object can reflect a pattern, the image obtained by the camera will include the pattern reflected by the object. The pattern reflected by the object generally includes patterns that the user does not want to appear in the image. For example, if a user takes a selfie with a smart phone while wearing sunglasses, he will see the phone in the obtained image.

Summary of the invention

This application provides an image processing method, device, electronic equipment, and readable storage medium.

The technical solutions adopted in the embodiments of this application are as follows:

In a first aspect, an embodiment of the present application provides an image processing method applied to an electronic device. The method includes: obtaining a to-be-processed image; detecting the to-be-processed image to determine the specular reflection area in the to-be-processed image ; According to the reference image, the pattern in the specular reflection area is replaced or corrected to modify the pattern in the specular reflection area.

In an optional embodiment, the image to be processed is obtained based on the first shooting direction, and the detecting the image to be processed to determine the specular reflection area in the image to be processed includes: obtaining a scene image, wherein , The scene image is obtained by shooting in a second shooting direction in the target environment where the image to be processed is obtained, the relationship between the second shooting direction and the first shooting direction and the difference between the incident light and the reflected light The relationship is the same; scene matching is performed between the scene image and the image to be processed; if there is an area similar to the scene of the scene image in the image to be processed, the area is regarded as the specular reflection area.

In an optional embodiment, the electronic device stores an object recognition model, and the detecting the to-be-processed image and determining the specular reflection area in the to-be-processed image includes: using the object recognition model Object recognition is performed on the to-be-processed image to determine whether the to-be-processed image includes an object with specular reflection; if so, the area where the object with specular reflection is located is taken as the specular reflection area.

In an alternative embodiment, the detecting the to-be-processed image and determining the specular reflection area in the to-be-processed image includes: receiving a selection operation input by a user, and taking the area corresponding to the selection operation as The specular reflection area.

In an optional embodiment, the replacing or correcting the pattern in the specular reflection area according to the reference image to modify the pattern in the specular reflection area includes: changing the pattern in the specular reflection area Replace with the pattern in the reference image; or, according to the matching result of the pattern in the specular reflection area and the pattern in the scene area, obtain the first pattern in the specular reflection area that is different from the pattern in the scene area. Area, a second area where the pattern in the scene area is different from the pattern in the specular reflection area, and the pattern in the first area is replaced with the pattern in the second area, wherein the scene area Is an area where the scene image is similar to the specular reflection area, the image to be processed is obtained based on the first shooting direction, and the scene image is taken in the second shooting direction in the target environment where the image to be processed is obtained, so The relationship between the second imaging direction and the first imaging direction is the same as the relationship between incident light and reflected light.

In a second aspect, an embodiment of the present application provides an image processing device, which is applied to electronic equipment, and the device includes: an obtaining module for obtaining a to-be-processed image; a detection module for detecting the to-be-processed image and determining The specular reflection area in the image to be processed; a processing module for replacing or correcting the pattern in the specular reflection area according to a reference image to modify the pattern in the specular reflection area.

In an optional implementation manner, the image to be processed is obtained based on the first shooting direction, and the detection module is used to: obtain a scene image, wherein the scene image is in the first environment where the image to be processed is obtained. It is obtained by shooting in a second shooting direction, and the relationship between the second shooting direction and the first shooting direction is the same as the relationship between incident light and reflected light; performing scene matching between the scene image and the image to be processed; If there is an area similar to the scene of the scene image in the image to be processed, the area is regarded as the specular reflection area.

In an optional implementation manner, an object recognition model is stored in the electronic device, and the detection module is configured to: perform object recognition on the to-be-processed image through the object recognition model to determine whether the image is in the to-be-processed image Whether an object with specular reflection is included; if it exists, the area where the object with specular reflection is located is taken as the specular reflection area.

In an optional implementation manner, the detection module is configured to: receive a selection operation input by a user, and use an area corresponding to the selection operation as the specular reflection area.

In an alternative embodiment, the processing module is configured to: replace the pattern in the specular reflection area with the pattern in the reference image; or, according to the pattern in the specular reflection area and the scene area As a result of pattern matching, a first area in the specular reflection area with a pattern different from that of the scene area, a second area in the scene area with a pattern different from the pattern in the specular reflection area is obtained, and all The pattern in the first area is replaced with the pattern in the second area, wherein the scene area is an area where the scene image is similar to the scene of the specular reflection area, and the image to be processed is obtained based on the first shooting direction, The scene image is captured in a second shooting direction in the target environment where the image to be processed is obtained, and the relationship between the second shooting direction and the first shooting direction and the relationship between incident light and reflected light the same.

In a third aspect, an embodiment of the present application provides an electronic device including a processor and a memory, the memory stores machine executable instructions that can be executed by the processor, and the processor can execute the machine executable instructions In order to realize the image processing method described in any one of the foregoing embodiments.

In a fourth aspect, an embodiment of the present application provides a readable storage medium having a computer program stored thereon, and the computer program, when executed by a processor, implements the image processing method according to any one of the foregoing embodiments.

The image processing method, device, electronic device, and readable storage medium provided by the embodiments of the application detect the obtained image to be processed, determine the specular reflection area in the image to be processed, and then use the reference image to determine the specular reflection area Perform replacement or correction processing on the pattern in to change the pattern in the specular reflection area. In this way, patterns that the user does not want to appear in the image can be removed, thereby improving image quality and improving user experience.

In order to make the above-mentioned objectives, features and advantages of the present application more obvious and understandable, preferred embodiments are specifically described below in conjunction with the accompanying drawings, which are described in detail as follows.

Description of the drawings

In order to more clearly describe the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings that need to be used in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application, and therefore do not It should be regarded as a limitation of the scope. For those of ordinary skill in the art, other related drawings can be obtained based on these drawings without creative work.

Fig. 1 is a schematic block diagram of an electronic device provided by an embodiment of the present application.

Fig. 2 is a schematic flowchart of an image processing method provided by an embodiment of the present application.

FIG. 3 is a schematic flowchart of the sub-steps included in step S120 in FIG. 2.

FIG. 4 is a schematic diagram of the relationship between the first shooting direction and the second shooting direction provided by an embodiment of the present application.

Fig. 5 is a schematic block diagram of an image processing apparatus provided by an embodiment of the present application.

Icons: 100-electronic equipment; 110-memory; 120-processor; 130-communication unit; 200-image processing device; 210-acquisition module; 220-detection module; 230-processing module.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present application in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application generally described and shown in the drawings herein may be arranged and designed in various different configurations.

Therefore, the following detailed description of the embodiments of the application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of the application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work shall fall within the protection scope of the present application.

It should be noted that the terms "first" and "second" and other relational terms are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply one of these entities or operations. There is any such actual relationship or order between. Moreover, the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes those that are not explicitly listed Other elements of, or also include elements inherent to this process, method, article or equipment. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other same elements in the process, method, article, or equipment that includes the element.

Please refer to FIG. 1, which is a block diagram of an electronic device 100 according to an embodiment of the present application. The electronic device 100 may be, but is not limited to, a smart phone, a video camera, a computer, and the like. The electronic device 100 may include a memory 110, a processor 120, and a communication unit 130. The components of the memory 110, the processor 120, and the communication unit 130 are directly or indirectly electrically connected to each other to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

Among them, the memory 110 is used to store programs or data. The memory 110 may be, but is not limited to, a random access memory (Random Access Memory, RAM), a read only memory (Read Only Memory, ROM), and a programmable read-only memory (Programmable Read-Only Memory, PROM). Erasable Programmable Read-Only Memory (EPROM), Electric Erasable Programmable Read-Only Memory (EEPROM), etc.

The processor 120 is used to read/write data or programs stored in the memory 110 and execute corresponding functions. For example, an image processing device 200 is stored in the memory 110, and the image processing device 200 includes at least one software function module that can be stored in the memory 110 in the form of software or firmware. The processor 120 executes various functional applications and data processing by running software programs and modules stored in the memory 110, such as the image processing apparatus 200 in the embodiment of the present application, and thus realizes the image processing in the embodiment of the present application. method.

The communication unit 130 is used to establish a communication connection between the electronic device 100 and other communication terminals through a network, and is used to send and receive data through the network.

It should be understood that the structure shown in FIG. 1 is only a schematic structural diagram of the electronic device 100, and the electronic device 100 may also include more or fewer components than those shown in FIG. Configuration. The components shown in FIG. 1 can be implemented by hardware, software, or a combination thereof.

Please refer to FIG. 2, which is a schematic flowchart of an image processing method provided by an embodiment of the present application. The method is applied to the electronic device 100. The following describes the implementation process of the image processing method.

In step S110, an image to be processed is obtained.

Step S120: Detect the image to be processed, and determine the specular reflection area in the image to be processed.

In step S130, the pattern in the specular reflection area is replaced or corrected according to the reference image, so as to change the pattern in the specular reflection area.

In this embodiment, after detecting and determining the specular reflection area in the image to be processed, the pattern in the specular reflection area is replaced or corrected according to the reference image to complete the modification of the pattern in the specular reflection area. As a result, patterns that the user does not want to appear in the image can be removed, thereby improving image quality and improving user experience.

Optionally, in this embodiment, the electronic device 100 may include a first camera, and the image to be processed is captured by the first camera. Alternatively, the electronic device 100 obtains the image to be processed by receiving an image sent by another device. Of course, it can be understood that the electronic device 100 may also obtain the to-be-processed image in other ways.

In an implementation of this embodiment, the specular reflection area can be obtained in the following manner. Please refer to FIG. 3, which is a schematic flowchart of the sub-steps included in step S120 in FIG. 2. Step S120 may include sub-step S121 to sub-step S123.

In sub-step S121, a scene image is obtained.

Sub-step S122, performing scene matching between the scene image and the image to be processed.

In sub-step S123, if there is an area similar to the scene of the scene image in the image to be processed, use this area as the specular reflection area.

In this embodiment, the image to be processed is obtained based on a first shooting direction, and the scene image is obtained by shooting in a second shooting direction in the target environment where the image to be processed is obtained. The relationship between the first shooting directions is the same as the relationship between incident light and reflected light. When light hits the reflective surface, the light is reflected by the reflective surface to the original medium. The incident light and the reflected light are located on both sides of the normal of the reflective surface, and the incident angle is equal to the reflection angle. That is to say, when the image to be processed captured in the first shooting direction includes the specular reflection area, the scene image is captured in the second shooting direction. As shown in FIG. 4, the first shooting direction and the second shooting direction are located at On both sides of the normal of the reflecting surface, the angle 1 between the first shooting direction and the normal is the same or close to the angle 2 between the second shooting direction and the normal. Wherein, the reflection surface in FIG. 4 represents the actual reflection surface corresponding to the specular reflection area in the image to be processed. For example, if the image to be processed includes the lens of sunglasses, the reflective surface in FIG. 4 represents the reflective surface of the lens. Of course, it can be understood that if the first shooting direction is perpendicular to the reflecting surface, the second shooting direction is opposite to the first shooting direction.

Optionally, the electronic device 100 or the device that obtains the image to be processed by shooting may include a first camera and a second camera with opposite shooting directions. After the image to be processed is obtained through the first camera, the image to be processed may be adjusted The shooting direction of the second camera, so that the scene image is obtained through the second camera.

For example, the electronic device 100 includes a front camera and a rear camera. After the user takes a selfie with the front camera, the image obtained by the front camera can be used as the image to be processed, and the shooting direction of the rear camera can be adjusted to obtain the scene image in the second shooting direction. image.

Optionally, after obtaining the image to be processed, the camera that has obtained the image to be processed may be adjusted from the first shooting direction to the second shooting direction, and then shooting again, and the image obtained by shooting again is used as the Describe the scene image.

After the scene image is obtained, scene matching may be performed on the scene image and the image to be processed to obtain the scene similarity, and it is determined whether the image to be processed includes an area similar to the scene image according to the scene similarity. If the scene similarity is not greater than the preset similarity, it can be determined that the image to be processed does not include specular reflection objects. If the scene similarity is greater than the preset similarity, it can be determined that the image to be processed includes an area similar to the scene image, and the area where the scene of the image to be processed is similar to the scene image in the scene image is used as the mirror Reflective area.

Among them, scene matching can be achieved through feature matching in the image. For example, if there is a cuboid building in a certain area of the image to be processed, and there is also a cuboid building in the scene image, it can be determined that the area where the cuboid building is in the image to be processed is Specular reflection area.

Optionally, in this embodiment, if the second shooting direction can be directly determined, then according to the second shooting direction and the current shooting direction, prompt information of angle movement is displayed, so that the user can directly change the shooting direction according to the information. Adjust to the second shooting direction.

Optionally, in this embodiment, if the second shooting direction cannot be directly determined, after obtaining the to-be-processed image in the first shooting direction, the user can be based on the specular reflection area in the to-be-processed image. Adjust the direction of the pattern corresponding to the actual scene, and then obtain an image in the adjusted direction. Then, according to the scene similarity and the scene similarity between the newly obtained image and the image to be processed, prompt information of the angle movement is displayed. This adjustment action is repeated, so that the user gradually adjusts the shooting direction to the second shooting direction, so as to obtain the scene image. Wherein, when the scene similarity is greater than the preset similarity, it can be determined that the image corresponding to the scene similarity is the scene image obtained in the second shooting direction, and the image to be processed corresponds to the scene similarity The area is the specular reflection area.

For example, if the scene similarity between the newly obtained image and the image to be processed is lower than the preset similarity, and the newly obtained image and the image to be processed have a partial scene match, the missing part can be determined , And prompt the user to move based on the missing part. For example, if a part of the pattern in the image to be processed is the same as the pattern on the left side of the newly acquired image, but the current scene similarity is lower than the preset similarity, it can be determined that the scene on the right side is missing. The user is prompted to move the shooting direction to the right; if the surrounding area is missing, the user is prompted to step back. Of course, the user can also determine which part is missing (for example, the right scene or the left scene), and then the electronic device 100 prompts the user with a specific movement mode (for example, to the right or to the left) according to the missing part determined by the user. The shooting direction is adjusted, and then the shooting direction is adjusted to the second shooting direction to obtain the scene image.

Optionally, if after adjusting the preset number of times or adjusting the preset angle, a scene image with a corresponding scene similarity greater than the preset similarity is still not obtained, the image obtained at a certain time may be directly used as the scene at random And determining that the specular reflection area is not included in the to-be-processed image.

Optionally, in this embodiment, if the second shooting direction cannot be determined directly, the user can adjust the direction according to the actual scene corresponding to the pattern in the specular reflection area in the image to be processed, and then obtain it from a different angle. Multiple images, so that the multiple images are combined into one image including the complete scene, that is, an image including the scene image is obtained. Or, when the missing part is determined (for example, it is determined that the surrounding area is missing), prompt information for prompting the user to take multiple images at different angles according to the actual scene can also be displayed, so as to synthesize the image including the scene image, and then according to the scene The image of the image and the image to be processed are matched to determine the specular reflection area.

In another implementation manner in this embodiment, common mirror-reflected objects (such as sunglasses, pupils, etc.) are model-trained through machine learning to obtain a trained object recognition model, and the object recognition model It is stored in the electronic device 100. The electronic device 100 can perform object recognition on the image to be processed through the object recognition model. If the specularly reflected object is not recognized, it can be determined that the to-be-processed image does not include the specularly reflected object. If an object with specular reflection is identified, it can be determined that the image to be processed includes an object with specular reflection, and the area where the object with specular reflection is located is regarded as the specular reflection area.

In another implementation of this embodiment, the electronic device 100 may also receive a selection operation input by the user for enclosing an area, and use the area corresponding to the selection operation as the specular reflection area.

In another implementation manner of this embodiment, the electronic device 100 may also determine the specular reflection area through at least two of the above three detection schemes. For example, if the specular reflection area obtained by scene matching or recognition through the object recognition model and the area corresponding to the user's selection operation are obtained, the area corresponding to the user's selection operation is taken as the specular reflection area of the image to be processed.

If the specular reflection area is determined by scene matching and object recognition model recognition, and the specular reflection area obtained by the two methods is different, the user can be prompted to input the selection operation, and the area specified by the user's selection operation is used as the waiting area. Process the specular reflection area in the image.

After the specular reflection area is determined, the pattern in the specular reflection area may be replaced with the pattern in the reference image, so as to remove the patterns in the specular reflection area that the user does not want to be displayed. Optionally, the reference image may be the scene image, or may be an image completely unrelated to the target environment in which the image to be processed is obtained.

For example, a user uses a smartphone to take a selfie in front of the four logos A, B, C, and D while wearing sunglasses. Under normal circumstances (when there is no block between the sunglasses and the logo), the sunglasses can reflect A, B, and C. Due to the occlusion of the mobile phone, the patterns on the sunglasses are the mobile phone and the two logos C and D. That is to say, the patterns in the specular reflection area of the image to be processed are the mobile phone and C, D. D two logos. If the reference image is a scene image with four logos of A, B, C, and D, the scene image can be scaled down according to the requirements, and all the patterns in the specular reflection area are replaced according to the reduced scene image, so that the replaced The patterns in the specular reflection area are four marks A, B, C, and D. The reference image can also be an image with a mountain peak or sea pattern, that is, it has nothing to do with the four marks A, B, C, and D, and then all the patterns in the specular reflection area are replaced according to the image in the same manner. Thus, the mobile phone pattern in the image to be processed can be removed.

After the specular reflection area is determined, an area similar to the specular reflection area in the scene image may be used as a scene area. Wherein, the scene area may be the scene image or a part of the scene image, which is determined according to actual conditions. Then, the pattern in the specular reflection area is matched with the pattern in the scene area to obtain a first area in which the pattern in the specular reflection area is different from the pattern in the scene area, the pattern in the scene area and the pattern in the scene area. The second area with a different pattern in the specular reflection area. Then replace the pattern in the first area with the pattern in the second area. In this way, the pattern in the first area in the original specular reflection area can be removed.

For example, a user uses a smartphone to take a selfie in front of the four logos A, B, C, and D while wearing sunglasses. Under normal circumstances (when there is no block between the sunglasses and the logo), the sunglasses can reflect A, B, and C. Due to the occlusion of the mobile phone, the patterns on the sunglasses are the mobile phone and the two logos C and D. That is to say, the patterns in the specular reflection area of the image to be processed are the mobile phone and C, D. D two logos. By matching the pattern of the specular reflection area with the scene area, it can be determined that the area where the mobile phone is located in the specular reflection area is the first area, and the area where the A and B marks are located is the second area, and the pattern in the first area is replaced After being the pattern in the second area, the pattern in the first area of the image to be processed is not a mobile phone, but the A and B logos.

In order to perform the corresponding steps in the above-mentioned embodiment and each possible manner, an implementation manner of the image processing apparatus 200 is given below. Optionally, the image processing apparatus 200 may use the components of the electronic device 100 shown in FIG. structure. Further, please refer to FIG. 5, which is a block diagram of an image processing apparatus 200 provided by an embodiment of the present application. It should be noted that the basic principles and technical effects of the image processing device 200 provided in this embodiment are the same as those in the above embodiments. For a brief description, for parts not mentioned in this embodiment, please refer to the above embodiments. In the corresponding content. In terms of functions, the image processing device 200 includes: an obtaining module 210, a detection module 220, and a processing module 230.

The obtaining module 210 is used to obtain an image to be processed.

The detection module 220 is configured to detect the image to be processed and determine the specular reflection area in the image to be processed.

The processing module 230 is configured to perform replacement or correction processing on the pattern in the specular reflection area according to the reference image, so as to modify the pattern in the specular reflection area.

Optionally, in the implementation of this embodiment, the image to be processed is obtained based on the first shooting direction, and the detection module 220 is configured to: obtain a scene image, wherein the scene image is obtained when the image to be processed is obtained. The target environment is captured in a second shooting direction, and the relationship between the second shooting direction and the first shooting direction is the same as the relationship between incident light and reflected light; The image is processed for scene matching; if there is an area similar to the scene of the scene image in the image to be processed, the area is regarded as the specular reflection area.

Optionally, in the implementation of this embodiment, an object recognition model is stored in the electronic device 100, and the detection module 220 is configured to perform object recognition on the image to be processed through the object recognition model to It is determined whether the image to be processed includes an object with specular reflection; if it exists, the area where the object with specular reflection is located is taken as the specular reflection area.

Optionally, in the implementation of this embodiment, the detection module 220 is configured to receive a selection operation input by a user, and use an area corresponding to the selection operation as the specular reflection area.

Optionally, in the implementation of this embodiment, the processing module 230 is configured to: replace the pattern in the specular reflection area with the pattern in the reference image; or, according to the pattern in the specular reflection area The pattern and the pattern in the scene area are matched to obtain a first area where the pattern in the specular reflection area is different from the pattern in the scene area, and the second area where the pattern in the scene area is different from the pattern in the specular reflection area is obtained. Second area, and replacing the pattern in the first area with the pattern in the second area, wherein the scene area is an area where the scene image is similar to the specular reflection area.

Optionally, the above-mentioned modules can be stored in the memory 110 shown in FIG. 1 in the form of software or firmware (Firmware) or solidified in the operating system (OS) of the electronic device 100, and can be processed by the processing shown in FIG. 1器120 executes. At the same time, data, program codes, etc. required to execute the above-mentioned modules may be stored in the memory 110.

In summary, the embodiments of the present application provide an image processing method, device, electronic device, and readable storage medium. After the image to be processed is obtained, the image to be processed is detected, the specular reflection area in the image to be processed is determined, and then the pattern in the specular reflection area is replaced or corrected according to the reference image to change the specular reflection area In the pattern. In this way, patterns that the user does not want to appear in the image can be removed, thereby improving image quality and improving user experience.

In the several embodiments provided in this application, it should be understood that the disclosed device and method may also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings show the possible implementation architectures, functions, and functions of the devices, methods, and computer program products according to multiple embodiments of the present application. operating. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or part of the code, and the module, program segment, or part of the code contains one or more functions for realizing the specified logic function. Executable instructions. It should also be noted that in some alternative implementations, the functions marked in the block may also occur in a different order from the order marked in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, or they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and/or flowchart, and the combination of the blocks in the block diagram and/or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or actions Or it can be realized by a combination of dedicated hardware and computer instructions.

In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist alone, or two or more modules may be integrated to form an independent part.

If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of the present application essentially or the part that contributes to the existing technology or the part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including Several instructions are used to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes. .

The above descriptions are only preferred embodiments of the application, and are not intended to limit the application. For those skilled in the art, the application can have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this application shall be included in the protection scope of this application.

Claims

An image processing method, characterized in that it is applied to an electronic device, and the method includes:

Obtain the image to be processed;

Detecting the image to be processed, and determining the specular reflection area in the image to be processed;

The pattern in the specular reflection area is replaced or corrected according to the reference image to modify the pattern in the specular reflection area.
The method according to claim 1, wherein the image to be processed is obtained based on a first shooting direction, and the detecting the image to be processed to determine the specular reflection area in the image to be processed comprises:

A scene image is obtained, wherein the scene image is obtained by shooting in a second shooting direction in the target environment where the image to be processed is obtained, and the relationship between the second shooting direction and the first shooting direction and the incident light The relationship between reflected light is the same;

Performing scene matching between the scene image and the image to be processed;

If there is an area similar to the scene of the scene image in the image to be processed, the area is regarded as the specular reflection area.
The method according to claim 2, wherein the electronic device comprises a first camera and a second camera with opposite shooting directions, and said obtaining a scene image comprises:

In the case where the image to be processed is obtained through the first camera, the shooting direction of the second camera is adjusted, and the scene image is obtained through the adjusted second camera.
The method according to claim 2, wherein the object recognition model is stored in the electronic device, and the detecting the to-be-processed image to determine the specular reflection area in the to-be-processed image comprises:

Performing scene matching between the scene image and the image to be processed to obtain scene similarity;

If the scene similarity is greater than the preset similarity, an area similar to the scene image in the to-be-processed image is used as the specular reflection area.
The method according to claim 1, wherein the object recognition model is stored in the electronic device, and the detecting the image to be processed to determine the specular reflection area in the image to be processed comprises:

Performing object recognition on the image to be processed by using the object recognition model to determine whether the image to be processed includes an object with specular reflection;

If it exists, the area where the object with specular reflection is located is taken as the specular reflection area.
The method according to claim 1, wherein the detecting the image to be processed and determining the specular reflection area in the image to be processed comprises:

The selection operation input by the user is received, and the area corresponding to the selection operation is used as the specular reflection area.
The method according to any one of claims 1 to 6, wherein the pattern in the specular reflection area is replaced or corrected according to a reference image to change the pattern in the specular reflection area, include:

Replace the pattern in the specular reflection area with the pattern in the reference image; or,

According to the matching result of the pattern in the specular reflection area and the pattern in the scene area, a first area in which the pattern in the specular reflection area is different from the pattern in the scene area, the pattern in the scene area and the mirror surface are obtained. Reflect a second area with a different pattern in the reflection area, and replace the pattern in the first area with the pattern in the second area, wherein the scene area is a scene image of a scene similar to that of the specular reflection area Area, the image to be processed is obtained based on a first shooting direction, the scene image is shot in a second shooting direction in the target environment where the image to be processed is obtained, and the second shooting direction is the same as the first shooting direction The relationship between is the same as that between incident light and reflected light.
An image processing device, characterized in that it is applied to electronic equipment, and the device includes:

The obtaining module is used to obtain the image to be processed;

The detection module is used to detect the image to be processed and determine the specular reflection area in the image to be processed;

The processing module is used for replacing or correcting the pattern in the specular reflection area according to the reference image, so as to change the pattern in the specular reflection area.
The device according to claim 8, wherein the image to be processed is obtained based on a first shooting direction, and the detection module is configured to:

A scene image is obtained, wherein the scene image is obtained by shooting in a second shooting direction in the target environment where the image to be processed is obtained, and the relationship between the second shooting direction and the first shooting direction and the incident light The relationship between reflected light is the same;

Performing scene matching between the scene image and the image to be processed;

If there is an area similar to the scene of the scene image in the image to be processed, the area is regarded as the specular reflection area.
The apparatus according to claim 8, wherein the electronic device stores an object recognition model, and the detection module is used for:

Performing object recognition on the image to be processed by using the object recognition model to determine whether the image to be processed includes an object with specular reflection;

If it exists, the area where the object with specular reflection is located is taken as the specular reflection area.
The device according to claim 8, wherein the detection module is used for:

The selection operation input by the user is received, and the area corresponding to the selection operation is used as the specular reflection area.
The device according to any one of claims 8-11, wherein the processing module is specifically:

Replace the pattern in the specular reflection area with the pattern in the reference image; or,

According to the matching result of the pattern in the specular reflection area and the pattern in the scene area, a first area in which the pattern in the specular reflection area is different from the pattern in the scene area, the pattern in the scene area and the mirror surface are obtained. Reflect a second area with a different pattern in the reflection area, and replace the pattern in the first area with the pattern in the second area, wherein the scene area is a scene image of a scene similar to that of the specular reflection area Region, the image to be processed is obtained based on a first shooting direction, the scene image is shot in a second shooting direction in the target environment where the image to be processed is obtained, and the second shooting direction is the same as the first shooting direction The relationship between is the same as that between incident light and reflected light.
An electronic device, comprising a processor and a memory, the memory storing machine executable instructions that can be executed by the processor, and the processor can execute the machine executable instructions to implement claim 1 -7 The image processing method described in any one.
A readable storage medium with a computer program stored thereon, wherein the computer program implements the image processing method according to any one of claims 1-7 when the computer program is executed by a processor.