WO2021114500A1

WO2021114500A1 - Image processing method and apparatus, electronic device, and computer-readable storage medium

Info

Publication number: WO2021114500A1
Application number: PCT/CN2020/078838
Authority: WO
Inventors: 周晨航
Original assignee: 上海传英信息技术有限公司
Priority date: 2019-12-09
Filing date: 2020-03-11
Publication date: 2021-06-17
Also published as: CN111127307A

Abstract

An image processing method and apparatus, an electronic device, and a computer-readable storage medium. The method comprises: acquiring image information to be processed including a sky; preprocessing the image information, wherein the preprocessing specifically comprises performing determination and segmentation with respect to the image information; performing further optimization processing according to a preprocessing result, wherein the optimization processing specifically comprises performing optimization processing on a sky region according to a preset rule, that is, redetermining the sky region such that the edge of a redetermined sky region is more accurate than the edge of the sky region identified during the preprocessing; and performing sky optimization processing on a portion in the image information corresponding to the redetermined sky region, so as to obtain an optimized image in which the edge of the sky region is accurately processed. Thus, the method improves the degree of fusion between the processed region and the image information, and improves the visual effect of the adapted and optimized image.

Description

Image processing method, device, electronic equipment and computer readable storage medium

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office on December 9, 2019, the application number is 201911249372.7, and the application name is "Image processing methods, devices, electronic equipment, and computer-readable storage media", and its entire contents Incorporated in this application by reference.

Technical field

This application relates to the field of data processing technology, and in particular to an image processing method, device, electronic equipment, and computer-readable storage medium.

Background technique

In interesting image processing, the sky in the image can be processed, for example, the sky area in the image can be replaced with a sky template of blue sky and white clouds.

In the prior art, usually only the sky area is recognized based on the trained sky recognition model, and processing such as template replacement is performed on the recognized sky area.

However, after template replacement often occurs in the prior art, the degree of fusion between the replaced template and the original image is poor, and the visual effect of the processed image is poor.

Summary of the invention

The embodiments of the present application provide an image processing method, device, electronic device, and computer-readable storage medium to solve the technical problem of poor fusion between the replaced template and the original image, resulting in poor visual effect of the processed image.

The first aspect of the embodiments of the present application provides an image processing method, including:

Obtain image information to be processed, image information to be processed;

Judging and segmenting the sky area of the image information;

The sky area is optimized according to preset rules to obtain an optimized image.

Optionally, the judging and segmenting the sky area of the image information includes: performing recognition processing on the image information to be processed to obtain the sky area and the non-sky area in the image information to be processed, and The sky edge transition zone between the sky area and the non-sky area; re-determine the sky area according to the sky area, the non-sky area, and the sky edge transition zone;

The optimizing processing of the sky area according to preset rules to obtain an optimized image includes: performing sky optimization processing on a portion corresponding to the re-determined sky area in the image information to be processed to obtain an optimized image .

Optionally, the re-determining the sky area according to the sky area, the non-sky area, and the sky edge transition zone includes:

Label the pixels in the sample image including the sky area and the non-sky area respectively;

Using AI semantic segmentation technology, training the pixels in the sample image that have been labeled to obtain the sky segmentation AI model;

Using the sky segmentation AI model to obtain a sky segmentation mask map corresponding to the sky area, the non-sky area, and the sky edge transition zone;

Refinement processing is performed on the sky edge of the sky segmentation mask map to obtain a sky segmentation mask map after edge refinement, so as to obtain the re-determined sky area.

Optionally, the performing refinement processing on the sky edge of the sky segmentation mask includes:

For each pixel point in the sky edge transition zone of the sky segmentation mask, the coordinates of the pixel point are obtained separately, and the coordinates are taken as the center point, and the local pixel similarity statistical method is used to calculate the center point. And the pixel points in the adjacent area of the center point perform edge determination.

Optionally, the performing sky optimization processing on the portion corresponding to the re-determined sky area in the image information to be processed includes:

The preset sky template is used to perform replacement processing on the portion corresponding to the re-determined sky area in the image information to be processed.

Optionally, said using a preset sky template to perform replacement processing on the portion corresponding to the re-determined sky area in the image information to be processed includes:

Adjust the position and/or size of the preset sky template according to the position and/or size of the re-determined sky area, and replace the adjusted sky template with the image information to be processed and re-determine The corresponding part of the sky area.

Identifying the scene corresponding to the image information to be processed, and searching for and acquiring the adjustment mode corresponding to the scene;

According to the adjustment mode, sky optimization processing is performed on the portion corresponding to the re-determined sky region in the image information to be processed.

A second aspect of the embodiments of the present application provides an image processing apparatus, including:

A segmentation module for judging and segmenting the sky area of the image information;

The optimization module is used for optimizing processing of the sky area according to preset rules to obtain an optimized image.

Optionally, the optimization module is specifically used for:

Obtaining a sky segmentation mask map corresponding to the sky area, the non-sky area, and the sky edge transition zone;

Optionally, it also includes:

The processing module is configured to adjust the sky area according to template replacement of the sky area or according to the scene corresponding to the image information to be processed.

A third aspect of the embodiments of the present application provides an electronic device, including: a processor, a memory, and a computer program; wherein the computer program is stored in the memory and is configured to be executed by the processor, and The computer program includes instructions for performing the method according to any one of the preceding first aspects.

A fourth aspect of the embodiments of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of the foregoing first aspects is implemented.

The beneficial effects of the embodiments of the present application relative to the prior art:

The embodiment of the application provides an image processing method and device. It is found that the reason for the poor fusion of the replacement template and the original picture in the prior art is that it is usually difficult to obtain an accurate image when recognizing the sky area in the prior art. The edge of the sky area may cause the non-sky area to be processed during the subsequent processing of the sky area, or the sky area may not be fully processed, resulting in poor fusion of the processed part with the original image. Therefore, in the embodiment of the present application, after acquiring the image information to be processed including the sky, the image information to be processed is preprocessed. Specifically, the image information to be processed is judged and segmented, and then the image information to be processed is determined and segmented according to the preprocessed image information. The result is further optimized. Specifically, the sky area is optimized according to the preset rules, that is, the sky area is re-determined, and the edge of the re-determined sky area is more accurate than the edge of the sky area identified in the preprocessing, and then processed After the sky area corresponding to the re-determined sky area in the image information is optimized for the sky, the optimized image obtained can achieve accurate processing of the edge of the sky area, so the processing area and the image information to be processed have a better fusion. The visual effect of the adapted and optimized image is better.

Description of the drawings

The drawings herein are incorporated into the specification and constitute a part of the specification, show embodiments that conform to the application, and are used together with the specification to explain the principle of the application.

Through the above drawings, the specific embodiments of the present application have been shown, which will be described in more detail later. These drawings and text description are not intended to limit the scope of the concept of the present application in any way, but to explain the concept of the present application for those skilled in the art by referring to specific embodiments.

FIG. 1 is a schematic flowchart of an image processing method provided by an embodiment of the application;

2 is a schematic diagram of a user interface of an image processing method provided by an embodiment of the application;

3 is a schematic diagram of image segmentation of the image processing method provided by an embodiment of the application;

FIG. 4 is a schematic diagram of the structure of the image processing device provided by this application.

Detailed ways

The exemplary embodiments will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the application as detailed in the appended claims.

The terms used in the embodiments of the present application are only for the purpose of describing specific embodiments, and are not intended to limit the present application. The singular forms of "a", "said" and "the" used in the embodiments of the present application and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings.

It should be understood that the term "and/or" used in this text is only an association relationship describing the associated objects, indicating that there can be three types of relationships, for example, A and/or B can mean that A alone exists, and both A and A exist at the same time. B, there are three cases of B alone. In addition, the character "/" in this text generally indicates that the associated objects before and after are in an "or" relationship.

It should be understood that the terms "first", "second", "third", "fourth", etc. (if any) in the description and claims of this application and the above-mentioned drawings are used to distinguish similar objects, and It does not have to be used to describe a specific order or sequence.

Depending on the context, the words "if" and "if" as used herein can be interpreted as "when" or "when" or "in response to determination" or "in response to detection". Similarly, depending on the context, the phrase "if determined" or "if detected (statement or event)" can be interpreted as "when determined" or "in response to determination" or "when detected (statement or event) )" or "in response to detection (statement or event)".

It should also be noted that the terms "include", "include" or any other variants thereof are intended to cover non-exclusive inclusion, so that a commodity or system that includes a series of elements not only includes those elements, but also includes those elements that are not explicitly listed. Other elements of, or also include elements inherent to this kind of commodity or system. If there are no more restrictions, the element defined by the sentence "including a..." does not exclude the existence of other identical elements in the commodity or system that includes the element.

In the research, the embodiment of the application reveals that the reason for the poor fusion between the replacement template and the original picture in the prior art is that it is usually difficult to obtain an accurate edge of the sky area when recognizing the sky area in the prior art, or it is understandable Because the edge of the sky area identified in the prior art is inaccurate, resulting in the subsequent processing of the sky area, if the edge exceeds the actual sky area, it may appear that the non-sky area is processed, or if the edge is included in the actual sky area. In the sky area, it may appear that the sky area is not fully processed, resulting in poor fusion of the processed part with the original picture.

Therefore, in the embodiment of the present application, after acquiring the image information to be processed including the sky, the image information to be processed is preprocessed first. Specifically, the image information to be processed is recognized to obtain the image information to be processed. The sky area, the non-sky area, and the sky edge transition zone between the sky area and the non-sky area in the information are then further optimized according to the preprocessing results. Specifically, according to the sky area and non-sky area, As well as the sky edge transition zone, the sky area is re-determined, and the edge of the re-determined sky area is more accurate than the edge of the sky area recognized in the preprocessing, and then the sky area corresponding to the re-determined sky area in the image information to be processed is optimized for the sky Then, in the optimized image obtained, accurate processing of the edge of the sky region can be realized, so the fusion degree of the processed region and the image information to be processed is better, and the visual effect of the adapted optimized image is better.

The image processing method in the embodiment of the present application may be applied to a terminal or a server, and the terminal may include: a mobile phone, a tablet computer, a notebook computer, a desktop computer, and other electronic devices, which is not specifically limited in the embodiment of the present application.

The sky optimization process described in the embodiments of the present application may be to replace the sky of the image information to be processed with a preset template, or to adjust the brightness, contrast, saturation, color, filter, etc. of the sky in the image information to be processed. The embodiments of the present application do not specifically limit this.

Hereinafter, some embodiments of the present application will be described in detail with reference to the accompanying drawings. In the case of no conflict, the following embodiments and features in the embodiments can be combined with each other.

As shown in FIG. 1, FIG. 1 is a schematic flowchart of an image processing method provided by an embodiment of the application. The method may specifically include:

S101: Acquire image information to be processed.

In the embodiment of the present application, the image information to be processed may be an image obtained by the electronic device from the network or locally.

The image information to be processed can also be an image input by the user that the electronic device receives. Illustratively, taking the electronic device as a mobile phone as an example, Figure 2 shows a user interface for receiving the image information to be processed by the user. By clicking a preset area in the user interface and inputting image information to be processed, the mobile phone can obtain the image information to be processed.

In the embodiments of the present application, the sky area contained in the image information to be processed can be determined according to actual application scenarios. For example, if the sky area contained in the image information to be processed is very small, the sky area is subsequently optimized. The optimization effect of the image information to be processed may not be obvious. Therefore, the user may be prompted to input the image information to be processed that contains the sky area occupying a proportion of the image information to be processed that is greater than the set value.

It can be understood that in the embodiment of the present application, as long as the image information to be processed includes the sky, there is no restriction on the sky, and the embodiment of the present application does not specifically limit this.

S102: Determine and segment the sky area of the image information.

Specifically, the image information to be processed may be identified to obtain the sky area, the non-sky area, and the sky edge transition zone between the sky area and the non-sky area in the image information to be processed .

In the embodiments of this application, any image recognition technology can be used to identify the image information to be processed. For example, a number of images containing the sky can be obtained in advance as a sample set, and the sky area and non-sky area of the image samples of the sample set can be obtained. Label, and then train the labeled sample to obtain a sky recognition model, and input the image information to be processed into the sky recognition model to obtain the sky area, non-sky area, and sky area and non-sky area in the image information to be processed The transition zone between the edges of the sky.

It should be noted that the sky edge transition zone is the boundary zone between the sky area and the non-sky area. In recognition, only the sky area and the non-sky area can be identified, and then the sky edge transition zone is obtained according to the boundary between the sky area and the non-sky area. The embodiments of the present application do not specifically limit this.

In a possible implementation, after identifying the sky area and the flying sky area, the boundary between the sky area and the non-sky area may be a narrow strip area, which may be called a dividing line, for example, a certain point of the dividing line may cover One or two pixels, as an example, to further determine the accurate dividing line later, you can include multiple pixels on both sides of the dividing line into the dividing line to obtain a sky edge transition zone that covers more pixels, such as an interface of the sky edge transition zone It can cover about 5-15 pixels, etc.

Exemplarily, as shown in FIG. 3, the sky regions 21, the non-sky regions 22, and the sky edge transition zone 23 can be segmented in FIG. 3. The sky regions can include the sun, white clouds, etc., and the balance outside the sky region can be processed. It can be called non-sky areas.

It can be understood that the sky area and the non-sky area obtained in this step may be inaccurate, so the sky area can be re-determined in the subsequent S103.

S103: Optimize the sky area according to a preset rule to obtain an optimized image.

Specifically, the sky area may be re-determined according to the sky area, the non-sky area, and the sky edge transition zone.

In the embodiment of the present application, any possible form may be used to re-determine the sky area according to the sky area, the non-sky area, and the sky edge transition zone to obtain a sky area that is more accurate than S102 .

Exemplarily, the image content contained in the sky edge transition zone can be re-identified, and after the accurate sky edge line is obtained, the accurate sky area can be re-determined through the accurate sky edge line.

In a possible implementation manner, the re-determining the sky area according to the sky area, the non-sky area, and the sky edge transition zone includes:

Labeling the pixels in the sample image including the sky area and the non-sky area respectively; using AI semantic segmentation technology to train the labeled pixels in the sample image to obtain a sky segmentation AI model; using the sky segmentation AI model Obtain the sky segmentation mask map corresponding to the sky area, the non-sky area, and the sky edge transition zone; refine the sky edge of the sky segmentation mask map to obtain the refined edge The sky is divided into a mask map to obtain the re-determined sky area.

In the embodiment of the present application, when labeling the sample image, the sky area, the non-sky area and the sky edge transition zone of the sample image can be assigned different values, and the labeling accuracy can be fine to the pixel level.

After the labeling process, AI semantic segmentation technology can be used to train the labeled sample images; for example, each pixel in the sample image can be labeled with a category label, such as people, trees, grass, sky, etc., and finally the sky area can be output. AI model of the sky segmentation of the mask map.

Further, the sky segmentation AI model is used to obtain a sky segmentation mask map corresponding to the sky area, the non-sky area, and the sky edge transition zone. Specifically, the mask image may be a binary image composed of 0 and 1. Exemplarily, when a mask is applied in a certain function, the 1 value area is processed, and the masked 0 value area is not included in the calculation. Through mask processing, the sky segmentation mask that needs to be processed can be obtained.

Further, refinement processing is performed on the sky edge of the sky segmentation mask map to obtain the sky segmentation mask map after edge refinement, to obtain the re-determined sky area.

For each pixel point in the sky edge transition zone of the sky segmentation mask, the coordinates of the pixel point are obtained separately, and the coordinates are used as the center point, and the local pixel similarity statistics method is used to calculate the center point. And the pixel points in the adjacent area of the center point perform edge determination to obtain an accurate edge of the sky.

In the example of this application, the coordinates of each pixel in the sky edge transition zone of the sky segmentation mask can be obtained. For each pixel, the coordinates of the pixel can be used as the center point, and the local voting method can be used to determine The center point is the exact edge in the adjacent area. Exemplarily, the similarity between the pixels of the center point and the pixels in the adjacent area of the center point can be calculated. If the similarity is low, the center point can be voted as an edge point, and if the similarity is high, the center point can be voted as the edge point. The central point is voted as a non-edge point. After the central point and the pixels in the adjacent area of the central point are voted separately, the pixel with a higher score can be determined as the edge pixel to obtain an accurate edge.

Further, sky optimization processing is performed on the re-determined sky region corresponding part in the image information to be processed to obtain an optimized image.

In the embodiment of the present application, after the re-determined accurate sky area is obtained, the sky can be optimized according to actual requirements to obtain an optimized image.

In a possible implementation manner, the performing sky optimization processing on the re-determined sky region corresponding part of the image information to be processed includes: adopting a preset sky template and re-determining the image information to be processed The corresponding part of the sky area is replaced.

In this embodiment of the application, the preset sky template may be a predefined standard sky template, such as a sunny white cloud sky template, a dark cloud sky template, etc. It is understandable that the sky area in the image information to be processed may not reach the preset sky template Therefore, a preset sky template is used to replace the portion corresponding to the re-determined sky area in the image information to be processed, and an image with a better sky effect can be obtained.

In the embodiment of the present application, the position of the preset sky template can be adjusted according to the position of the re-determined sky area, so that the preset sky template is exactly located in the re-determined sky area, thereby avoiding the replacement effect caused by the improper location of the preset sky template Poor problem.

In the embodiment of the present application, the size of the preset sky template can be adjusted according to the size of the re-determined sky area, so that the size of the preset sky template matches the size of the re-determined sky area, thereby avoiding the improper size of the preset sky template The resulting picture violation problem.

In another possible implementation manner, the performing sky optimization processing on the portion corresponding to the re-determined sky area in the image information to be processed includes:

Identify the scene corresponding to the image information to be processed, and search for and acquire the adjustment mode corresponding to the scene; according to the adjustment mode, perform sky optimization processing on the re-determined sky region corresponding part of the image information to be processed.

In the embodiment of the present application, first, an algorithm can be used to identify the scene corresponding to the image information to be processed, for example, different scenes such as sunny days, cloudy days, night scenes, sunsets, grasslands, etc. can be identified. Then you can find the response adjustment mode, and realize the customized sky optimization processing.

Exemplarily, the following lists several automatic adjustments to different sky image styles according to different scenes. The adjustment method is not limited to smart superimposing filters, or adjusting the brightness, contrast, saturation, color, and filters of the sky area. Wait.

If it is detected as a sunny scene, adjust the color of the sky area to blue sky with less clouds, and enhance the sun in the picture. Or intelligently overlay a sunny sky filter in the sky area.

If it is detected as a cloudy scene, adjust the color of the sky area to dark and cloudy, and superimpose different cloud types to form different special effects. Or intelligently overlay a cloudy sky filter in the sky area.

If it is detected as a sunset scene, adjust the sky area to the corresponding sunset color, and enhance the saturation of the image. Or intelligently overlay a sunset sky filter in the sky area.

If it is detected as a night scene, add special effects such as aurora or starlight in the sky area, and enhance the moon in the sky. Or intelligently overlay a night sky filter in the sky area.

If it is detected as a grassland scene, adjust the sky area to blue sky and white clouds, and increase the contrast of the image to make the image more transparent. Or intelligently overlay a grassland sky filter in the sky area.

It can be understood that in specific implementation, it is also possible to replace the sky template matching the scene according to different scenes, or perform different intensities and different types of image enhancements on the segmented sky area and non-sky area, so as to realize the image to be processed. Information optimization. The comparison of the embodiments of this application is not specifically limited.

In summary, an image processing method and device are provided in the embodiments of the present application, and it is found that the reason for the poor fusion between the replacement template and the original picture in the prior art is that in the prior art, when recognizing the sky area, It is usually difficult to obtain an accurate edge of the sky area. As a result, in the subsequent processing of the sky area, the non-sky area may be processed, or the sky area may not be fully processed, resulting in poor fusion of the processed part with the original image. Therefore, in the embodiment of the present application, after acquiring the image information to be processed including the sky, the image information to be processed is preprocessed first. Specifically, the image information to be processed is recognized to obtain the image information to be processed. The sky area, the non-sky area, and the sky edge transition zone between the sky area and the non-sky area in the information are then further optimized according to the preprocessing results. Specifically, according to the sky area and non-sky area, As well as the sky edge transition zone, the sky area is re-determined, and the edge of the re-determined sky area is more accurate than the edge of the sky area recognized in the preprocessing, and then the sky area corresponding to the re-determined sky area in the image information to be processed is optimized for the sky Then, in the optimized image obtained, accurate processing of the edge of the sky region can be realized, so the fusion degree of the processed region and the image information to be processed is better, and the visual effect of the adapted optimized image is better.

FIG. 4 is a schematic structural diagram of an embodiment of an image processing apparatus provided by this application. As shown in Figure 4, the image processing apparatus provided in this embodiment includes:

The segmentation module 31 is used to determine and segment the sky area of the image information to be processed;

The optimization module 32 is configured to optimize processing of the sky area according to preset rules to obtain an optimized image.

Optionally, the optimization module is specifically used for:

For each pixel point in the sky edge transition zone of the sky segmentation mask, the coordinates of the pixel point are obtained separately, and the coordinates are used as the center point, and the local pixel similarity statistics method is used to calculate the center point. And the pixel points in the adjacent area of the center point perform edge determination.

Optionally, it also includes:

The image processing apparatus provided in each embodiment of the present application can be used to execute the method shown in each corresponding embodiment described above, and the implementation manner and principle are the same, and details are not repeated here.

An embodiment of the present application further provides an electronic device, including: a processor, a memory, and a computer program; wherein the computer program is stored in the memory and is configured to be executed by the processor, and the computer program It includes instructions for executing the method as described in any of the foregoing embodiments.

An embodiment of the present application also provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed, the method as described in any one of the foregoing embodiments is implemented.

The reader should understand that, in the description of this specification, reference to the description of the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples", etc. means to combine the embodiments or examples. The described specific features, structures, or characteristics are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above-mentioned terms are not necessarily directed to the same embodiment or example. Moreover, the described specific features, structures or characteristics can be combined in any one or more embodiments or examples in an appropriate manner. In addition, those skilled in the art can combine and combine the different embodiments or examples and the features of the different embodiments or examples described in this specification without contradicting each other.

Those skilled in the art can clearly understand that for the convenience and conciseness of description, the specific working process of the device and unit described above can refer to the corresponding process in the foregoing method embodiment, which will not be repeated here.

In the several embodiments provided in this application, it should be understood that the disclosed device and method may be implemented in other ways. For example, the device embodiments described above are only illustrative. For example, the division of units is only a logical function division, and there may be other divisions in actual implementation, for example, multiple units or components can be combined or integrated. To another system, or some features can be ignored, or not implemented.

The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments of the present application.

In addition, the functional units in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or software functional unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer readable storage medium. Based on this understanding, the technical solution of this application is essentially or the part that contributes to the existing technology, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium. It includes several instructions to make a computer device (which may be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods in the various embodiments of the present application. The aforementioned storage media include: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), magnetic disks or optical disks and other media that can store program codes. .

It should also be understood that in each embodiment of the present application, the size of the sequence number of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not correspond to the difference in the embodiments of the present application. The implementation process constitutes any limitation.

The above are only specific implementations of this application, but the scope of protection of this application is not limited to this. Any person skilled in the art can easily think of various equivalent modifications or changes within the technical scope disclosed in this application. Replacement, these modifications or replacements shall be covered within the scope of protection of this application. Therefore, the protection scope of this application shall be subject to the protection scope of the claims.

Claims

An image processing method, which includes:

Obtain the image information to be processed;

Judging and segmenting the sky area of the image information;

The sky area is optimized according to preset rules to obtain an optimized image.
The method according to claim 1, wherein said determining and segmenting the sky area of the image information comprises:

Performing recognition processing on the image information to be processed to obtain the sky area, the non-sky area, and the sky edge transition zone between the sky area and the non-sky area in the image information to be processed; according to the The sky area, the non-sky area, and the sky edge transition zone are re-determined for the sky area.
The method according to claim 2, wherein the optimizing processing of the sky area according to a preset rule to obtain an optimized image comprises:

Perform sky optimization processing on the re-determined sky region corresponding part in the image information to be processed to obtain an optimized image.
The method according to claim 2, wherein the re-determining the sky area according to the sky area, the non-sky area, and the sky edge transition zone comprises:

Label the pixels in the sample image including the sky area and the non-sky area respectively;

The AI semantic segmentation technology is used to train the labeled sample images to obtain the sky segmentation AI model.
The method according to claim 4, further comprising:

Using the sky segmentation AI model to obtain a sky segmentation mask map corresponding to the sky area, the non-sky area, and the sky edge transition zone;

Refinement processing is performed on the sky edge of the sky segmentation mask map to obtain a sky segmentation mask map after edge refinement, so as to obtain the re-determined sky area.
The method according to claim 5, wherein the thinning processing of the sky edge of the sky segmentation mask map comprises:

For each pixel point in the sky edge transition zone of the sky segmentation mask, the coordinates of the pixel point are obtained separately, and the coordinates are taken as the center point, and the local pixel similarity statistics method is used to calculate the center point. And the pixel points in the adjacent area of the center point perform edge determination.
The method according to claim 1, wherein the performing sky optimization processing on the re-determined sky region corresponding part in the image information to be processed includes:

The preset sky template is used to perform replacement processing on the portion corresponding to the re-determined sky area in the image information to be processed.
8. The method according to claim 7, wherein said using a preset sky template to perform replacement processing on the portion corresponding to the re-determined sky area in the image information to be processed comprises:

Adjust the position and/or size of the preset sky template according to the position and/or size of the re-determined sky area, and replace the adjusted sky template with the image information to be processed and re-determine The corresponding part of the sky area.
The method according to claim 1, wherein the performing sky optimization processing on the re-determined sky region corresponding part in the image information to be processed includes:

Identifying the scene corresponding to the image information to be processed, and searching for and acquiring the adjustment mode corresponding to the scene;

According to the adjustment mode, sky optimization processing is performed on the portion corresponding to the re-determined sky region in the image information to be processed.
An image processing device, which includes:

The segmentation module is used to determine and segment the sky area of the image information to be processed;

The optimization module is used for optimizing processing of the sky area according to preset rules to obtain an optimized image.
The device according to claim 10, wherein the optimization module is specifically configured to:

Obtaining a sky segmentation mask map corresponding to the sky area, the non-sky area, and the sky edge transition zone;

For each pixel point in the sky edge transition zone of the sky segmentation mask, the coordinates of the pixel point are obtained separately, and the coordinates are taken as the center point, and the local pixel similarity statistics method is used to calculate the center point. And the pixel points in the adjacent area of the center point perform edge determination.
The device according to claim 10 or 11, further comprising:

The processing module is configured to adjust the sky area according to template replacement of the sky area or according to the scene corresponding to the image information to be processed.
An electronic device, including:

A processor, a memory, and a computer program; wherein the computer program is stored in the memory and is configured to be executed by the processor, and the computer program includes a computer program for executing any one of claims 1 Method of instruction.
A computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed, the method according to any one of claim 1 is realized.