CN110910400A

CN110910400A - Image processing method, image processing device, storage medium and electronic equipment

Info

Publication number: CN110910400A
Application number: CN201911040153.8A
Authority: CN
Inventors: 朱理; 李文哲; 谢存煌
Original assignee: Beijing Sankuai Online Technology Co Ltd
Current assignee: Beijing Sankuai Online Technology Co Ltd
Priority date: 2019-10-29
Filing date: 2019-10-29
Publication date: 2020-03-24

Abstract

The disclosure relates to an image processing method, an image processing device, a storage medium and an electronic device, which are used for improving the accuracy of image area division and further obtaining a better image area replacement result. The method comprises the following steps: performing semantic segmentation processing on an original image, and overlapping images before and after the semantic segmentation to obtain a target image; based on the color distance, performing binary clustering on target pixel points corresponding to a first region and all pixel points corresponding to a second region in the target image to obtain two pixel clusters; determining a pixel cluster corresponding to the first region and a pixel cluster corresponding to the second region according to the color distance between the clustering centers of the two pixel clusters and the pixel mean value of the target region; and dividing the pixel points in the target image according to the two pixel clusters so as to remove the pixel points belonging to the second region from the pixel points included in the first region.

Description

Image processing method, image processing device, storage medium and electronic equipment

Technical Field

The present disclosure relates to image processing technologies, and in particular, to an image processing method, an apparatus, a storage medium, and an electronic device.

Background

In daily life, more and more users have demands for beautifying images. For example, in an image captured in cloudy or haze weather, a user usually replaces the sky with a blue sky to improve the beautification degree of the image. In the process of replacing the sky with the blue sky, whether the division of the sky area is accurate or not directly influences the subsequent replacement effect.

In the related technology, a semantic segmentation technology is mainly used for segmenting sky pixel points in an image, and then the pixels are replaced by corresponding pixels of a blue-sky image, so that sky replacement is realized. However, the method cannot achieve the leaf level, and the sky area near the leaf branch in the image cannot be replaced, so that the replaced sky and the leaves have obvious visual obtrusive feeling, and the beautifying requirement of a user on the image cannot be well met.

Disclosure of Invention

The present disclosure provides an image processing method, an image processing apparatus, a storage medium, and an electronic device, so as to improve accuracy of image region division and obtain a better image region replacement result.

In order to achieve the above object, in a first aspect, the present disclosure provides an image processing method, the method comprising:

performing semantic segmentation processing on an original image, and overlapping images before and after the semantic segmentation to obtain a target image;

performing binary clustering on a target pixel point corresponding to a first region and all pixel points corresponding to a second region in a target image based on color distance to obtain two pixel clusters, wherein the pixel points belonging to the second region exist in the first region;

determining a pixel class cluster corresponding to the first region and a pixel class cluster corresponding to the second region according to the color distance between the clustering centers of the two pixel class clusters and the pixel mean value of a target region, wherein the target region is the first region or the second region;

and dividing the pixel points in the target image according to the two pixel clusters so as to remove the pixel points belonging to the second region from the pixel points included in the first region.

Optionally, the method further comprises:

performing image expansion processing on the semantically segmented image aiming at the second region;

and determining pixel points belonging to the first region in all different pixel points between the image subjected to the image expansion processing and the image subjected to the semantic segmentation as the target pixel points.

Optionally, a difference between the number of the target pixel points and the number of the pixel points in the second region is within a preset difference range.

Optionally, the target image further includes a third region excluding the first region and the second region, and the method further includes:

determining a spatial distance between each pixel point in the pixel cluster corresponding to the first region and each pixel point in the non-second region;

determining a non-second region pixel point closest to the pixel point according to the space distance;

in the image after semantic segmentation, if the pixel value of the pixel point in the non-second region is the same as the pixel value in the third region, determining that the pixel point is the pixel point in the third region, so as to remove the pixel point in the pixel cluster corresponding to the first region.

Optionally, the determining, for each pixel point in the pixel class cluster corresponding to the first region, a spatial distance between the pixel point and each pixel point in the non-second region includes:

and determining the spatial distance between the pixel point and each pixel point in the image region corresponding to the target pixel point aiming at each pixel point in the pixel cluster corresponding to the first region.

Optionally, the target pixel points are all pixel points in the first region.

Optionally, the performing binary clustering on the target pixel point in the first region and all pixel points in the second region in the target image based on the color distance to obtain two pixel clusters includes:

determining the color distance between the target pixel point in the first region and each pixel point in all the pixel points in the second region according to the pixel values of the target pixel point in the first region and all the pixel points in the second region under an LAB image channel;

and performing dichotomous clustering on the target pixel points in the first region and all the pixel points in the second region in the target image based on the color distance to obtain two pixel clusters.

In a second aspect, an embodiment of the present disclosure further provides an image processing apparatus, including:

the preprocessing module is used for performing semantic segmentation processing on the original image and superposing the images before and after the semantic segmentation to obtain a target image;

the clustering module is used for performing binary clustering on a target pixel point corresponding to a first region and all pixel points corresponding to a second region in a target image based on color distance to obtain two pixel clusters, wherein the pixel points belonging to the second region exist in the first region;

a determining module, configured to determine a pixel class cluster corresponding to the first region and a pixel class cluster corresponding to the second region according to color distances between the clustering centers of the two pixel class clusters and a pixel mean value of a target region, where the target region is the first region or the second region;

and the dividing module is used for dividing the pixel points in the target image according to the two pixel cluster types so as to remove the pixel points belonging to the second area from the pixel points included in the first area.

In a third aspect, the present disclosure also provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method of any one of the first aspect.

In a fourth aspect, the present disclosure also provides an electronic device, including:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to implement the steps of the method of any one of the first aspect.

By the technical scheme, the original image can be subjected to semantic segmentation processing, and the images before and after the semantic segmentation are superposed to obtain the target image. Then, clustering can be performed based on the color distance, and target pixel points in the first region and all pixel points in the second region are classified, so that pixel points belonging to the second region are removed from the pixel points included in the first region, and accurate division of image regions is achieved. If the technical scheme is applied to a scene that the sky is replaced by the blue sky, the sky area and the non-sky area can be accurately distinguished, the problem that the sky area near the leaf branch in the image cannot be replaced is solved, transition between the replaced sky and the leaves is natural, and the beautifying requirement of a user on the image is better met.

Additional features and advantages of the disclosure will be set forth in the detailed description which follows.

Drawings

The accompanying drawings, which are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the disclosure without limiting the disclosure. In the drawings:

FIG. 1 is an original image to be processed;

FIG. 2 is an image obtained by processing the image shown in FIG. 1 and performing image region replacement according to a semantic segmentation technique in the related art;

FIG. 3 is a flow chart illustrating a method of image processing according to an exemplary embodiment of the present disclosure;

FIG. 4 is an image obtained by semantically segmenting the image shown in FIG. 1;

FIG. 5 is an enlarged view of a portion of FIG. 4;

FIG. 6 is an enlarged view of a partial region corresponding to FIG. 5 after image processing of FIG. 4 according to an image processing method in another exemplary embodiment of the present disclosure;

FIG. 7 is an image resulting from image processing of FIG. 1 according to an image processing method in another exemplary embodiment of the present disclosure;

FIG. 8 is an image resulting from image processing of FIG. 1 according to an image processing method in another exemplary embodiment of the present disclosure;

FIG. 9 is a flowchart illustrating an image processing method according to another exemplary embodiment of the present disclosure;

fig. 10 is a block diagram illustrating an image processing apparatus according to an exemplary embodiment of the present disclosure;

FIG. 11 is a block diagram illustrating an electronic device in accordance with an exemplary embodiment of the present disclosure;

fig. 12 is a block diagram illustrating an electronic device according to another exemplary embodiment of the present disclosure.

Detailed Description

The following detailed description of specific embodiments of the present disclosure is provided in connection with the accompanying drawings. It should be understood that the detailed description and specific examples, while indicating the present disclosure, are given by way of illustration and explanation only, not limitation.

In the related technology, a semantic segmentation technology is mainly used for segmenting sky pixel points in an image, and then the pixels are replaced by corresponding pixels of a blue-sky image, so that sky replacement is realized. However, the method cannot achieve the leaf level, and the sky area near the leaf branch in the image cannot be replaced, so that the replaced sky and the leaves have obvious visual obtrusive feeling, and the beautifying requirement of a user on the image cannot be well met. For example, fig. 1 is an original image, and fig. 2 is an image obtained by processing the image shown in fig. 1 according to a semantic segmentation technique in the related art and performing image region replacement. Comparing fig. 1 and fig. 2, it can be seen that details are not processed in the semantically segmented image, the tree is segmented as a whole, a trunk, branches, leaves, and the like are unified as the tree without distinction, and the segmentation at the leaf level is not achieved, and partial pixel points at the edge of the tree region (for example, the pixel points at the upper right corner shown in fig. 1) are wrongly divided into the sky region in fig. 2, so that an obvious visual obtrusive feeling exists between the replaced sky and the leaves, and the requirement of the user for beautifying the image cannot be well met.

In view of this, embodiments of the present disclosure provide an image processing method, an image processing apparatus, a storage medium, and an electronic device, so as to solve the problems in the related art, and improve the accuracy of dividing an image area, thereby achieving replacement of the image area more accurately and better meeting the requirement of users for beautifying the image.

First, it is explained that the image processing method in the embodiment of the present disclosure may be applied to a client having an image processing function, such as a camera, a video camera, a computer, a mobile phone, and a Pad, or may also be applied to a server, which is not limited in the embodiment of the present disclosure. If the image processing method is applied to a server, the server may first receive an image sent by a client, and then process the received image according to the image processing method of the embodiment of the present disclosure. It should be understood that the image in the embodiment of the present disclosure may be a picture that is captured by a camera of the client and then stored in the client, may also be a picture that is downloaded and stored from a network by the client, may also be a certain frame of picture captured from a video stored in the client, and the like, which is not limited in the embodiment of the present disclosure.

Fig. 3 is a flowchart illustrating an image processing method according to an exemplary embodiment of the present disclosure. Referring to fig. 3, the image processing method may include:

step S301, performing semantic segmentation processing on the original image, and superposing the images before and after the semantic segmentation to obtain a target image.

Step S302, based on the color distance, performing dichotomous clustering on a target pixel point corresponding to a first region and all pixel points corresponding to a second region in the target image to obtain two pixel clusters, wherein the pixel points belonging to the second region exist in the first region.

Step S303, determining a pixel cluster corresponding to the first region and a pixel cluster corresponding to the second region according to the color distance between the clustering centers of the two pixel clusters and the pixel mean value of the target region. Wherein the target area is a first area or a second area.

Step S304, dividing the pixel points in the target image according to the two pixel cluster types so as to remove the pixel points belonging to the second area from the pixel points included in the first area.

In order to make those skilled in the art understand the technical solutions provided by the embodiments of the present disclosure, the following detailed descriptions of the above steps are provided.

In step S301, the original image shown in fig. 1 may be processed by a semantic segmentation technique to obtain an initial region division result. For example, the image may be first pyramid pooled such that the image is converted to a fixed size feature vector. If the area to be segmented is larger in the image, for example, the sky area and the tree area shown in fig. 1 are larger in the image, a pyramid scale with a larger receptive field may be selected for processing. After the feature vectors of the images are normalized to be uniform in length and combined into a feature map, the feature vectors before pooling can be obtained through upsampling, and then the feature vectors before and after pooling are fused to obtain a semantic segmentation result as shown in fig. 4. Referring to fig. 4, the original image shown in fig. 1 may be divided into three parts of a sky area a, a tree area B, and other areas C through a semantic segmentation process.

However, comparing fig. 1 and 4, it can be seen that the image after semantic segmentation lacks detail processing. For example, semantic segmentation divides a tree as a whole, and unites a trunk, branches, leaves, and the like as the tree without distinction, and does not achieve the division at the leaf level, and part of pixel points at the edge of the tree region are wrongly divided into sky regions. Therefore, in order to realize finer region division and obtain more accurate image processing results, after the semantic segmentation processing is performed on the original image, the images before and after the semantic segmentation can be overlapped to obtain a target image, so that the subsequent image processing steps can be realized through the target image. For example, the images shown in fig. 1 and 4, respectively, may be superimposed to obtain a target image, and then the subsequent image processing steps may be performed on the target image.

Further, in the subsequent step S302, binary clustering may be performed on the target pixel point in the first region and all pixel points in the second region in the target image based on the color distance, so as to obtain two pixel clusters.

The color distance refers to the difference between the colors of the two pixel points, the larger the color distance is, the larger the color difference between the two pixel points is, otherwise, the smaller the color distance is, the closer the colors of the two pixel points are. In a possible manner, the color distance between each pixel point in the target pixel point in the first region and each pixel point in all the pixel points in the second region can be determined according to the pixel values of the target pixel point in the first region and all the pixel points in the second region under the LAB image channel. And then performing dichotomous clustering on the target pixel point in the first region and all the pixel points in the second region based on the color distance to obtain two pixel clusters.

That is to say, in the embodiment of the present disclosure, the color distance between the pixels can be obtained through the pixel values of the pixels under each image channel by directly using different characteristics of the pixels expressed in the color space. Because the LAB image channel respectively represents three image channels of lightness, red-green color difference and blue-yellow color difference, and is based on the perception of human eyes on colors, the color distance between the pixels is obtained through the pixel values of the pixels under the LAB image channel to carry out binary clustering, which is more beneficial to the fine segmentation of leaf holes, thereby obtaining more accurate image processing results.

For example, the target image may be an image obtained by superimposing the images shown in fig. 1 and fig. 4, respectively, and accordingly, the first region may be a sky region in the target image, and the second region may be a tree region in the target image. And then performing dichotomous clustering on target pixel points corresponding to the sky area and all pixel points in the tree area based on the color distance.

In one possible approach, the target pixel may be all pixels in the first region.

Referring to fig. 1 and 4, after semantic segmentation processing, the thin and long leaves in the tree region (e.g., the part of the leaves shown at the upper right of fig. 1) are misclassified to the sky region in fig. 4. In the embodiment of the present disclosure, in order to avoid the above situation that the slender sparse leaves at the ends of the leaves, the bald slender branches, and the like are classified into other regions by mistake, the target pixel points may be determined as all the pixel points of the first region to perform color clustering, so as to implement division of the image region. For example, in the above example, the first region is a sky region, and the second region is a tree region, all pixel points in the sky region may be used as target pixel points, and then color clustering may be performed according to all pixel points in the sky region and all pixel points in the tree region, so as to implement image region division. For example, fig. 5 is a partial enlarged view of fig. 4, and fig. 6 is a partial enlarged view corresponding to fig. 5 after image processing is performed on fig. 4 in the clustering manner described above. Referring to fig. 5 and 6, after the image area is divided in the above manner, the slender and sparse leaves and branches in the tree area are accurately divided into the tree area, so that a more accurate image processing result can be obtained.

In another possible manner, the target pixel point may also be determined by:

firstly, aiming at the second area, the image expansion processing is carried out on the semantically segmented image. And then, determining pixel points belonging to the first region in all different pixel points between the image subjected to the image expansion processing and the image subjected to the semantic segmentation as target pixel points. In this case, the target pixel may be a pixel around the second region that belongs to the first region.

For example, as shown in fig. 4, the semantically segmented image is obtained by taking a first region as a sky region a and a second region as a tree region B, and then performing image expansion processing on the image according to a preset kernel by taking the tree region B as a foreground and taking all other regions except the tree region B as backgrounds, that is, the sky region a and the other regions C as backgrounds. And then, determining pixel points belonging to the sky area A in all different pixel points between the image subjected to the image expansion processing and the image subjected to the semantic segmentation as target pixel points.

The preset kernel may be a template or a mask for image expansion processing, and may be set according to an actual situation, which is not limited in this disclosure. The specific process of the image expansion process is similar to that in the related art, and is not described herein again. It should be understood that by determining the kernels to be different values, different image expansion processing results can be obtained, so that different numbers of target pixel points can be obtained.

In a possible manner, in order to avoid too few selected target pixels, clustering accuracy is improved, so that a more accurate region division result is obtained, and a difference value between the number of the target pixels and the number of pixels in the first region can be within a preset difference value range. In this case, the kernel used for the image expansion processing may be a kernel that makes a difference between the number of target pixel points and the number of pixel points in the first region be within a preset difference range.

The preset difference range may be set according to actual conditions, and is not limited in the embodiment of the present disclosure. It should be understood that the smaller the preset difference range is set, the closer the number of the target pixel point and the pixel point in the first region is, otherwise, the larger the difference between the number of the target pixel point and the pixel point in the first region is.

For example, an image after the semantic segmentation is checked may be randomly determined to perform image expansion processing, and if the difference between the number of target pixels and the number of pixels in the first region is not within the preset difference range after the image expansion processing, the value of the kernel may be adjusted to perform the image expansion processing again until the difference between the number of target pixels and the number of pixels in the first region is within the preset difference range. Specifically, if the number of target pixel points is less than the difference between the numbers of pixel points in the first region, the kernel value may be increased, and otherwise, the kernel value may be decreased.

After the target pixel point is determined, the target pixel point in the first region and all the pixel points in the second region can be used as a pixel set to perform binary clustering to obtain two pixel clusters. Because the clustering is unsupervised, the clustering result is only to separate the pixel points of the first region and the pixel points of the second region into two types, but it cannot be determined which pixel cluster the pixel points of the first region or the second region specifically correspond to. Therefore, in order to determine the pixel class cluster specifically corresponding to the pixel point of the first region or the second region, the pixel class cluster corresponding to the first region and the pixel class cluster corresponding to the second region may be determined according to the color distance between the cluster center of the two pixel class clusters and the pixel mean value of the target region. The target area may be the first area or the second area.

Taking the target area as the first area as an example, a first target cluster center with a smaller color distance from the pixel mean of the first area and a second target cluster center with a larger color distance from the pixel mean of the first area may be determined among the cluster centers of the two pixel clusters. And then determining all pixel points included in the pixel cluster corresponding to the first target clustering center as pixel points in the first region, and determining all pixel points included in the pixel cluster corresponding to the second target clustering center as pixel points in the second region.

It should be understood that, in other possible manners, the pixel mean of all the pixels in the second region may be determined first, and then the color distance between each cluster center and the pixel mean is calculated, where a cluster center with a smaller color distance corresponds to the second region, and a cluster center with a larger color distance corresponds to the first region. In this way, the image area to which the pixel class cluster specifically corresponds can also be determined. When the present disclosure is implemented, a user may select a target area as the first area or the second area by himself, which is not limited in the embodiments of the present disclosure.

After determining the pixel clusters corresponding to the first region and the second region respectively, dividing the pixel points of the target image by using the pixel clusters so as to remove the pixel points belonging to the second region from the pixel points included in the first region.

For example, the image processing procedure is performed with the image obtained by superimposing the images shown in fig. 1 and 4 as a target image, with the sky area in the target image being a first area, the tree area being a second area, and the sky area (first area) being a target area, so that the area division result shown in fig. 7 can be obtained. Referring to fig. 7, compared with the image processing result after semantic segmentation shown in fig. 1, part of pixel points at the edge of the tree region in the image shown in fig. 7 are accurately divided into the tree region, and the region division result is finer, so that a more accurate image processing result can be obtained.

In other possible cases, the target image may further include a third region excluding the first region and the second region, and accordingly, a spatial distance between each pixel point in the pixel cluster corresponding to the first region and each pixel point in the non-second region may be determined. And then, according to the space distance, determining a non-second region pixel point closest to the pixel point. And finally, in the image after semantic segmentation, if the pixel value of the pixel point in the non-second region is the same as the pixel value in the third region, determining that the pixel point is the pixel point in the third region, and removing the pixel point from the pixel cluster corresponding to the first region. The spatial distance may be an euclidean distance between pixels, or may be a distance between pixels on a user-defined coordinate axis, and the like, which is not limited in the embodiment of the present disclosure.

Referring to fig. 7, leaf hole pixels in the tree region are already separated from the tree region (a blank portion in the tree region is the leaf hole pixels), but since the background of the tree region includes not only the sky region but also the building region, these hole pixels may not all belong to the sky region. For more precise area division, the distance between each hole pixel and the non-tree area can be calculated one by one based on the space distance, and the non-tree pixel point with the nearest hole pixel distance is found. When the pixel value of the non-tree pixel point is the same as that of the sky area, the corresponding hole pixel point is marked as sky, otherwise, the hole pixel point is marked as non-sky.

It should be understood that the image after semantic segmentation is a grayscale image, and therefore in the process of performing the above processing based on the image after semantic segmentation, in the same image region in the image, for example, in any one of the first region, the second region, and the third region, the pixel values of the pixels are the same, so that the pixel values of the pixels in the non-second region can be compared with the pixel values of the third region to determine whether the pixels in the non-second region belong to the third region, and then corresponding pixels are removed from the pixel cluster corresponding to the first region, so as to obtain a more refined region division result.

Referring to fig. 8, a part of the leaf hole pixels in the tree region are divided into non-sky regions, i.e., the leaf hole pixels shown as black in fig. 8. Compared with the image processing result of fig. 7, the accuracy of region division is further improved, so that a more accurate pixel point replacement result can be obtained.

Further, in order to reduce the complexity of calculation, pixel points around the second area can be selected to perform the calculation process of the spatial distance. Specifically, the spatial distance between the pixel point and each pixel point in the image region corresponding to the target pixel point may be determined for each pixel point in the pixel class cluster corresponding to the first region. It should be understood that, in the embodiment of the present disclosure, the calculation process of the spatial distance between the pixels is similar to that in the related art, and is not described herein again.

The image processing method of the present disclosure is explained below by another exemplary embodiment. Referring to fig. 9, the method may include the steps of:

step S901, performing semantic segmentation processing on the original image, and superimposing the images before and after the semantic segmentation to obtain a target image.

In step S902, the semantic-segmented image is subjected to image expansion processing for the second region.

Step S903, determining pixel points belonging to the first area in all different pixel points between the image after the image expansion processing and the image after the semantic segmentation as target pixel points. And the difference value between the number of the target pixel points and the number of the pixel points in the second area is within a preset range.

Step S904, determining a color distance between the target pixel point in the first region and each of all pixel points in the second region according to the pixel values of the target pixel point in the first region and all pixel points in the second region under the LAB image channel, respectively.

Step S905, based on the color distance, performing dichotomous clustering on the target pixel points in the first region and all the pixel points in the second region in the target image to obtain two pixel clusters. And the first area has pixel points belonging to the second area.

Step S906, determining a pixel cluster corresponding to the first area and a pixel cluster corresponding to the second area according to the color distance between the clustering centers of the two pixel clusters and the pixel mean value of the target area. Wherein the target area is a first area or a second area.

Step S907, according to the two pixel clusters, dividing the pixel points in the target image to remove the pixel points belonging to the second region from the pixel points included in the first region.

Step S908 is to determine the target pixel points as all pixel points in the first region, and execute steps S904 to S907.

In step S909, for each pixel point in the pixel cluster corresponding to the first region, a spatial distance between the pixel point and each pixel point in the image region corresponding to the target pixel point is determined.

In step S910, according to the spatial distance, a non-second area pixel point closest to the pixel point is determined.

Step S911, in the image after semantic segmentation, if the pixel value of the pixel point in the non-second region is the same as the pixel value in the third region, it is determined that the pixel point is the pixel point in the third region, so as to remove the pixel point in the pixel cluster corresponding to the first region.

The detailed description of the above steps is given above for illustrative purposes, and will not be repeated here. It will also be appreciated that for simplicity of explanation, the above-described method embodiments are all presented as a series of acts or combination of acts, but those skilled in the art will recognize that the present disclosure is not limited by the order of acts or combination of acts described above. Further, those skilled in the art will also appreciate that the embodiments described above are preferred embodiments and that the steps involved are not necessarily required for the present disclosure.

After the image is processed by the image processing mode, the accurate division result of the first region and the second region can be obtained, so that when pixel point replacement is carried out on the first region or the second region, a more accurate pixel point replacement result can be obtained. For example, the original image is the image shown in fig. 1, the first region is a sky region, and the second region is a tree region, so that the result of dividing the sky region and the tree region shown in fig. 8 can be obtained. In this case, if the operation of replacing the sky area with the blue sky is performed, according to the sky area shown in fig. 8, the pixel points corresponding to the sky area in the image and the blue sky template image shown in fig. 1 may be determined, and then the pixel points corresponding to the sky area in the blue sky template image in fig. 1 may be replaced with the pixel points corresponding to the sky area in the blue sky template image, respectively. Compared with the mode in the prior art, the mode disclosed by the invention can improve the image processing efficiency, can also avoid replacing pixel points close to the sky color in a non-sky area with blue-sky pixel points, and better meets the beautifying requirement of a user on the image.

Based on the same inventive concept, referring to fig. 10, an embodiment of the present disclosure further provides an image processing apparatus 1000. The image processing device can be a part or all of an electronic device through software, hardware or a combination of the software and the hardware, and comprises:

the preprocessing module 1001 is configured to perform semantic segmentation processing on an original image, and superimpose images before and after the semantic segmentation to obtain a target image;

a clustering module 1002, configured to perform binary clustering on a target pixel point corresponding to a first region and all pixel points corresponding to a second region in a target image based on a color distance to obtain two pixel clusters, where a pixel point belonging to the second region exists in the first region;

a determining module 1003, configured to determine, according to color distances between the clustering centers of the two pixel clusters and a pixel mean of a target region, a pixel cluster corresponding to the first region and a pixel cluster corresponding to the second region, where the target region is the first region or the second region;

a dividing module 1004, configured to divide the pixel points in the target image according to the two pixel class clusters, so as to remove the pixel points belonging to the second region from the pixel points included in the first region.

Optionally, the apparatus further comprises:

the expansion processing module is used for carrying out image expansion processing on the semantically segmented image aiming at the second area;

and the first pixel point determining module is used for determining pixel points belonging to the first area in all different pixel points between the image subjected to the image expansion processing and the image subjected to the semantic segmentation as the target pixel points.

Optionally, the target image further includes a third region excluding the first region and the second region, and the apparatus further includes:

the distance determining module is used for determining the spatial distance between each pixel point in the pixel cluster corresponding to the first region and each pixel point in the non-second region;

the second pixel point determining module is used for determining a non-second area pixel point closest to the pixel point according to the space distance;

and a third pixel point determining module, configured to determine, in the semantically segmented image, that a pixel point in the non-second region is a pixel point in the third region if a pixel value of the pixel point is the same as a pixel value in the third region, so as to remove the pixel point from a pixel cluster corresponding to the first region.

Optionally, the distance determining module is configured to:

Optionally, the target pixel points are all pixel points in the first region.

Optionally, the clustering module 1002 is configured to:

With regard to the apparatus in the above-described embodiment, the specific manner in which each module performs the operation has been described in detail in the embodiment related to the method, and will not be elaborated here.

Based on the same inventive concept, an embodiment of the present disclosure further provides an electronic device, including:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to implement the steps of any of the image processing methods described above.

In one possible approach, a block diagram of the electronic device is shown in FIG. 11. Referring to fig. 11, the electronic device 1100 may include: a processor 1101, a memory 1102. The electronic device 1100 may also include one or more of a multimedia component 1103, an input/output (I/O) interface 1104, and a communications component 1105.

The processor 1101 is configured to control the overall operation of the electronic device 1100, so as to complete all or part of the steps in the image processing method. The memory 1102 is used to store various types of data to support operation at the electronic device 1100, such as instructions for any application or method operating on the electronic device 1100, as well as application-related data, such as contact data, messaging, images, audio, video, and so forth. The Memory 1102 may be implemented by any type or combination of volatile and non-volatile Memory devices, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read-Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic Memory, flash Memory, magnetic disk, or optical disk. The multimedia components 1103 may include screen and audio components. Wherein the screen may be, for example, a touch screen and the audio component is used for outputting and/or inputting audio signals. For example, the audio component may include a microphone for receiving external audio signals. The received audio signal may further be stored in the memory 1102 or transmitted through the communication component 1105. The audio assembly also includes at least one speaker for outputting audio signals. The I/O interface 1104 provides an interface between the processor 1101 and other interface modules, such as a keyboard, mouse, buttons, and the like. These buttons may be virtual buttons or physical buttons. The communication component 1105 provides for wired or wireless communication between the electronic device 1100 and other devices. Wireless communication, such as Wi-Fi, bluetooth, Near Field Communication (NFC), 2G, 3G, 4G, NB-IOT, eMTC, or other 5G, etc., or a combination of one or more of them, which is not limited herein. The corresponding communication component 1105 can therefore include: Wi-Fi module, Bluetooth module, NFC module, etc.

In an exemplary embodiment, the electronic Device 1100 may be implemented by one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), Field Programmable Gate Arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the image Processing methods described above.

In another exemplary embodiment, there is also provided a computer readable storage medium comprising program instructions which, when executed by a processor, implement the steps of the image processing method described above. For example, the computer readable storage medium may be the memory 1102 described above comprising program instructions executable by the processor 1101 of the electronic device 1100 to perform the image processing method described above.

In another possible approach, the electronic device may also be provided as a server. Referring to fig. 12, the electronic device 1200 includes a processor 1222, which may be one or more in number, and a memory 1232 for storing computer programs executable by the processor 1222. The computer programs stored in memory 1232 may include one or more modules that each correspond to a set of instructions. Further, the processor 1222 may be configured to execute the computer program to perform the image processing method described above.

Additionally, electronic device 1200 may also include a power component 1226 and a communication component 1250, the power component 1226 may be configured to perform power management of the electronic device 1200, and the communication component 1250 may be configured to enable communication, e.g., wired or wireless communication, of the electronic device 1200. In addition, the electronic device 1200 may also include input/output (I/O) interfaces 1258. The electronic device 1200 may operate based on an operating system stored in memory 1232, such as Windows Server, Mac OS XTM, UnixTM, Linux, and the like.

In another exemplary embodiment, there is also provided a computer readable storage medium comprising program instructions which, when executed by a processor, implement the steps of the image processing method described above. For example, the computer readable storage medium may be the memory 1232 described above that includes program instructions executable by the processor 1222 of the electronic device 1200 to perform the image processing method described above.

In another exemplary embodiment, a computer program product is also provided, which comprises a computer program executable by a programmable apparatus, the computer program having code portions for performing the image processing method described above when executed by the programmable apparatus.

The preferred embodiments of the present disclosure are described in detail with reference to the accompanying drawings, however, the present disclosure is not limited to the specific details of the above embodiments, and various simple modifications may be made to the technical solution of the present disclosure within the technical idea of the present disclosure, and these simple modifications all belong to the protection scope of the present disclosure.

It should be noted that, in the foregoing embodiments, various features described in the above embodiments may be combined in any suitable manner, and in order to avoid unnecessary repetition, various combinations that are possible in the present disclosure are not described again.

In addition, any combination of various embodiments of the present disclosure may be made, and the same should be considered as the disclosure of the present disclosure, as long as it does not depart from the spirit of the present disclosure.

Claims

1. An image processing method, characterized in that the method comprises:

2. The method of claim 1, further comprising:

3. The method according to claim 2, wherein the difference between the number of target pixels and the number of pixels in the second region is within a preset difference range.

4. The method of claim 2 or 3, wherein the target image further comprises a third region in addition to the first and second regions, the method further comprising:

5. The method according to claim 4, wherein the determining, for each pixel point in the pixel class cluster corresponding to the first region, a spatial distance between the pixel point and each pixel point in a non-second region comprises:

6. The method of claim 1, wherein the target pixel is all pixels in the first region.

7. The method according to any one of claims 1 to 3, wherein the performing dichotomous clustering on the target pixel point in the first region and all pixel points in the second region in the target image based on the color distance to obtain two pixel clusters comprises:

8. An image processing apparatus, characterized in that the apparatus comprises:

9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 1 to 7.

10. An electronic device, comprising:

a memory having a computer program stored thereon;

a processor for executing the computer program in the memory to carry out the steps of the method of any one of claims 1 to 7.