CN114463367A - Image processing method and device - Google Patents

Image processing method and device Download PDF

Info

Publication number
CN114463367A
CN114463367A CN202210151644.5A CN202210151644A CN114463367A CN 114463367 A CN114463367 A CN 114463367A CN 202210151644 A CN202210151644 A CN 202210151644A CN 114463367 A CN114463367 A CN 114463367A
Authority
CN
China
Prior art keywords
image
pixel
region
edge
area
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202210151644.5A
Other languages
Chinese (zh)
Inventor
易成
罗程
李斌
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202210151644.5A priority Critical patent/CN114463367A/en
Publication of CN114463367A publication Critical patent/CN114463367A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/12—Edge-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/181—Segmentation; Edge detection involving edge growing; involving edge linking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image

Abstract

The embodiment of the application discloses an image processing method, which comprises the steps of determining a main body area, an edge undetermined area and a background area of a target object in an image to be processed according to a classification label obtained by performing semantic segmentation on the image to be processed, setting an identification value of the main body area as a first identification value, and setting an identification value of the background area as a second identification value. And calculating the pixel characteristic similarity between the pixel point of the edge undetermined region and the adjacent pixel point according to the pixel points adjacent to the edge undetermined region in the main body region and the background region, and determining the identification value of the pixel point in the edge undetermined region according to the similarity calculation result, the first identification value and the second identification value. Whether the pixel point belongs to the target object can be determined according to the proximity degree of the identification value, the first identification value and the second identification value, so that the edge part of the target object can be accurately determined, the determination precision of the region where the target object is located in the image to be processed is improved, and the method is suitable for the image background removal scene with high precision requirements.

Description

Image processing method and device
The application provides divisional application to a Chinese patent application with the application number of 201910364702.0, the application date of 2019, 04 and 30, and the invention name of 'an image processing method and device'.
Technical Field
The present application relates to the field of image processing, and in particular, to an image processing method and apparatus.
Background
The image background removal mainly refers to removing a background except for a desired target object in an image, for example, in a portrait image, if the target object is a portrait, the image background removal for the image refers to removing pixel points except for the portrait in the image as the background.
Semantic segmentation techniques based on neural networks can identify classification labels for each pixel from an image, which can identify, for example, a person, cat, bicycle, etc. Therefore, currently, most of common image background removal methods employ an image semantic segmentation technology, and by using the technology, a classification label of each pixel point in an image is identified, and a pixel point (i.e., a background) with a classification label inconsistent with a classification to which a target object belongs is removed.
However, the conventional semantic segmentation technology has a problem that the recognition accuracy of the classification label of the edge pixel of the target object is not high, that is, the determined semantic label of the edge pixel of the target object often does not conform to the actual classification of the object, for example, the classification label of the edge pixel of the portrait does not identify the classification of the portrait, so that the pixel which actually belongs to the portrait is recognized as the background and removed when the background is removed.
Therefore, the edge recognition accuracy of the target object in the image is low in the traditional mode of removing the image background through the traditional semantic segmentation technology, and the method is difficult to be suitable for the image background removing scene with high accuracy requirement.
Disclosure of Invention
In order to solve the technical problem, the application provides an image processing method, which can accurately determine the edge part of a target object, and improve the determination precision of the region where the target object is located in an image to be processed, so that the method can be suitable for an image background removal scene with high precision requirement.
The embodiment of the application discloses the following technical scheme:
in a first aspect, an embodiment of the present application provides an image processing method, where the method includes:
determining a main body area, an edge undetermined area and a background area of a target object in the image to be processed, which correspond to the main body area, the edge undetermined area and the background area of the target object, according to the classification labels of the pixel points obtained by semantic segmentation of the image to be processed; the identification value of the main body area is set as a first identification value, and the identification value of the background area is set as a second identification value;
calculating pixel feature similarity between pixel points in the edge undetermined region and adjacent pixel points according to pixel points in the main region and the background region which are adjacent to the edge undetermined region;
determining the identification value of the pixel point in the edge pending area according to the similarity calculation result of the pixel point in the edge pending area, the first identification value and the second identification value;
and determining the edge contour of the target object in the image to be processed according to the identification values of the pixel points in the edge region to be determined.
In a second aspect, an embodiment of the present application provides an image processing apparatus, which includes a first determining unit, a calculating unit, a second determining unit, and a third determining unit:
the first determining unit is used for determining a main body area, an edge undetermined area and a background area of a target object in the image to be processed according to the classification label of the pixel point obtained by semantic segmentation of the image to be processed; the identification value of the main body area is set as a first identification value, and the identification value of the background area is set as a second identification value;
the calculating unit is used for calculating the pixel characteristic similarity between the adjacent pixel points in the edge undetermined region according to the pixel points adjacent to the edge undetermined region in the main body region and the background region;
the second determining unit is configured to determine an identification value of a pixel point in the edge to-be-determined region according to a similarity calculation result of the pixel point in the edge to-be-determined region, and the first identification value and the second identification value;
the third determining unit is configured to determine an edge contour of the target object in the image to be processed according to the identification value of the pixel point in the edge region to be determined.
In a third aspect, an embodiment of the present application provides an apparatus for image processing, where the apparatus includes a processor and a memory:
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to perform the image processing method of the first aspect according to instructions in the program code.
In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium for storing program code for executing the image processing method according to the first aspect.
According to the technical scheme, for the image to be processed including the target object, the main body area, the edge undetermined area and the background area corresponding to the target object in the image to be processed can be determined according to the classification label obtained by performing semantic segmentation operation on the image to be processed, the identification value of the main body area is set as the first identification value, and the identification value of the background area is set as the second identification value. And according to the pixel points adjacent to the edge undetermined area in the main body area and the background area, calculating the pixel characteristic similarity between the pixel point in the edge undetermined area and the adjacent pixel point, and according to the similarity calculation result between the pixel point in the edge undetermined area and the adjacent pixel point, and the first identification value and the second identification value, determining the identification value of the pixel point in the edge undetermined area. Because the pixel characteristics among the pixel points belonging to the target object are closer to the pixel points in the background area, the similarity calculation result determined by any pixel point in the edge undetermined area can reflect the possibility that the pixel point belongs to the target object or the background. And because the pixel points in the background area and the main body area are respectively provided with different identification values, whether the pixel point belongs to a target object can be determined according to the identification value determined by the pixel point in the edge undetermined area and the proximity degree of the identification value, the first identification value and the second identification value, so that the edge part of the target object can be accurately determined, the determination precision of the area where the target object is located in the image to be processed is improved, and the method and the device can be suitable for the image background removal scene with high precision requirements.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without inventive exercise.
FIG. 1 is a diagram illustrating the effect of recognizing edges by a conventional semantic segmentation technique;
fig. 2 is an exemplary diagram of an application scenario of an image processing method according to an embodiment of the present application;
fig. 3 is a flowchart of an image processing method according to an embodiment of the present application;
fig. 4 is a diagram illustrating an example of the FCN structure provided in an embodiment of the present application;
FIG. 5 is an exemplary diagram of an intermediate processed image provided by an embodiment of the present application;
FIG. 6a is a diagram illustrating an example of a process for generating a certificate photo according to a determined edge profile according to an embodiment of the present application;
fig. 6b is a flowchart of a method for determining an intermediate processed image corresponding to an image to be processed according to an embodiment of the present disclosure;
FIG. 7 is a diagram of a first example of an image provided by an embodiment of the present application;
FIG. 8 is a diagram of a second example of an image provided by an embodiment of the present application;
fig. 9 is a diagram illustrating a second pixel region according to an embodiment of the present disclosure;
FIG. 10 is a diagram illustrating a second pixel region after an expansion process according to an embodiment of the present application;
FIG. 11 is a flowchart illustrating the generation of a certificate photo using social software according to an embodiment of the present disclosure;
FIG. 12 is an exemplary diagram of an interface for generating a certificate photo using social software according to an embodiment of the present application;
fig. 13a is a structural diagram of an image processing apparatus according to an embodiment of the present application;
fig. 13b is a block diagram of an image processing apparatus according to an embodiment of the present application;
fig. 14 is a block diagram of a server according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described below with reference to the accompanying drawings.
The traditional semantic segmentation technology has low identification precision on classification labels of edge pixel points of a target object, namely, the determined semantic labels of the edge pixel points of the target object often do not accord with the actual classification of the object, so that the pixel points which actually belong to the target object are identified as the background and removed together when the background is removed.
For example, as shown in fig. 1, fig. 1 shows an effect diagram of identifying an edge by using a conventional semantic segmentation technique, in which a first column is an original image including different target objects (target objects are, for example, airplanes, birds, chairs, etc.), a second column is an effect diagram of identifying an edge by using a full convolution network-8 s (FCN-8 s), a third column is an effect diagram of identifying an edge by using a hole convolution network (called "scaled Conv"), and a fourth column is an actual case of an edge of a target object. It can be seen from the figure that the difference between the edge identified by the conventional semantic segmentation technology and the actual situation precision of the edge is very obvious, for example, in the case that the target object is a chair, the armrest of the chair recognizes the pixel point of the armrest as the background because the semantic label of the determined pixel point of the edge of the target object does not conform to the actual classification of the object, so that the armrest of the chair is removed as the background when the background is removed.
It should be noted that the image background removal can be applied to many scenes, for example, scenes in which a user generates a personalized photo by removing the background. In this scenario, the certificate photo has a background of a different color, the user may sometimes have a need to temporarily replace the background, or the user may wish to generate the certificate photo based on a personal life photo. The method has the advantages that the user can conveniently generate the certificate photo, the user is prevented from needing to shoot the certificate photo in the photo studio, the original background can be removed in an image background removing mode, and a new background is superposed, so that the certificate photo meeting the user requirement can be quickly obtained.
However, the requirement for the identification accuracy of the edge pixel point of the target object by generating the certificate photo in the image background removal mode is very high, and in order to solve the technical problem and meet the high accuracy requirement, an embodiment of the present application provides an image processing method, which further determines an edge undetermined area on the basis of a semantic segmentation technology, so as to further accurately identify the edge contour of the target object by calculating the pixel feature similarity between the pixel point in the edge undetermined area and an adjacent pixel point, thereby accurately determining the edge contour of the target object in the image to be processed.
The method can be applied to an image processing device, which can be a terminal device, for example, a device with an image processing function, such as an intelligent terminal, a computer, a Personal Digital Assistant (PDA), a tablet computer, and the like.
The image processing device can also be a server, the server is a device for providing image processing service for the terminal device, the terminal device can upload the image to be processed to the server, the server obtains the edge contour of the target object in the image to be processed by using the method provided by the embodiment of the application, and the processing result is returned to the terminal device. The server may be an independent server or a server in a cluster.
In order to facilitate understanding of the technical solution of the present application, the following describes a method for processing an image provided in the embodiment of the present application by taking a terminal device as an example, in combination with an actual application scenario.
Referring to fig. 2, fig. 2 is a diagram illustrating an application scenario of an image processing method, where the scenario includes a terminal device 201, and the terminal device 201 may acquire an image to be processed, where the image to be processed includes a target object, and the target object is an object required by a user in the image to be processed. For example, the image to be processed includes a portrait, a flower, a tree, and the like, and if the user needs the portrait, the portrait is a target object, and the flower, the tree, and the like can be used as a background.
The terminal device 201 performs semantic segmentation on the image to be processed to obtain the classification label of the pixel point, and can determine the corresponding main body region, edge undetermined region and background region of the target object in the image to be processed according to the classification label. The main body area and the background area are accurately determined areas, the main body area is an area where a main body except for an edge portion of the target object is located in the image to be processed, and the background area is an area where a background which needs to be removed relative to the target object is located in the image to be processed. The identification value of the main body area is set as a first identification value, the identification value of the background area is set as a second identification value, and the first identification value and the second identification value are used for determining the identification values of the pixel points in the edge to-be-determined area.
The edge pending area is an area where an edge portion corresponding to the target object is located in the image to be processed, and the edge area may include a pixel point actually belonging to the background in addition to a pixel point actually belonging to the edge portion of the target object. Therefore, the terminal device 201 needs to further identify the pixel points in the edge pending area, determine which pixel points actually belong to the target object, and which pixel points actually belong to the background, so as to accurately determine the edge contour of the target object.
Specifically, the terminal device 201 calculates the pixel feature similarity between the pixel point in the edge undetermined region and the adjacent pixel point according to the pixel point adjacent to the edge undetermined region in the main body region and the background region in the image to be processed, and thus determines the identification value of the pixel point in the edge undetermined region according to the calculation result of the similarity between the pixel point in the edge undetermined region and the adjacent pixel point, the first identification value and the second identification value. In this embodiment, the adjacent pixel points are pixel points which are in the main body region and the background region and are closer to the edge undetermined region, and the adjacent pixel points are not pixel points which are adjacent to a certain pixel point in the edge undetermined region in a strict sense, and the adjacent pixel points of the certain pixel point in the undetermined region can be pixel points around the certain pixel point in the undetermined region.
Because the pixel characteristics among the pixel points belonging to the target object are closer to the pixel points in the background area, the similarity calculation result determined by any pixel point in the edge undetermined area can reflect the possibility that the pixel point belongs to the target object or the background. If the similarity degree of the pixel characteristics between a certain pixel point and an adjacent pixel point in the main body area is higher, the possibility that the pixel point actually belongs to the target object is higher; if the similarity degree of the pixel characteristics between a certain pixel point and an adjacent pixel point in the background area is higher, the possibility that the pixel point actually belongs to the background is higher.
The known body region and background region are accurate and have corresponding identification values (first identification value and second identification value), respectively. And the identification values of the pixel points in the edge to-be-determined region are generated based on the similarity calculation results of the pixel points and the adjacent pixel points in the main region and the background region, and the identification values respectively corresponding to the main region and the background region. Therefore, the degree of proximity between the identification value determined by the pixel point in the edge undetermined area and the first identification value and the second identification value can reflect whether the pixel point in the edge undetermined area actually belongs to the target object or the background. If the identification value of the pixel point is the first identification value or is close to the first identification value, the pixel point actually belongs to the target object, and if the identification value of the pixel point is the second identification value or is close to the second identification value, the pixel point actually belongs to the background.
Next, an image processing method provided by an embodiment of the present application will be described with reference to the drawings.
Referring to fig. 3, fig. 3 shows a flow chart of an image processing method comprising:
s301, according to the classification labels of the pixel points obtained by semantic segmentation of the image to be processed, determining a main body area, an edge undetermined area and a background area of a target object in the image to be processed.
The terminal equipment acquires the image to be processed, and then semantically divides the image to be processed to obtain the classification label of the pixel point. The classification label is used for representing the probability that the pixel point belongs to a certain class object, for example, the classification label represents the probability that the pixel point belongs to a person, a bicycle, a chair, and the like.
The to-be-processed image may be obtained in a plurality of manners, for example, the to-be-processed image may be obtained by a user by taking a picture through a terminal device, or may be obtained by the user by selecting a picture from a local place (album, storage, etc.).
In this embodiment, the FCN may be used to perform semantic segmentation on the image to be processed, and the FCN is shown in fig. 4. The last three layers (corresponding to 4096, 4096, 21 in the upper figure) of the Convolutional Neural network (CNN for short) are replaced by 1 × 1 convolution kernel corresponding to the multi-channel convolution layer (i.e. full convolution) with the same vector length, so as to obtain the FCN. Where 21 is the dimension of the finally generated classification label. Of course, the network used for semantic segmentation may also be other semantic segmentation networks, which is not limited in this embodiment.
The probability matrix with the same resolution as that of the image to be processed can be obtained through semantic segmentation, the probability value of each point in the matrix represents the classification label of the pixel point at the corresponding position in the image to be processed, and when the probability corresponding to a certain classification label of the pixel point reaches a threshold value, the pixel point can be considered to correspond to the category corresponding to the classification label.
In a possible implementation manner, the implementation manner of S201 may be to determine an intermediate processed image corresponding to the image to be processed according to a classification label of a pixel point obtained by semantic segmentation of the image to be processed.
The intermediate processing image is used for identifying a main body area, an edge undetermined area and a background area of a target object in the image to be processed, the identification value of the main body area is set as a first identification value, and the identification value of the background area is set as a second identification value. The first identification value and the second identification value are different identification values, so that the main body area and the background area are distinguished through different identification values.
The identification value may be represented in different forms, for example, the identification value may be represented by a transparency value (alpha, α), and in order to clearly distinguish the main body region from the background region, the first identification value may be represented by a maximum transparency value, such as 1, and the second identification value may be represented by a minimum transparency value, such as 0.
In this case, the determined intermediate processed image may be represented by trimap as shown in fig. 5. In fig. 5, a white area indicates a main body area 501, a gray area indicates an edge pending area 502, and a black area indicates a background area 503.
Of course, the identification value may also be represented by a number without special meaning, for example, the first identification value may be 50, the second identification value may be 0, and so on.
S302, according to the pixel points adjacent to the edge undetermined area in the main body area and the background area, calculating the pixel characteristic similarity between the pixel points in the edge undetermined area and the adjacent pixel points.
And aiming at each pixel point in the edge undetermined area, searching adjacent pixel points in a main body area and a background area in the image to be processed, and calculating the pixel characteristic similarity between the pixel point in the edge undetermined area and the adjacent pixel points. The adjacent pixel points may be, for example, a plurality of pixel points near the periphery of the pixel point of the edge undetermined area. The pixel characteristics identify the relevant characteristics of the content of the pixel points in the image to be processed.
It should be noted that, in order to calculate the similarity of the pixel features, the pixel features may be constructed for each pixel point in the edge undetermined area, and the pixel features may include different information. The color information may be color values of Red (Red, abbreviated as R), Green (Green, abbreviated as G), and Blue (Blue, abbreviated as B) of the pixel point in the image to be processed; the position information can be coordinate values of x and y of the pixel points in the image to be processed.
The color information may be R, G, B color values, and may be color spaces such as Hue, Saturation, and lightness (HSV).
Taking the example that the pixel feature includes a color value of R, G, B and coordinate values of X and y, a pixel feature X (r, g, b, X, y) is constructed, and a kernel function k (i, j) is defined as k (i, j) ═ 1- | X (i) -X (j) |/C, and the similarity calculation result can be represented by a similarity matrix a, where C is the maximum feature distance of X and | | represents a norm using a feature euclidean distance L2. Then, the result of calculating the similarity between the pixel points in the edge undetermined region can be expressed as: a. theijAnd k (i, j), wherein i represents the pixel position of the pixel point in the edge undetermined area, and j represents the pixel position of the adjacent pixel point of the pixel position corresponding to i.
S303, determining the identification value of the pixel point in the edge pending area according to the similarity calculation result of the pixel point in the edge pending area, the first identification value and the second identification value.
It can be understood that the proximity of the identification value of the pixel point in the edge undetermined area to the first identification value and the second identification value can determine whether the pixel point belongs to the target object, so that the edge part of the target object can be accurately determined. And in the process of determining the identification values of the pixel points in the edge to-be-determined region, the identification value proximity degree of the adjacent pixel points is related to the similarity degree of the pixel characteristics of the adjacent pixel points. For example, the higher the similarity degree of the pixel characteristics of the adjacent pixel points is, the closer the identification value of the pixel point in the edge undetermined region is to the identification value of the region where the adjacent pixel point is located.
For example, the first identification value is 1, the second identification value is 0, and if the degree of similarity between a certain pixel point in the edge pending region and the pixel characteristics of the adjacent pixel point in the main body region is higher, the closer the identification value of the pixel point in the edge pending region is to the first identification value 1, for example, the identification value is determined to be 0.9. If the similarity degree of the pixel feature of a certain pixel point in the edge pending region and an adjacent pixel point in the background region is higher, the identification value of the pixel point in the edge pending region is closer to the second identification value 0, for example, the identification value is determined to be 0.1.
In some cases, if the similarity degree of the pixel features of the adjacent pixel points is high enough, for example, the target condition is satisfied, it may be directly determined that the identification value of the pixel point in the edge pending area is the same as the identification value of the adjacent pixel point. The target condition may be that the degree of similarity reaches a first preset threshold.
For example, the first identification value is 1, the second identification value is 0, and if the degree of similarity between a certain pixel point in the edge pending region and the pixel feature of the adjacent pixel point in the main body region satisfies the target condition (the target condition is that the degree of similarity reaches 90%), then the identification values of the pixel points in the edge pending region are the same as the identification values of the adjacent pixel points in the main body region, and the identification value of the pixel point in the edge pending region is determined to be the first identification value 1. If the similarity degree of the pixel characteristics of a certain pixel point in the edge pending area and an adjacent pixel point in the background area meets the target condition (the similarity degree reaches 90% under the target condition), the identification value of the pixel point in the edge pending area is the same as the identification value of the adjacent pixel point in the background area, and the identification value of the pixel point in the edge pending area is determined to be a second identification value 0.
S304, determining the edge contour of the target object in the image to be processed according to the identification values of the pixel points in the edge region to be determined.
The known body region and background region are accurate and have corresponding identification values, respectively. And the identification values of the pixel points in the edge to-be-determined region are generated based on the similarity calculation results of the pixel points and the adjacent pixel points in the main region and the background region, and the identification values respectively corresponding to the main region and the background region. Therefore, the degree of proximity between the identification value determined by the pixel point in the edge undetermined area and the first identification value and the second identification value can reflect whether the pixel point in the edge undetermined area actually belongs to the target object or the background. Therefore, the edge part of the target object can be accurately determined, and the edge contour of the target object in the image to be processed is further determined.
And if the proximity degree of the identification value of a certain pixel point in the edge pending area to the first identification value is higher than the proximity degree of the identification value of the pixel point to the second identification value, determining that the pixel point in the edge pending area actually belongs to the target object, otherwise, determining that the pixel point in the edge pending area actually belongs to the background.
For example, the first identification value is 1, the second identification value is 0, the edge undetermined area includes A, B, C pixel points, the identification value of the a pixel point is 0.9, the identification value of the B pixel point is 0.7, and the identification value of the C pixel point is 0.1. The proximity degree of the identification value 0.9 of the pixel A to the first identification value 1 is higher than that of the identification value to the second identification value 0, so that the fact that the pixel A actually belongs to the target object can be determined; the proximity degree of the identification value 0.7 of the B pixel point to the first identification value 1 is higher than that of the B pixel point to the second identification value 0, so that the B pixel point can be determined to actually belong to a target object; the proximity degree of the identification value 0.1 of the C pixel point to the first identification value 1 is lower than that of the identification value to the second identification value 0, so that the C pixel point can be determined to actually belong to the background.
After the edge contour of the target object in the image to be processed is determined, the original background can be removed according to the edge contour, and the image with the determined edge contour is overlapped with the new background actually required by the user, so that an image meeting the user requirement is generated, for example, a certificate photo with the new background replaced is generated. A process of generating a certificate photo according to the determined edge contour can be shown in fig. 6a, where 601 is an image to be processed, 602 is an image in which an edge contour is determined, that is, an Alpha Mask, 603 is a new background actually required by a user, and 604 is a finally generated certificate photo.
According to the technical scheme, for the image to be processed including the target object, the main body area, the edge undetermined area and the background area corresponding to the target object in the image to be processed can be determined according to the classification label obtained by performing semantic segmentation operation on the image to be processed, the identification value of the main body area is set as the first identification value, and the identification value of the background area is set as the second identification value. And according to the pixel points adjacent to the edge undetermined area in the main body area and the background area, calculating the pixel characteristic similarity between the pixel point in the edge undetermined area and the adjacent pixel point, and according to the similarity calculation result between the pixel point in the edge undetermined area and the adjacent pixel point, and the first identification value and the second identification value, determining the identification value of the pixel point in the edge undetermined area. Because the pixel characteristics among the pixel points belonging to the target object are closer to the pixel points in the background area, the similarity calculation result determined by any pixel point in the edge undetermined area can reflect the possibility that the pixel point belongs to the target object or the background. And because the pixel points in the background area and the main body area are respectively provided with different identification values, whether the pixel point belongs to a target object can be determined according to the identification value determined by the pixel point in the edge undetermined area and the proximity degree of the identification value, the first identification value and the second identification value, so that the edge part of the target object can be accurately determined, the determination precision of the area where the target object is located in the image to be processed is improved, and the method and the device can be suitable for the image background removal scene with high precision requirements.
Next, a detailed description will be given of a manner of generating the intermediate processed image in S301.
Referring to fig. 6b, fig. 6b is a flowchart of a method for determining an intermediate processed image corresponding to an image to be processed, the method comprising:
s601, according to the classification label, determining an undetermined area of the corresponding target object in the image to be processed.
The classification label can reflect the probability that the pixel point belongs to a certain class object, namely, the pixel point in the image to be processed can be identified to belong to people, bicycles, chairs and the like according to the classification label. And if the target object is a person, the region in which the person is located is the pending region of the target object.
And S602, performing binarization processing on the image to be processed to obtain a first image.
The image to be processed may include an undetermined region and other regions outside the undetermined region, and in order to distinguish the two regions, binarization processing may be performed on the image to be processed, and different pixel values are set for pixel points of the area to be processed in the first image and pixel points outside the area to be determined. For example, the binarization processing may be to set the pixel value of a pixel point on the image to 0 or 255, so as to obtain a first image, which is a black-and-white image in this case.
In some cases, the pixel value of the pixel point in the to-be-determined region may be set to 255, the pixel value of the pixel point outside the to-be-determined region may be set to 0, and the obtained first image is shown in fig. 7, where a white region is the to-be-determined region 701, and a black region is the region 702 outside the to-be-determined region.
S603, carrying out pixel value blurring processing on the edge part of the to-be-determined area in the first image to obtain a second image.
The blurring process may be a gaussian blurring process, or may be other blurring processes, which is not limited in this embodiment. Through the fuzzy processing of the pixel values, the pixel values of the pixel points near the edge of the undetermined area can be changed gradually, for example, the pixel values of the pixel points in the undetermined area are a first value, the pixel values of the pixel points outside the undetermined area (for example, a non-undetermined area) are a second value, the pixel values of the pixel points in the fuzzy processed area (near the edge of the undetermined area) are between the first value and the second value, the closer the pixel values of the pixel points in the area to the undetermined area are to the first value, and the closer the pixel values of the pixel points to the non-undetermined area are to the second value.
The second image obtained after the blurring processing can be as shown in fig. 8, and at this time, the pixel values of the pixel points in the second image are no longer only two values, and may also include other pixel values. If the pixel value of the pixel point of the undetermined area is 255, the white area in fig. 8 represents an area 801 where the pixel point with the pixel value of 255 is located, and the area 801 is the position where the target object main body is located, which is equivalent to the undetermined area 701 in fig. 7; the pixel value of the pixel point outside the undetermined area is 0, the black area in fig. 8 indicates an area 803 where the pixel point with the pixel value of 0 is located, and the area 803 is the position of the background, which is equivalent to the non-undetermined area 702 in fig. 7; the pixel values of the pixel points in the blurred second image may further include other pixel values between 0 and 255, so that the pixel values of the pixel points near the edge of the to-be-determined region gradually change, and the gradual change portion near the edge of the to-be-determined region in fig. 8 represents a region (a blurred region) 802 where the pixel points with pixel values between 0 and 255 are located.
After the second image is obtained, the second pixel region may be determined in the first pixel region where the pixel value blurring processing is performed according to the pixel value distribution of the pixel points in the second image. The first pixel region is a blurred region 802, for example, a region where pixel points corresponding to pixel values between 0 and 255 are located; the second pixel region is a region where a pixel point whose pixel value satisfies a certain condition, for example, a region where a pixel point whose pixel value is greater than 10 and less than 245 is determined as the second pixel region. The resulting second pixel region is shown in fig. 9, where the white region is a second pixel region 902 and the black region includes a region 901 and a region 903. The region 901 is a position where the target subject is located, and corresponds to the region 801 in fig. 8. The area 903 is the position of the background, and corresponds to the area 803 in fig. 8.
S604, determining the intermediate processing image according to the pixel value distribution of the pixel points in the second image.
It should be noted that, if the second pixel region is determined in the first pixel region subjected to the pixel value blurring process, one possible implementation manner of S604 is to determine the intermediate processing image according to the pixel value distribution of the pixel points in the second image and the second pixel region.
It should be noted that the second pixel region determined from the first pixel region of the pixel value blurring process may be relatively narrow, so that some edge pixel points actually belonging to the target object are not located in the second pixel region and located outside the target object and may be directly regarded as belonging to the background. In this case, in order to make the second pixel region cover all edge pixel points as much as possible, before determining the intermediate processed image according to the pixel value distribution of the pixel points in the second image and the second pixel region, the second pixel region may be subjected to expansion processing.
The dilation process may refer to expanding outward around on the basis of the original pixel area (e.g., the second pixel area) such that the processed pixel area is larger than the original pixel area. The second pixel region after the expansion process is shown in fig. 10, in which the white region is a second pixel region 1002 after the expansion process, and the black region includes a region 1001 and a region 1003. The region 1001 is a position where the target subject is located, and corresponds to the region 901 in fig. 9. The area 1003 is where the background is located, and corresponds to the area 903 in fig. 9. The second pixel region 1002 after the expansion process has a larger area than the second pixel region 902 in fig. 9.
In order to avoid the excessive calculation amount caused by the excessively wide second pixel region when the dilation processing is performed on the second pixel region, the intermediate processing image is determined according to the pixel value distribution of the pixel points in the second image and the second pixel region, for example, the pixel value distribution of the second image and the second pixel region after the dilation processing are mixed, and the intermediate processing image is determined according to the mixed pixel value distribution.
Next, how to determine the identification values of the pixel points in the edge pending area in S303 will be described.
If the identification value is the alpha value of the pixel point in the edge undetermined area, the determined alpha of each pixel point needs to satisfy the following formula: i ═ α F + (1- α) B
Wherein, I is the pixel value of the pixel point in the image to be processed, F is the pixel value of the pixel point in the main body region, B is the pixel value of the pixel point in the background region, and α is the alpha (identification value) of the pixel point.
Calculating alpha is equivalent to solving the formula I ═ α F + (1- α) B.
The formula I ═ α F + (1- α) B original can be transformed into solving a matrix equation as shown below:
Figure BDA0003510690490000131
wherein, the similarity matrix AijThe diagonal matrix D can be obtained by the method described in the previous embodimentii＝∑jAij，L＝Dii-AijM is a first mark of the main body area or a second mark of the background area, and lambda is a constraint parameter.
Since A can be calculated by the aforementioned similarity calculation procedureijAccording to formula Dii＝∑jAijD can be calculatedii(D) According to the formula L ═ Dii-AijAnd L, m is known, and λ is known (for example, it may be taken as 100), then the equation may be solved
Figure BDA0003510690490000132
Alpha value (α) of each pixel point.
It is to be understood that, in a scene in which the personalized photo is generated based on a single photo, since the personalized photo is required to be an image embodying a human face, the image to be processed in S301 may be an image embodying a human face.
In order to avoid that the image to be processed is not qualified and thus an unqualified identification photo is obtained, before S301 is executed, whether the image to be processed is qualified (for example, whether the image is a portrait face image) or not may be checked, and if the image to be processed is not qualified, a corresponding prompt may pop up, for example, the user is prompted to "please upload a face photo".
Specifically, in the process of acquiring the image to be processed, whether the image to be processed contains one face or not may be detected through face detection, and if there is no or more than one face frame, an error is returned to prompt the user to take a single frontal face picture. And then, judging whether the face of the image to be processed meets the requirement that the face orientation of the certificate photo is a front face or not through face orientation detection and a predefined orientation threshold, and if not, returning an error and prompting a user to shoot an independent front face photo. In general, if the orientation angle of the x/y/z axis of the face is determined to be not more than 5 degrees through face orientation detection and a predefined orientation threshold, the face orientation of the identification photo in the image to be processed is considered to be the front face.
It should be noted that the certificate photo may have certain requirements on the proportion of the face in the certificate photo and the position in the certificate photo, in this case, the face frame position of the original photo can be centered and expanded in proportion, a cut picture conforming to the proportion and position of the face of the certificate photo is generated, and the picture is taken as an image to be processed.
Next, the method provided by the embodiment of the present application will be described with reference to an actual application scenario. In the application scene, a user wants to remove the image background of the user's own life photograph, so as to generate a blue background certificate photograph. The method comprises the step of generating the certificate photo by the application on the terminal equipment, wherein the method provided by the embodiment of the application is used for generating the certificate photo. For example, the function of generating the certificate photo is realized by social software, and the social software can be WeChat, enterprise WeChat, QQ and the like.
In this embodiment, taking the social software implementation of generating a certificate photo as an example, referring to fig. 11, the method includes:
s1101, the user opens the social software.
S1102, selecting an original picture from the photo album through a picture selection inlet of the social software by the user.
The selected original picture can be seen as 1201 in fig. 12.
S1103, the social software checks whether the original photo is in compliance, and if so, S1104 is executed.
And S1104, the social software cuts the original photo according to the portrait proportion requirement to obtain the image to be processed.
The target object in the image to be processed is a user portrait.
S1105, the social software determines an edge contour corresponding to the target object according to the image to be processed, and removes the original background of the image to be processed according to the edge contour.
The method for determining the edge contour corresponding to the target object by the social software according to the image to be processed may refer to the description of the embodiment corresponding to fig. 3, and details are not repeated here.
And S1106, the social software performs certificate photo background superposition, so that the certificate photo is generated.
An interface diagram for generating a certificate photograph is shown at 1202 in FIG. 12.
It should be noted that the color value of the certificate photo background used when the certificate photo background is superimposed may be a default background color value, and if the default background color value does not meet the user requirement, the user may select the color value by himself or herself, for example, the user may select a background such as white, blue, red, and the like in the interface shown in 1202.
S1107, the user selects different beauty parameters to adjust the generated certificate photo.
And S1108, outputting the adjusted certificate photo.
Referring to 1203 in FIG. 12, the adjusted certificate photograph is shown in 1204 in FIG. 12.
And S1109, the user saves or shares the adjusted certificate photo through social software.
S1103-S1105 may be executed by social software (client), or may be executed by a server, which is not limited in this embodiment. S1106-S1107 may be performed by social software.
Based on the image processing method provided by the foregoing embodiment, the present embodiment provides an image processing apparatus, see fig. 13a, which includes a first determining unit 1301, a calculating unit 1302, a second determining unit 1303, and a third determining unit 1304:
the first determining unit 1301 is configured to determine, according to a classification label of a pixel point obtained by semantic segmentation of an image to be processed, a main region, an edge undetermined region, and a background region corresponding to a target object in the image to be processed; the identification value of the main body area is set as a first identification value, and the identification value of the background area is set as a second identification value;
the calculating unit 1302 is configured to calculate, according to the pixel points adjacent to the edge undetermined region in the main body region and the background region, pixel feature similarity between the adjacent pixel points and the edge undetermined region;
the second determining unit 1303 is configured to determine, according to the similarity calculation result of the pixel points in the edge to-be-determined region, the first identification value and the second identification value, the identification values of the pixel points in the edge to-be-determined region;
the third determining unit 1304 is configured to determine an edge contour of the target object in the image to be processed according to the identification values of the pixel points in the edge to-be-determined region.
In a possible implementation manner, in the process of determining the identification values of the pixels in the edge to-be-determined region, the proximity degree of the identification values of the adjacent pixels is related to the similarity degree of the pixel features of the adjacent pixels.
In a possible implementation manner, if the similarity degree of the pixel characteristics of the adjacent pixel points satisfies a target condition, the identification values of the adjacent pixel points are the same.
In one possible implementation, the pixel characteristics include color information and position information.
In a possible implementation manner, the first determining unit 1301 is configured to:
determining an intermediate processing image corresponding to the image to be processed according to a classification label of a pixel point obtained by semantic segmentation of the image to be processed, wherein the intermediate processing image is used for identifying a main body region, an edge undetermined region and a background region of the target object corresponding to the image to be processed.
In a possible implementation manner, the first determining unit 1301 is configured to:
determining an undetermined area of a corresponding target object in the image to be processed according to the classification label;
obtaining a first image by carrying out binarization processing on the image to be processed; the pixel values of the pixel points of the undetermined area in the first image are different from the pixel values of the pixel points outside the undetermined area;
obtaining a second image by carrying out pixel value blurring processing on the edge part of the region to be determined in the first image;
and determining the intermediate processing image according to the pixel value distribution of the pixel points in the second image.
In a possible implementation manner, the first determining unit 1301 is further configured to:
determining a second pixel area in the first pixel area subjected to the pixel value blurring processing according to the pixel value distribution of the pixel points in the second image;
and determining the intermediate processing image according to the pixel value distribution of the pixel points in the second image and the second pixel area.
In a possible implementation manner, the first determining unit 1301 is further configured to:
and performing expansion processing on the second pixel region.
According to the technical scheme, for the image to be processed including the target object, the main body area, the edge undetermined area and the background area corresponding to the target object in the image to be processed can be determined according to the classification label obtained by performing semantic segmentation operation on the image to be processed, the identification value of the main body area is set as the first identification value, and the identification value of the background area is set as the second identification value. And according to the pixel points adjacent to the edge undetermined area in the main body area and the background area, calculating the pixel characteristic similarity between the pixel point in the edge undetermined area and the adjacent pixel point, and according to the similarity calculation result between the pixel point in the edge undetermined area and the adjacent pixel point, and the first identification value and the second identification value, determining the identification value of the pixel point in the edge undetermined area. Because the pixel characteristics among the pixel points belonging to the target object are closer to the pixel points in the background area, the similarity calculation result of any pixel point in the edge undetermined area can show the possibility that the pixel point belongs to the target object or the background. And because the pixel points in the background area and the main body area are respectively provided with different identification values, whether the pixel point belongs to a target object can be determined according to the identification value determined by the pixel point in the edge undetermined area and the proximity degree of the identification value, the first identification value and the second identification value, so that the edge part of the target object can be accurately determined, the determination precision of the area where the target object is located in the image to be processed is improved, and the method and the device can be suitable for the image background removal scene with high precision requirements.
The embodiment of the application also provides an image processing device, which is described below with reference to the accompanying drawings. Referring to fig. 13b, an embodiment of the present application provides an image processing apparatus 1300, where the apparatus 1300 may be a terminal apparatus, and the terminal apparatus may be any intelligent terminal including a mobile phone, a tablet computer, a Personal Digital Assistant (PDA), a Point of Sales (POS), a vehicle-mounted computer, and the terminal apparatus is a mobile phone:
fig. 13b is a block diagram illustrating a partial structure of a mobile phone related to a terminal device provided in an embodiment of the present application. Referring to fig. 13b, the handset includes: radio Frequency (RF) circuit 1310, memory 1320, input unit 1330, display unit 1340, sensor 1350, audio circuit 1360, wireless fidelity (WiFi) module 1370, processor 1380, and power supply 1390. Those skilled in the art will appreciate that the handset configuration shown in fig. 13b is not intended to be limiting and may include more or fewer components than shown, or some components may be combined, or a different arrangement of components.
The following describes the components of the mobile phone in detail with reference to fig. 13 b:
RF circuit 1310 may be used for receiving and transmitting signals during a message transmission or call, and in particular, for processing received downlink information of a base station by processor 1380; in addition, the data for designing uplink is transmitted to the base station. The memory 1320 may be used to store software programs and modules, and the processor 1380 executes various functional applications and data processing of the cellular phone by operating the software programs and modules stored in the memory 1320. The memory 1320 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like.
The input unit 1330 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the cellular phone. Specifically, the input unit 1330 may include a touch panel 1331 and other input devices 1332.
The display unit 1340 may be used to display information input by a user or information provided to the user and various menus of the cellular phone. The display unit 1340 may include a display panel 1341. The handset may also include at least one sensor 1350, such as light sensors, motion sensors, and other sensors.
Audio circuitry 1360, speaker 1361, microphone 1362 may provide an audio interface between the user and the cell phone.
The processor 1380 is a control center of the mobile phone, connects various parts of the entire mobile phone using various interfaces and lines, and performs various functions of the mobile phone and processes data by operating or executing software programs and/or modules stored in the memory 1320 and calling data stored in the memory 1320.
In this embodiment, the processor 1380 included in the terminal device further has the following functions:
determining a main body area, an edge undetermined area and a background area of a target object in the image to be processed, which correspond to the main body area, the edge undetermined area and the background area of the target object, according to the classification labels of the pixel points obtained by semantic segmentation of the image to be processed; the identification value of the main body area is set as a first identification value, and the identification value of the background area is set as a second identification value;
calculating pixel feature similarity between pixel points in the edge undetermined region and adjacent pixel points according to pixel points in the main region and the background region which are adjacent to the edge undetermined region;
determining the identification value of the pixel point in the edge pending area according to the similarity calculation result of the pixel point in the edge pending area, the first identification value and the second identification value;
and determining the edge contour of the target object in the image to be processed according to the identification values of the pixel points in the edge region to be determined.
The image Processing device provided by the embodiment of the present application may be a server, please refer to fig. 14, fig. 14 is a block diagram of a server 1400 provided by the embodiment of the present application, and the server 1400 may have a relatively large difference due to different configurations or performances, and may include one or more Central Processing Units (CPUs) 1422 (e.g., one or more processors) and a memory 1432, and one or more storage media 1430 (e.g., one or more mass storage devices) for storing an application 1442 or data 1444. Memory 1432 and storage media 1430, among other things, may be transient or persistent storage. The program stored on storage medium 1430 may include one or more modules (not shown), each of which may include a sequence of instructions operating on a server. Still further, a central processor 1422 may be disposed in communication with storage medium 1430 for executing a series of instruction operations on storage medium 1430 on server 1400.
The Server 1400 may also include one or more power supplies 1426, one or more wired or wireless network interfaces 1450, one or more input-output interfaces 1458, and/or one or more operating systems 1441, such as a Windows ServerTM，Mac OS XTM，UnixTM,LinuxTM，FreeBSDTMAnd so on.
The steps performed by the server in the above embodiment may be based on the server structure shown in fig. 14.
The CPU 1422 is configured to perform the following steps:
determining a main body area, an edge undetermined area and a background area of a target object in the image to be processed, which correspond to the main body area, the edge undetermined area and the background area of the target object, according to the classification labels of the pixel points obtained by semantic segmentation of the image to be processed; the identification value of the main body area is set as a first identification value, and the identification value of the background area is set as a second identification value;
calculating pixel feature similarity between pixel points in the edge undetermined region and adjacent pixel points according to pixel points in the main region and the background region which are adjacent to the edge undetermined region;
determining an identification value of a pixel point in the edge pending area according to a similarity calculation result of the pixel point in the edge pending area, the first identification value and the second identification value;
and determining the edge contour of the target object in the image to be processed according to the identification values of the pixel points in the edge region to be determined.
The embodiment of the present application provides a computer-readable storage medium, which is used for storing a program code, and the program code is used for executing the image processing method described in the foregoing embodiment.
The terms "first," "second," "third," "fourth," and the like in the description of the application and the above-described figures, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the application described herein are, for example, capable of operation in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.
In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.
The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: various media capable of storing program codes, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk.
The above embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions in the embodiments of the present application.

Claims (13)

1. An image processing method, characterized in that the method comprises:
determining a main body area, an edge undetermined area and a background area of a target object in the image to be processed, which correspond to the main body area, the edge undetermined area and the background area of the target object, according to the classification labels of the pixel points obtained by semantic segmentation of the image to be processed; the identification value of the main body area is set as a first identification value, and the identification value of the background area is set as a second identification value;
calculating pixel feature similarity between pixel points in the edge undetermined region and adjacent pixel points according to pixel points in the main region and the background region which are adjacent to the edge undetermined region;
determining the identification values of the pixels in the edge pending region according to the similarity calculation result of the pixels in the edge pending region, the first identification value and the second identification value, wherein in the process of determining the identification values of the pixels in the edge pending region, the proximity degree of the identification values of the adjacent pixels is related to the similarity degree of the pixel characteristics of the adjacent pixels;
and determining the edge contour of the target object in the image to be processed according to the identification values of the pixel points in the edge region to be determined.
2. The method according to claim 1, wherein if the similarity degree of the pixel characteristics of the adjacent pixels meets a target condition, the identification values of the adjacent pixels are the same.
3. The method of claim 1 or 2, wherein the pixel characteristics comprise color information and location information.
4. The method according to claim 1 or 2, wherein the determining a main region, an edge undetermined region and a background region of a target object in the image to be processed corresponding to the classification label of the pixel point obtained by semantic segmentation according to the image to be processed comprises:
determining an intermediate processing image corresponding to the image to be processed according to a classification label of a pixel point obtained by semantic segmentation of the image to be processed, wherein the intermediate processing image is used for identifying a main body region, an edge undetermined region and a background region of the target object corresponding to the image to be processed.
5. The method according to claim 4, wherein the determining the intermediate processed image corresponding to the image to be processed according to the classification label of the pixel point obtained by semantic segmentation of the image to be processed comprises:
determining an undetermined area of a corresponding target object in the image to be processed according to the classification label;
obtaining a first image by carrying out binarization processing on the image to be processed; the pixel values of the pixel points in the undetermined area in the first image are different from the pixel values of the pixel points outside the undetermined area;
obtaining a second image by carrying out pixel value blurring processing on the edge part of the region to be determined in the first image;
and determining the intermediate processing image according to the pixel value distribution of the pixel points in the second image.
6. The method of claim 5, wherein after obtaining the second image, the method further comprises:
determining a second pixel area in the first pixel area subjected to the pixel value blurring processing according to the pixel value distribution of the pixel points in the second image;
determining the intermediate processing image according to the pixel value distribution of the pixel points in the second image comprises:
and determining the intermediate processing image according to the pixel value distribution of the pixel points in the second image and the second pixel area.
7. The method of claim 6, wherein prior to said determining the intermediate processed image based on the pixel value distribution of the pixels in the second image and the second pixel region, the method further comprises:
and performing expansion processing on the second pixel region.
8. An image processing apparatus characterized by comprising a first determining unit, a calculating unit, a second determining unit, and a third determining unit:
the first determining unit is used for determining a main body area, an edge undetermined area and a background area of a target object in the image to be processed according to the classification label of the pixel point obtained by semantic segmentation of the image to be processed; the identification value of the main body area is set as a first identification value, and the identification value of the background area is set as a second identification value;
the calculating unit is used for calculating the pixel characteristic similarity between the adjacent pixel points in the edge undetermined region according to the pixel points adjacent to the edge undetermined region in the main body region and the background region;
the second determining unit is configured to determine the identification value of the pixel point in the edge to-be-determined region according to the similarity calculation result of the pixel point in the edge to-be-determined region, the first identification value and the second identification value, wherein in the process of determining the identification value of the pixel point in the edge to-be-determined region, the proximity degree of the identification value of the adjacent pixel point is related to the similarity degree of the pixel feature of the adjacent pixel point;
the third determining unit is configured to determine an edge contour of the target object in the image to be processed according to the identification value of the pixel point in the edge region to be determined.
9. The apparatus according to claim 8, wherein the identification values of the neighboring pixels are the same if the similarity degree of the pixel characteristics of the neighboring pixels satisfies a target condition.
10. The apparatus according to claim 8 or 9, wherein the first determining unit is configured to:
determining an intermediate processing image corresponding to the image to be processed according to a classification label of a pixel point obtained by semantic segmentation of the image to be processed, wherein the intermediate processing image is used for identifying a main body region, an edge undetermined region and a background region of the target object corresponding to the image to be processed.
11. The apparatus of claim 10, wherein the first determining unit is configured to:
determining an undetermined area of a corresponding target object in the image to be processed according to the classification label;
obtaining a first image by carrying out binarization processing on the image to be processed; the pixel values of the pixel points of the undetermined area in the first image are different from the pixel values of the pixel points outside the undetermined area;
obtaining a second image by carrying out pixel value blurring processing on the edge part of the region to be determined in the first image;
and determining the intermediate processing image according to the pixel value distribution of the pixel points in the second image.
12. An apparatus for image processing, the apparatus comprising a processor and a memory:
the memory is used for storing program codes and transmitting the program codes to the processor;
the processor is configured to perform the image processing method of claims 1-7 according to instructions in the program code.
13. A computer-readable storage medium characterized in that the computer-readable storage medium is configured to store a program code for executing the image processing method of claims 1 to 7.
CN202210151644.5A 2019-04-30 2019-04-30 Image processing method and device Pending CN114463367A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210151644.5A CN114463367A (en) 2019-04-30 2019-04-30 Image processing method and device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910364702.0A CN110097570B (en) 2019-04-30 2019-04-30 Image processing method and device
CN202210151644.5A CN114463367A (en) 2019-04-30 2019-04-30 Image processing method and device

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CN201910364702.0A Division CN110097570B (en) 2019-04-30 2019-04-30 Image processing method and device

Publications (1)

Publication Number Publication Date
CN114463367A true CN114463367A (en) 2022-05-10

Family

ID=67446697

Family Applications (2)

Application Number Title Priority Date Filing Date
CN201910364702.0A Active CN110097570B (en) 2019-04-30 2019-04-30 Image processing method and device
CN202210151644.5A Pending CN114463367A (en) 2019-04-30 2019-04-30 Image processing method and device

Family Applications Before (1)

Application Number Title Priority Date Filing Date
CN201910364702.0A Active CN110097570B (en) 2019-04-30 2019-04-30 Image processing method and device

Country Status (1)

Country Link
CN (2) CN110097570B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111192276B (en) * 2019-12-18 2024-04-09 中国平安财产保险股份有限公司 Image processing method, device, electronic equipment and storage medium
CN111914769B (en) * 2020-08-06 2024-01-26 腾讯科技（深圳）有限公司 User validity determination method, device, computer readable storage medium and equipment

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8805653B2 (en) * 2010-08-11 2014-08-12 Seiko Epson Corporation Supervised nonnegative matrix factorization
CN102903128B (en) * 2012-09-07 2016-12-21 北京航空航天大学 The video image content editor's transmission method kept based on Similarity of Local Characteristic Structure
CN105654458A (en) * 2014-11-14 2016-06-08 华为技术有限公司 Method and device for processing images
CN104680482A (en) * 2015-03-09 2015-06-03 华为技术有限公司 Method and device for image processing
CN104680185B (en) * 2015-03-15 2018-04-17 西安电子科技大学 Hyperspectral image classification method based on boundary point reclassification
CN105631880B (en) * 2015-12-31 2019-03-22 百度在线网络技术（北京）有限公司 Lane line dividing method and device
CN105681686A (en) * 2015-12-31 2016-06-15 北京奇艺世纪科技有限公司 Image processing method and system
US10296812B2 (en) * 2017-01-04 2019-05-21 Qualcomm Incorporated Systems and methods for mapping based on multi-journey data
CN107123150B (en) * 2017-03-25 2021-04-30 复旦大学 Global color contrast detection and saliency map segmentation method
EP3438929B1 (en) * 2017-08-04 2020-07-08 Université de Liège Foreground and background detection method
CN108537799B (en) * 2018-03-21 2021-03-23 广西师范大学 Color sampling method based on pixel type and weight statistics
CN109344724B (en) * 2018-09-05 2020-09-25 深圳伯奇科技有限公司 Automatic background replacement method, system and server for certificate photo
CN109472200B (en) * 2018-09-29 2022-04-05 深圳市锦润防务科技有限公司 Intelligent sea surface rubbish detection method, system and storage medium
CN109685822A (en) * 2019-01-11 2019-04-26 恒信东方文化股份有限公司 A kind of automatically stingy drawing method

Also Published As

Publication number Publication date
CN110097570B (en) 2022-03-01
CN110097570A (en) 2019-08-06

Similar Documents

Publication Publication Date Title
CN110473141B (en) Image processing method, device, storage medium and electronic equipment
US11308351B2 (en) Method and apparatus for recognizing sequence in image, electronic device, and storage medium
CN110874594A (en) Human body surface damage detection method based on semantic segmentation network and related equipment
CN107633526A (en) A kind of image trace point acquisition methods and equipment, storage medium
US20210342632A1 (en) Image processing method and apparatus, electronic device, and storage medium
CN113132704B (en) Image processing method, device, terminal and storage medium
CN110188865B (en) Information processing method and device, electronic equipment and storage medium
US20230401833A1 (en) Method, computer device, and storage medium, for feature fusion model training and sample retrieval
CN109658330A (en) A kind of color development method of adjustment and device
CN112035042A (en) Application program control method and device, electronic equipment and readable storage medium
US20220084165A1 (en) System and method for single-modal or multi-modal style transfer and system for random stylization using the same
CN114463367A (en) Image processing method and device
CN112084959B (en) Crowd image processing method and device
CN111354059A (en) Image processing method and device
CN110910400A (en) Image processing method, image processing device, storage medium and electronic equipment
CN107292817B (en) Image processing method, device, storage medium and terminal
CN113486785A (en) Video face changing method, device, equipment and storage medium based on deep learning
CN109903252B (en) Image processing method and device, electronic equipment and storage medium
CN108491780A (en) Image landscaping treatment method, apparatus, storage medium and terminal device
CN111290684A (en) Image display method, image display device and terminal equipment
CN107992894B (en) Image recognition method, image recognition device and computer-readable storage medium
CN107730443B (en) Image processing method and device and user equipment
CN113420769A (en) Image mask recognition, matting and model training method and device and electronic equipment
KR100793285B1 (en) System and method for image noise reduction with filter matrix and computer readable medium stored thereon computer executable instruction for performing the method
EP4287068A1 (en) Model training method, scene recognition method, and related device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination