WO2022017163A1 - 图像处理方法、装置、设备及存储介质 - Google Patents

图像处理方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2022017163A1
WO2022017163A1 PCT/CN2021/104481 CN2021104481W WO2022017163A1 WO 2022017163 A1 WO2022017163 A1 WO 2022017163A1 CN 2021104481 W CN2021104481 W CN 2021104481W WO 2022017163 A1 WO2022017163 A1 WO 2022017163A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
candidate
probability
area
segmentation
Prior art date
Application number
PCT/CN2021/104481
Other languages
English (en)
French (fr)
Inventor
张瑞
程培
俞刚
傅斌
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to EP21846696.9A priority Critical patent/EP4092623A4/en
Publication of WO2022017163A1 publication Critical patent/WO2022017163A1/zh
Priority to US17/887,994 priority patent/US20220398742A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G06V10/267—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion by performing operations on regions, e.g. growing, shrinking or watersheds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/776—Validation; Performance evaluation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20076—Probabilistic image processing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Definitions

  • the present application relates to the technical field related to computer vision in artificial intelligence, and in particular, to an image processing method, apparatus, device and storage medium.
  • Artificial Intelligence is a comprehensive technology of computer science. By studying the design principles and implementation methods of various intelligent machines, the machines have the functions of perception, reasoning and decision-making. Artificial intelligence technology is a comprehensive discipline covering a wide range of fields, such as computer vision technology, natural language processing technology, and machine learning/deep learning. With the development of technology, artificial intelligence technology will be applied in more fields , and play an increasingly important value.
  • image segmentation is an important research direction, which can segment the foreground image area and the background image area from the original image, so that the foreground image area or the background image area can be processed accordingly. Face recognition, replace background image areas, add image effects, and more.
  • the embodiment of the present application provides an image processing method, including:
  • the second segmentation model is used to perform region segmentation and identification on the reconstituted image, so as to obtain the target foreground image region and the target background image region of the original image.
  • the embodiment of the present application also provides an image processing apparatus, including:
  • an identification module configured to perform preliminary segmentation and identification of the original image by using the first segmentation model to obtain candidate foreground image regions and candidate background image regions of the original image;
  • a reorganization module used for reorganizing the candidate foreground image area, the candidate background image area and the original image to obtain a reorganized image; the pixels in the reorganized image and the pixels in the original image are have a one-to-one correspondence;
  • the recognition module is further configured to perform region segmentation and recognition on the reorganized image by using the second segmentation model to obtain the target foreground image region and the target background image region of the original image.
  • Embodiments of the present application also provide a computer device, including: a processor and a memory;
  • the above-mentioned memory is used for storing a computer program
  • the above-mentioned processor is used for calling the above-mentioned computer program to execute the steps of the above-mentioned method.
  • Embodiments of the present application further provide a non-volatile computer-readable storage medium, where the computer-readable storage medium stores a computer program, and the computer program includes program instructions, and the program instructions, when executed by a processor, execute the above-mentioned program instructions. steps of the method.
  • FIG. 1 is a schematic diagram of the architecture of an image processing system provided by the present application.
  • FIG. 2a is a schematic diagram of an interaction scene between various devices of an image processing system provided by the present application.
  • 2b is a schematic diagram of an interaction scene between various devices of an image processing system provided by the present application.
  • FIG. 3 is a schematic flowchart of an image processing method provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a scenario for obtaining a probability of a candidate region type provided by an embodiment of the present application
  • FIG. 5 is a schematic diagram of a scene of adjusting and segmenting an original image to obtain a target foreground image area and a target background image area of the original image provided by an embodiment of the present application;
  • Fig. 6 is a kind of scene schematic diagram of the target foreground image area and the target background image area of the original image obtained by adjusting and segmenting the original image provided by the embodiment of the present application;
  • FIG. 7 is a schematic diagram of a scene provided by an embodiment of the present application in which a first candidate segmentation model is adjusted to obtain a first segmentation model
  • FIG. 8 is a schematic diagram of a scene provided by an embodiment of the present application in which a second candidate segmentation model is adjusted to obtain a second segmentation model;
  • FIG. 9 is a schematic structural diagram of an image processing apparatus provided by an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of a computer device provided by an embodiment of the present application.
  • the image processing methods provided in the embodiments of the present application mainly relate to computer vision technologies in artificial intelligence, and specifically relate to image segmentation technologies in computer vision technologies.
  • an image processing system for implementing the image processing method of the present application is introduced. As shown in FIG. 1 , the image processing system includes a server 10 and at least one terminal 11 .
  • the terminal 11 may refer to a user-oriented terminal, that is, the terminal 11 may refer to a terminal used by the user to acquire the original image and send the image to the server 10 .
  • the original image may refer to an image to be subjected to image segmentation processing, and the original image may be captured by the terminal 11 , or the original image may be downloaded from the network by the terminal 11 .
  • the server 10 may refer to a back-end service device for image processing, and may specifically be used to obtain an original image from the terminal 11, and perform segmentation processing on the original image to obtain a target foreground image area or a target background image area of the original image.
  • the target foreground image area may be Refers to the area that includes the target object, the target background image area may refer to the area that does not include the target object, and the target object may refer to people, buildings, animals, objects, and so on.
  • the above-mentioned process of segmenting the original image may be implemented by the terminal 11 or the server 10.
  • the following description in this application takes the server 10 segmenting the original image as an example.
  • the process of segmenting the original image by the terminal 11 is described below.
  • the process of dividing the original image by the server 10 may be referred to, which is not repeated in this application.
  • the server 10 may be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or may provide cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, Cloud servers for basic cloud computing services such as cloud communications, middleware services, domain name services, security services, Content Delivery Network (CDN), and big data and artificial intelligence platforms.
  • the terminal 11 may be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, etc., but is not limited thereto.
  • the terminal and the server may be directly or indirectly connected through wired or wireless communication, which is not limited in this application.
  • the terminal 11 can download an image from the network, or obtain an image by shooting, use the image as an original image, and send the original image to the server 10;
  • the original image can be a person image
  • the original image It includes a foreground image area 12 and a background image area 13, the foreground image area 12 refers to the area including the person, and the background image area 13 refers to the area behind the person, for example, the background image area 13 includes plants.
  • the server 10 can receive the original image sent by the terminal 11, input the original image into the first segmentation model, and perform preliminary segmentation and identification on the original image to obtain the candidate region type probability of the pixels in the original image; Segmentation identification may refer to the preliminary identification processing of the original image, and the candidate region type probability includes a probability used to indicate that a pixel in the original image belongs to a background image region, that is, a candidate background probability; or, the candidate region type probability includes a probability used to indicate The probability that the pixels in the original image belong to the foreground image region, that is, the candidate foreground probability; or, the candidate region type probability includes the candidate background probability and the candidate foreground probability.
  • the original image may be segmented according to the type probability of the candidate region to obtain a candidate background image region and a candidate foreground image region of the original image.
  • some lines are missing in the candidate background image area and the candidate foreground image area, that is, there is a problem of information loss in the candidate foreground image area and the candidate background image area, that is, the accuracy comparison of the segmentation results obtained by the first segmentation model Therefore, the segmentation result of the first segmentation model needs to be optimized. Specifically, as shown in FIG.
  • the server 10 may reorganize the original image, the candidate background image area, and the candidate foreground image area to obtain a reorganized image, and use the second segmentation model to perform region segmentation and identification on the reorganized image to obtain the reorganized image.
  • the area segmentation adjustment parameter of the pixel point in the image that is, the area segmentation adjustment parameter may refer to a parameter used to optimize the segmentation result of the first segmentation model.
  • the original image can be adjusted and segmented according to the region segmentation adjustment parameters and the candidate region type probability to obtain the target foreground image region and the target background image region of the original image. Comparing the candidate background image area and the candidate foreground image area in Fig. 2a with the target background image area and the target foreground image area in Fig. 2b, it can be seen that the boundary between the target background image area and the target foreground image area is smoother and more stable. accurate.
  • the first segmentation model is used to perform preliminary segmentation processing on the original image
  • the second segmentation model is used to optimize the segmentation result of the first segmentation model, which can improve the accuracy of image segmentation.
  • the second segmentation model is used to optimize the segmentation result of the first segmentation model, which can improve the accuracy of image segmentation.
  • more information is provided for the area segmentation and identification process of the second segmentation model, so as to avoid the process of segmenting the original image.
  • the problem of information loss of the original image can further improve the accuracy of image segmentation.
  • FIG. 3 is a schematic flowchart of an image processing method provided by an embodiment of the present application.
  • the method may be executed by a computer device, and the computer device may refer to the service device 10 or the terminal 11 in FIG. 1 .
  • the image processing method may include the following steps S101 to S104 .
  • S101 Use a first segmentation model to perform preliminary segmentation and identification on the original image, and obtain candidate foreground image regions and candidate background image regions of the original image.
  • the foreground image area may refer to an area including a target object, and the target object may refer to at least one of characters, buildings, objects, plants, animals, etc., that is, the target object may refer to an object of interest to the user; the background image area It may refer to the area where the person or object behind the target object is located, that is, the background image area is the area where the object that the user is not interested in is located.
  • the first segmentation model can be used to perform preliminary segmentation and identification of the original image, and the candidate foreground image area and the candidate background image area of the original image can be obtained;
  • the model for performing preliminary segmentation and identification of the original image, and the candidate foreground image area and the candidate background image area may be obtained by performing preliminary segmentation processing on the original image.
  • the above step S101 may include the following s01-s02.
  • the first segmentation model can be used to segment and identify the original image to obtain the candidate region type probability of the pixels in the original image; that is, the original image is input to In the first segmentation model, the original image is segmented and identified to obtain the candidate region type probability of the pixels in the original image.
  • the computer device obtains the candidate region type probability of the pixel in the original image
  • the original image can be segmented according to the candidate region type probability to obtain candidate foreground image regions and candidate background image regions of the original image.
  • the candidate region type probability includes a candidate foreground probability, and the candidate foreground probability is used to reflect the probability that the corresponding pixel in the original image belongs to the foreground image region; the computer device may determine the probability that the candidate foreground probability in the original image is greater than the foreground probability threshold. Pixels are determined as foreground pixels, and pixels whose candidate foreground probability is less than or equal to the foreground probability threshold in the original image are determined as background pixels; further, the area belonging to the foreground pixels is segmented from the original image as the foreground image area , segment the area belonging to the background pixels from the original image as the background image area.
  • the candidate region type probability includes a candidate foreground probability.
  • the computer device can obtain the sum of the candidate foreground probabilities of the pixels in the original image as the first probability value;
  • the ratio between the candidate foreground probability of the pixel point and the first probability value, the pixel point whose ratio is greater than the first foreground ratio threshold value is determined as the foreground pixel point, and the pixel point whose ratio is less than or equal to the first foreground ratio threshold value in the original image , determined as background pixels; further, segment the area belonging to the foreground pixels from the original image as the foreground image area, and segment the area belonging to the background pixels from the original image as the background image area.
  • the ratio between the candidate foreground probability of the pixel point in the original image and the first probability value can be expressed by the following formula (1).
  • F i represents the ratio between the candidate foreground probability of the ith pixel in the original image and the first probability value
  • K represents the number of pixels in the original image
  • FG i represents the number of pixels in the original image.
  • the candidate region type probability includes a candidate background probability, and the candidate background probability is used to reflect the probability that the corresponding pixel in the original image belongs to the background image region; the computer device may determine the probability of the candidate background in the original image that is smaller than the background probability threshold. Pixels are determined as foreground pixels, and pixels whose background probability is greater than or equal to the background probability threshold in the original image are determined as background pixels; further, the area belonging to the foreground pixels is segmented from the original image as the foreground image area , segment the area belonging to the background pixels from the original image as the background image area.
  • the candidate region type probability includes the candidate background probability.
  • the computer device can obtain the sum of the candidate background probabilities of the pixels in the original image as the second probability value; The ratio between the candidate background probability of the pixel point and the second probability value, the pixel point whose ratio is greater than the first background ratio threshold value is determined as the background pixel point, and the pixel point whose ratio is less than or equal to the first background ratio threshold value in the original image , determined as foreground pixels; further, segment the area belonging to the background pixels from the original image as the background image area, and segment the area belonging to the foreground pixels from the original image as the foreground image area.
  • the candidate region type probability includes a candidate background probability and a candidate foreground probability.
  • the computer device may select one of the candidate background probability and the candidate foreground probability, and perform segmentation processing on the original image according to the selected probability to obtain the original image.
  • S102 Recombining the candidate foreground image area, the candidate background image area, and the original image to obtain a reorganized image; there is a one-to-one correspondence between the pixels in the reorganized image and the pixels in the original image.
  • the computer device can perform the segmentation on the candidate foreground image area, the candidate background image area and the original image. Recombining to obtain a reorganized image, that is, performing fusion processing on the candidate foreground image area, the candidate background image area and the original image, and using the image obtained by fusion processing as the reorganized image. Compared with the original image, the reconstructed image can provide more information, that is, the boundary information lost in the segmentation process of the original image can be compensated, which is beneficial to improve the accuracy of image segmentation.
  • the size of the reconstructed image is the same as that of the original image, that is, there is a one-to-one correspondence between the pixels in the reconstructed image and the pixels in the original image, that is, the two pixels in the original image and the reconstructed image have the same position information There is a corresponding relationship between them.
  • the coordinate system is established by taking the lower left corner of the original image and the reconstructed image as the coordinate origin, and it is determined that there is a corresponding relationship between the two pixels with the same position coordinates in the original image and the reconstructed image.
  • the position coordinates in the original image are determined as ( There is a corresponding relationship between the pixel points of 1, 2) and the pixel points whose position coordinates are (1, 2) in the reconstructed image.
  • the computer equipment can use the second segmentation model to optimize the segmentation result of the first segmentation model, and obtain the target background image area and the target foreground image area of the original image; that is, the target background image area and the target background image area The image area and the candidate background image area are optimized.
  • the above step S103 may include the following s03-s04.
  • the computer equipment can use the second segmentation model to perform region segmentation and identification on the reorganized image, and obtain the region segmentation adjustment parameters of the pixels in the reorganized image, that is, input the reorganized image into the second segmentation model, and then perform the segmentation and identification of the pixels in the reorganized image.
  • the reorganized image is identified by region segmentation, and the region segmentation adjustment parameters of the pixels of the reorganized image are obtained.
  • the second segmentation model may refer to a model for performing optimization processing on the segmentation result of the first segmentation model, and may also be referred to as a fine segmentation model, where the segmentation result may refer to the above candidate region type probability.
  • the original image can be adjusted and segmented according to the area segmentation adjustment parameters and the candidate area type probability to obtain the target foreground image area of the original image.
  • the target background image area that is, the candidate background image area and the candidate foreground image area can be adjusted according to the area segmentation adjustment parameters and the candidate area type probability, and the adjusted candidate background image area is used as the target background image area of the original image. , take the adjusted candidate foreground image area as the target foreground image area of the original image.
  • the second segmentation model may be a depth refinement model, which is formed by stacking multiple modules with depthwise separable convolution as the basic structure.
  • the first segmentation The binary segmentation model can be formed by stacking three modules with depthwise separable convolution as the basic structure.
  • the size of the feature map (that is, the reorganized image) of the second segmentation model remains unchanged during the process of region segmentation and identification, which can avoid reorganization.
  • the size of the image is adjusted, resulting in the loss of information in the reconstructed image.
  • the original image is preliminarily segmented and identified by the first segmentation model to obtain the candidate region type probability of the pixels in the original image, and the original image is segmented according to the candidate region type probability to obtain the candidate region of the original image.
  • the background image area and the candidate background image area; that is, the initial segmentation processing of the original image can be realized through the first segmentation model.
  • a recombined image can be obtained by recombining the original image, the candidate background image area and the candidate background image area, which can compensate the boundary information lost in the initial segmentation process of the original image, which is beneficial to the optimal segmentation of the original image.
  • the processing process provides rich information; the boundary between the background image area and the foreground image area of the original image is smoother and clearer, and the accuracy of the original image segmentation is improved.
  • the second segmentation model can be used to perform regional segmentation and identification of the reconstituted image to obtain the regional segmentation adjustment parameters of the pixels in the reconstituted image, which can be adjusted according to the regional segmentation parameters and the candidate region type probability , the original image is adjusted and segmented to obtain the target foreground image area and the target background image area of the original image.
  • the segmentation prior information is provided for the second segmentation model through the first segmentation model, that is, the segmentation prior information here may refer to the probability of the candidate region type, the candidate background image region, and the candidate foreground image region.
  • the segmentation result of the segmentation model is optimized to improve the accuracy of image segmentation.
  • the first segmentation model includes a feature extraction layer and a segmentation layer
  • step s01 may include the following steps s11 and s12.
  • segmentation layer uses the segmentation layer to perform segmentation and identification on the structural feature information and the semantic feature information, and obtain the candidate region type probability of the pixel point in the original image.
  • the first segmentation model includes a feature extraction layer and a segmentation layer
  • the feature extraction layer can be used to extract the features of the original image
  • the segmentation layer can be used to perform candidate segmentation processing on the original image .
  • the feature extraction layer can be used to perform feature extraction on the original image to obtain structural feature information and semantic feature information of pixels in the original image.
  • the structural feature information can refer to the shallow features of the original image, that is, the structure
  • the feature information can be used to reflect the contour information (ie texture information) in the original image
  • the semantic feature information can refer to the deep features of the original image, that is, the semantic feature information can be used to reflect the object information in the original image, and the object information includes the type of object, size, color, etc.
  • the computer equipment can use the segmentation layer to segment and identify the structural feature information and the semantic feature information to obtain the candidate region type probability of the pixel in the original image;
  • the feature information is fused, the fused feature information is segmented and identified, and the candidate region type probability of the pixel point in the original image is obtained.
  • Both the above-mentioned first segmentation model and the second segmentation model may be models constructed based on CNN (Convolutional Neural Networks, convolutional neural networks), for example: VGGNet network (Visual Geometry Group Network, a kind of convolutional neural network), ResNet network (residual network) and AlexNet network (a convolutional neural network), etc.; it can also be a model constructed based on FCN (Fully Convolutional Networks, full neural network), which is not limited.
  • CNN Convolutional Neural Networks, convolutional neural networks
  • VGGNet network Visual Geometry Group Network, a kind of convolutional neural network
  • ResNet network residual network
  • AlexNet network a convolutional neural network
  • the first segmentation model may be a semantic segmentation model with a depthwise separable convolution structure and the ability to identify foreground image regions and background image regions.
  • the first segmentation model may include a feature extraction layer and a segmentation identification layer, the feature extraction layer may be an encoder, and the segmentation identification layer may be a decoder.
  • the encoder can be formed by stacking modules with depthwise separable convolution as the basic structure, and the decoder can be formed by using a deconvolution structure.
  • the feature transfer of enables the decoder to fuse different features, segment and identify the semantic feature information in the shallow feature and the structural feature information in the deep feature, and obtain the type probability of the candidate region.
  • the "encoder-decoder" network structure based on depthwise separable convolution enables the first segmentation model to greatly reduce the amount of computation and parameters of the network, while ensuring the effect of image segmentation.
  • step S102 may include the following steps s21-s24.
  • the computer equipment may reorganize the candidate background image area, the candidate foreground image area and the original image; specifically, the computer equipment may fuse the candidate background image area with the original image to obtain a first fusion image , that is, according to the position of the candidate background image area in the original image, the candidate background image area is fused with the original image to obtain the first fusion image.
  • the candidate foreground image region and the original image can be fused to obtain the second fusion image; according to the positions of the candidate foreground image region and the background image region in the original image, The candidate foreground image area is fused with the candidate background image area to obtain a third fusion image.
  • the original image, the first fused image, the second fused image and the third fused image are fused to obtain the recombined image; by recombining the candidate background image area, the candidate foreground image area and the original image, the Compensate the boundary information lost in the initial segmentation process of the original image, which is beneficial to optimize the segmentation process of the original image and provide rich information; make the boundary between the background image area and the foreground image area of the original image smoother , clear, and improve the accuracy of the original image segmentation.
  • step s04 may include the following steps s31-s32.
  • the computer device may perform optimal segmentation processing on the original image according to the output result of the second segmentation model to obtain the target foreground image area and the target background image area of the original image. Specifically, as shown in FIG. 5 , the computer device can adjust the probability of the candidate region type according to the region segmentation adjustment parameter to obtain the target region type probability, that is, the target region type probability is the probability of the candidate region type determined by the second segmentation model. After optimization, the accuracy of the target region type probability is higher. Therefore, the original image can be adjusted and segmented according to the type probability of the target area to obtain the target foreground image area and the target background image area of the original image.
  • the target area type probability includes a target foreground probability, and the target foreground probability is used to reflect the probability that the corresponding pixel in the original image belongs to the foreground image area; the computer device may set the target foreground probability in the original image to be greater than the foreground probability.
  • the pixels with the threshold value are determined as foreground pixels, and the pixels whose target foreground probability is less than or equal to the foreground probability threshold in the original image are determined as background pixels; further, the area belonging to the foreground pixels is segmented from the original image as the foreground.
  • Image area segment the area belonging to the background pixels from the original image as the background image area.
  • the target area type probability includes the target foreground probability.
  • the computer device can obtain the sum of the target foreground probability of the pixels in the original image as the third probability value; The ratio between the target foreground probability of the pixel and the third probability value, the pixel with the ratio greater than the second foreground ratio threshold is determined as the foreground pixel, and the ratio in the original image is less than or equal to the second foreground ratio threshold. , determined as background pixels; further, segment the area belonging to the foreground pixels from the original image as the foreground image area, and segment the area belonging to the background pixels from the original image as the background image area.
  • the target area type probability includes a target background probability, and the target background probability is used to reflect the probability that the corresponding pixel in the original image belongs to the background image area; the computer device may make the target background probability in the original image smaller than the background probability
  • the pixels with the threshold value are determined as foreground pixels, and the pixels whose target background probability in the original image is greater than or equal to the background probability threshold are determined as background pixels; further, the area belonging to the foreground pixels is divided from the original image as the foreground.
  • Image area segment the area belonging to the background pixels from the original image as the background image area.
  • the target area type probability includes the target background probability.
  • the computer device can obtain the sum of the target background probability of the pixels in the original image as the fourth probability value;
  • the ratio between the target background probability of the pixel point and the fourth probability value, the pixel point whose ratio is greater than the second background ratio threshold value is determined as the background pixel point, and the pixel point whose ratio is less than or equal to the second background ratio threshold value in the original image , determined as foreground pixels; further, segment the area belonging to the background pixels from the original image as the background image area, and segment the area belonging to the foreground pixels from the original image as the foreground image area.
  • the target area type probability includes a target background probability and a target foreground probability.
  • the computer device may select one of the target background probability and the target foreground probability, and adjust and segment the original image according to the selected probability to obtain The target foreground image area of the original image and the target background image area.
  • the region segmentation adjustment parameters include foreground segmentation adjustment parameters, background segmentation adjustment parameters and offset values;
  • the candidate region type probability includes candidate foreground probability and candidate background probability;
  • step s31 may include the following steps s41-s43.
  • s42 Generate a target foreground probability according to the probability sum and the offset value; obtain a target background probability according to the target foreground probability.
  • the area segmentation adjustment parameters include foreground segmentation adjustment parameters, background segmentation adjustment parameters and offset values, and the foreground segmentation adjustment parameters are used to reflect the accuracy of the foreground probability of the corresponding pixel point candidate in the original image (that is, the confidence level). degree), that is, the larger the foreground segmentation adjustment parameter of the pixel point, the lower the accuracy of the candidate foreground probability of the pixel point, and the greater the adjustment strength of the candidate foreground probability of the pixel point; the more the foreground segmentation adjustment parameter of the pixel point is If the value is small, it indicates that the higher the accuracy of the candidate foreground probability of the pixel point, the smaller the adjustment strength of the candidate foreground probability of the pixel point.
  • the background segmentation adjustment parameter is used to reflect the accuracy (ie confidence) of the candidate background probability of the corresponding pixel point in the original image, that is, the larger the background segmentation adjustment parameter of the pixel point, the more accurate the candidate background probability of the pixel point.
  • the offset value may refer to a balance parameter between the candidate background probability and the candidate foreground probability, and is used to fine-tune the candidate background probability and the candidate foreground probability.
  • the computer device can use the foreground segmentation adjustment parameter and the background segmentation adjustment parameter to perform a weighted sum of the candidate foreground probability and the candidate background probability to obtain a probability sum, that is, the foreground segmentation adjustment parameter is used for the candidate foreground.
  • the probability is adjusted, the background segmentation adjustment parameter is used to adjust the candidate background probability, and the sum of the adjusted candidate foreground probability and the adjusted candidate background probability is determined as the probability sum.
  • the target foreground probability can be generated according to the probability and the offset value; the target background probability can be obtained according to the target foreground probability; the target foreground probability and the target background probability can be determined as the target area type probability.
  • the target area type probability can be expressed by the following formula (2).
  • formula Pred (2) i is an objective foreground probability of the i th pixel in the original image
  • a i, b i, c i denote the foreground i-th pixel recombinant image segmentation adjustment parameters, background Segmentation adjustment parameters and offset values
  • FG i and BG i respectively represent the candidate foreground probability and the candidate background probability of the ith pixel in the original image.
  • 1-Pred i can be used as the target background probability of the ith pixel in the original image.
  • the ith pixel of the reconstructed image corresponds to the ith pixel of the original image, that is, the position information of the ith pixel of the reconstructed image is the same as the position information of the ith pixel of the original image.
  • the method may include the following steps s51-s53.
  • the computer device may acquire a background image, and the background image may refer to an image used to replace the background image area of the original image.
  • the background image includes grass and clouds; the pixels in the background image
  • the target background probability of the pixels in the original image can be determined as The target background probability of the corresponding pixel in the background image.
  • the computer device can adjust the background image according to the target background probability to obtain the background replacement area, that is, the pixels whose corresponding target background probability is greater than the background probability threshold in the background image are regarded as background pixels, and the target corresponding to the pixels in the background image are used as background pixels. Pixels whose background probability is less than or equal to the background probability threshold are regarded as foreground pixels.
  • the color information of the background pixels in the background image can be adjusted according to the color information of the target foreground image area (or, the background pixels in the background image can be blurred, such as reducing the transparency), and the foreground pixels in the background image can be adjusted.
  • Points are removed to obtain the background replacement area; the background replacement area and the target foreground area can be spliced to obtain the original image after replacing the background.
  • the background replacement area is obtained, so that the background replacement area and the target foreground area can be better spliced, even if the color and size of the background replacement area are more in line with the color and size of the target foreground area respectively , and make the original image after replacing the background smoother and clearer.
  • the person in the original image is still located in the foreground image area of the original image after replacing the background, and the grass in the background image is located in the background image area of the original image after replacing the background.
  • the original image after replacing the background can be expressed by the following formula (3).
  • RAW can refer to the original image
  • BACK can refer to the background image
  • Pred final represents the original image after replacing the background
  • Pred*RAW represents the target foreground image area obtained by dividing the original image by the target foreground probability of the original image
  • ( 1-Pred)*BACK represents the background replacement area obtained by segmenting (that is, adjusting) the background image by the target background probability of the original image
  • Pred represents the target foreground probability matrix formed by the target foreground probability of all pixels in the original image.
  • the method may further include the following steps s61-s64.
  • s61 Acquire a first candidate segmentation model and a sample image set, where the sample image set includes the sample image and the marked region type probability of the pixels in the sample image.
  • the first candidate segmentation model uses the first candidate segmentation model to predict the above-mentioned sample image, and obtain the predicted region type probability of the pixel point in the sample image, which is used as the first predicted region type probability.
  • the computer device may train the first candidate segmentation model to obtain the first segmentation model, specifically, may obtain the first candidate segmentation model and a sample image set, where the sample image set includes the sample image and the sample Annotated area type probability of pixel points in the image; the sample image set may include sample images with various target objects, such as sample images with people, sample images with animals, and sample images with buildings, etc.; The marked region type probability of the pixel point can be obtained by manually marking the sample image. Further, the first candidate segmentation model can be used to predict the above-mentioned sample image, and the predicted region type probability of the pixel in the sample image can be obtained as the first predicted region type probability.
  • the computer device may adjust the first candidate segmentation model according to the first predicted region type probability and the marked region type probability, and determine the adjusted first candidate segmentation model as the first segmentation model. By adjusting the first candidate segmentation model, the image segmentation accuracy of the first candidate segmentation model can be improved.
  • the above-mentioned step s63 may include the following steps s71-s76.
  • s72 Acquire the rate of change between the probabilities of the first predicted region types of the pixel points in the sample image as the first gradient rate of change.
  • s73 Obtain the rate of change between the probabilities of the marked region types of the pixel points in the sample image, as the second gradient rate of change.
  • s75 Determine a total loss value of the first candidate segmentation model according to the boundary loss value and the original loss value.
  • the computer device may determine the original loss value of the first candidate segmentation model according to the first predicted region type probability and the labeled region type probability, and the original loss value is used to reflect the first
  • a candidate segmentation model outputs the accuracy of the first predicted region type probability. Further, the rate of change between the first predicted region type probabilities of adjacent pixels in the sample image can be obtained, and as the first gradient change rate, the difference between the marked region type probabilities of adjacent pixels in the sample image can be obtained. rate of change, as the second gradient rate of change.
  • the first gradient change rate can be used to reflect the speed of change between the first prediction region type probabilities, usually the first prediction region type probability of the pixels at the boundary between the foreground image region and the background image region of the sample image is usually slow. Therefore, the first gradient change rate can be used to reflect the segmentation recognition accuracy of the first candidate segmentation model for the pixel points at the boundary between the foreground image area and the background image area in the sample image. Therefore, the boundary loss value of the first candidate segmentation model can be determined according to the first gradient change rate and the second gradient change rate, that is, the boundary loss value is used to reflect the boundary segmentation recognition accuracy of the first candidate segmentation model; further , the total loss value of the first candidate segmentation model can be determined according to the boundary loss value and the original loss value.
  • the first candidate segmentation model is based on the total loss value. Split the model to adjust. The first candidate segmentation model is adjusted by the boundary loss value and the original loss value to obtain the first segmentation model, which improves the accuracy and clarity of the boundary segmentation between the foreground image area and the background image area of the image by the first segmentation model.
  • the total loss value of the first candidate segmentation model can be represented by the following formula (4).
  • L 1 represents the total loss value of the first candidate segmentation model
  • L ce represents the original loss value of the first candidate segmentation model
  • L grad represents the boundary loss value of the first candidate segmentation model, which can also be referred to as Gradient loss value.
  • the original loss value L ce of the first candidate segmentation model can be represented by the following formula (5).
  • p i represents a marked area on the probabilities of the i-th pixel of the sample image
  • q i denotes the prediction region on the probabilities of the i-th pixel of the sample image
  • K represents the pixel samples in an image quantity.
  • the boundary loss value L grad of the first candidate segmentation model can be expressed by the following formula (6).
  • G (q i) represents the rate of change of a first gradient, i.e., represents the probability of a first type of prediction region of the sample image pixel gradient
  • G (p i) denotes the rate of change of the second gradient, i.e., Represents the gradient of the type probability of the labeled area of the pixel point in the sample image
  • S, S T represent the gradient operator of the pixel point in the sample image in the x-axis and y-axis directions respectively
  • S T is the transpose of S
  • S can be used It is represented by the following formula (7).
  • the method may further include the following steps s81-s86.
  • s81 Obtain a second candidate segmentation model and a target prediction region type probability, where the target prediction region type probability is the first prediction region type probability output when the total loss value of the first candidate segmentation model is in a convergent state.
  • the second candidate segmentation model may be trained according to the first candidate segmentation model.
  • the computer device can obtain the second candidate segmentation model and the target prediction region type probability; the target prediction region type probability is the output first prediction region type when the total loss value of the first candidate segmentation model is in a convergent state.
  • the probability that is, the probability of the target predicted region type is the outputted probability of the first predicted region type when the prediction accuracy of the first candidate segmentation model is relatively high.
  • the sample image can be segmented according to the target prediction area type probability to obtain the foreground image area and the background image area of the sample image, and the sample image, the foreground image area and the background image area of the sample image can be reorganized to obtain the sample image. Reorganize the image.
  • the above-mentioned second candidate segmentation model is used to predict the above-mentioned sample recombination image, and the second predicted area type probability is obtained; if the second predicted area type probability is relatively close to the marked area type probability, it indicates the prediction of the second candidate segmentation model.
  • the accuracy is relatively high; if the difference between the second predicted region type probability and the marked region type probability is relatively large, it indicates that the prediction accuracy of the second candidate segmentation model is relatively low. Therefore, the second candidate segmentation model is adjusted according to the second predicted region type probability and the marked region type probability, and by adjusting the second segmentation model, the image segmentation accuracy of the second candidate segmentation model can be improved.
  • the above step s84 includes the following steps s88-s89.
  • the computer equipment can use the second candidate segmentation model to predict the sample reorganized image, and obtain the prediction area segmentation adjustment parameters of the pixels in the sample reorganized image; the prediction area segmentation adjustment parameters are used for first.
  • the target prediction region type probability output by the candidate segmentation model is adjusted.
  • the prediction area segmentation adjustment parameter is used to adjust the target prediction area type probability to obtain the second prediction area type probability.
  • the above step s85 may include the following steps s91-s92.
  • the computer device may determine a segmentation loss value of the second candidate segmentation model according to the second predicted region type probability and the labeled region type probability, where the segmentation loss value is used to reflect the Image segmentation accuracy. Therefore, if the segmentation loss value satisfies the convergence condition, indicating that the image segmentation accuracy of the second candidate segmentation model is relatively high, the second candidate segmentation model is used as the second segmentation model. If the segmentation loss value does not satisfy the convergence condition, it indicates that the image segmentation accuracy of the second candidate segmentation model is relatively low, then the second candidate segmentation model is adjusted according to the segmentation loss value; by adjusting the second candidate segmentation model, The image segmentation accuracy of the second candidate segmentation model can be improved.
  • the segmentation loss value of the second candidate segmentation model can be represented by the following formula (8).
  • w i and p i respectively represent the second prediction region type probability and label region type probability of the ith pixel in the sample image
  • L 2 represents the segmentation loss value of the second candidate segmentation model
  • FIG. 9 is a schematic structural diagram of an image processing apparatus provided by an embodiment of the present application.
  • the above image processing apparatus may be a computer program (including program code) running in a computer device, for example, the image processing apparatus is an application software; the apparatus may be used to execute corresponding steps in the methods provided in the embodiments of the present application.
  • the image processing apparatus may include: an identification module 901 and a reorganization module 902 .
  • the identification module 901 is used to perform preliminary segmentation and identification on the original image by using the first segmentation model, and obtain candidate foreground image areas and candidate background image areas of the original image;
  • the reorganization module 902 is used to reorganize the candidate foreground image area, the candidate background image area and the original image to obtain a reorganized image; the pixel points in the reorganized image and the pixel points in the original image are different. There is a one-to-one correspondence between them;
  • the identification module 901 is further configured to perform region segmentation and identification on the reorganized image by using the second segmentation model to obtain the target foreground image region and the target background image region of the original image.
  • the identification module 901 uses the first segmentation model to perform preliminary segmentation and identification of the original image, and a specific implementation manner to obtain candidate foreground image regions and candidate background image regions of the original image includes:
  • the original image is segmented according to the type probability of the candidate region to obtain candidate foreground image regions and candidate background image regions of the original image.
  • the above-mentioned first segmentation model includes a feature extraction layer and a segmentation layer
  • the identification module 901 uses the first segmentation model to segment and identify the above-mentioned original image, and a specific implementation manner of obtaining the candidate region type probability of the pixel point in the above-mentioned original image includes:
  • the above-mentioned structural feature information and the above-mentioned semantic feature information are segmented and identified by the above-mentioned segmentation layer, and the probability of the candidate region type of the pixel point in the above-mentioned original image is obtained.
  • the reorganization module 902 reorganizes the above-mentioned candidate foreground image area, the above-mentioned candidate background image area and the above-mentioned original image, and the specific implementation manner of obtaining the reorganized image includes:
  • the above-mentioned candidate background image area is fused with the above-mentioned original image to obtain a first fusion image
  • the above-mentioned original image, the above-mentioned first fused image, the above-mentioned second fused image, and the above-mentioned third fused image are fused to obtain the above-mentioned reconstructed image.
  • the identification module 901 uses the second segmentation model to perform region segmentation and identification on the reorganized image, and a specific implementation manner to obtain the target foreground image region and the target background image region of the original image includes:
  • the original image is adjusted and segmented to obtain the target foreground image area and the target background image area of the original image.
  • the above-mentioned identification module 901 performs adjustment and segmentation processing on the above-mentioned original image according to the above-mentioned area segmentation adjustment parameters and the above-mentioned candidate area type probability, so as to obtain the specific implementation manner of the target foreground image area and the target background image area of the above-mentioned original image.
  • the above-mentioned original image is adjusted and segmented to obtain a target foreground image area and a target background image area of the above-mentioned original image.
  • the area segmentation adjustment parameters include a foreground segmentation adjustment parameter, a background segmentation adjustment parameter, and an offset value;
  • the candidate area type probability includes a candidate foreground probability and a candidate background probability;
  • the above-mentioned identification module 901 adjusts the above-mentioned candidate area type probability according to the above-mentioned area segmentation adjustment parameter, and the specific implementation manner of obtaining the target area type probability includes:
  • the above-mentioned candidate foreground probability and the above-mentioned candidate background probability are weighted and summed to obtain a probability sum
  • the target foreground probability is generated; the target background probability is obtained according to the above target foreground probability;
  • the above target area type probability is determined according to the above target foreground probability and the above target background probability.
  • the apparatus further includes: an acquisition module 906, configured to acquire a background image;
  • the adjustment module 903 is configured to adjust the background image according to the target background probability to obtain a background replacement area; splicing the background replacement area and the target foreground area to obtain the original image after replacing the background.
  • the obtaining module 906 is further configured to obtain the first candidate segmentation model and a sample image set, where the sample image set includes the sample image and the marked region type probability of the pixels in the sample image;
  • the above-mentioned apparatus further includes: a prediction module 904, configured to use the above-mentioned first candidate segmentation model to predict the above-mentioned sample image, and obtain the predicted region type probability of the pixels in the above-mentioned sample image, as the first predicted region type probability;
  • the above-mentioned adjustment module 903 is further configured to adjust the above-mentioned first candidate segmentation model according to the above-mentioned first predicted area type probability and the above-mentioned marked area type probability;
  • the apparatus further includes: a determination module 905, configured to determine the adjusted first candidate segmentation model as the above-mentioned first segmentation model.
  • the above-mentioned adjustment module 903 adjusts the above-mentioned first candidate segmentation model according to the above-mentioned first predicted area type probability and the above-mentioned marked area type probability, a specific implementation manner includes:
  • the above-mentioned first candidate segmentation model is adjusted according to the above-mentioned total loss value.
  • the obtaining module 906 is further configured to obtain the second candidate segmentation model and the target prediction region type probability.
  • the target prediction region type probability is that when the total loss value of the first candidate segmentation model is in a convergent state, the Output first predicted region type probability;
  • the above-mentioned identification module 901 is further configured to segment the above-mentioned sample image according to the above-mentioned target prediction area type probability, so as to obtain a foreground image area and a background image area of the above-mentioned sample image;
  • the above-mentioned reorganization module 902 is further configured to perform reorganization processing on the above-mentioned sample image, the foreground image area and the background image area of the above-mentioned sample image, to obtain a sample reorganized image;
  • the above-mentioned prediction module 904 is further configured to use the above-mentioned second candidate segmentation model to predict the above-mentioned sample recombination image to obtain the second predicted region type probability;
  • the above-mentioned adjustment module 903 is further configured to adjust the above-mentioned second candidate segmentation model according to the above-mentioned second predicted area type probability and the above-mentioned marked area type probability;
  • the aforementioned determining module 905 is further configured to determine the adjusted second candidate segmentation model as the aforementioned second segmentation model.
  • the prediction module 904 uses the second candidate segmentation model to predict the sample reorganized image, and a specific manner for obtaining the probability of the second predicted region type includes:
  • the above-mentioned target prediction area type probability is adjusted by using the above prediction area segmentation adjustment parameter to obtain the second prediction area type probability.
  • the specific implementation manner for the adjustment module 903 to adjust the second candidate segmentation model according to the second predicted region type probability and the labeled region type probability includes:
  • the above-mentioned second candidate segmentation model is adjusted according to the above-mentioned segmentation loss value.
  • the steps involved in the image processing method shown in FIG. 3 may be performed by modules in the image processing apparatus shown in FIG. 9 .
  • steps S101 and S103 shown in FIG. 3 may be performed by the identification module 901 shown in FIG. 9
  • step S102 shown in FIG. 3 may be performed by the reorganization module 902 shown in FIG. 9 .
  • the modules in the image processing apparatus shown in FIG. 9 can be respectively or all combined into one or several units to form, or some of the unit(s) can be further divided into functional units. Multiple smaller subunits can implement the same operation without affecting the realization of the technical effects of the embodiments of the present application.
  • the above modules are divided based on logical functions. In practical applications, the function of one module may also be implemented by multiple units, or the functions of multiple modules may be implemented by one unit. In other embodiments of the present application, the image processing apparatus may also include other units, and in practical applications, these functions may also be implemented with the assistance of other units, and may be implemented by cooperation of multiple units.
  • a general-purpose computer device such as a computer
  • a general-purpose computer device may be executed by running on a central processing unit (CPU), a random access storage medium (RAM), a read only storage medium (ROM), etc. processing elements and storage elements.
  • a computer program (including program code) capable of executing the steps involved in the corresponding methods as shown in FIG. 3 and FIG. 7 , to construct the image processing apparatus as shown in FIG. 9 , and to realize the images of the embodiments of the present application Approach.
  • the above-mentioned computer program can be recorded on, for example, a computer-readable recording medium, loaded in the above-mentioned computing device via the computer-readable recording medium, and executed therein.
  • FIG. 10 is a schematic structural diagram of a computer device provided by an embodiment of the present application.
  • the above-mentioned computer device 1000 may include: a processor 1001 , a network interface 1004 and a memory 1005 , in addition, the above-mentioned computer device 1000 may further include: a user interface 1003 , and at least one communication bus 1002 .
  • the communication bus 1002 is used to realize the connection and communication between these components.
  • the user interface 1003 may include a display screen (Display) and a keyboard (Keyboard), and the user interface 1003 may also include a standard wired interface and a wireless interface.
  • the network interface 1004 may include a standard wired interface, a wireless interface (eg, a WI-FI interface).
  • the memory 1005 can be a high-speed RAM memory, or a non-volatile memory, such as at least one disk memory.
  • the memory 1005 may also be at least one storage device located remotely from the aforementioned processor 1001 .
  • the memory 1005 as a computer-readable storage medium may include an operating system, a network communication module, a user interface module, and an image processing application program.
  • the network interface 1004 can provide a network communication function; the user interface 1003 is mainly used to provide an input interface for the user; and the processor 1001 can be used to call the image processing application stored in the memory 1005 A program to implement the image processing method described in each of the above embodiments.
  • the computer device 1000 described in this embodiment of the present application may execute the description of the above image processing method in the foregoing FIG. 3 and the corresponding embodiment, and may also execute the foregoing image processing apparatus in the foregoing embodiment corresponding to FIG. 9 . description, which will not be repeated here. In addition, the description of the beneficial effects of using the same method will not be repeated.
  • the embodiment of the present application further provides a computer-readable storage medium, and the computer program executed by the image processing apparatus mentioned above is stored in the computer-readable storage medium, and the computer program described above is stored in the computer-readable storage medium.
  • Including program instructions when the processor executes the program instructions, it can execute the description of the image processing method in the embodiment corresponding to FIG. 3 above, and therefore will not be repeated here.
  • the description of the beneficial effects of using the same method will not be repeated.
  • program instructions may be deployed and executed on one computer device, or on multiple computer devices located at one site, or alternatively, distributed across multiple sites and interconnected by a communication network.
  • Executed on a blockchain multiple computer devices distributed in multiple locations and interconnected by a communication network can form a blockchain network.
  • the above-mentioned storage medium may be a magnetic disk, an optical disk, a read-only memory (Read-Only Memory, ROM) or a random access memory (Random Access Memory, RAM) and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

本申请实施例公开了一种图像处理方法、装置、设备及存储介质，属于人工智能中的计算机视觉相关的技术领域。其中，方法包括：采用第一分割模型对原始图像进行初步分割识别，得到所述原始图像的候选前景图像区域和候选背景图像区域；对所述候选前景图像区域、所述候选背景图像区域以及所述原始图像进行重组，得到重组图像；所述重组图像中的像素点与所述原始图像中的像素点之间具有一一对应关系；采用第二分割模型对所述重组图像进行区域分割识别，得到所述原始图像的目标前景图像区域和目标背景图像区域。

Description

图像处理方法、装置、设备及存储介质
本申请要求于2020年7月23日提交中国专利局、申请号为2020107186929，发明名称为“图像处理方法、装置及设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及人工智能中的计算机视觉相关的技术领域，具体涉及一种图像处理方法、装置、设备及存储介质。
背景技术
人工智能(Artificial Intelligence，AI)是计算机科学的一个综合技术，通过研究各种智能机器的设计原理与实现方法，使机器具有感知、推理与决策的功能。人工智能技术是一门综合学科，涉及领域广泛，例如计算机视觉技术、自然语言处理技术以及机器学习/深度学习等几大方向，随着技术的发展，人工智能技术将在更多的领域得到应用，并发挥越来越重要的价值。
在基于人工智能的计算机视觉技术中，图像分割是一个重要的研究方向，能够从原始图像中分割出前景图像区域和背景图像区域，以便对前景图像区域或者背景图像区域进行相应的处理，例如人脸识别、替换背景图像区域、添加图像特效等等。
技术内容
本申请实施例提供了一种图像处理方法，包括：
采用第一分割模型对原始图像进行初步分割识别，得到所述原始图像的候选前景图像区域和候选背景图像区域；
对所述候选前景图像区域、所述候选背景图像区域以及所述原始图像进行重组，得到重组图像；所述重组图像中的像素点与所述原始图像中的像素点之间具有一一对应关系；
采用第二分割模型对所述重组图像进行区域分割识别，得到所述原始图像的目标前景图像区域和目标背景图像区域。
本申请实施例还提供了一种图像处理装置，包括：
识别模块，用于采用第一分割模型对原始图像进行初步分割识别，得到所述原始图像的候选前景图像区域和候选背景图像区域；
重组模块，用于对所述候选前景图像区域、所述候选背景图像区域以及所述原始图像进行重组，得到重组图像；所述重组图像中的像素点与所述原始图像中的像素点之间具有一一对应关系；
所述识别模块，还用于采用第二分割模型对所述重组图像进行区域分割识别，得到所 述原始图像的目标前景图像区域和目标背景图像区域。
本申请实施例还提供了一种计算机设备，包括：处理器及存储器；
其中，上述存储器用于存储计算机程序，上述处理器用于调用上述计算机程序，以执行上述方法的步骤。
本申请实施例还提供了一种非易失性计算机可读存储介质，上述计算机可读存储介质存储有计算机程序，上述计算机程序包括程序指令，上述程序指令当被处理器执行时，以执行上述方法的步骤。
附图简要说明
图1是本申请提供的一种图像处理系统的架构示意图；
图2a是本申请提供的一种图像处理系统的各个设备之间交互场景示意图；
图2b是本申请提供的一种图像处理系统的各个设备之间交互场景示意图；
图3是本申请实施例提供的一种图像处理方法的流程示意图；
图4是本申请实施例提供的一种获取候选区域类型概率的场景示意图；
图5是本申请实施例提供的一种对原始图像调整分割得到原始图像的目标前景图像区域以及目标背景图像区域的场景示意图；
图6是本申请实施例提供的一种对原始图像调整分割得到原始图像的目标前景图像区域以及目标背景图像区域的场景示意图；
图7是本申请实施例提供的一种对第一候选分割模型进行调整得到第一分割模型的场景示意图；
图8是本申请实施例提供的一种对第二候选分割模型进行调整得到第二分割模型的场景示意图；
图9是本申请实施例提供的一种图像处理装置的结构示意图；
图10是本申请实施例提供的一种计算机设备的结构示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请实施例提供的图像处理方法主要涉及人工智能中的计算机视觉技术，具体涉及计算机视觉技术中的图像分割技术。首先介绍用于实现本申请的图像处理方法的图像处理系统，如图1所示，该图像处理系统中包括服务器10、至少一个终端11。
其中，终端11可以是指面向用户的终端，即终端11可以是指用户用于获取原始图像，并将图像发送至服务器10的终端。原始图像可以是指需要进行图像分割处理的图像，该原始图像可以是由终端11所拍摄得到的，或者，该原始图像可以是由终端11从网络中下载得到的。服务器10可以是指图像处理的后端服务设备，具体可用于从终端11中获取原始图像，对原始图像进行分割处理，得到原始图像的目标前景图像区域或目标背景图像区域，目标前景图像区域可以是指包括目标对象的区域，目标背景图像区域可以是指不包括目标对象的区域，目标对象可以是指人物、建筑物、动物以及物品等等。
在一些实施例中，上述对原始图像的分割处理过程可以由终端11或服务器10来实现，本申请下面以服务器10对原始图像进行分割处理为例进行说明，终端11对原始图像的分割处理过程可以参考服务器10对原始图像进行分割处理过程，本申请对此不再赘述。
其中，服务器10可以是独立的一个物理服务器，也可以是多个物理服务器构成的服务器集群或者分布式系统，还可以是提供云服务、云数据库、云计算、云函数、云存储、网络服务、云通信、中间件服务、域名服务、安全服务、内容分发网络(Content Delivery Network，CDN)、以及大数据和人工智能平台等基础云计算服务的云服务器。终端11可以是智能手机、平板电脑、笔记本电脑、台式计算机、智能音箱、智能手表等，但并不局限于此。终端以及服务器可以通过有线或无线通信方式进行直接或间接地连接，本申请在此不做限制。
在具体实现中，如图2a中，终端11可以从网络下载图像，或者，拍摄得到图像，将该图像作为原始图像，将原始图像发送至服务器10；该原始图像可以为人物图像，该原始图像中包括前景图像区域12以及背景图像区域13，前景图像区域12是指包括人物的区域，背景图像区域13是指人物后方的区域，例如，背景图像区域13包括植物。相应地，服务器10可以接收终端11所发送的原始图像，将该原始图像输入至第一分割模型，对该原始图像进行初步分割识别得到该原始图像中的像素点的候选区域类型概率；该初步分割识别可以是指对原始图像的初步识别处理，该候选区域类型概率包括用于指示原始图像中的像素点属于背景图像区域的概率，即候选背景概率；或者，候选区域类型概率包括用于指示原始图像中的像素点属于前景图像区域的概率，即候选前景概率；或者，候选区域类型概率包括候选背景概率以及候选前景概率。
在服务器10获取到候选区域类型概率后，可以根据该候选区域类型概率对该原始图像进行分割处理，得到该原始图像的候选背景图像区域以及候选前景图像区域。如图2a中，候选背景图像区域以及候选前景图像区域中缺少部分线条，即候选前景图像区域以及候选背景图像区域中存在信息丢失的问题，即通过第一分割模型得到的分割结果的准确度 比较低，因此，需要对第一分割模型的分割结果进行优化处理。具体的，如图2b中，服务器10可对该原始图像、该候选背景图像区域以及候选前景图像区域进行重组，得到重组图像，采用第二分割模型对该重组图像进行区域分割识别，得到该重组图像中的像素点的区域分割调整参数，即该区域分割调整参数可以是指用于优化第一分割模型的分割结果的参数。进一步，可根据该区域分割调整参数以及候选区域类型概率，对该原始图像进行调整分割处理，得到该原始图像的目标前景图像区域和目标背景图像区域。将图2a中的候选背景图像区域以及候选前景图像区域，与图2b中的目标背景图像区域以及目标前景图像区域相比可知，目标背景图像区域与目标前景图像区域之间的边界更加平滑、更精确。
可见，本申请中，通过第一分割模型对原始图像进行初步分割处理，并通过第二分割模型对第一分割模型的分割结果进行优化，可提高对图像分割的准确度。另外，通过对该原始图像、该候选背景图像区域以及候选前景图像区域进行重组，为第二分割模型的区域分割识别过程提供更多信息量，避免在对原始图像进行分割处理的过程中，导致原始图像的信息丢失的问题，可进一步提高对图像分割的准确度。
基于上述的描述，请参见图3，是本申请实施例提供的一种图像处理方法的流程示意图。该方法可由计算机设备来执行，该计算机设备可以是指图1中的服务设备10或者终端11，如图3所示，该图像处理方法可以包括如下步骤S101～S104。
S101、采用第一分割模型对原始图像进行初步分割识别，得到该原始图像的候选前景图像区域和候选背景图像区域。
该前景图像区域可以是指包括目标对象的区域，该目标对象可以是指人物、建筑、物品、植物、动物等中的至少一种，即目标对象可以是指用户感兴趣的对象；背景图像区域可以是指目标对象的后方的人或物所在的区域，即背景图像区域为用户不感兴趣的对象所在的区域。
在计算机设备获取到原始图像后，可以采用第一分割模型对该原始图像进行初步分割识别，得到该原始图像的候选前景图像区域以及候选背景图像区域；该第一分割模型可以是指用于对原始图像进行初步分割识别的模型，该候选前景图像区域以及候选背景图像区域可以是指对该原始图像进行初步分割处理得到的。
在一些实施例中，上述步骤S101可包括如下s01～s02。
s01、采用第一分割模型对该原始图像进行分割识别，得到该原始图像中的像素点的候选区域类型概率。
s02、根据该候选区域类型概率对该原始图像进行分割处理，得到该原始图像的候选前景图像区域和候选背景图像区域。
在步骤s01～s02中，在计算机设备获取到原始图像后，可以采用第一分割模型对该原始图像进行分割识别，得到该原始图像中的像素点的候选区域类型概率；即将该原始图像输入至第一分割模型中，对该原始图像进行分割识别，得到该原始图像中的像素点的候选区域类型概率。在计算机设备获取到该原始图像中的像素点的候选区域类型概率后，可以根据该候选区域类型概率对该原始图像进行分割出来，得到该原始图像的候选前景图像区域以及候选背景图像区域。
在一些实施例中，该候选区域类型概率包括候选前景概率，候选前景概率用于反映原始图像中对应像素点属于前景图像区域的概率；计算机设备可以将原始图像中候选前景概率大于前景概率阈值的像素点，确定为前景像素点，将原始图像中候选前景概率小于或等于前景概率阈值的像素点，确定为背景像素点；进一步，从原始图像中分割属于前景像素点的区域，作为前景图像区域，从原始图像中分割属于背景像素点的区域，作为背景图像区域。
在一些实施例中，候选区域类型概率包括候选前景概率，计算机设备获取到候选前景概率后，可以获取原始图像中的像素点的候选前景概率之和，作为第一概率值；获取原始图像中的像素点的候选前景概率与第一概率值之间的比值，将比值大于第一前景比值阈值的像素点，确定为前景像素点，将原始图像中比值小于或等于第一前景比值阈值的像素点，确定为背景像素点；进一步，从原始图像中分割属于前景像素点的区域，作为前景图像区域，从原始图像中分割属于背景像素点的区域，作为背景图像区域。其中，原始图像中的像素点的候选前景概率与第一概率值之间的比值可以采用如下公式(1)表示。
Figure PCTCN2021104481-appb-000001
在公式(1)中，F i表示原始图像中第i个像素点的候选前景概率与第一概率值之间的比值，K表示原始图像中的像素点的数量，FG i表示原始图像中的像素点的候选前景概率。
在一些实施例中，该候选区域类型概率包括候选背景概率，候选背景概率用于反映原始图像中对应像素点属于背景图像区域的概率；计算机设备可以将原始图像中候选背景概率小于背景概率阈值的像素点，确定为前景像素点，将原始图像中候选背景概率大于或等于背景概率阈值的像素点，确定为背景像素点；进一步，从原始图像中分割属于前景像素点的区域，作为前景图像区域，从原始图像中分割属于背景像素点的区域，作为背景图像区域。
在一些实施例中，候选区域类型概率包括候选背景概率，计算机设备获取到候选背景概率后，可以获取原始图像中的像素点的候选背景概率之和，作为第二概率值；获取原始 图像中的像素点的候选背景概率与第二概率值之间的比值，将比值大于第一背景比值阈值的像素点，确定为背景像素点，将原始图像中比值小于或等于第一背景比值阈值的像素点，确定为前景像素点；进一步，从原始图像中分割属于背景像素点的区域，作为背景图像区域，从原始图像中分割属于前景像素点的区域，作为前景图像区域。
在一些实施例中，该候选区域类型概率包括候选背景概率和候选前景概率，计算机设备可以选择候选背景概率和候选前景概率中的一种，根据所选择的概率对原始图像进行分割处理，得到原始图像的候选前景图像区域以及候选背景图像区域。
S102、对该候选前景图像区域、该候选背景图像区域以及该原始图像进行重组，得到重组图像；该重组图像中的像素点与该原始图像中的像素点之间具有一一对应关系。
由于对原始图像进行分割处理过程中，容易导致候选前景图像区域与候选背景图像区域之间的边界信息丢失，因此，计算机设备可以对该候选前景图像区域、该候选背景图像区域以及该原始图像进行重组，得到重组图像，即对该候选前景图像区域、该候选背景图像区域以及该原始图像进行融合处理，将融合处理得到的图像作为重组图像。该重组图像相比原始图像，能够提供更多信息量，即可实现对原始图像在分割处理过程中所丢失的边界信息量进行补偿，有利于提高对图像分割的准确度。其中，该重组图像与原始图像的尺寸相同，即重组图像中的像素点与原始图像中的像素点之间具有一一对应关系，即原始图像和重组图像中具有相同位置信息的两个像素点之间具有对应关系。例如，均以原始图像和重组图像的左下角为坐标原点建立坐标系，确定原始图像和重组图像中具有相同位置坐标的两个像素点之间具有对应关系，如确定原始图像中位置坐标为(1,2)的像素点，与重组图像中位置坐标为(1,2)的像素点之间具有对应关系。
S103、采用第二分割模型对该重组图像进行区域分割识别，得到该原始图像的目标前景图像区域和目标背景图像区域。
计算机设备可以采用第二分割模型对第一分割模型的分割结果进行优化处理，得到原始图像的目标背景图像区域以及目标前景图像区域；即目标背景图像区域以及目标背景图像区域可以是指对候选前景图像区域以及候选背景图像区域进行优化得到的。
在一些实施例中，上述步骤S103可包括如下s03～s04。
s03、采用第二分割模型对该重组图像进行区域分割识别，得到该重组图像中的像素点的区域分割调整参数。
s04、根据该区域分割调整参数以及该候选区域类型概率，对该原始图像进行调整分割处理，得到该原始图像的目标前景图像区域和目标背景图像区域。
步骤s03～s04中，计算机设备可以采用第二分割模型对该重组图像进行区域分割识别， 得到该重组图像中的像素点的区域分割调整参数，即将该重组图像输入该第二分割模型中，对该重组图像进行区域分割识别，得到该重组图像的像素点的区域分割调整参数。该第二分割模型可以是指用于对第一分割模型的分割结果进行优化处理的模型，也可以称为细分割模型，此处分割结果可以是指上述候选区域类型概率。在计算机设备获取到重组图像中的像素点的区域分割调整参数后，可以根据该区域分割调整参数以及该候选区域类型概率，对该原始图像进行调整分割处理，得到该原始图像的目标前景图像区域和目标背景图像区域，即可以根据该区域分割调整参数以及该候选区域类型概率，对候选背景图像区域以及候选前景图像区域进行调整，将调整后的候选背景图像区域作为原始图像的目标背景图像区域，将调整后的候选前景图像区域作为原始图像的目标前景图像区域。
其中，第二分割模型可以为深度细化模型，由具有深度可分离卷积为基础结构的多个模块堆叠而成，考虑到第二分割模型的复杂度以及区域分割识别的准确度，该第二分割模型可以是由具有深度可分离卷积为基础结构的三个模块堆叠而成，第二分割模型在区域分割识别过程中特征图(即重组图像)的尺寸大小不变，可避免对重组图像的尺寸大小进行调整，导致重组图像的信息量丢失。
本申请实施例中，通过第一分割模型对原始图像进行初步分割识别，得到原始图像中的像素点的候选区域类型概率，根据该候选区域类型概率对原始图像进行分割处理，得到原始图像的候选背景图像区域和候选背景图像区域；即通过第一分割模型可实现对原始图像的初步分割处理。进一步，可以通过对原始图像、候选背景图像区域和候选背景图像区域进行重组，得到重组图像，可实现对原始图像在初步分割处理过程所丢失的边界信息进行补偿，有利于对原始图像进行优化分割处理过程提供丰富的信息量；使原始图像的背景图像区域与前景图像区域之间的边界更加平滑、清晰，提高对原始图像分割的准确度。在计算机设备获取到重组图像后，可以采用第二分割模型对该重组图像进行区域分割识别，得到重组图像中的像素点的区域分割调整参数，可以根据该区域分割调整参数以及该候选区域类型概率，对该原始图像进行调整分割处理，得到该原始图像的目标前景图像区域和目标背景图像区域。即通过第一分割模型为第二分割模型提供分割先验信息，即此处分割先验信息可以是指候选区域类型概率、候选背景图像区域以及候选前景图像区域，通过第二分割模型对第一分割模型的分割结果进行优化处理，可提高图像分割的准确度。
在一些实施例中，该第一分割模型包括特征提取层以及分割层，步骤s01可包括如下步骤s11和s12。
s11、采用该特征提取层对该原始图像进行特征提取，得到该原始图像中的像素点的结构特征信息以及语义特征信息。
s12、采用该分割层对该结构特征信息以及该语义特征信息进行分割识别，得到该原始图像中的像素点的候选区域类型概率。
在步骤s11和s12中，如图4所示，该第一分割模型包括特征提取层以及分割层，该特征提取层可以用于提取原始图像的特征，分割层可用于对原始图像进行候选分割处理。具体的，可采用该特征提取层对该原始图像进行特征提取，得到该原始图像中的像素点的结构特征信息以及语义特征信息，该结构特征信息可以是指原始图像的浅层特征，即结构特征信息可用于反映原始图像中的轮廓信息(即纹理信息)；语义特征信息可以是指原始图像的深层特征，即语义特征信息可用于反映原始图像中的对象信息，对象信息包括对象的类型、尺寸、颜色等等。进一步，计算机设备可采用该分割层对该结构特征信息以及该语义特征信息进行分割识别，得到该原始图像中的像素点的候选区域类型概率；即采用该分割层对该结构特征信息以及该语义特征信息进行融合，对融合后的特征信息进行分割识别，得到该原始图像中的像素点的候选区域类型概率。
上述第一分割模型以及第二分割模型均可以是基于CNN(Convolutional Neural Networks，卷积神经网络)所构建的模型，例如：VGGNet网络(Visual Geometry Group Network，一种卷积神经网络)、ResNet网络(残差网络)以及AlexNet网络(一种卷积神经网络)，等等；也可以是基于FCN(Fully Convolutional Networks，全神经网络)所构建的模型，对此不作限定。
例如，该第一分割模型可以为具有深度可分离卷积结构，且具有识别前景图像区域和背景图像区域能力的语义分割模型。该第一分割模型可包括特征提取层以及分割识别层，该特征提取层可以为一个编码器，该分割识别层可以为一个解码器。其中，编码器可以是以深度可分离卷积为基础结构的模块堆叠而成，解码器可以是采用反卷积结构构成的，编码器和解码器之间通过跳跃链接进行浅层特征和深层特征的特征传递，使解码器能够融合不同的特征，对浅层特征中的语义特征信息和深层特征中的结构特征信息进行分割识别，得到候选区域类型概率。这样以深度可分离卷积为基础结构的“编码器-解码器”网络结构，使第一分割模型极大地减少了网络的计算量和参数量，同时可确保图像分割效果。
在一些实施例中，步骤S102可包括如下步骤s21～s24。
s21、将该候选背景图像区域与该原始图像进行融合，得到第一融合图像。
s22、将该候选前景图像区域与该原始图像进行融合，得到第二融合图像。
s23、将该候选前景图像区域与该候选背景图像区域进行融合，得到第三融合图像。
s24、将该原始图像、该第一融合图像、该第二融合图像以及该第三融合图像进行融合，得到该重组图像。
在步骤s21～s24中，计算机设备可以对候选背景图像区域、候选前景图像区域以及原始图像进行重组；具体的，计算机设备可以对该候选背景图像区域与该原始图像进行融合，得到第一融合图像，即按照候选背景图像区域在原始图像中的位置，对该候选背景图像区域与该原始图像进行融合，得到第一融合图像。可以按照候选前景图像区域在原始图像中的位置，对该候选前景图像区域与该原始图像进行融合，得到第二融合图像；可以按照候选前景图像区域以及背景图像区域在原始图像中的位置，对该候选前景图像区域与该候选背景图像区域进行融合，得到第三融合图像。然后，对该原始图像、该第一融合图像、该第二融合图像以及该第三融合图像进行融合，得到该重组图像；通过对候选背景图像区域、候选前景图像区域以及原始图像进行重组，可实现对原始图像在初步分割处理过程所丢失的边界信息进行补偿，有利于对原始图像进行优化分割处理过程提供丰富的信息量；使原始图像的背景图像区域与前景图像区域之间的边界更加平滑、清晰，提高对原始图像分割的准确度。
在一些实施例中，步骤s04可包括如下步骤s31～s32。
s31、根据该区域分割调整参数对该候选区域类型概率进行调整，得到目标区域类型概率。
s32、根据该目标区域类型概率，对该原始图像进行调整分割处理，得到该原始图像的目标前景图像区域和目标背景图像区域。
在步骤s31～s32中，计算机设备可以根据第二分割模型的输出结果，对原始图像进行优化分割处理，得到该原始图像的目标前景图像区域和目标背景图像区域。具体的，如图5所示，计算机设备可以根据该区域分割调整参数对该候选区域类型概率进行调整，得到目标区域类型概率，即该目标区域类型概率是由第二分割模型对候选区域类型概率进行优化得到的，目标区域类型概率的准确度更高。因此，可以根据该目标区域类型概率，对该原始图像进行调整分割处理，得到该原始图像的目标前景图像区域和目标背景图像区域。
在一些实施例中，该目标区域类型概率包括目标前景概率，目标前景概率是用于反映原始图像中对应的像素点属于前景图像区域的概率；计算机设备可以将原始图像中目标前景概率大于前景概率阈值的像素点，确定为前景像素点，将原始图像中目标前景概率小于或等于前景概率阈值的像素点，确定为背景像素点；进一步，从原始图像中分割属于前景像素点的区域，作为前景图像区域，从原始图像中分割属于背景像素点的区域，作为背景图像区域。
在一些实施例中，目标区域类型概率包括目标前景概率，计算机设备获取到目标前景概率后，可以获取原始图像中的像素点的目标前景概率之和，作为第三概率值；获取原始 图像中的像素点的目标前景概率与第三概率值之间的比值，将比值大于第二前景比值阈值的像素点，确定为前景像素点，将原始图像中比值小于或等于第二前景比值阈值的像素点，确定为背景像素点；进一步，从原始图像中分割属于前景像素点的区域，作为前景图像区域，从原始图像中分割属于背景像素点的区域，作为背景图像区域。
在一些实施例中，该目标区域类型概率包括目标背景概率，目标背景概率是用于反映原始图像中对应的像素点属于背景图像区域的概率；计算机设备可以将原始图像中目标背景概率小于背景概率阈值的像素点，确定为前景像素点，将原始图像中目标背景概率大于或等于背景概率阈值的像素点，确定为背景像素点；进一步，从原始图像中分割属于前景像素点的区域，作为前景图像区域，从原始图像中分割属于背景像素点的区域，作为背景图像区域。
在一些实施例中，目标区域类型概率包括目标背景概率，计算机设备获取到目标背景概率后，可以获取原始图像中的像素点的目标背景概率之和，作为第四概率值；获取原始图像中的像素点的目标背景概率与第四概率值之间的比值，将比值大于第二背景比值阈值的像素点，确定为背景像素点，将原始图像中比值小于或等于第二背景比值阈值的像素点，确定为前景像素点；进一步，从原始图像中分割属于背景像素点的区域，作为背景图像区域，从原始图像中分割属于前景像素点的区域，作为前景图像区域。
在一些实施例中，该目标区域类型概率包括目标背景概率和目标前景概率，计算机设备可以选择目标背景概率和目标前景概率中的一种，根据所选择的概率对原始图像进行调整分割处理，得到原始图像的目标前景图像区域以及目标背景图像区域。
在此实施例中，该区域分割调整参数包括前景分割调整参数、背景分割调整参数以及偏移值；该候选区域类型概率包括候选前景概率以及候选背景概率；步骤s31可包括如下步骤s41～s43。
s41、采用该前景分割调整参数、该背景分割调整参数，对该候选前景概率以及该候选背景概率进行加权求和，得到概率和。
s42、根据该概率和以及该偏移值，生成目标前景概率；根据该目标前景概率获取目标背景概率。
s43、根据该目标前景概率以及该目标背景概率，确定该目标区域类型概率。
在步骤s41～s43中，该区域分割调整参数包括前景分割调整参数、背景分割调整参数以及偏移值，该前景分割调整参数用于反映原始图像中对应像素点候选前景概率的准确度(即置信度)，即像素点的前景分割调整参数越大，表明该像素点的候选前景概率的准确度越低，对该像素点的候选前景概率的调整力度越大；像素点的前景分割调整参数越小， 表明该像素点的候选前景概率的准确度越高，对该像素点的候选前景概率的调整力度越小。同理，该背景分割调整参数用于反映原始图像中对应像素点候选背景概率的准确度(即置信度)，即像素点的背景分割调整参数越大，表明该像素点的候选背景概率的准确度越低，对该像素点的候选背景概率的调整力度越大；像素点的背景分割调整参数越小，表明该像素点的候选背景概率的准确度越高，对该像素点的候选背景概率的调整力度越小。偏移值可以是指候选背景概率与候选前景概率之间的平衡参数，用于实现对候选背景概率与候选前景概率进行微调。具体应用中，计算机设备可以采用该前景分割调整参数、该背景分割调整参数，对该候选前景概率以及该候选背景概率进行加权求和，得到概率和，即采用该前景分割调整参数对该候选前景概率进行调整，采用背景分割调整参数对该候选背景概率进行调整，将调整后的候选前景概率以及调整后的候选背景概率之和，确定为概率和。进一步，可根据该概率和以及该偏移值，生成目标前景概率；根据该目标前景概率获取目标背景概率；将该目标前景概率以及该目标背景概率，确定为该目标区域类型概率。
在一些实施例中，目标区域类型概率可以采用如下公式(2)表示。
Pred i＝a i*FG i+b i*BG i+c i     (2)
其中，公式(2)中的Pred i表示原始图像中第i个像素点的目标前景概率，a i、b i、c i分别表示重组图像中的第i个像素点的前景分割调整参数、背景分割调整参数以及偏移值；FG i、BG i分别表示原始图像中第i个像素点的候选前景概率以及候选背景概率。在计算得到原始图像中的第i个像素点的目标前景概率后，可以将1-Pred i作为原始图像中第i个像素点的目标背景概率。其中，重组图像的第i个像素点与原始图像的第i个像素点相对应，即重组图像的第i个像素点的位置信息与原始图像的第i个像素点的位置信息相同。
在一个实施例中，该方法可包括如下步骤s51～s53。
s51、获取背景图像。
s52、根据该目标背景概率对该背景图像进行调整，得到背景替换区域。
s53、对该背景替换区域以及该目标前景区域进行拼接，得到替换背景后的原始图像。
在步骤s51～s53中，在识别到原始图像的目标背景图像区域和目标前景图像区域之后，采用原始图像替换背景操作。具体的，计算机设备可以获取背景图像，该背景图像可以是指用于替换原始图像的背景图像区域的图像，如图6所示，该背景图像中包括草以及云朵；该背景图像中的像素点与原始图像的中的像素点之间具有一一对应关系，即背景图像和原始图像中具有相同位置信息的像素点之间具有对应关系，即可以将原始图像中像素点的目标背景概率确定为背景图像中对应像素点的目标背景概率。计算机设备可以根据该目标背景概率对该背景图像进行调整，得到背景替换区域，即将背景图像中对应目标背景概率大 于背景概率阈值的像素点，作为背景像素点，将背景图像中对应像素点的目标背景概率小于或等于背景概率阈值的像素点，作为前景像素点。可以根据目标前景图像区域的颜色信息对背景图像中的背景像素点的颜色信息进行调整(或者，可以对背景图像中的背景像素点虚化处理，如降低透明度)，并将背景图像中前景像素点移除，得到背景替换区域；可对背景替换区域以及该目标前景区域进行拼接，得到替换背景后的原始图像。通过根据目标背景概率对该背景图像进行调整，得到背景替换区域，使背景替换区域与目标前景区域能够更好的拼接，即使背景替换区域的颜色、大小分别与目标前景区域的颜色、大小更加契合，并使替换背景后的原始图像更加平滑、清晰。如图6中，原始图像中的人物仍然位于替换背景后的原始图像的前景图像区域中，背景图像中的草位于替换背景后的原始图像的背景图像区域中。
在一些实施例中，替换背景后的原始图像可以采用如下公式(3)表示。
Pred final＝Pred*RAW+(1-Pred)*BACK   (3)
其中，RAW可以是指原始图像，BACK可以是指背景图像，Pred final表示替换背景后的原始图像，Pred*RAW表示通过原始图像的目标前景概率对原始图像进行分割得到的目标前景图像区域，(1-Pred)*BACK表示通过原始图像的目标背景概率对背景图像进行分割(即调整)得到的背景替换区域；Pred表示原始图像中所有像素点的目标前景概率所构成的目标前景概率矩阵。
在一个实施例中，该方法可进一步包括如下步骤s61～s64。
s61、获取第一候选分割模型以及样本图像集，该样本图像集包括样本图像以及该样本图像中的像素点的标注区域类型概率。
s62、采用该第一候选分割模型对上述样本图像进行预测，得到该样本图像中的像素点的预测区域类型概率，作为第一预测区域类型概率。
s63、根据该第一预测区域类型概率以及该标注区域类型概率，对该第一候选分割模型进行调整。
s64、将调整后的第一候选分割模型确定为该第一分割模型。
在步骤s61～s64中，计算机设备可以对第一候选分割模型进行训练，得到第一分割模型，具体的，可以获取第一候选分割模型以及样本图像集，该样本图像集包括样本图像以及该样本图像中的像素点的标注区域类型概率；样本图像集中可包括具有多种目标对象的样本图像，如包括具有人物的样本图像、具有动物的样本图像以及具有建筑的样本图像等等；样本图像中的像素点的标注区域类型概率可以是指人工对样本图像进行标注得到的。进一步，可采用该第一候选分割模型对上述样本图像进行预测，得到该样本图像中的像素 点的预测区域类型概率，作为第一预测区域类型概率，如果该第一预测区域类型概率与该标注区域类型概率比较接近，则表明该第一候选分割模型的预测准确度比较高；如果该第一预测区域类型概率与该标注区域类型概率相差比较大，则表明该第一候选分割模型的预测准确度比较低。因此，计算机设备可以根据该第一预测区域类型概率以及该标注区域类型概率，对该第一候选分割模型进行调整，将调整后的第一候选分割模型确定为该第一分割模型。通过对第一候选分割模型进行调整，可以提高第一候选分割模型的图像分割准确度。
在一些实施例中，上述步骤s63可包括如下步骤s71～s76。
s71、根据该第一预测区域类型概率以及该标注区域类型概率，确定该第一候选分割模型的原始损失值。
s72、获取该样本图像中像素点的第一预测区域类型概率之间的变化率，作为第一梯度变化率。
s73、获取该样本图像中像素点的标注区域类型概率之间的变化率，作为第二梯度变化率。
s74、根据该第一梯度变化率以及该第二梯度变化率，确定该第一候选分割模型的边界损失值。
s75、根据该边界损失值以及该原始损失值确定该第一候选分割模型的总损失值。
s76、若该总损失值不满足收敛条件，则根据该总损失值对该第一候选分割模型进行调整。
在步骤s71～s76中，如图7中，计算机设备可以根据该第一预测区域类型概率以及该标注区域类型概率，确定该第一候选分割模型的原始损失值，该原始损失值用于反映第一候选分割模型输出第一预测区域类型概率的准确度。进一步，可以获取该样本图像中的相邻像素点的第一预测区域类型概率之间的变化率，作为第一梯度变化率，获取该样本图像中相邻像素点的标注区域类型概率之间的变化率，作为第二梯度变化率。第一梯度变化率可用于反映第一预测区域类型概率之间变化快慢，通常样本图像的前景图像区域与背景图像区域之间的边界处的像素点的第一预测区域类型概率之间通常是缓慢变化的；因此，第一梯度变化率可用于体现第一候选分割模型对样本图像中的前景图像区域与背景图像区域之间的边界处的像素点的分割识别准确度。因此，可根据该第一梯度变化率以及该第二梯度变化率，确定该第一候选分割模型的边界损失值，即边界损失值用于反映第一候选分割模型的边界分割识别准确度；进一步，可根据该边界损失值以及该原始损失值确定该第一候选分割模型的总损失值。若该总损失值满足收敛条件，表明该第一候选分割模型的边 界分割识别准确度以及输出的第一预测区域类型概率的准确度比较高，则可以将第一候选分割模型作为第一分割模型；若该总损失值不满足收敛条件，表明该第一候选分割模型的边界分割识别准确度以及输出的第一预测区域类型概率的准确度比较低，则根据该总损失值对该第一候选分割模型进行调整。通过边界损失值以及原始损失值对第一候选分割模型进行调整，得到第一分割模型，提高第一分割模型对图像的前景图像区域与背景图像区域的边界分割的准确度、以及清晰度。
在一些实施例中，该第一候选分割模型的总损失值可以采用如下公式(4)表示。
L 1＝L ce+L grad   (4)
在公式(4)中，L 1表示第一候选分割模型的总损失值，L ce表示第一候选分割模型的原始损失值，L grad表示第一候选分割模型的边界损失值，也可以称为梯度损失值。
其中，第一候选分割模型的原始损失值L ce可以采用如下公式(5)表示。
Figure PCTCN2021104481-appb-000002
在公式(5)中，p i表示样本图像的第i个像素点的标注区域类型概率，q i表示样本图像的第i个像素点的预测区域类型概率，K表示样本图像中的像素点的数量。
其中，第一候选分割模型的边界损失值L grad可以采用如下公式(6)表示。
Figure PCTCN2021104481-appb-000003
在公式(6)中，G(q i)表示第一梯度变化率，即表示该样本图像中像素点的第一预测区域类型概率的梯度，G(p i)表示第二梯度变化率，即表示该样本图像中像素点的标注区域类型概率的梯度，S、S T分别表示样本图像中的像素点在x轴和y轴方向的梯度算子，S T为S的转置，S可以采用如下公式(7)表示。
Figure PCTCN2021104481-appb-000004
在一些实施例中，该方法可进一步包括如下步s81～s86。
s81、获取第二候选分割模型，以及目标预测区域类型概率，该目标预测区域类型概率为该第一候选分割模型的总损失值处于收敛状态时，所输出的第一预测区域类型概率。
s82、根据该目标预测区域类型概率对该样本图像进行分割，得到该样本图像的前景图像区域以及背景图像区域。
s83、对该样本图像、该样本图像的前景图像区域以及背景图像区域进行重组处理，得到样本重组图像。
s84、采用该第二候选分割模型对该样本重组图像进行预测，得到第二预测区域类型概率。
s85、根据该第二预测区域类型概率以及该标注区域类型概率对该第二候选分割模型进行调整。
s86、将调整后的第二候选分割模型确定为该第二分割模型。
在步骤s81～s86中，图8所示，在第一候选分割模型训练完成后，可以根据第一候选分割模型对第二候选分割模型进行训练。具体的，计算机设备可以获取第二候选分割模型，以及目标预测区域类型概率；该目标预测区域类型概率为该第一候选分割模型的总损失值处于收敛状态时，所输出的第一预测区域类型概率，即该目标预测区域类型概率为第一候选分割模型的预测准确度比较高时，所输出的第一预测区域类型概率。可以根据该目标预测区域类型概率对该样本图像进行分割，得到该样本图像的前景图像区域以及背景图像区域，对该样本图像、该样本图像的前景图像区域以及背景图像区域进行重组处理，得到样本重组图像。采用上述第二候选分割模型对上述样本重组图像进行预测，得到第二预测区域类型概率；如果该第二预测区域类型概率与该标注区域类型概率比较接近，则表明该第二候选分割模型的预测准确度比较高；如果该第二预测区域类型概率与该标注区域类型概率相差比较大，则表明该第二候选分割模型的预测准确度比较低。因此，根据该第二预测区域类型概率以及该标注区域类型概率对该第二候选分割模型进行调整，通过对第二分割模型进行调整，可以提高第二候选分割模型的图像分割准确度。
在一些实施例中，上述步骤s84包括如下步骤s88～s89。
s88、采用该第二候选分割模型对该样本重组图像进行预测，得到该样本重组图像中像素点的预测区域分割调整参数。
s89、采用该预测区域分割调整参数对该目标预测区域类型概率进行调整，得到第二预测区域类型概率。
上述步骤s88～s89中，计算机设备可以采用该第二候选分割模型对该样本重组图像进行预测，得到上述样本重组图像中像素点的预测区域分割调整参数；预测区域分割调整参数用于对第一候选分割模型输出的目标预测区域类型概率进行调整。然后，采用该预测区域分割调整参数对该目标预测区域类型概率进行调整，得到第二预测区域类型概率。
在一些实施例中，上述步骤s85可包括如下步骤s91～s92。
s91、根据该第二预测区域类型概率以及该标注区域类型概率，确定该第二候选分割模型的分割损失值。
s92、若该分割损失值不满足收敛条件，则根据该分割损失值对该第二候选分割模型 进行调整。
在步骤s91～s92中，计算机设备可以根据该第二预测区域类型概率以及该标注区域类型概率，确定该第二候选分割模型的分割损失值，该分割损失值用于反映第二候选分割模型的图像分割准确度。因此，若该分割损失值满足收敛条件，表明第二候选分割模型的图像分割准确度比较高，则将第二候选分割模型作为第二分割模型。若该分割损失值不满足收敛条件，表明第二候选分割模型的图像分割准确度比较低，则根据该分割损失值对该第二候选分割模型进行调整；通过对第二候选分割模型进行调整，可以提高第二候选分割模型的图像分割准确度。
在一些实施例中，该第二候选分割模型的分割损失值可以采用如下公式(8)表示。
Figure PCTCN2021104481-appb-000005
在公式(8)中，w i、p i分别表示样本图像中第i个像素点的第二预测区域类型概率以及标注区域类型概率，L 2表示该第二候选分割模型的分割损失值。
请参见图9，是本申请实施例提供的一种图像处理装置的结构示意图。上述图像处理装置可以是运行于计算机设备中的一个计算机程序(包括程序代码)，例如该图像处理装置为一个应用软件；该装置可以用于执行本申请实施例提供的方法中的相应步骤。如图9所示，该图像处理装置可以包括：识别模块901、重组模块902。
识别模块901，用于采用第一分割模型对原始图像进行初步分割识别，得到所述原始图像的候选前景图像区域和候选背景图像区域；
重组模块902，用于对所述候选前景图像区域、所述候选背景图像区域以及所述原始图像进行重组，得到重组图像；所述重组图像中的像素点与所述原始图像中的像素点之间具有一一对应关系；
所述识别模块901，还用于采用第二分割模型对所述重组图像进行区域分割识别，得到所述原始图像的目标前景图像区域和目标背景图像区域。
在一些实施例中，所述识别模块901采用第一分割模型对原始图像进行初步分割识别，得到所述原始图像的候选前景图像区域和候选背景图像区域的具体实现方式包括：
采用所述第一分割模型对所述原始图像进行分割识别，得到所述原始图像中的像素点的候选区域类型概率；
根据所述候选区域类型概率对所述原始图像进行分割处理，得到所述原始图像的候选前景图像区域和候选背景图像区域。
在一些实施例中，上述第一分割模型包括特征提取层以及分割层；
在一些实施例中，识别模块901采用第一分割模型对上述原始图像进行分割识别，得到上述原始图像中的像素点的候选区域类型概率的具体实现方式包括：
采用上述特征提取层对上述原始图像进行特征提取，得到上述原始图像中的像素点的结构特征信息以及语义特征信息；
采用上述分割层对上述结构特征信息以及上述语义特征信息进行分割识别，得到上述原始图像中的像素点的候选区域类型概率。
重组模块902对上述候选前景图像区域、上述候选背景图像区域以及上述原始图像进行重组，得到重组图像的具体实现方式包括：
将上述候选背景图像区域与上述原始图像进行融合，得到第一融合图像；
将上述候选前景图像区域与上述原始图像进行融合，得到第二融合图像；
将上述候选前景图像区域与上述候选背景图像区域进行融合，得到第三融合图像；
将上述原始图像、上述第一融合图像、上述第二融合图像以及上述第三融合图像进行融合，得到上述重组图像。
在一些实施例中，所述识别模块901采用第二分割模型对所述重组图像进行区域分割识别，得到所述原始图像的目标前景图像区域和目标背景图像区域的具体实现方式包括：
采用所述第二分割模型对所述重组图像进行区域分割识别，得到所述重组图像中的像素点的区域分割调整参数；
根据所述区域分割调整参数以及所述候选区域类型概率，对所述原始图像进行调整分割处理，得到所述原始图像的目标前景图像区域和目标背景图像区域。
在一些实施例中，上述识别模块901根据上述区域分割调整参数以及上述候选区域类型概率，对上述原始图像进行调整分割处理，得到上述原始图像的目标前景图像区域和目标背景图像区域的具体实现方式包括：
根据上述区域分割调整参数对上述候选区域类型概率进行调整，得到目标区域类型概率；
根据上述目标区域类型概率，对上述原始图像进行调整分割处理，得到上述原始图像的目标前景图像区域和目标背景图像区域。
在一些实施例中，上述区域分割调整参数包括前景分割调整参数、背景分割调整参数以及偏移值；上述候选区域类型概率包括候选前景概率以及候选背景概率；
上述识别模块901根据上述区域分割调整参数对上述候选区域类型概率进行调整，得到目标区域类型概率的具体实现方式包括：
采用上述前景分割调整参数、上述背景分割调整参数，对上述候选前景概率以及上述 候选背景概率进行加权求和，得到概率和；
根据上述概率和以及上述偏移值，生成目标前景概率；根据上述目标前景概率获取目标背景概率；
根据上述目标前景概率以及上述目标背景概率，确定上述目标区域类型概率。
在一些实施例中，所述装置还包括：获取模块906，用于获取背景图像；
调整模块903，用于根据上述目标背景概率对上述背景图像进行调整，得到背景替换区域；对上述背景替换区域以及上述目标前景区域进行拼接，得到替换背景后的原始图像。
在一些实施例中，上述获取模块906，还用于获取第一候选分割模型以及样本图像集，上述样本图像集包括样本图像以及上述样本图像中的像素点的标注区域类型概率；
上述装置还包括：预测模块904，用于采用上述第一候选分割模型对上述样本图像进行预测，得到上述样本图像中的像素点的预测区域类型概率，作为第一预测区域类型概率；
在一些实施例中，上述调整模块903，还用于根据上述第一预测区域类型概率以及上述标注区域类型概率，对上述第一候选分割模型进行调整；
所述装置还包括：确定模块905，用于将调整后的第一候选分割模型确定为上述第一分割模型。
在一些实施例中，上述调整模块903根据上述第一预测区域类型概率以及上述标注区域类型概率，对上述第一候选分割模型进行调整的具体实现方式包括：
根据上述第一预测区域类型概率以及上述标注区域类型概率，确定上述第一候选分割模型的原始损失值；
获取上述样本图像中像素点的第一预测区域类型概率之间的变化率，作为第一梯度变化率；
获取上述样本图像中像素点的标注区域类型概率之间的变化率，作为第二梯度变化率；
根据上述第一梯度变化率以及上述第二梯度变化率，确定上述第一候选分割模型的边界损失值；
根据上述边界损失值以及上述原始损失值确定上述第一候选分割模型的总损失值；
若上述总损失值不满足收敛条件，则根据上述总损失值对上述第一候选分割模型进行调整。
在一些实施例中，获取模块906，还用于获取第二候选分割模型，以及目标预测区域类型概率，上述目标预测区域类型概率为上述第一候选分割模型的总损失值处于收敛状态时，所输出的第一预测区域类型概率；
上述识别模块901，还用于根据上述目标预测区域类型概率对上述样本图像进行分割， 得到上述样本图像的前景图像区域以及背景图像区域；
上述重组模块902，还用于对上述样本图像、上述样本图像的前景图像区域以及背景图像区域进行重组处理，得到样本重组图像；
上述预测模块904，还用于采用上述第二候选分割模型对上述样本重组图像进行预测，得到第二预测区域类型概率；
上述调整模块903，还用于根据上述第二预测区域类型概率以及上述标注区域类型概率对上述第二候选分割模型进行调整；
上述确定模块905，还用于将调整后的第二候选分割模型确定为上述第二分割模型。
在一些实施例中，上述预测模块904采用所述第二候选分割模型对所述样本重组图像进行预测，得到第二预测区域类型概率的具体方式包括：
采用上述第二候选分割模型对上述样本重组图像进行预测，得到上述样本重组图像中像素点的预测区域分割调整参数；
采用上述预测区域分割调整参数对上述目标预测区域类型概率进行调整，得到第二预测区域类型概率。
在一些实施例中，上述调整模块903根据上述第二预测区域类型概率以及上述标注区域类型概率对上述第二候选分割模型进行调整的具体实现方式包括：
根据上述第二预测区域类型概率以及上述标注区域类型概率，确定上述第二候选分割模型的分割损失值；
若上述分割损失值不满足收敛条件，则根据上述分割损失值对上述第二候选分割模型进行调整。
根据本申请的一个实施例，图3所示的图像处理方法所涉及的步骤可由图9所示的图像处理装置中的模块来执行。例如，图3中所示的步骤S101以及S103可由图9中的识别模块901来执行，图3中所示的步骤S102可由图9中的重组模块902来执行。
根据本申请的一个实施例，图9所示的图像处理装置中的模块可以分别或全部合并为一个或若干个单元来构成，或者其中的某个(些)单元还可以再拆分为功能上更小的多个子单元，可以实现同样的操作，而不影响本申请的实施例的技术效果的实现。上述模块是基于逻辑功能划分的，在实际应用中，一个模块的功能也可以由多个单元来实现，或者多个模块的功能由一个单元实现。在本申请的其它实施例中，图像处理装置也可以包括其它单元，在实际应用中，这些功能也可以由其它单元协助实现，并且可以由多个单元协作实现。
根据本申请的一个实施例，可以通过在包括中央处理单元(CPU)、随机存取存储介质 (RAM)、只读存储介质(ROM)等处理元件和存储元件的例如计算机的通用计算机设备上运行能够执行如图3以及图7中所示的相应方法所涉及的各步骤的计算机程序(包括程序代码)，来构造如图9中所示的图像处理装置，以及来实现本申请实施例的图像处理方法。上述计算机程序可以记载于例如计算机可读记录介质上，并通过计算机可读记录介质装载于上述计算设备中，并在其中运行。
请参见图10，是本申请实施例提供的一种计算机设备的结构示意图。如图10所示，上述计算机设备1000可以包括：处理器1001，网络接口1004和存储器1005，此外，上述计算机设备1000还可以包括：用户接口1003，和至少一个通信总线1002。其中，通信总线1002用于实现这些组件之间的连接通信。其中，用户接口1003可以包括显示屏(Display)、键盘(Keyboard)，用户接口1003还可以包括标准的有线接口、无线接口。网络接口1004可以包括标准的有线接口、无线接口(如WI-FI接口)。存储器1005可以是高速RAM存储器，也可以是非易失性的存储器(non-volatile memory)，例如至少一个磁盘存储器。存储器1005还可以是至少一个位于远离前述处理器1001的存储装置。如图10所示，作为一种计算机可读存储介质的存储器1005中可以包括操作系统、网络通信模块、用户接口模块以及图像处理应用程序。
在图10所示的计算机设备1000中，网络接口1004可提供网络通讯功能；而用户接口1003主要用于为用户提供输入的接口；而处理器1001可以用于调用存储器1005中存储的图像处理应用程序，以实现上述各个实施例所述的图像处理方法。
应当理解，本申请实施例中所描述的计算机设备1000可执行前文图3以及所对应实施例中对上述图像处理方法的描述，也可执行前文图9所对应实施例中对上述图像处理装置的描述，在此不再赘述。另外，对采用相同方法的有益效果描述，也不再进行赘述。
此外，这里需要指出的是：本申请实施例还提供了一种计算机可读存储介质，且上述计算机可读存储介质中存储有前文提及的图像处理装置所执行的计算机程序，且上述计算机程序包括程序指令，当上述处理器执行上述程序指令时，能够执行前文图3对应实施例中对上述图像处理方法的描述，因此，这里将不再进行赘述。另外，对采用相同方法的有益效果描述，也不再进行赘述。对于本申请所涉及的计算机可读存储介质实施例中未披露的技术细节，请参照本申请方法实施例的描述。
作为示例，上述程序指令可被部署在一个计算机设备上执行，或者被部署位于一个地点的多个计算机设备上执行，又或者，在分布在多个地点且通过通信网络互连的多个计算机设备上执行，分布在多个地点且通过通信网络互连的多个计算机设备可以组成区块链网络。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分流程，是可以通过计算机程序来指令相关的硬件来完成，上述的程序可存储于一计算机可读取存储介质中，该程序在执行时，可包括如上述各方法的实施例的流程。其中，上述的存储介质可为磁盘、光盘、只读存储器(Read-Only Memory，ROM)或随机存储器(Random Access Memory，RAM)等。
以上所揭露的仅为本申请较佳实施例而已，当然不能以此来限定本申请之权利范围，因此依本申请权利要求所作的等同变化，仍属本申请所涵盖的范围。

Claims (16)

  1. 一种图像处理方法，包括：
    采用第一分割模型对原始图像进行初步分割识别，得到所述原始图像的候选前景图像区域和候选背景图像区域；
    对所述候选前景图像区域、所述候选背景图像区域以及所述原始图像进行重组，得到重组图像；所述重组图像中的像素点与所述原始图像中的像素点之间具有一一对应关系；
    采用第二分割模型对所述重组图像进行区域分割识别，得到所述原始图像的目标前景图像区域和目标背景图像区域。
  2. 如权利要求1所述的方法，所述采用第一分割模型对原始图像进行初步分割识别，得到所述原始图像的候选前景图像区域和候选背景图像区域，包括：
    采用所述第一分割模型对所述原始图像进行分割识别，得到所述原始图像中的像素点的候选区域类型概率；所述候选区域类型概率用于指示所述原始图像中的像素点属于前景图像区域的概率和/或属于背景图像区域的概率；
    根据所述候选区域类型概率对所述原始图像进行分割处理，得到所述原始图像的候选前景图像区域和候选背景图像区域。
  3. 如权利要求2所述的方法，所述第一分割模型包括特征提取层以及分割层，所述采用第一分割模型对所述原始图像进行分割识别，得到所述原始图像中的像素点的候选区域类型概率，包括：
    采用所述特征提取层对所述原始图像进行特征提取，得到所述原始图像中的像素点的结构特征信息以及语义特征信息；
    采用所述分割层对所述结构特征信息以及所述语义特征信息进行分割识别，得到所述原始图像中的像素点的候选区域类型概率。
  4. 如权利要求2或3所述的方法，所述采用第二分割模型对所述重组图像进行区域分割识别，得到所述原始图像的目标前景图像区域和目标背景图像区域，包括：
    采用所述第二分割模型对所述重组图像进行区域分割识别，得到所述重组图像中的像素点的区域分割调整参数；
    根据所述区域分割调整参数以及所述候选区域类型概率，对所述原始图像进行调整分割处理，得到所述原始图像的目标前景图像区域和目标背景图像区域。
  5. 如权利要求4所述的方法，所述根据所述区域分割调整参数以及所述候选区域类型概率，对所述原始图像进行调整分割处理，得到所述原始图像的目标前景图像区域和目标背景图像区域，包括：
    根据所述区域分割调整参数对所述候选区域类型概率进行调整，得到目标区域类型概率；
    根据所述目标区域类型概率，对所述原始图像进行调整分割处理，得到所述原始图像的目标前景图像区域和目标背景图像区域。
  6. 如权利要求5所述的方法，所述区域分割调整参数包括前景分割调整参数、背景分割调整参数以及偏移值；所述候选区域类型概率包括候选前景概率以及候选背景概率；
    所述根据所述区域分割调整参数对所述候选区域类型概率进行调整，得到目标区域类型概率，包括：
    采用所述前景分割调整参数、所述背景分割调整参数，对所述候选前景概率以及所述候选背景概率进行加权求和，得到概率和；
    根据所述概率和以及所述偏移值，生成目标前景概率；根据所述目标前景概率获取目标背景概率；
    根据所述目标前景概率以及所述目标背景概率，确定所述目标区域类型概率。
  7. 如权利要求6所述的方法，所述方法还包括：
    获取背景图像；所述背景图像用于替换所述原始图像中的背景图像区域；
    根据所述目标背景概率对所述背景图像进行调整，得到背景替换区域；
    对所述背景替换区域以及所述目标前景区域进行拼接，得到替换背景后的原始图像。
  8. 如权利要求1所述的方法，所述对所述候选前景图像区域、所述候选背景图像区域以及所述原始图像进行重组，得到重组图像，包括：
    将所述候选背景图像区域与所述原始图像进行融合，得到第一融合图像；
    将所述候选前景图像区域与所述原始图像进行融合，得到第二融合图像；
    将所述候选前景图像区域与所述候选背景图像区域进行融合，得到第三融合图像；
    将所述原始图像、所述第一融合图像、所述第二融合图像以及所述第三融合图像进行融合，得到所述重组图像。
  9. 如权利要求1所述的方法，所述方法还包括：
    获取第一候选分割模型以及样本图像集，所述样本图像集包括样本图像以及所述样本图像中的像素点的标注区域类型概率；
    采用所述第一候选分割模型对所述样本图像进行预测，得到所述样本图像中的像素点的预测区域类型概率，作为第一预测区域类型概率；
    根据所述第一预测区域类型概率以及所述标注区域类型概率，对所述第一候选分割模型进行调整；
    将调整后的第一候选分割模型确定为所述第一分割模型。
  10. 如权利要求9所述的方法，所述根据所述第一预测区域类型概率以及所述标注区域类型概率，对所述第一候选分割模型进行调整，包括：
    根据所述第一预测区域类型概率以及所述标注区域类型概率，确定所述第一候选分割模型的原始损失值；
    获取所述样本图像中像素点的第一预测区域类型概率之间的变化率，作为第一梯度变化率；
    获取所述样本图像中像素点的标注区域类型概率之间的变化率，作为第二梯度变化率；
    根据所述第一梯度变化率以及所述第二梯度变化率，确定所述第一候选分割模型的边界损失值；
    根据所述边界损失值以及所述原始损失值确定所述第一候选分割模型的总损失值；
    若所述总损失值不满足收敛条件，则根据所述总损失值对所述第一候选分割模型进行调整。
  11. 如权利要求10所述的方法，所述方法还包括：
    获取第二候选分割模型，以及目标预测区域类型概率，所述目标预测区域类型概率为所述第一候选分割模型的总损失值处于收敛状态时，所输出的第一预测区域类型概率；
    根据所述目标预测区域类型概率对所述样本图像进行分割，得到所述样本图像的前景图像区域以及背景图像区域；
    对所述样本图像、所述样本图像的前景图像区域以及背景图像区域进行重组处理，得到样本重组图像；
    采用所述第二候选分割模型对所述样本重组图像进行预测，得到第二预测区域类型概率；
    根据所述第二预测区域类型概率以及所述标注区域类型概率对所述第二候选分割模型进行调整；
    将调整后的第二候选分割模型确定为所述第二分割模型。
  12. 如权利要求11所述的方法，所述采用所述第二候选分割模型对所述样本重组图像进行预测，得到第二预测区域类型概率，包括：
    采用所述第二候选分割模型对所述样本重组图像进行预测，得到所述样本重组图像中像素点的预测区域分割调整参数；
    采用所述预测区域分割调整参数对所述目标预测区域类型概率进行调整，得到第二预测区域类型概率。
  13. 如权利要求11或12所述的方法，所述根据所述第二预测区域类型概率以及所述标注区域类型概率对所述第二候选分割模型进行调整，包括：
    根据所述第二预测区域类型概率以及所述标注区域类型概率，确定所述第二候选分割模型的分割损失值；
    若所述分割损失值不满足收敛条件，则根据所述分割损失值对所述第二候选分割模型进行调整。
  14. 一种图像处理装置，包括：
    识别模块，用于采用第一分割模型对原始图像进行初步分割识别，得到所述原始图像的候选前景图像区域和候选背景图像区域；
    重组模块，用于对所述候选前景图像区域、所述候选背景图像区域以及所述原始图像进行重组，得到重组图像；所述重组图像中的像素点与所述原始图像中的像素点之间具有一一对应关系；
    所述识别模块，还用于采用第二分割模型对所述重组图像进行区域分割识别，得到所述原始图像的目标前景图像区域和目标背景图像区域。
  15. 一种计算机设备，包括：处理器、存储器以及网络接口；
    所述处理器与存储器、网络接口相连，其中，网络接口用于提供数据通信功能，所述存储器用于存储程序代码，所述处理器用于调用所述程序代码，以执行如权利要求1至13任一项所述的方法。
  16. 一种非易失性计算机可读存储介质，其中所述存储介质中存储有机器可读指令，所述机器可读指令可以由处理器执行以完成如权利要求1至13任一项所述的方法。
PCT/CN2021/104481 2020-07-23 2021-07-05 图像处理方法、装置、设备及存储介质 WO2022017163A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP21846696.9A EP4092623A4 (en) 2020-07-23 2021-07-05 IMAGE PROCESSING APPARATUS AND METHOD, DEVICE AND STORAGE MEDIA
US17/887,994 US20220398742A1 (en) 2020-07-23 2022-08-15 Image processing method and apparatus, device and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010718692.9A CN111768425B (zh) 2020-07-23 2020-07-23 图像处理方法、装置及设备
CN202010718692.9 2020-07-23

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/887,994 Continuation US20220398742A1 (en) 2020-07-23 2022-08-15 Image processing method and apparatus, device and storage medium

Publications (1)

Publication Number Publication Date
WO2022017163A1 true WO2022017163A1 (zh) 2022-01-27

Family

ID=72727075

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/104481 WO2022017163A1 (zh) 2020-07-23 2021-07-05 图像处理方法、装置、设备及存储介质

Country Status (4)

Country Link
US (1) US20220398742A1 (zh)
EP (1) EP4092623A4 (zh)
CN (1) CN111768425B (zh)
WO (1) WO2022017163A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114419321A (zh) * 2022-03-30 2022-04-29 珠海市人民医院 一种基于人工智能的ct图像心脏分割方法及系统

Families Citing this family (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111768425B (zh) * 2020-07-23 2021-08-10 腾讯科技（深圳）有限公司 图像处理方法、装置及设备
CN111951157A (zh) * 2020-09-02 2020-11-17 深圳传音控股股份有限公司 图像处理方法、设备及存储介质
CN112966730A (zh) * 2021-03-01 2021-06-15 创新奇智(上海)科技有限公司 车辆伤损识别方法、装置、设备及存储介质
CN113012188A (zh) * 2021-03-23 2021-06-22 影石创新科技股份有限公司 图像融合方法、装置、计算机设备和存储介质
CN113486716B (zh) * 2021-06-04 2022-06-14 电子科技大学长三角研究院(衢州) 机场场面目标分割方法及其系统
CN113724276B (zh) * 2021-08-04 2024-05-28 香港中文大学（深圳） 一种息肉图像的分割方法和装置
CN115761152B (zh) * 2023-01-06 2023-06-23 深圳星坊科技有限公司 普通光源下图像处理及三维重建方法、装置和计算机设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180308267A1 (en) * 2017-04-24 2018-10-25 Intel Corporation Separately Processing Regions or Objects or Interest from a Render Engine to a Display Engine or a Display Panel
CN109934812A (zh) * 2019-03-08 2019-06-25 腾讯科技（深圳）有限公司 图像处理方法、装置、服务器及存储介质
CN110956654A (zh) * 2019-12-02 2020-04-03 Oppo广东移动通信有限公司 图像处理方法、装置、设备及存储介质
CN111127486A (zh) * 2019-12-25 2020-05-08 Oppo广东移动通信有限公司 图像分割方法、装置、终端及存储介质
CN111768425A (zh) * 2020-07-23 2020-10-13 腾讯科技（深圳）有限公司 图像处理方法、装置及设备

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA2647018A1 (en) * 1999-12-28 2001-07-05 Sony Corporation Signal processing device and method, and recording medium
US20030156759A1 (en) * 2002-02-19 2003-08-21 Koninklijke Philips Electronics N.V. Background-foreground segmentation using probability models that can provide pixel dependency and incremental training
US20130308856A1 (en) * 2012-01-12 2013-11-21 Google Inc. Background Detection As An Optimization For Gesture Recognition
US9584814B2 (en) * 2014-05-15 2017-02-28 Intel Corporation Content adaptive background foreground segmentation for video coding
CN106846336B (zh) * 2017-02-06 2022-07-15 腾讯科技（上海）有限公司 提取前景图像、替换图像背景的方法及装置
CN108734712B (zh) * 2017-04-18 2020-12-25 北京旷视科技有限公司 背景分割的方法、装置及计算机存储介质
CN108038857B (zh) * 2017-12-25 2018-10-12 北京航空航天大学 一种基于语义信息与边缘约束的前景目标检测方法
CN111292333B (zh) * 2018-12-07 2024-05-17 北京京东尚科信息技术有限公司 用于分割图像的方法和装置
CN110335277A (zh) * 2019-05-07 2019-10-15 腾讯科技（深圳）有限公司 图像处理方法、装置、计算机可读存储介质和计算机设备
CN110544258B (zh) * 2019-08-30 2021-05-25 北京海益同展信息科技有限公司 图像分割的方法、装置、电子设备和存储介质
CN111369582B (zh) * 2020-03-06 2023-04-07 腾讯科技（深圳）有限公司 图像分割方法、背景替换方法、装置、设备及存储介质
CN111428726B (zh) * 2020-06-10 2020-09-11 中山大学 基于图神经网络的全景分割方法、系统、设备及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180308267A1 (en) * 2017-04-24 2018-10-25 Intel Corporation Separately Processing Regions or Objects or Interest from a Render Engine to a Display Engine or a Display Panel
CN109934812A (zh) * 2019-03-08 2019-06-25 腾讯科技（深圳）有限公司 图像处理方法、装置、服务器及存储介质
CN110956654A (zh) * 2019-12-02 2020-04-03 Oppo广东移动通信有限公司 图像处理方法、装置、设备及存储介质
CN111127486A (zh) * 2019-12-25 2020-05-08 Oppo广东移动通信有限公司 图像分割方法、装置、终端及存储介质
CN111768425A (zh) * 2020-07-23 2020-10-13 腾讯科技（深圳）有限公司 图像处理方法、装置及设备

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
See also references of EP4092623A4 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114419321A (zh) * 2022-03-30 2022-04-29 珠海市人民医院 一种基于人工智能的ct图像心脏分割方法及系统
CN114419321B (zh) * 2022-03-30 2022-07-08 珠海市人民医院 一种基于人工智能的ct图像心脏分割方法及系统

Also Published As

Publication number Publication date
EP4092623A1 (en) 2022-11-23
CN111768425B (zh) 2021-08-10
CN111768425A (zh) 2020-10-13
US20220398742A1 (en) 2022-12-15
EP4092623A4 (en) 2023-08-30

Similar Documents

Publication Publication Date Title
WO2022017163A1 (zh) 图像处理方法、装置、设备及存储介质
CN111368685B (zh) 关键点的识别方法、装置、可读介质和电子设备
CN111739027B (zh) 一种图像处理方法、装置、设备及可读存储介质
EP4137991A1 (en) Pedestrian re-identification method and device
EP3815047A1 (en) Image colorization based on reference information
CN111325271B (zh) 图像分类方法及装置
US20220222796A1 (en) Image processing method and apparatus, server, and storage medium
CN113704531A (zh) 图像处理方法、装置、电子设备及计算机可读存储介质
CN110490959B (zh) 三维图像处理方法及装置、虚拟形象生成方法以及电子设备
CN114821734A (zh) 一种驱动虚拟人物表情的方法和装置
US20220358675A1 (en) Method for training model, method for processing video, device and storage medium
US20230047748A1 (en) Method of fusing image, and method of training image fusion model
CN111080746B (zh) 图像处理方法、装置、电子设备和存储介质
CN114511041B (zh) 模型训练方法、图像处理方法、装置、设备和存储介质
CN114332590B (zh) 联合感知模型训练、联合感知方法、装置、设备和介质
CN113724132B (zh) 图像风格迁移处理方法、装置、电子设备及存储介质
CN117095019B (zh) 一种图像分割方法及相关装置
CN111898544B (zh) 文字图像匹配方法、装置和设备及计算机存储介质
CN112862840B (zh) 图像分割方法、装置、设备及介质
CN114299105A (zh) 图像处理方法、装置、计算机设备及存储介质
CN117649358B (zh) 图像处理方法、装置、设备及存储介质
CN116597098B (zh) 三维重建方法、装置、电子设备和计算机可读存储介质
CN116740540B (zh) 一种数据处理方法、装置、设备以及计算机可读存储介质
CN116935425A (zh) 表格图像识别方法、装置、设备及计算机存储介质
CN118097327A (zh) 模型训练方法、视频全景分割方法、装置、设备、介质及产品

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21846696

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2021846696

Country of ref document: EP

Effective date: 20220818

NENP Non-entry into the national phase

Ref country code: DE