CN115705648A

CN115705648A - Image processing method, device, equipment and storage medium

Info

Publication number: CN115705648A
Application number: CN202110941688.3A
Authority: CN
Inventors: 李成; 徐鹏; 刘阳兴
Original assignee: Wuhan TCL Group Industrial Research Institute Co Ltd
Current assignee: Wuhan TCL Group Industrial Research Institute Co Ltd
Priority date: 2021-08-17
Filing date: 2021-08-17
Publication date: 2023-02-17

Abstract

The invention discloses an image processing method, an image processing device, image processing equipment and a storage medium, wherein the method comprises the following steps: determining a first foreground masking layout of a target image according to the target image to be processed, then determining a second foreground masking layout of the target image according to a preset pixel parameter threshold for image edge region noise processing, the first foreground masking layout and the target image, and finally obtaining an image region corresponding to a target foreground object in the target image according to the second foreground masking layout and the target image. By adopting the embodiment provided by the invention, the second foreground masking layout with the better target image can be determined according to the preset pixel parameter threshold, the determined first foreground masking layout and the target image, so that high-precision and high-efficiency matting processing can be realized according to the second foreground masking layout with the better target image.

Description

Image processing method, device, equipment and storage medium

Technical Field

The present invention relates to the field of image processing technologies, and in particular, to an image processing method, apparatus, device, and storage medium.

Background

Matting is one of the most common operations in image processing, which separates a portion of an image from an original image into individual layers.

The existing image matting methods realize the functions of changing background, blurring background and the like of a target image to be processed through Alpha Matte (foreground masking), but the method has the problems of large workload, low image matting efficiency and low precision.

Therefore, a matting method capable of improving matting efficiency and matting accuracy is needed.

Disclosure of Invention

The embodiment of the invention aims to provide an image processing method, an image processing device, image processing equipment and a storage medium, and aims to solve the technical problems of low matting efficiency and low matting accuracy.

In a first aspect, to achieve the above object, an embodiment of the present invention provides an image processing method, including:

determining a first foreground montage picture of a target image according to the target image to be processed;

determining a second foreground masking image of the target image according to a preset pixel parameter threshold value for processing noise of an image edge region, the first foreground masking image and the target image;

and obtaining an image area corresponding to a target foreground object in the target image according to the second foreground masking image and the target image.

Further, the step of determining the second foreground masking image of the target image according to the preset pixel parameter threshold for processing the noise in the image edge region, the first foreground masking image and the target image includes:

performing threshold segmentation on the first foreground masking layout according to the first threshold and the second threshold respectively to obtain a corresponding first mask and a corresponding second mask;

respectively carrying out edge region noise elimination processing on the first mask and the second mask to obtain a corresponding first noise elimination result and a corresponding second noise elimination result;

determining a first Trimap of the target image according to the first noise elimination result and the second noise elimination result;

and determining a second foreground montage of the target image according to the first Trimap and the target image.

Further, the performing edge noise removal processing on the first mask and the second mask to obtain corresponding first noise removal result and second noise removal result includes:

performing expansion processing on the first mask to obtain a first noise elimination result after the expansion processing;

carrying out corrosion treatment on the second shade to obtain a second noise elimination result after the corrosion treatment;

determining a first Trimap of the target image according to the first noise elimination result and the second noise elimination result, including:

and performing superposition operation on the first noise elimination result and the second noise elimination result, and determining and obtaining a first Trimap of the target image.

Further, the determining a second foreground montage of the target image according to the first Trimap and the target image includes:

determining a third foreground montage of the target image according to the first Trimap and the target image;

performing threshold segmentation on the third foreground masking layout according to the first threshold and the second threshold respectively to obtain a corresponding third mask and a corresponding fourth mask;

performing expansion processing on the third mask to obtain a third noise elimination result after the expansion processing;

carrying out corrosion treatment on the fourth shade to obtain a fourth noise elimination result after the corrosion treatment;

performing superposition operation on the third noise elimination result and the fourth noise elimination result, and determining and obtaining a second Trimap of the target image;

and determining a second foreground masking image of the target image according to the second Trimap and the target image.

Further, the determining a first foreground masking map of the target image according to the target image to be processed includes:

inputting the target image to be processed into a trained sectional image model to obtain a first foreground masking image of the target image output by the sectional image model;

the cutout model is formed by training a cutout model to be trained by the following training method:

acquiring a training sample set; the training samples in the training sample set comprise a plurality of input images containing target foreground objects and label images of foreground montage patterns corresponding to the input images;

and training the cutout model to be trained according to the training samples in the training sample set, and optimizing model parameters based on a set target loss function to generate the cutout model after training.

Further, the target loss function value of the target loss function is a weighted sum of the loss function values of the minimum absolute value error function, the synthetic loss function, the gradient loss function, and the laplacian pyramid loss function;

the difference between the first threshold and the second threshold is proportional to the weight of the loss function value of the minimum absolute value error function in the target loss function value.

Further, the obtaining an image region corresponding to a target foreground object in the target image according to the second foreground mask and the target image includes:

and adding the second foreground masking layout to the layer corresponding to the target image to obtain an image area corresponding to a target foreground object in the target image.

In a second aspect, to solve the same technical problem, an embodiment of the present invention provides an image processing apparatus including:

the first determining module is used for determining a first foreground masking image of a target image according to the target image to be processed;

the second determining module is used for determining a second foreground masking image of the target image according to a preset pixel parameter threshold value used for image edge region noise processing, the first foreground masking image and the target image;

and the processing module is used for obtaining an image area corresponding to the target foreground object in the target image according to the second foreground montage and the target image.

In a third aspect, to solve the same technical problem, an embodiment of the present invention provides an electronic device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, where the memory is coupled to the processor, and the processor implements the image processing method according to any one of the above items when executing the computer program.

In a fourth aspect, to solve the same technical problem, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program, wherein when the computer program runs, an apparatus in which the computer-readable storage medium is located is controlled to execute any one of the image processing methods described above.

The embodiment of the invention provides an image processing method, an image processing device, image processing equipment and a storage medium, wherein the method comprises the following steps: the method comprises the steps of determining a first foreground montage of a target image according to the target image to be processed, then determining a second foreground montage of the target image according to a preset pixel parameter threshold value used for image edge region noise processing, the first foreground montage and the target image, and finally obtaining an image region corresponding to a target foreground object in the target image according to the second foreground montage and the target image. By adopting the embodiment provided by the invention, the second foreground masking layout with the better target image can be determined according to the preset pixel parameter threshold, the determined first foreground masking layout and the target image, so that high-precision and high-efficiency matting processing can be realized according to the second foreground masking layout with the better target image.

Drawings

FIG. 1 is a schematic flowchart of an image processing method according to an embodiment of the present invention;

FIG. 2a is a schematic diagram of a target image with a portrait provided by an embodiment of the present invention;

fig. 2b is a schematic diagram of a first foreground montage of a target image according to an embodiment of the present invention;

fig. 3 is a schematic flow chart of automatically generating a Trimap with unequal width according to an embodiment of the present invention;

FIG. 4 is a schematic flow chart of an image processing method according to an embodiment of the present invention;

fig. 5 is a schematic diagram of a foreground montage layout after multiple iterations according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of a target image after background replacement according to an embodiment of the present invention;

FIG. 7 is a schematic diagram of an embodiment of an image processing apparatus;

FIG. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention;

fig. 9 is another schematic structural diagram of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments of the present invention, are within the scope of protection of the present invention.

It should be understood that the various steps recited in the method embodiments of the present disclosure may be performed in a different order, and/or performed in parallel. Moreover, method embodiments may include additional steps and/or omit performing the illustrated steps. The scope of the present disclosure is not limited in this respect.

The term "include" and variations thereof as used herein are open-ended, i.e., "including but not limited to". The term "based on" is "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments". Relevant definitions for other terms will be given in the following description.

It should be noted that the drawings combined in the embodiments of the present invention include a plurality of grayscale drawings, where the grayscale drawings include grayscales represented by objects with different luminances, which are only used as different depth of field information of different objects in the grayscale drawings, that is, the larger the pixel value in the grayscale drawings, the closer the distance from the camera that captures the grayscale drawings is; the smaller the pixel value in the grayscale map, the farther the distance from the camera that captured the grayscale map.

Referring to fig. 1, fig. 1 is a schematic flow chart of an image processing method according to an embodiment of the present invention, and as shown in fig. 1, the image processing method according to the embodiment of the present invention includes steps 101 to 104;

step 101, determining a first foreground masking image of a target image according to the target image to be processed.

In this embodiment, the target image to be processed is an image to be scratched which contains a target foreground object, where the target foreground object refers to an object to be scratched or an object to be subjected to background replacement, and may be a human image, an animal image, an article image, or the like. In actual use, the user can select according to specific situations.

Optionally, by performing manual operation on a photo shop or other image modifying application, a first foreground masking image (Alpha mate) of the target image to be processed can be obtained, and the target image to be processed can also be processed through the trained matting model, so as to obtain the first foreground masking image output by the model.

Specifically, step 101 may specifically be: and inputting the target image to be processed into the trained cutout model to obtain a first foreground cutout image of the target image output by the cutout model.

In the embodiment of the present invention, the target foreground object is determined as a portrait in the embodiment of the present invention, please refer to fig. 2a and fig. 2b, fig. 2a is a schematic diagram of a target image including a portrait provided in the embodiment of the present invention, fig. 2b is a schematic diagram of a first foreground sketch of the target image provided in the embodiment of the present invention, and as shown in fig. 2a and fig. 2b, when a user inputs the target image shown in fig. 2a into the trained matte model provided in the embodiment, the trained matte model outputs the first foreground sketch corresponding to the target image shown in fig. 2 b. Similarly, the user can also obtain the first foreground masking image corresponding to the target image on the masking application such as Photoshop and the like in a manual operation mode.

In this embodiment, the trained cutout model is trained by the cutout model to be trained by the following training method: acquiring a training sample set, wherein training samples in the training sample set comprise a plurality of input images containing target foreground objects and label images of foreground montage pictures corresponding to each input image; training a cutout model to be trained according to training samples in a training sample set, and optimizing model parameters based on a set target loss function to generate the cutout model after training.

It should be noted that, when a user mainly wants to obtain a foreground montage containing a portrait image, a large number of images containing the portrait, such as 5 thousands of images, need to be obtained, then the foreground montage corresponding to the input images containing the portrait can be quickly obtained through a photo shop or other image-modifying tool to obtain 5 thousands of label images, then the 5 thousands of images containing the portrait are used as the input of the model, and the parameters of the to-be-trained matting model are adjusted in real time until convergence according to the loss function between each output of the model and the corresponding label image.

Optionally, the target loss function adopted by the cutout model is a weighted sum of a minimum absolute value error function (L1 loss function), a synthetic loss function, a gradient loss function, and a laplacian pyramid loss function.

And 102, determining a second foreground masking image of the target image according to a preset pixel parameter threshold for image edge region noise processing, the first foreground masking image and the target image.

In this embodiment, the preset pixel parameter threshold at least includes a first threshold and a second threshold, and the first threshold and the second threshold are two different thresholds. Specifically, step 102 specifically includes: the method comprises the steps of performing threshold segmentation on a first foreground masking layout according to a first threshold and a second threshold respectively to obtain a corresponding first mask and a corresponding second mask, then performing edge region noise elimination processing on the first mask and the second mask respectively to obtain a corresponding first noise elimination result and a corresponding second noise elimination result, determining a first Trimap of a target image according to the first noise elimination result and the second noise elimination result, and finally determining a second foreground masking layout of the target image according to the first Trimap and the target image.

It should be noted that, in this embodiment, the first foreground masking map can also be processed according to a pixel parameter threshold: and performing threshold segmentation on the first foreground mask map by using a threshold to obtain a binarization mask (mask), and then generating a first trimap by performing corrosion expansion on the mask, or determining a transition region range according to Euclidean distance from the mask to an edge, or amplifying, reducing and overlapping images.

In some embodiments, the first mask is obtained by performing threshold segmentation on the first foreground masking map by using a first threshold: pixels with a gray level greater than the first threshold value are marked as white, and pixels with a gray level lower than the first threshold value are marked as black, so that a binarized image (first mask) is obtained.

Similarly, the second mask can be obtained by adopting the threshold segmentation method.

The mask mainly serves to block (wholly or partially) a processed image so as to control an image processing area or a processing process.

Optionally, values of the preset first threshold and the preset second threshold are close to two ends of a value range of each pixel position of the first foreground masking image, for example, if the value range is [0, 255], the first threshold and the second threshold are respectively set to 10 and 245.

Further, the difference between the first threshold and the second threshold is proportional to the weight occupied by the minimum absolute value error function in the objective loss function, for example, if the value range is still [0, 255], when the weight occupied by the minimum absolute value error function in the objective loss function is smaller, the first threshold and the second threshold are respectively set to 50 and 205; when the weight of the minimum absolute value error function in the target loss function is large, the first threshold and the second threshold are set to 5 and 250, respectively.

In one embodiment, the first threshold is smaller than the second threshold, and therefore, the edge region noise removal processing is performed on the first mask and the second mask respectively, and the manner of obtaining the corresponding first noise removal result and the second noise removal result is specifically: performing expansion processing on the first mask to obtain a first noise elimination result after the expansion processing; and carrying out corrosion treatment on the second shade to obtain a second noise elimination result after the corrosion treatment.

In another embodiment, the edge region noise elimination process may be performed on the first mask and the second mask by: the first mask is enlarged and the second mask is reduced. Specifically, the method further includes other manners of performing edge region noise elimination processing on the first mask and the second mask, for example, combining all background points in the first mask, which are in contact with the target foreground object, into the target foreground object to enlarge the target, filling holes in the target foreground object, eliminating boundary points of the target foreground object in the second mask, reducing the target foreground object, eliminating noise points, and the like, which is not limited herein.

In another embodiment, the determining the first Trimap of the target image according to the first noise elimination result and the second noise elimination result specifically includes: and performing superposition operation on the first noise elimination result and the second noise elimination result, and determining and obtaining a first Trimap of the target image.

Specifically, carry out the expansion result that the expansion process obtained with first shade, and the corrosion result that the second shade corroded the processing and obtains adds, obtains the wider and unequal width Trimap of transition region, compares in the equal width Trimap that prior art generated, and the unequal width Trimap of this application more can embody target foreground object edge pixel, like embodying the hair that the portrait is wandering to the precision of matting has been improved.

Referring to fig. 3, fig. 3 is a schematic flow chart of automatically generating a Trimap with unequal widths according to an embodiment of the present invention, as shown in fig. 3, when a Trimap with unequal widths of a target image 10 needs to be obtained, the target image 10 needs to be input into a trained matting model provided in the above embodiment to obtain a first foreground layout 11 corresponding to the target image 10, then threshold segmentation is performed on the first foreground layout 11 by using a first threshold, such as 5, and a second threshold, such as 250, respectively to obtain a first mask 110 obtained by performing threshold segmentation on the first threshold, and a second mask 120 obtained by performing threshold segmentation on the second threshold, then expansion processing is performed on the first mask 110, corrosion processing is performed on the second mask 120 to obtain an expansion result 111 and a corrosion result 121, and then the expansion result 111 and the corrosion result 121 are superimposed to obtain a Trimap20 with unequal widths of the target image 10.

As an optional embodiment, in order to obtain a better second foreground montage, the embodiment further provides the following manner: determining a third foreground mask of the target image according to the first Trimap and the target image, performing threshold segmentation on the third foreground mask according to a first threshold and a second threshold respectively to obtain a corresponding third mask and a fourth mask, performing expansion processing on the third mask to obtain a third noise elimination result after expansion processing, performing corrosion processing on the fourth mask to obtain a fourth noise elimination result after corrosion processing, performing superposition operation on the third noise elimination result and the fourth noise elimination result to determine and obtain a second Trimap of the target image, and determining a second foreground mask of the target image according to the second Trimap and the target image.

By carrying out the optimization processing provided by the embodiment on the first foreground masking layout, a better second foreground masking layout can be obtained, so that the better second foreground masking layout is adopted for matting, and the matting accuracy can be effectively improved.

It can be understood that, in order to obtain the optimal second foreground montage territory, the highest matting accuracy is realized, iterative optimization processing can be continued according to the mode of obtaining the optimal second foreground montage territory, and the optimal foreground montage territory can be obtained by continuously optimizing.

And 103, obtaining an image area corresponding to the target foreground object in the target image according to the second foreground masking image and the target image.

In this embodiment, according to the second foreground sketch map, the target image can be subjected to matting processing, background replacement, background blurring and other processing, and because the second foreground sketch map is determined through the wide and unequal-width Trimap of the transition region and the target image, the unequal-width Trimap can effectively improve the accuracy of matting, therefore, when the second foreground sketch map is adopted to process the target image, the accuracy of matting can be effectively improved, the matting effect of the image is better, and the user experience is improved.

In summary, an embodiment of the present invention provides an image processing method, including: the method comprises the steps of determining a first foreground montage of a target image according to the target image to be processed, then determining a second foreground montage of the target image according to a preset pixel parameter threshold value used for image edge region noise processing, the first foreground montage and the target image, and finally obtaining an image region corresponding to a target foreground object in the target image according to the second foreground montage and the target image. By adopting the embodiment provided by the invention, the second foreground masking layout with the better target image can be determined according to the preset pixel parameter threshold, the determined first foreground masking layout and the target image, so that high-precision and high-efficiency matting processing can be realized according to the second foreground masking layout with the better target image.

Referring to fig. 4, fig. 4 is another schematic flow chart of the image processing method according to the embodiment of the present invention, and as shown in fig. 4, the image processing method according to the embodiment of the present invention includes steps 201 to 212;

step 201, inputting a target image to be processed into the trained matting model to obtain a first foreground masking image of the target image output by the matting model.

In this embodiment, the trained matting model is configured to calculate and output a first foreground masking image of a target image according to the target image containing a target foreground object.

Step 202, performing threshold segmentation on the first foreground masking map according to a first threshold and a second threshold respectively to obtain a corresponding first mask and a corresponding second mask.

In this embodiment, the first threshold and the second threshold are two different thresholds, and values of the first threshold and the second threshold are close to two ends of a value range of each pixel position of the first foreground masking image, for example, if the value range is [0, 255], the first threshold and the second threshold are set to 10 and 245, respectively.

Step 203, performing dilation processing on the first mask to obtain a first noise elimination result after dilation processing.

And 204, carrying out corrosion treatment on the second shade to obtain a second noise elimination result after the corrosion treatment.

And step 205, performing superposition operation on the first noise elimination result and the second noise elimination result, and determining and obtaining a first Trimap of the target image.

And step 206, determining a third foreground masking map of the target image according to the first Trimap and the target image.

And step 207, performing threshold segmentation on the third foreground masking image according to the first threshold and the second threshold respectively to obtain a corresponding third mask and a corresponding fourth mask.

And step 208, performing expansion processing on the third mask to obtain a third noise elimination result after the expansion processing.

Step 209, perform etching process on the fourth mask to obtain a fourth noise removal result after the etching process.

And step 210, performing superposition operation on the third noise elimination result and the fourth noise elimination result, and determining and obtaining a second Trimap of the target image.

And step 211, determining a second foreground masking image of the target image according to the second Trimap and the target image.

And step 212, adding the second foreground montage into a layer corresponding to the target image to obtain an image area corresponding to the target foreground object in the target image.

In this embodiment, the Trimap of the target image is continuously updated in an iterative manner, so that a continuously optimized foreground masking map can be obtained, and the matting accuracy can be continuously improved. In this embodiment, the number of times of performing optimization iteration on the Trimap of the target image is 2.

Optionally, the specific number of iterations is not limited to the number of iterations provided in the above embodiment, and can be performed for 3 times, 4 times or more. When the method is actually used, a user can select different iteration times according to specific conditions to optimize the foreground Mongolian picture.

Referring to fig. 2a, fig. 2b, and fig. 5, fig. 5 is a schematic diagram of a foreground montage after multiple iterations according to an embodiment of the present invention, as shown in fig. 2a, fig. 2b, and fig. 5, fig. 2a is a target image, when a user inputs the target image into a trained cutout model, a first foreground montage as shown in fig. 2b is obtained, and then a Trimap after multiple iterations is obtained through an iteration manner provided by the above embodiment, so that a better foreground montage as shown in fig. 5 corresponding to the target image can be obtained according to the Trimap, and further, the better foreground montage as shown in fig. 5 is adopted, so that the accuracy of the cutout can be improved to the greatest extent; meanwhile, the Trimap generation is automatic in the whole process, manual manufacturing by a user is not needed, therefore, time for manufacturing the Trimap in a large amount can be saved, and the matting efficiency is improved to a great extent.

In this embodiment, please refer to fig. 2a and fig. 6, and fig. 6 is a schematic diagram of a target image after background replacement according to an embodiment of the present invention, and as shown in fig. 2a and fig. 6, when obtaining a better foreground montage field obtained by iteration according to the above embodiment, a user may directly perform background replacement on the target image shown in fig. 2a according to the better foreground montage field, and finally obtain the target image after background replacement as shown in fig. 6.

According to the method described in the foregoing embodiment, the embodiment will be further described from the perspective of an image processing apparatus, which may be specifically implemented as an independent entity, or may be implemented by being integrated in an electronic device, such as a terminal, where the terminal may include a mobile phone, a tablet computer, and the like.

Referring to fig. 7, fig. 7 is a schematic structural diagram of an image processing apparatus according to an embodiment of the present invention, and as shown in fig. 7, an image processing apparatus 700 according to an embodiment of the present invention includes:

the first determining module 701 is configured to determine a first foreground masking image of a target image according to the target image to be processed.

In this embodiment, the first determining module 701 is specifically configured to input the target image to be processed into the trained matting model, so as to obtain the first foreground masking image of the target image output by the matting model.

The cutout model is formed by training a cutout model to be trained by the following training method: acquiring a training sample set; the training samples in the training sample set comprise a plurality of input images containing target foreground objects and label images of foreground montage patterns corresponding to the input images; and training the matting model to be trained according to the training samples in the training sample set, and optimizing model parameters based on a set target loss function to generate the trained matting model.

Optionally, the target loss function value of the target loss function is a weighted sum of the loss function values of the minimum absolute value error function, the synthetic loss function, the gradient loss function, and the laplacian pyramid loss function; the difference between the first threshold and the second threshold is proportional to the weight of the loss function value of the minimum absolute value error function in the target loss function value.

A second determining module 702, configured to determine a second foreground montage of the target image according to a preset pixel parameter threshold for image edge region noise processing, the first foreground montage, and the target image.

In this embodiment, the second determining module 702 is specifically configured to perform threshold segmentation on the first foreground masking layout according to the first threshold and the second threshold, respectively, to obtain a corresponding first mask and a corresponding second mask; respectively carrying out edge region noise elimination processing on the first mask and the second mask to obtain a corresponding first noise elimination result and a corresponding second noise elimination result; determining a first Trimap of the target image according to the first noise elimination result and the second noise elimination result; and determining a second foreground montage of the target image according to the first Trimap and the target image.

In some embodiments, the second determining module 702 is further configured to perform dilation processing on the first mask to obtain a first noise removal result after dilation processing; and carrying out corrosion treatment on the second shade to obtain a second noise elimination result after the corrosion treatment.

In other embodiments, the second determining module 702 is further specifically configured to perform a superposition operation on the first noise removal result and the second noise removal result, and determine and obtain a first Trimap of the target image.

And the processing module 703 is configured to obtain an image area corresponding to a target foreground object in the target image according to the second foreground mask layout and the target image.

As an alternative embodiment, the second determining module 702 is further specifically configured to determine a third foreground montage of the target image according to the first Trimap and the target image; performing threshold segmentation on the third foreground masking layout according to the first threshold and the second threshold respectively to obtain a corresponding third mask and a corresponding fourth mask; performing expansion processing on the third mask to obtain a third noise elimination result after the expansion processing; carrying out corrosion treatment on the fourth shade to obtain a fourth noise elimination result after the corrosion treatment; performing superposition operation on the third noise elimination result and the fourth noise elimination result, and determining and obtaining a second Trimap of the target image; and determining a second foreground montage of the target image according to the second Trimap and the target image.

In a specific implementation, each of the modules and/or units may be implemented as an independent entity, or may be implemented as one or several entities by any combination, where the specific implementation of each of the modules and/or units may refer to the foregoing method embodiment, and specific achievable beneficial effects also refer to the beneficial effects in the foregoing method embodiment, which are not described herein again.

In addition, referring to fig. 8, fig. 8 is a schematic structural diagram of an electronic device according to an embodiment of the present invention, where the electronic device may be a mobile terminal such as a smart phone and a tablet computer. As shown in fig. 8, the electronic device 800 includes a processor 801, a memory 802. The processor 801 is electrically connected to the memory 802.

The processor 801 is a control center of the electronic device 800, connects various parts of the entire electronic device using various interfaces and lines, and performs various functions of the electronic device 800 and processes data by running or loading an application program stored in the memory 802 and calling data stored in the memory 802, thereby performing overall monitoring of the electronic device 800.

In this embodiment, the processor 801 in the electronic device 800 loads instructions corresponding to processes of one or more application programs into the memory 802, and the processor 801 executes the application programs stored in the memory 802 according to the following steps, so as to implement various functions:

determining a second foreground masking image of the target image according to a preset pixel parameter threshold value for processing noise in the image edge region, the first foreground masking image and the target image;

and obtaining an image area corresponding to the target foreground object in the target image according to the second foreground masking image and the target image.

The electronic device 800 can implement the steps in any embodiment of the image processing method provided in the embodiment of the present invention, and therefore, the beneficial effects that can be achieved by any image processing method provided in the embodiment of the present invention can be achieved, which are detailed in the foregoing embodiments and will not be described herein again.

Referring to fig. 9, fig. 9 is another schematic structural diagram of an electronic device according to an embodiment of the present invention, and as shown in fig. 9, fig. 9 is a specific structural block diagram of the electronic device according to the embodiment of the present invention, where the electronic device may be used to implement the image processing method provided in the foregoing embodiment. The electronic device 900 may be a mobile terminal such as a smart phone or a notebook computer.

The RF circuit 910 is used for receiving and transmitting electromagnetic waves, and interconverting the electromagnetic waves and electrical signals, so as to communicate with a communication network or other devices. RF circuit 910 may include various existing circuit elements for performing these functions, such as an antenna, a radio frequency transceiver, a digital signal processor, an encryption/decryption chip, a Subscriber Identity Module (SIM) card, memory, and so forth. The RF circuit 910 may communicate with various networks such as the internet, an intranet, a wireless network, or with other devices over a wireless network. The wireless network may include a cellular telephone network, a wireless local area network, or a metropolitan area network. The Wireless network described above may use various Communication standards, protocols and technologies, including but not limited to Global System for Mobile Communication (GSM), enhanced Mobile Communication (Enhanced Data GSM Environment, EDGE), wideband Code Division Multiple Access (WCDMA), code Division Multiple Access (CDMA), time Division Multiple Access (TDMA), wireless Fidelity (Wi-Fi) (e.g., institute of electrical and electronics engineers standard IEEE802.11 a, IEEE802.11 b, IEEE802.1 g and/or IEEE802.11 n), voice over Internet Protocol (VoIP), world wide Internet Access (micro for Access, max), other suitable protocols for Wireless messaging, and other instant messaging protocols, including any other protocols that are currently developed, and even those suitable for instant messaging.

The memory 920 may be used to store software programs and modules, such as program instructions/modules corresponding to the image processing method in the above-described embodiment, and the processor 980 executes various functional applications and data processing by running the software programs and modules stored in the memory 920, so as to implement the following functions:

The memory 920 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 920 can further include memory located remotely from the processor 980, which can be connected to the electronic device 900 over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The input unit 930 may be used to receive input numeric or character information and generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control. In particular, the input unit 930 may include a touch-sensitive surface 931 as well as other input devices 932. Touch-sensitive surface 931, also referred to as a touch screen or touch pad, may collect user touch operations (e.g., user operations on or near touch-sensitive surface 931 using a finger, stylus, or any other suitable object or attachment) and drive the corresponding connecting device according to a predetermined program. Alternatively, the touch sensitive surface 931 may include both a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 980, and can receive and execute commands sent by the processor 980. In addition, the touch sensitive surface 931 may be implemented in various types, such as resistive, capacitive, infrared, and surface acoustic wave. The input unit 930 may comprise other input devices 932 in addition to the touch-sensitive surface 931. In particular, other input devices 932 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

The display unit 940 may be used to display information input by or provided to the user and various graphical user interfaces of the electronic device 900, which may be made up of graphics, text, icons, video, and any combination thereof. The Display unit 940 may include a Display panel 941, and optionally, the Display panel 941 may be configured in the form of an LCD (Liquid Crystal Display), an OLED (Organic Light-Emitting Diode), or the like. Further, the touch-sensitive surface 931 may overlay the display panel 941, and when a touch operation is detected on or near the touch-sensitive surface 931, the touch operation is transmitted to the processor 980 to determine the type of touch event, and the processor 980 then provides a corresponding visual output on the display panel 941 according to the type of touch event. Although the touch-sensitive surface 931 and the display panel 941 are shown as two separate components to implement input and output functions, in some embodiments, the touch-sensitive surface 931 and the display panel 941 may be integrated to implement input and output functions.

The electronic device 900 may also include at least one sensor 950, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor may include an ambient light sensor that may adjust the brightness of the display panel 941 according to the brightness of ambient light, and a proximity sensor that may generate an interrupt when the folder is closed or closed. As one of the motion sensors, the gravity acceleration sensor can detect the magnitude of acceleration in each direction (generally, three axes), can detect the magnitude and direction of gravity when the mobile phone is stationary, and can be used for applications of recognizing the posture of the mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer and tapping), and the like; as for other sensors such as a gyroscope, a barometer, a hygrometer, a thermometer, and an infrared sensor, which may be further configured to the electronic device 900, detailed descriptions thereof are omitted.

Audio circuitry 960, speaker 961, microphone 962 may provide an audio interface between a user and electronic device 900. The audio circuit 960 may transmit the electrical signal converted from the received audio data to the speaker 961, and convert the electrical signal into a sound signal for output by the speaker 961; on the other hand, the microphone 962 converts the collected sound signal into an electric signal, converts the electric signal into audio data after being received by the audio circuit 960, and outputs the audio data to the processor 980 for processing, and then transmits the audio data to another terminal via the RF circuit 910, or outputs the audio data to the memory 920 for further processing. The audio circuit 960 may also include an earbud jack to provide communication of a peripheral headset with the electronic device 900.

The electronic device 900, via the transport module 970 (e.g., wi-Fi module), may assist the user in receiving requests, sending messages, etc., which provides the user with wireless broadband internet access. Although the transmission module 970 is shown in the drawings, it is understood that it does not belong to the essential constitution of the electronic device 900 and may be omitted entirely as needed within the scope not changing the essence of the invention.

The processor 980 is a control center of the electronic device 900, connects various parts of the entire cellular phone using various interfaces and lines, and performs various functions of the electronic device 900 and processes data by operating or executing software programs and/or modules stored in the memory 920 and calling data stored in the memory 920, thereby integrally monitoring the electronic device. Optionally, processor 980 may include one or more processing cores; in some embodiments, the processor 980 may integrate an application processor, which primarily handles operating systems, user interfaces, applications, etc., and a modem processor, which primarily handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 980.

The electronic device 900 also includes a power supply 990 (e.g., a battery) that provides power to the various components and, in some embodiments, may be logically coupled to the processor 980 via a power management system that provides management of charging, discharging, and power consumption. Power supply 990 may also include any component of one or more dc or ac power sources, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, and the like.

Although not shown, the electronic device 900 further includes a camera (e.g., a front camera, a rear camera), a bluetooth module, etc., which are not described in detail herein. Specifically, in this embodiment, the display unit of the electronic device is a touch screen display, the mobile terminal further includes a memory, and one or more programs, where the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs include instructions for:

determining a first foreground masking image of a target image according to the target image to be processed;

determining a second foreground montage picture of the target image according to a preset pixel parameter threshold value used for image edge region noise processing, the first foreground montage picture and the target image;

In specific implementation, the above modules may be implemented as independent entities, or may be combined arbitrarily, and implemented as the same or several entities, and specific implementations of the above modules may refer to the foregoing method embodiment, which is not described herein again.

It will be understood by those skilled in the art that all or part of the steps of the methods of the above embodiments may be performed by instructions or by associated hardware controlled by the instructions, which may be stored in a computer readable storage medium and loaded and executed by a processor. To this end, an embodiment of the present invention provides a storage medium, in which a plurality of instructions are stored, where the instructions can be loaded by a processor to execute the steps of any embodiment of the image processing method provided in the embodiment of the present invention.

Wherein the storage medium may include: read Only Memory (ROM), random Access Memory (RAM), magnetic or optical disks, and the like.

Since the instructions stored in the storage medium can execute the steps in any embodiment of the image processing method provided in the embodiment of the present invention, the beneficial effects that can be achieved by any image processing method provided in the embodiment of the present invention can be achieved, which are detailed in the foregoing embodiments and will not be described herein again.

The foregoing detailed description has provided an image processing method, apparatus, device and storage medium provided in an embodiment of the present application, and specific examples have been applied herein to explain the principles and implementations of the present application, and the description of the foregoing embodiments is only used to help understand the method and its core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present application. Moreover, it will be apparent to those skilled in the art that various modifications and adaptations can be made without departing from the principles of the invention, and such modifications and adaptations are intended to be within the scope of the invention.

Claims

1. An image processing method, characterized by comprising:

and obtaining an image area corresponding to a target foreground object in the target image according to the second foreground montage picture and the target image.

2. The image processing method according to claim 1, wherein the preset pixel parameter threshold at least includes a first threshold and a second threshold, and the determining the second foreground mask of the target image according to the preset pixel parameter threshold for image edge region noise processing, the first foreground mask and the target image includes:

and determining a second foreground masking image of the target image according to the first Trimap and the target image.

3. The image processing method as claimed in claim 2, wherein the first threshold is smaller than the second threshold, and the performing edge noise removal processing on the first mask and the second mask respectively to obtain a corresponding first noise removal result and a corresponding second noise removal result comprises:

4. The image processing method of claim 3, wherein the determining a second foreground montage of the target image according to the first Trimap and the target image comprises:

5. The image processing method of claim 1, wherein the determining a first foreground montage of the target image according to the target image to be processed comprises:

inputting the target image to be processed into a trained matting model to obtain a first foreground masking image of the target image output by the matting model;

acquiring a training sample set; the training samples in the training sample set comprise a plurality of input images containing target foreground objects and label images of foreground montage pictures corresponding to the input images;

6. The image processing method of claim 5, wherein the target loss function value of the target loss function is a weighted sum of the loss function values of a minimum absolute value error function, a synthetic loss function, a gradient loss function, and a laplacian pyramid loss function;

7. The image processing method according to claim 1, wherein obtaining an image region corresponding to a target foreground object in the target image according to the second foreground mask and the target image comprises:

8. An image processing apparatus characterized by comprising:

the first determining module is used for determining a first foreground montage picture of a target image according to the target image to be processed;

9. An electronic device comprising a processor, a memory and a computer program stored in the memory and configured to be executed by the processor, the memory being coupled to the processor and the processor implementing the image processing method of any of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program, wherein the computer program, when executed, controls an apparatus in which the computer-readable storage medium is located to perform the image processing method according to any one of claims 1 to 7.