WO2021135676A1

WO2021135676A1 - Photographing background blurring method, mobile terminal, and storage medium

Info

Publication number: WO2021135676A1
Application number: PCT/CN2020/128657
Authority: WO
Inventors: 李鹏
Original assignee: 武汉Tcl集团工业研究院有限公司
Priority date: 2019-12-30
Filing date: 2020-11-13
Publication date: 2021-07-08

Abstract

A photographing background blurring method, a mobile terminal, and a storage medium. The method comprises: obtaining a depth image, and preprocessing and segmenting the depth image; blurring a segmented image to obtain an image subjected to background blurring. According to the method, a captured image is processed, so that the captured image has the effect of multi-level background blurring, and the effect of background blurring is improved.

Description

Method for photographing background blurring, mobile terminal and storage medium

priority

This disclosure requires that it be submitted to the Chinese Patent Office on December 30, 2019, the application number is "201911397607.7", the title of the invention is "a method and device for background blurring of pictures, computer equipment, and storage media" and the application date is The priority of a Chinese patent invention filed with the Chinese Patent Office on February 10, 2020, with the application number "202010085333.4" and the invention title "A method for blurring the background of a photograph, a mobile terminal and a storage medium", the entire content of which is incorporated by reference In this disclosure.

Technical field

The present disclosure relates to the technical field of image processing, and in particular to a method for blurring the background of a photograph, a mobile terminal and a storage medium.

Background technique

Among the current smartphone camera functions, the dual-camera-based camera background blur (the background blur is to make the depth of field shallow and focus on the subject) is becoming more and more popular.

At present, the main steps of background blurring are: using binocular cameras to generate a depth map (the depth map is the main image and sub-image taken by the binocular camera to synthesize an image with distance information, and each pixel in the image represents the distance from the scene to the camera. , Also known as depth, so it is called depth map), based on the depth map to segment the foreground and background; and then classify the background according to the depth value, each level is subjected to different blur and smoothing processing and superimposed with the foreground to achieve the background blur effect.

However, the foreground and background segmentation based on the depth map, and the multi-level blurring and superposition of the background based on the depth map have certain defects. The estimated depth map of the dual camera may be inaccurate (the depth map is taken by the main and sub cameras of the binocular camera). Image estimated), resulting in poor or wrong segmentation of the foreground and background, and the use of depth distance information (ie, depth value) hierarchical multi-level blur will make the adjacent areas appear inconsistent with the blur level, the foreground and the blur The superimposed background will appear inconsistent in the transition at the edge or have a halo effect; the above-mentioned defects will eventually cause the visual effect of the blurred picture to be poor.

Therefore, the existing technology needs to be improved and developed.

Public content

The main purpose of the present disclosure is to provide a method for blurring the background of a photograph, a mobile terminal, and a storage medium, which aims to solve the problem of inaccurate segmentation of the foreground and background in the prior art and the inconsistency of the blur levels between adjacent background blocks. The edges that overlap with the background have halos or unnatural transitions.

In order to achieve the above objective, the present disclosure provides a method for blurring the background of a photograph, and the method includes the following steps:

Acquire a depth map, and perform preprocessing and segmentation processing on the depth map;

The segmented picture is blurred to obtain a picture with a blurred background.

In the method for blurring the background of a photograph, the acquiring a depth map includes:

Obtain the first picture and the second picture;

The first picture and the second picture are combined into a depth map.

In the method for blurring a photographed background, the preprocessing and segmentation of the depth map includes:

The depth map is preprocessed, and the preprocessed depth map is segmented to obtain a foreground mask and a background mask.

In the method for blurring the background of a photograph, said performing blurring processing on the segmented picture includes:

Respectively determining a foreground depth value parameter and a background depth value parameter according to the foreground mask and the background mask;

The size of the blur kernel used for blurring is determined according to the foreground depth value parameter and the background depth value parameter, and the background blurring is performed on the first picture according to the blur kernel size.

In the method for blurring a photographed background, the first picture is a picture taken by a first camera group of a mobile terminal, and the first camera group includes one or more cameras;

The second picture is a picture taken by a second camera group of the mobile terminal, and the second camera group includes one or more cameras;

At least one of the cameras of the first camera group is different from the cameras of the second camera group.

In the method for blurring a photographed background, the preprocessing includes:

Performing edge-preserving filtering processing on the depth map;

The depth map after edge-preserving filtering processing is subjected to a median filtering operation.

In the method for blurring the background of a photograph, the segmentation of the preprocessed depth map to obtain a foreground mask and a background mask specifically includes:

Performing segmentation processing on the depth map after the median filtering operation;

The foreground mask and the background mask are obtained according to a preset segmentation threshold.

In the method for blurring a photographed background, wherein the obtaining the foreground mask and the background mask according to a preset segmentation threshold specifically includes:

Classify pixels with pixel values greater than the segmentation threshold in the depth map as the foreground mask;

The pixel points in the depth map whose pixel values are less than or equal to the segmentation threshold are classified as the background mask.

In the method for blurring a photographed background, wherein the determining a foreground depth value parameter and a background depth value parameter respectively according to the foreground mask and the background mask specifically includes:

According to the foreground mask and the background mask, the mean value of the foreground depth value and the mean square error of the foreground depth value, as well as the maximum value of the background depth value and the minimum value of the background depth value are respectively counted.

The method for blurring a photographed background, wherein the determining the size of the blur kernel used for blurring according to the foreground depth value parameter and the background depth value parameter specifically includes:

It is determined that the blur kernel of the foreground is a Gaussian kernel, then the blur radius Fg_r of the current pixel is:

Among them, Fg_level is the blur level, depthVal is the depth value, Fg_mean is the mean value of the foreground depth value, and Fg_std is the mean square error of the foreground depth value;

The size of the fuzzy kernel is (2r+1)*(2r+1), and the calculation formula of each weight is as follows:

Fg_k=exp(-Fg_r*dist);

Among them, dist is the Euclidean distance between the kernel midpoint (r,r) and its neighbors;

It is determined that the blur kernel of the background is a defocus blur kernel, then the blur radius Bg_r of the current pixel is:

Bg_r=Bg_level*(depthVal-Bg_min)/(Bg_max-Bg_min+1);

Among them, Bg_level is the number of blur levels, Bg_max is the maximum background depth value, and Bg_min is the minimum background depth value;

Bg_k=a*dist+b;

Among them, a and b represent weight values, and 0<a<1,b>1.

In the method for blurring the background of a photograph, the Fg_level is 2, the Bg_level is 11, a=0.1, and b=10.

In the method for blurring a photographed background, the background blurring is a pixel-by-pixel background blurring.

The method for blurring the background of photographing, wherein the method for blurring the background of photographing in the present disclosure further includes:

Preprocess the depth map to get the foreground image and background image;

Blur the background picture to obtain a blurred background picture;

Blur the foreground picture to obtain a transition zone between the foreground picture and the background picture;

The foreground picture and the blurred background picture are superimposed and merged in the transition zone to obtain a picture with a blurred background.

In the method for blurring the background of a photograph, the preprocessing of the depth map to obtain the foreground image and the background image includes:

Performing edge-preserving filtering on the depth image to obtain a first depth image;

Performing median filtering on the first depth picture to obtain a second depth picture;

Performing level set segmentation on the second depth picture to obtain a foreground mask and a background mask;

According to the foreground mask and the background mask, the foreground picture and the background picture are obtained.

The method for blurring the background of a photograph, wherein the blurring the background picture to obtain a blurring background picture includes:

Stretch and transform the background picture to obtain a first background picture;

Performing point-by-point multi-level blurring on the first background picture to obtain a second background picture;

Performing an inverse stretch transformation on the second background picture to obtain the blurred background picture.

In the method for blurring the background of a photograph, the step-by-point multi-level blurring of the first background picture to obtain the second background picture includes:

Calculating the coordinates of the centroid point of the foreground picture according to the foreground picture;

Calculating the linear distance between each pixel point in the first background picture and the centroid point according to the coordinates of the pixel points in the first background picture and the coordinates of the centroid point;

According to the straight line distance and the blur degree value, the blur radius of the pixel in the first background picture is calculated.

The method for blurring a photographed background, wherein, after calculating a blur radius of a pixel in the first background picture according to the linear distance and the value of the blur degree, the method further includes:

Generating a blur kernel of the pixel according to the blur radius of the pixel;

According to the blur kernel of the pixel point, blur processing is performed on the pixel point.

In the method for blurring a photographed background, wherein the superimposing the foreground picture and the blurring background picture and fusing them in the transition zone to obtain a picture with a blurring background includes:

In the non-transition zone area, replace the corresponding pixels in the captured picture with the pixels in the foreground picture or the blurred background picture;

In the transition zone area, according to the blur value, the pixel value of the foreground picture, and the pixel value of the blurred background picture, the fused pixel value of the transition zone area is obtained, wherein the blur value is the result of the erosion and blurring of the foreground picture Pixel values.

In the method for blurring the background of a photograph, before the preprocessing of the depth map to obtain the foreground image and the background image, the method further includes:

Obtain the photographed picture and the depth map.

In addition, in order to achieve the above object, the present disclosure also provides a mobile terminal, wherein the mobile terminal includes: a memory, a processor, and a camera background blur program stored in the memory and running on the processor When the photographing background blurring program is executed by the processor, the steps of the aforementioned photographing background blurring method are implemented.

In addition, in order to achieve the above-mentioned object, the present disclosure also provides a storage medium, wherein the storage medium stores a photographing background blurring program, and the photographing background blurring program is executed by a processor to realize the aforementioned photographing background blurring. The steps of the chemical method.

The present disclosure performs preprocessing and segmentation processing on the depth map by acquiring the depth map; performing blurring processing on the segmented picture to obtain a picture with a blurry background. The present disclosure processes the taken pictures, so that the taken pictures have the effect of multi-level blurring of the background, and the effect of the background blurring can be improved.

Description of the drawings

FIG. 1 is a flowchart of a preferred embodiment of a method for blurring a photographed background of the present disclosure;

2 is a flowchart of the first blurring method in the preferred embodiment of the photo background blurring method of the present disclosure;

FIG. 3 is a depth map after preprocessing in the first blurring method in the preferred embodiment of the photographing background blurring method of the present disclosure;

4 is an effect diagram after background blurring in the first blurring method in the preferred embodiment of the photographing background blurring method of the present disclosure;

FIG. 5 is a flowchart of the second blurring mode in the preferred embodiment of the photo background blurring method of the present disclosure;

FIG. 6 is a schematic diagram of the operating environment of the preferred embodiment of the mobile terminal of the present disclosure.

Detailed ways

In order to make the objectives, technical solutions, and advantages of the present disclosure clearer and clearer, the present disclosure will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present disclosure, but not used to limit the present disclosure.

As shown in FIG. 1, the method for blurring the background of a photograph according to a preferred embodiment of the present disclosure includes the following steps:

Step S100: Obtain a depth map, and perform preprocessing and segmentation processing on the depth map;

Step S200: Perform blurring processing on the divided picture to obtain a picture with a blurred background.

The first blurring method in the photographing background blurring method according to the preferred embodiment of the present disclosure, as shown in FIG. 2, the photographing background blurring method includes the following steps:

Step S10: Synthesize the first picture and the second picture into a depth map.

First obtain a first picture and a second picture. The first picture is a picture taken by a first camera group of a mobile terminal. The first camera group includes one or more cameras; the second picture is a picture taken by a mobile terminal. A picture taken by a second camera group of the terminal, where the second camera group includes one or more cameras; at least one of the cameras of the first camera group is different from the cameras of the second camera group.

The photo background blurring method of the present disclosure is applied to a mobile terminal (such as the most commonly used smart phone, but also other smart devices with dual cameras to take photos), the mobile terminal includes a main camera (that is, the first camera group) and The most obvious effect of the secondary camera (that is, the second camera group), dual camera or binocular camera is the excellent background blur effect. First, take pictures through the main camera and the secondary camera, and obtain two The first picture and the second picture taken at different angles (it should be noted that in this disclosure, the first picture and the second picture taken at different angles can also be obtained by two devices at the same time stamp, for example, two The first picture and the second picture of different angles obtained by a mobile phone at the same time, the present disclosure is preferably the first picture and the second picture of different angles obtained by the main camera and the auxiliary camera on a mobile terminal at the same time), for example, taken by the main camera Is the first picture, the second picture is taken by the secondary camera, and the first picture and the second picture are synthesized into a depth map. The depth map is the main picture and the auxiliary picture taken by a binocular camera with a distance The image of the information (each pixel value represents the distance from the object to the xy plane of the camera), and the depth image is also called the range image, which refers to the distance from the image collector to the scene The distance (depth) of the point is used as the image of the pixel value, which directly reflects the geometric shape of the visible surface of the scene; and the depth map is estimated from the pictures taken by the primary and secondary cameras of the binocular camera.

In step S20, the depth map is preprocessed, and the preprocessed depth map is segmented to obtain a foreground mask and a background mask.

After synthesizing the depth map, the depth map needs to be preprocessed. The purpose of the preprocessing is to obtain a uniform depth map, that is, to preprocess the depth map to make the depth map more uniform and consistent (uniform The consistent feature is that the depth value in the depth map changes smoothly, and the local area has a smaller change in depth value, as shown in Figure 3.

Wherein, the preprocessing specifically includes: performing edge-preserving filtering processing on the depth map, and the edge-preserving filtering processing is used to protect the edges of the image (that is, the depth map), and the edge-preserving filtering is a filtering method. The edge information in the image can be effectively preserved in the process. For example, the edge-preserving filter can be processed by the edge-preserving filter; the depth map after the edge-preserving filter processing is subjected to the median filtering operation, and the median filtering operation is used To remove image burrs, median filtering is a non-linear smoothing technique, which sets the gray value of each pixel to the median of the gray values of all pixels in a certain neighborhood window at that point. Value filtering is a nonlinear signal processing technology that can effectively suppress noise based on the sorting statistical theory. The basic principle of median filtering is to use the value of a point in a digital image or digital sequence with the value of each point in a neighborhood of that point. The median value is replaced, so that the surrounding pixel values are close to the true value, thereby eliminating isolated noise points.

Further, the depth map after the median filtering operation is segmented, using OTSU (Otsu Method or Maximum Between-Class Variance Method, an efficient algorithm for binarizing the image, using a threshold to divide the image into foreground and Background) Roughly segment the depth map, obtain a preset segmentation threshold (T), and obtain the foreground mask and the background mask according to the preset segmentation threshold.

Wherein, the segmentation threshold (T) is a pixel value, for example: the pixels of an image are between 0 and 255, if T=128, the pixels smaller than T or the same are classified as background, and larger than T The pixels are classified as foreground.

Therefore, in the present disclosure, pixels in the depth map with pixel values greater than the segmentation threshold are classified as the foreground mask; pixels in the depth map with pixel values less than or equal to the segmentation threshold are classified as The background mask.

Wherein, the foreground mask (Fg_mask) represents a logo image, in which pixels only have 0 and 1, a pixel of 1 indicates that the point belongs to the foreground, and a pixel of 0 indicates that the point belongs to the background; the background mask (Bg_mask) ) Is the inverse of the foreground mask (Fg_mask).

Step S30: Determine a foreground depth value parameter and a background depth value parameter according to the foreground mask and the background mask, respectively.

According to the foreground mask and the background mask, the mean value of the foreground depth value and the mean square error of the foreground depth value, as well as the maximum value of the background depth value and the minimum value of the background depth value are counted respectively; that is, the foreground depth value parameter includes the mean value of the foreground depth value (Fg_mean) and the mean square error of the foreground depth value (Fg_std). The background depth value parameter includes a maximum background depth value (Bg_max) and a minimum background depth value (Bg_min).

Further, the corresponding depth map pixel can be obtained according to the position of the pixel with the flag of 1 in the foreground mask (Fg_mask), the pixel values of all the foreground pixels in the depth map are counted, the average value is calculated, and the variance is calculated, and the background mask After obtaining the pixel values of all background pixels in the depth map, the mask (Bg_mask) can obtain the maximum background depth value and the minimum background depth value.

Step S40: Determine a blur kernel size for blurring according to the foreground depth value parameter and the background depth value parameter, and perform background blurring on the first picture according to the blur kernel size.

Wherein, the background blur is a pixel-by-pixel background blur.

Determine the blur level of the rough foreground and the rough background. Most of the rough foreground and the foreground overlap, and may not overlap at the edge. The foreground is more accurate than the rough foreground. The relationship between the rough background and the background is also the same. ; And determine the type and size of the multi-level virtual core, the size of the virtual core is inconsistent called multi-level virtual core, such as 3x3, 5x5, 7x7 and other square cores of different sizes.

Further, the determining the size of the blur kernel used for blurring according to the foreground depth value parameter and the background depth value parameter specifically includes:

(1) Determine that the blur kernel of the foreground is a Gaussian kernel (the Gaussian kernel means that the weight value in the kernel obeys a two-dimensional discrete Gaussian distribution), then the blur radius Fg_r of the current pixel is:

Among them, Fg_level is the blur level, depthVal is the depth value, Fg_mean is the mean value of the foreground depth value, Fg_std is the mean square error of the foreground depth value; Fg_level is 2;

The size of the fuzzy kernel (a fuzzy kernel is a square with a weight value) is (2r+1)*(2r+1), where each weight (each weight represents a different weight value in a kernel, the weight is a value, the reaction The calculation formula of the importance of a certain pixel is as follows:

Fg_k=exp(-Fg_r*dist);

(2) Determine that the background blur kernel is a defocus blur kernel (the defocus blur kernel is in a square kernel, the weight value in the inscribed circle is not 0, and the weight value outside the inscribed circle is 0) , The blur radius Bg_r of the current pixel is:

Bg_r=Bg_level*(depthVal-Bg_min)/(Bg_max-Bg_min+1);

Among them, Bg_level is the number of blur levels, Bg_max is the maximum background depth value, Bg_min is the minimum background depth value; Bg_level is 11;

The size of the fuzzy kernel (a fuzzy kernel is a square with a weight value) is (2r+1)*(2r+1), and the calculation formula for each weight is as follows:

Bg_k=a*dist+b;

Among them, a and b represent weight values, which are used to control the influence of the dist value, and 0<a<1, b>1, a=0.1, b=10.

The first picture taken by the main camera of the mobile terminal (the picture of the secondary camera does not need to be processed, and the function of the secondary camera is to cooperate with the main camera to generate a depth map), and perform pixel-by-pixel blurring according to the corresponding depth map traversal , The blur kernel of each pixel depends on the corresponding depth value depthVal, and the blur kernel is selected as described above.

Further, because the pixel-by-pixel traversal is relatively time-consuming, the blur operation between pixels is independent, and multi-threaded parallel acceleration calculation can be used to obtain the background blur effect map, as shown in FIG. 4. The present disclosure can improve the visual effect of the blurred background of the dual-camera photography of the mobile terminal, and reduce the memory consumption of the mobile terminal.

The present disclosure directly performs blur processing based on different blur kernel sizes on the original images (such as the first picture and the second picture), the depth of the foreground area is small, and the blur kernel size is basically 1, which is essentially equivalent to no blurring; Even if there is an abnormal depth value, a small blur core (3x3) will not affect the final visual effect; do not do the layered multi-level blur processing superimposition of the background, avoiding the halo or the edge of the foreground and the background superimposed Transition unnatural phenomenon, real-time multi-level background blur effect.

Further, the inventor found through research that in the field of smartphone photography, the existing background blurring in the processing of captured photos mainly uses the depth map synthesized by the main camera and the sub-camera to segment the foreground and background from the depth map, and then The background and foreground after layered multi-level background blur are superimposed to generate a background blur effect. However, the layered multi-level background blur using depth map and the superposition of foreground and background have certain defects, which affect the effect of photo shooting. Can not meet the real-time nature of mobile phone photography.

In order to solve the above-mentioned problems, in the embodiments of the present disclosure, the effect of background blur is mainly improved in three aspects: 1. Through the stretching operation and reverse stretching operation of the pixels of the background picture, the blurred background has a certain degree of Spot effect; 2. Taking the center of the foreground contour as the center of the circle, the circle is diffused outward to increase the radius of the blur core to blur point by point, achieving the purpose of multi-level blur but not layering, and solves the multi-level blur Improve the unnatural problem of edge transition of background superimposition; 3. The method of etching and Gaussian blur smoothing of the foreground mask generates a transition zone between the foreground and the background, and merges the foreground and background of the transition zone, effectively alleviating the abrupt edge transition and the appearance of black Issues such as edges and halos.

Hereinafter, various non-limiting embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

The embodiment of the present disclosure provides a second blurring method in the background blurring method for photographing. As shown in FIG. 5, the method includes:

S1. Preprocess the depth map to obtain the foreground image and the background image.

In the embodiment of the present disclosure, before preprocessing the depth map, the captured picture and the depth map are acquired. In step S1, the depth map is preprocessed, the foreground image F_Img and the background image B_Img are segmented, and the corresponding foreground mask F_mask and background mask B_mask are obtained, and the foreground image and background image are obtained according to the foreground mask F_mask and the background mask B_mask. Among them, the depth map may be a picture obtained by taking a picture and processing it. The foreground picture can be the subject in the captured picture, the object close to the lens, the object that the user wants to take, or the focus of the mobile phone camera, etc. The background picture is the picture excluding the foreground area in the picture. For example, if a person is photographed, the area of the person in the picture is the foreground picture, and the rest of the area is the background picture.

In an optional embodiment of the present disclosure, step S1 includes:

S11. Perform edge-preserving filtering on the depth image to obtain a first depth image.

S12. Perform median filtering on the first depth picture to obtain a second depth picture.

S13. Perform level set segmentation on the second depth picture to obtain a foreground mask and a background mask.

In the embodiment of the present disclosure, in order to remove the burrs in the first depth picture, median filtering is performed on the first depth picture in step S12. By performing level set segmentation on the second depth image, the contour of the foreground area is obtained, and the foreground mask and the background mask are obtained according to the contour of the foreground area. The first depth picture and the second depth picture are both depth pictures during processing.

S14. Obtain the foreground picture and the background picture according to the foreground mask and the background mask.

S2. Blur the background picture to obtain a blurred background picture.

In the embodiment of the present disclosure, the blurred background picture is a blurred background picture. In step S2, the pixelValue in the background image that exceeds the first threshold is stretched and transformed, and then the stretched background image is subjected to point-by-point multi-level blurring, and finally the blurring background image is reverse-stretched Transform to get a blurred background image.

In an optional embodiment of the present disclosure, step S2 includes:

S21: Perform stretching transformation on the background picture to obtain a first background picture.

In the embodiment of the present disclosure, the first background picture is a background picture after stretching and transformation. Perform stretching transformation on the pixel values in the background image that exceed the first threshold, where the representation form of the stretching transformation is as follows:

pixelValue=a*(pixelValue-threshold) ^b +threshold

Wherein: threshold is the first threshold, the values of a, b, and threshold depend on the specific spot effect to be achieved, where b>1 and a>1, for example: a=2, b=2, and threshold=128.

S22. Perform point-by-point multi-level blurring on the first background picture to obtain a second background picture.

In the embodiment of the present disclosure, the second background picture is a background picture that is blurred by point-by-point multi-level blur. According to the foreground contour, the centroid point of the foreground contour is calculated, and the linear distance between each pixel point and the centroid point in the stretched background image is calculated.

In the embodiments of the present disclosure, a point-by-point multi-level blurring method is used to solve the problem that the hierarchical multi-level blurring using depth distance information in the prior art will cause inconsistent blur levels in adjacent areas, and more blur levels will cause The processing time is too long to meet the real-time performance of the mobile phone, which makes the background blurry and the transition more natural.

In an optional embodiment of the present disclosure, step S22 includes:

S221. Calculate the coordinates of the centroid point of the foreground picture according to the foreground picture.

In the embodiment of the present disclosure, the centroid point (x0, y0) of the foreground contour can be calculated according to the contour of the foreground in the foreground picture.

S222: Calculate the linear distance between each pixel point in the first background picture and the centroid point according to the coordinates of the pixel points in the first background picture and the coordinates of the centroid point.

In the embodiment of the present disclosure, the first background picture is a stretched background picture, and it is calculated according to the coordinates (i, j) of the pixel points in the first background picture and the coordinates (x0, y0) of the centroid point The linear distance between each pixel point (i, j) and the centroid point (x0, y0) in the first background picture.

S223: Calculate the blur radius of the pixel in the first background picture according to the straight line distance and the blur degree value.

In the embodiment of the present disclosure, the centroid point (x0, y0) of the foreground contour is calculated according to the contour of the foreground, and the linear distance dist between all (i, j) and (x0, y0) is calculated. The characterization form of the radius of the pixel point blur is:

radius=baseRadius*(dist _{i, j} /max Dist)

Among them, disti, j is the distance between the pixel point of the background image coordinate (i, j) and (x0, y0), maxDist is the maximum value of all straight-line distances, baseRadius is the value of the degree of blur, baseRadius is taken according to the degree of blur Value, for example: baseRadius can be 15.

In an optional embodiment of the present disclosure, after step S223, the method further includes:

S224: Generate a blur kernel of the pixel point according to the blur radius of the pixel point.

S225: Perform blur processing on the pixel according to the blur kernel of the pixel.

In the embodiment of the present disclosure, the pixels in the first background picture are traversed, the corresponding blur kernel is generated according to the blur radius of the pixel, and the pixel is blurred. The pixel-by-pixel traversal is relatively time-consuming, and the blur operation between pixels is independent, and multi-threaded parallel processing can be used to accelerate processing.

S23. Perform inverse stretch transformation on the second background picture to obtain the blurred background picture.

In the embodiment of the present disclosure, the second background picture is a background picture that is blurred by point-by-point multi-level blur. The characterization form of the inverse stretch transformation is the inverse function of the above-mentioned stretch transformation expression.

S3. Blur the foreground picture to obtain a transition zone between the foreground picture and the background picture.

In the embodiment of the present disclosure, the mask F_mask of the foreground area is processed by etching and Gaussian blur, and a transition zone with pixel values ranging from 0 to 255 (not including 0 and 255) is generated at the edge of the mask and normalized To 0~1, the transition zone is the edge area adjacent to the foreground and the background. The foreground pixel value of the binary mask image is 255, and the background pixel value is 0. A smoothing is performed on the edge part adjacent to the foreground and background. Processing, such as Gaussian blur processing, makes the pixel values of the foreground and background edge parts between 0 and 255. The area where the pixel value is between 0 and 255 is called the transition zone.

S4. Superimpose the foreground picture and the blurred background picture and merge them in the transition zone to obtain a picture with a blurred background.

In an optional embodiment of the present disclosure, step S4 includes:

For example: in the non-transition zone area of the taken picture, the pixels belonging to the foreground picture are replaced with the pixels of the corresponding position in the foreground picture, and the pixels belonging to the background picture in the non-transition zone area of the taken picture are replaced with the background The pixels at the corresponding positions in the picture are replaced. After the replacement is completed, the background of the picture in the non-transition zone is the blurred background.

In the embodiment of the present disclosure, the pixel value of the F_mask processed in step S3 of the pixel of the transition zone area is w, the pixel value of the transition zone area is F_pixel in the foreground image, and the pixel point of the transition zone area is The pixel value of the blurred background image is B_pixel, and the pixel value of the pixel after fusion in the transition zone is FB_pixel=W*F_pixel+(1-w)*B_pixel.

In the embodiment of the present disclosure, the processed foreground and background are merged by generating a transition zone between the foreground and the background, so as to alleviate the abrupt edge transition and avoid the problem of black borders and halos appearing on the boundary between the foreground and the background.

Further, as shown in FIG. 6, based on the aforementioned method for blurring the background of a photograph, the present disclosure also provides a mobile terminal correspondingly, and the mobile terminal includes a processor 10, a memory 20 and a display 30. FIG. 6 only shows part of the components of the mobile terminal, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

The mobile terminal also includes a first camera group and a second camera group; the first camera group and the second camera group are used to acquire two first pictures and second pictures taken from different angles; the first The picture is a picture taken by the first camera group, the first camera group includes one or more cameras; the second picture is a picture taken by the second camera group, the second camera group includes One or more cameras; at least one of the cameras of the first camera group is different from the cameras of the second camera group.

Wherein, the memory 20 may be an internal storage unit of the mobile terminal in some embodiments, such as a hard disk or a memory of the mobile terminal. In other embodiments, the memory 20 may also be an external storage device of the mobile terminal, such as a plug-in hard disk equipped on the mobile terminal, a smart media card (SMC), and a secure digital (Secure Digital). Digital, SD) card, flash card, etc. Further, the memory 20 may also include both an internal storage unit of the mobile terminal and an external storage device. The memory 20 is used to store application software and various types of data installed in the mobile terminal, such as the program code of the installed mobile terminal. The memory 20 can also be used to temporarily store data that has been output or will be output. In one embodiment, the memory 20 stores a photographing background blurring program 40, and the photographing background blurring program 40 can be executed by the processor 10, so as to realize the photographing background blurring method in the present disclosure.

The processor 10 may be a central processing unit (CPU), microprocessor or other data processing chip in some embodiments, and is used to run the program code or process data stored in the memory 20, for example Perform the method of blurring the background of the photograph and so on.

In some embodiments, the display 30 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch device, etc. The display 30 is used for displaying information on the mobile terminal and for displaying a visualized user interface. The components 10-30 of the mobile terminal communicate with each other via a system bus.

In one embodiment, when the processor 10 executes the photographing background blurring program 40 in the memory 20, the following steps are implemented:

The segmented picture is blurred to obtain a picture with a blurred background.

The acquiring depth map includes:

Obtain the first picture and the second picture;

The first picture and the second picture are combined into a depth map.

The preprocessing and segmentation processing of the depth map includes:

The blurring processing of the divided picture includes:

The first picture is a picture taken by a first camera group of the mobile terminal, and the first camera group includes one or more cameras;

The preprocessing includes:

Performing edge-preserving filtering processing on the depth map;

The segmentation of the preprocessed depth map to obtain a foreground mask and a background mask specifically includes:

The obtaining the foreground mask and the background mask according to a preset segmentation threshold specifically includes:

The respectively determining the foreground depth value parameter and the background depth value parameter according to the foreground mask and the background mask specifically includes:

The determining the size of the blur kernel used for blurring according to the foreground depth value parameter and the background depth value parameter specifically includes:

Fg_k=exp(-Fg_r*dist);

Bg_r=Bg_level*(depthVal-Bg_min)/(Bg_max-Bg_min+1);

Bg_k=a*dist+b;

Among them, a and b represent weight values, and 0<a<1,b>1.

Wherein, the Fg_level is 2, the Bg_level is 11, a=0.1, and b=10.

Wherein, the background blur is a pixel-by-pixel background blur.

The photo background blurring method further includes:

Preprocess the depth map to get the foreground image and background image;

Blur the background picture to obtain a blurred background picture;

The preprocessing of the depth map to obtain the foreground image and the background image includes:

The blurring the background picture to obtain a blurring background picture includes:

The step-by-point multi-level blurring of the first background picture to obtain the second background picture includes:

After calculating the blur radius of the pixel in the first background picture according to the straight line distance and the blur degree value, the method includes:

The superimposing the foreground picture and the blurred background picture and fusing them in the transition zone to obtain a blurred background picture includes:

Obtain the photographed picture and the depth map.

The present disclosure also provides a storage medium, wherein the storage medium stores a photographing background blurring program, and the photographing background blurring program is executed by a processor to realize the steps of the photographing background blurring method as described above.

In summary, the present disclosure provides a method for blurring a photographed background, a mobile terminal, and a storage medium. The method includes: acquiring a depth map, performing preprocessing and segmentation processing on the depth map; and performing a virtual image on the segmented picture. After the bokeh process, the picture with the blurred background is obtained. The present disclosure processes the taken pictures, so that the taken pictures have the effect of multi-level blurring of the background, and the effect of the background blurring can be improved.

Of course, those of ordinary skill in the art can understand that all or part of the processes in the methods of the above-mentioned embodiments can be implemented by instructing relevant hardware (such as a processor, a controller, etc.) through a computer program, and the program can be stored in a computer program. In a computer-readable storage medium, the program may include the processes of the foregoing method embodiments when executed. The storage medium mentioned may be a memory, a magnetic disk, an optical disk, and the like.

It should be understood that the application of the present disclosure is not limited to the above examples, and those of ordinary skill in the art can make improvements or changes based on the above description, and all these improvements and changes should fall within the protection scope of the appended claims of the present disclosure.

Claims

A method for blurring a photographed background, wherein the method for blurring a photographed background includes:

Acquire a depth map, and perform preprocessing and segmentation processing on the depth map;

The segmented picture is blurred to obtain a picture with a blurred background.
The method for blurring a photographed background according to claim 1, wherein said acquiring a depth map comprises:

Obtain the first picture and the second picture;

The first picture and the second picture are combined into a depth map.
The method for blurring a photographed background according to claim 1, wherein the preprocessing and segmentation processing of the depth map comprises:

The depth map is preprocessed, and the preprocessed depth map is segmented to obtain a foreground mask and a background mask.
The method for blurring a photographed background according to claim 3, wherein said performing blurring processing on the segmented picture comprises:

Respectively determining a foreground depth value parameter and a background depth value parameter according to the foreground mask and the background mask;

The size of the blur kernel used for blurring is determined according to the foreground depth value parameter and the background depth value parameter, and the background blurring is performed on the first picture according to the blur kernel size.
The method for blurring a photographic background according to claim 2, wherein the first picture is a picture taken by a first camera group of a mobile terminal, and the first camera group includes one or more cameras;

The second picture is a picture taken by a second camera group of the mobile terminal, and the second camera group includes one or more cameras;

At least one of the cameras of the first camera group is different from the cameras of the second camera group.
The method for blurring a photographed background according to claim 3, wherein the preprocessing comprises:

Performing edge-preserving filtering processing on the depth map;

The depth map after edge-preserving filtering processing is subjected to a median filtering operation.
The method for blurring a photographed background according to claim 6, wherein the segmenting the preprocessed depth map to obtain a foreground mask and a background mask specifically includes:

Performing segmentation processing on the depth map after the median filtering operation;

The foreground mask and the background mask are obtained according to a preset segmentation threshold.
8. The method for blurring a photographed background according to claim 7, wherein the obtaining the foreground mask and the background mask according to a preset segmentation threshold specifically includes:

Classify pixels with pixel values greater than the segmentation threshold in the depth map as the foreground mask;

The pixel points in the depth map whose pixel values are less than or equal to the segmentation threshold are classified as the background mask.
The method for blurring a photographed background according to any one of claims 4, wherein said determining a foreground depth value parameter and a background depth value parameter respectively according to the foreground mask and the background mask specifically includes:

According to the foreground mask and the background mask, the mean value of the foreground depth value and the mean square error of the foreground depth value, as well as the maximum value of the background depth value and the minimum value of the background depth value are respectively counted.
The method for blurring a photographed background according to claim 9, wherein the determining the size of the blur kernel used for blurring according to the foreground depth value parameter and the background depth value parameter specifically includes:

It is determined that the blur kernel of the foreground is a Gaussian kernel, then the blur radius Fg_r of the current pixel is:

Among them, Fg_level is the blur level, depthVal is the depth value, Fg_mean is the mean value of the foreground depth value, and Fg_std is the mean square error of the foreground depth value;

The size of the fuzzy kernel is (2r+1)*(2r+1), and the calculation formula of each weight is as follows:

Fg_k=exp(-Fg_r*dist);

Among them, dist is the Euclidean distance between the kernel midpoint (r,r) and its neighbors;

It is determined that the blur kernel of the background is a defocus blur kernel, then the blur radius Bg_r of the current pixel is:

Bg_r=Bg_level*(depthVal-Bg_min)/(Bg_max-Bg_min+1);

Among them, Bg_level is the number of blur levels, Bg_max is the maximum background depth value, and Bg_min is the minimum background depth value;

The size of the fuzzy kernel is (2r+1)*(2r+1), and the calculation formula of each weight is as follows:

Bg_k=a*dist+b;

Among them, a and b represent weight values, and 0<a<1,b>1.
10. The method for blurring a photographic background according to claim 10, wherein the Fg_level is 2, the Bg_level is 11, a=0.1, and b=10.
The method for blurring a photographed background according to claim 1, wherein the background blurring is a pixel-by-pixel background blurring.
The method for blurring a photographed background according to claim 1, wherein the method for blurring a photographed background further comprises:

Preprocess the depth map to get the foreground image and background image;

Blur the background picture to obtain a blurred background picture;

Blur the foreground picture to obtain a transition zone between the foreground picture and the background picture;

The foreground picture and the blurred background picture are superimposed and merged in the transition zone to obtain a picture with a blurred background.
The method according to claim 13, wherein said preprocessing the depth map to obtain the foreground picture and the background picture comprises:

Performing edge-preserving filtering on the depth image to obtain a first depth image;

Performing median filtering on the first depth picture to obtain a second depth picture;

Performing level set segmentation on the second depth picture to obtain a foreground mask and a background mask;

According to the foreground mask and the background mask, the foreground picture and the background picture are obtained.
The method according to claim 13, wherein said blurring said background picture to obtain a blurred background picture comprises:

Stretch and transform the background picture to obtain a first background picture;

Performing point-by-point multi-level blurring on the first background picture to obtain a second background picture;

Performing an inverse stretch transformation on the second background picture to obtain the blurred background picture.
The method according to claim 15, wherein said performing point-by-point multi-level blurring on the first background picture to obtain the second background picture comprises:

Calculating the coordinates of the centroid point of the foreground picture according to the foreground picture;

Calculating the linear distance between each pixel point in the first background picture and the centroid point according to the coordinates of the pixel points in the first background picture and the coordinates of the centroid point;

According to the straight line distance and the blur degree value, the blur radius of the pixel in the first background picture is calculated.
The method according to claim 16, wherein after calculating the blur radius of the pixel in the first background image according to the straight line distance and the blur degree value, the method further comprises:

Generating a blur kernel of the pixel according to the blur radius of the pixel;

According to the blur kernel of the pixel point, blur processing is performed on the pixel point.
The method according to claim 13, wherein the superimposing the foreground picture and the blurred background picture and fusing them in the transition zone to obtain a picture with a blurred background comprises:

In the non-transition zone area, replace the corresponding pixels in the captured picture with the pixels in the foreground picture or the blurred background picture;

In the transition zone area, according to the blur value, the pixel value of the foreground picture, and the pixel value of the blurred background picture, the fused pixel value of the transition zone area is obtained, wherein the blur value is the result of the erosion and blurring of the foreground picture Pixel values.
The method according to claim 13, wherein before said preprocessing the depth map to obtain the foreground picture and the background picture, the method further comprises:

Obtain the photographed picture and the depth map.
A mobile terminal, wherein the mobile terminal includes a memory, a processor, and a photographing background blurring program stored in the memory and running on the processor, and the photographing background blurring program is When the processor is executed, the steps of the method for blurring the photographic background according to any one of claims 1-19 are realized.
A storage medium, wherein the storage medium stores a photographing background blurring program, and when the photographing background blurring program is executed by a processor, the method for implementing the photographing background blurring method according to any one of claims 1-19 step.