WO2021068729A1

WO2021068729A1 - Image synthesis method and apparatus, electronic device, and computer readable storage medium

Info

Publication number: WO2021068729A1
Application number: PCT/CN2020/116292
Authority: WO
Inventors: 吕烨华
Original assignee: 北京字节跳动网络技术有限公司
Priority date: 2019-10-10
Filing date: 2020-09-18
Publication date: 2021-04-15
Also published as: CN112651909A; CN112651909B

Abstract

Disclosed are an image synthesis method and apparatus, an electronic device, and a computer readable storage medium. The image synthesis method comprises: acquiring an original image frame from an image, the original image frame comprising a target object; segmenting a target object image from the original image frame; performing blurring processing on the target object image to obtain a first image; acquiring a historical first image from a cache queue, the historical first image being obtained by processing an original image frame prior to the original image frame; placing the first image in the cache queue; acquiring a noisy image frame, the noisy image frame being a greyscale image; and, on the basis of the original image frame, the historical first image, and the noisy image frame, generating a synthesised image. The present method solves the technical problem in the prior art of being unable to simply and quickly generate video effects.

Description

Image synthesis method, device, electronic equipment and computer readable storage medium

Cross-references to related applications

This application claims the priority of a Chinese patent application filed on October 10, 2019 with the application number 201910959995.7 and the invention title "Image synthesis method, device, electronic equipment and computer readable storage medium". The full text of the application is approved The reference is incorporated in this application.

Technical field

The present disclosure relates to the field of image synthesis, and in particular to an image synthesis method, device, electronic equipment, and computer-readable storage medium.

Background technique

With the development of computer networks and the popularization of smart phones, ordinary users can no longer be satisfied with just using monotonous pictures and words to express their emotions. Video is deeply loved by users for presenting more abundant and diverse content and forms and the intuitive feelings it brings, and it is gradually becoming popular. It is gradually becoming a trend for ordinary users to make original videos. But on the other hand, the original self-portrait video expression is plain and tasteless. At the same time, we can see that the use of video special effects in film and television works is becoming more and more abundant, and the form of content expression is also more diversified. It can be said that video persistence Effectiveness is the support and guarantee of a successful film and television work.

However, the current video special effects production is generally completed by post-production after recording the video. The displayed special effects are fixed and can only be played until the end according to the pre-time logic; and the threshold of post-production is high, and ordinary users cannot generate it quickly. A special effect or a more complex special effect. Therefore, how to generate video effects simply and quickly becomes a technical problem to be solved urgently.

Summary of the invention

The content of the invention is provided to introduce concepts in a brief form, and these concepts will be described in detail in the following specific embodiments. The content of the invention is not intended to identify the key features or essential features of the technical solution required to be protected, nor is it intended to be used to limit the scope of the technical solution required to be protected.

In the first aspect, an embodiment of the present disclosure provides an image synthesis method, including:

Obtaining an original image frame from an image source, wherein the original image frame includes a target object;

Segmenting the target object image from the original image frame;

Performing blur processing on the target object image to obtain a first image;

Acquiring a historical first image in the buffer queue, where the historical first image is obtained by processing an original image frame before the original image frame;

Putting the first image in the buffer queue;

Acquiring a noise image frame, where the noise image frame is a grayscale image;

A composite image is generated according to the original image frame, the historical first image, and the noise image frame.

In a second aspect, an embodiment of the present disclosure provides an image synthesis device, including:

An original image frame obtaining module, configured to obtain an original image frame from an image source, wherein the original image frame includes a target object;

A target object image segmentation module, configured to segment the target object image from the original image frame;

The blur processing module is used to perform blur processing on the target object image to obtain the first image;

The historical first image acquisition module is configured to acquire the historical first image in the buffer queue, where the historical first image is obtained by processing the original image frame before the original image frame;

The first image buffer module is configured to put the first image into the buffer queue;

A noise image frame acquisition module, configured to acquire a noise image frame, where the noise image frame is a grayscale image;

The synthesis module is used to generate a synthesized image according to the original image frame, the first historical image, and the noise image frame.

In a third aspect, an embodiment of the present disclosure provides an electronic device, including: at least one processor; and,

A memory that is communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processing The device can execute any one of the image synthesis methods described in the first aspect.

In a fourth aspect, embodiments of the present disclosure provide a non-transitory computer-readable storage medium, characterized in that the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to make a computer execute the aforementioned first aspect Any of the image synthesis methods described above.

The present disclosure discloses an image synthesis method, device, electronic equipment, and computer-readable storage medium. The image synthesis method includes: obtaining an original image frame from an image source, wherein the original image frame includes a target object; segmenting the target object image from the original image frame; and blurring the target object image to obtain the first An image; obtain a historical first image in the buffer queue, the historical first image is obtained by processing the original image frame before the original image frame; put the first image in the buffer queue; obtain a noise image Frame, the noise image frame is a grayscale image; a composite image is generated according to the original image frame, the historical first image, and the noise image frame. Through the above method, the technical problem that the video effect cannot be generated simply and quickly in the prior art is solved.

The above description is only an overview of the technical solutions of the present disclosure. In order to understand the technical means of the present disclosure more clearly, they can be implemented in accordance with the content of the specification, and to make the above and other objectives, features and advantages of the present disclosure more obvious and understandable. In the following, the preferred embodiments are cited in conjunction with the drawings, and the detailed description is as follows.

Description of the drawings

The above and other features, advantages, and aspects of the embodiments of the present disclosure will become more apparent in conjunction with the accompanying drawings and with reference to the following specific implementations. Throughout the drawings, the same or similar reference signs indicate the same or similar elements. It should be understood that the drawings are schematic and the originals and elements are not necessarily drawn to scale.

FIG. 1 is a flowchart of an embodiment of an image synthesis method provided by the present disclosure;

2 is a flow chart of a specific example of step S104 in the embodiment of the image synthesis method provided by the present disclosure;

FIG. 3 is a flow chart of a specific example of step S107 in the embodiment of the image synthesis method provided by the present disclosure;

4 is a flow chart of a specific example of step S301 in the embodiment of the image synthesis method provided by the present disclosure;

FIG. 5 is a flow chart of a specific example of step S402 in the embodiment of the image synthesis method provided by the present disclosure;

6 is a flow chart of a specific example of step S302 in the embodiment of the image synthesis method provided by the present disclosure;

FIG. 7 is a flowchart of another specific example of step S302 in the embodiment of the image synthesis method provided by the present disclosure

FIG. 8 is a schematic structural diagram of an embodiment of an image synthesis device provided by an embodiment of the disclosure;

Fig. 9 is a schematic structural diagram of an electronic device provided according to an embodiment of the present disclosure.

Detailed ways

Hereinafter, embodiments of the present disclosure will be described in more detail with reference to the accompanying drawings. Although some embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided for Have a more thorough and complete understanding of this disclosure. It should be understood that the drawings and embodiments of the present disclosure are only used for exemplary purposes, and are not used to limit the protection scope of the present disclosure.

It should be understood that the steps recorded in the method embodiments of the present disclosure may be executed in a different order, and/or executed in parallel. In addition, method implementations may include additional steps and/or omit to perform the illustrated steps. The scope of the present disclosure is not limited in this respect.

The term "including" and its variations as used herein are open-ended includes, that is, "including but not limited to". The term "based on" is "based at least in part on." The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one additional embodiment"; the term "some embodiments" means "at least some embodiments." Related definitions of other terms will be given in the following description.

It should be noted that the concepts of “first” and “second” mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units. Or interdependence.

It should be noted that the modifications of “a” and “a plurality of” mentioned in the present disclosure are illustrative and not restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as “one or Multiple".

The names of messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes, and are not used to limit the scope of these messages or information.

FIG. 1 is a flowchart of an embodiment of an image synthesis method provided by an embodiment of the disclosure. The image synthesis method provided in this embodiment may be executed by an image synthesis device, and the image synthesis device may be implemented as software, or as software and With a combination of hardware, the image synthesis device can be integrated in a certain device in the image synthesis system, such as an image synthesis server or an image synthesis terminal device. As shown in Figure 1, the method includes the following steps:

Step S101: Obtain an original image frame from an image source;

In the present disclosure, the original image frame includes a target object, where the target object is a preset object to be detected, such as a portrait of a person, an animal, and the like.

In the present disclosure, the original image frame is the current image frame of the original image obtained from the image source, and the current image frame is the image frame played by the original image at the current moment.

Optionally, in this step, the original image frame is an image frame received from an image sensor. The image sensor refers to various devices that can collect images, and typical image sensors are video cameras, cameras, cameras, and so on. In this embodiment, the image sensor may be a camera on a terminal device, such as a front or rear camera on a smart phone, and the image collected by the camera may be directly displayed on the display screen of the mobile phone.

Optionally, in this step, the original image frame is an image frame that receives a video image from a memory, and the video image is a pre-recorded video and is stored in the memory. Optionally, the storage is a local storage or a network storage. Before this step, the video image sent from the memory is received and displayed on the display device of the device.

In this step, the acquisition of the original image frame is one of the multiple frames in the original image, and specifically refers to the image frame being played at the current moment. The image frames are played at a certain speed to form the original image. Typically, 30 image frames per second are played. When the original image frame starts to be played, the image frame is replaced every 33 milliseconds, such as 0 seconds at the beginning of the playback. The original image frame is the first frame of the entire original image, and when the first second is played, the original image frame is the 31st frame of the entire original image, and so on. It can be understood that the frame rate (the number of frames played per second) of the original image can be any value, and the above examples are merely examples and do not constitute any limitation to the present disclosure.

Step S102, segmenting a target object image from the original image frame;

The target object may be any preset object that needs to be recognized from the original image, and the typical target object may be a portrait.

Any target recognition algorithm can be used to recognize the target object in the original image. Typically, based on deep learning, if the target object is a portrait, the neural network can be trained using images with portraits, and the trained neural network can be used to perform training on the first image frame in the acquired video image. Classification to determine whether the first image frame contains a portrait, when it is determined that the first image frame contains a portrait, a portrait detection algorithm is used to detect the key points of the portrait to determine the outline of the portrait; or it can be used to mark people The neural network is trained on the image of the outer frame, and the trained neural network is used to perform the regression of the outer frame on each image frame in the video image to reduce the scope of the portrait, and then use the portrait detection algorithm to detect the key points of the portrait To determine the contour of the portrait; or use the image marked with the key points of the portrait to train the neural network, and use the trained neural network to perform portrait key point regression on each image frame in the video image to determine the image frame Whether a portrait is included and the outline of the portrait is determined according to the key points.

It is understandable that the above-mentioned portraits and recognition methods are only examples, and do not constitute a limitation to the present disclosure. In fact, the target object and a suitable recognition algorithm for the target object can be pre-selected according to the effect to be achieved and the scene.

After determining that the original image frame includes a target object, the target object is segmented from the original image frame to generate a target object image. While judging whether the target object is included in the original image frame, it will be determined whether the original image frame contains certain features of the target object. Based on these features, the outline of the target object can be determined so as to The target object is segmented from the first image frame.

Optionally, the above identification and segmentation can also be the same step, typically using a trained convolutional neural network to classify each pixel in the first image frame to determine whether it is a pixel in the portrait , When all the pixels are classified, if the first image frame includes a portrait, the image of the portrait has also been segmented.

Step S103, performing blur processing on the target object image to obtain a first image;

Optionally, in the present disclosure, the blurring processing is Gaussian blurring processing. After blurring the target object image, a first image is obtained. The edges of the target object in the first image may be affected by the blurring processing. The cause spreads outward, producing a divergent effect.

Optionally, said performing blur processing on the target object image to obtain the first image includes:

Obtain fuzzy parameters;

Gaussian blurring is performed on the target object image according to the blurring parameter to obtain a first image.

Wherein, the blur parameter determines the degree of Gaussian blur processing. Typically, the blur parameter is a blur radius and/or a weight matrix. The fuzzy parameters can be acquired through a configuration file or through a human-computer interaction interface, which will not be repeated here.

Acquiring the size attribute of the target object;

Control the processing range of the blur processing according to the size attribute to perform the blur processing on the target object image to obtain the first image.

Typically, the target object is a portrait, and the size attribute is the width or height of the circumscribing frame of the face in the face. The control of the processing range of the fuzzy processing according to the size attribute includes the parameter of the processing range according to the following formula:

Scale=a+(b-a)*(facewidth-facewidthmin)/(facewidthmax-facewidthmin)

Where a and b are preset constant values, where a≥0, b≥0, and b>a; facewidthmin and facewidthmax are preset constant values, facewidth is the size attribute of the target object, and facewidth≥facewidthmin .

After obtaining the above range parameter Scale, when performing blur processing, regardless of the processing method, multiply the range parameter in the calculation. Since the range parameter is related to the size of the target object, the blur can be controlled by the size of the target object. The range in turn controls the effect range of the final composite image.

It is understandable that the blur processing in the present disclosure may also be other blur processing, whose purpose is to produce a divergent effect of the target object image. The above-mentioned Gaussian blur is only an example and does not constitute a limitation of the present disclosure.

Step S104: Obtain the first historical image in the cache queue;

In the present disclosure, a buffer queue is preset to store the first image obtained from the historical original image frame. Typically, the original image frame before the current time has been obtained before the original image frame at the current time is obtained, and the same After the processing of step S103, the corresponding first images are obtained. These images become historical first images in the present disclosure. The size of the cache queue determines the number of historical first images. The typical length of the cache queue is 6, then The buffer queue buffers the first images corresponding to the original image frames at 6 moments before the current moment, that is, 6 historical first images.

Optionally, the historical first image is the result of processing the image in the cache queue. In this case, the obtaining the historical first image in the cache queue includes:

Step S201: Obtain multiple original historical first images in the cache queue;

Step S202: Obtain the transparency coefficients of the multiple original historical first images;

Step S203: Multiply the transparency of the multiple original historical first images by the corresponding transparency coefficients to obtain multiple adjusted transparency;

Step S204, calculating the maximum adjustment transparency of the plurality of adjustment transparency;

Step S205: Generate the historical first image according to the maximum adjusted transparency.

Typically, the length of the cache queue is set to 6. In step S201, six original historical first images in the cache queue are acquired. In this optional embodiment, the original historical first image is before the current moment. The first image corresponding to the original image frame. Obtain the transparency coefficients of the multiple original historical first images. Typically, the transparency coefficients are based on the time sequence of the cache. The earlier the original historical first image corresponds to the smaller the transparency coefficient, as in this example, the cache The original historical first images at the 6 buffer positions in the queue are buffer1, buffer2, buffer3, buffer4, buffer5, and buffer6, among which the time of buffer6 is the earliest, and the time of buffer1 is the latest, and the corresponding transparency coefficients are 1, 0.8, 0.6, 0.4, 0.2, 0, the transparency can be adjusted by multiplying the transparency of the original historical first image by the transparency coefficient buffer1.a*1, buffer2.a*0.8, buffer3.a*0.6, buffer4.a*0.4, buffer5 .a*0.2, buffer6.a*0, where buffer1.a represents the transparency of the original historical first image at the first buffer location, and so on. Then calculate the maximum adjustment transparency a.max=max(buffer1.a*1,buffer2.a*0.8,buffer3.a*0.6,buffer4.a*0.4,buffer5.a*0.2,buffer6. a*0), using the maximum adjusted transparency of each pixel on the original historical first image as the transparency of the historical first image to generate the historical first image. Optionally, the multiple original historical first images are mixed to obtain a mixed image of the historical first image, and then the transparency of each pixel in the mixed image is set to the maximum adjusted transparency of the pixel to generate The first image.

Step S105: Put the first image into the buffer queue;

In this step, the first image at the current moment is put into the buffer queue.

Optionally, the putting the first image into the cache queue includes: sequentially moving the historical first images in the cache queue one position forward; putting the first image into the cache The end of the queue. The cache queue includes a plurality of cache locations, and the historical first image is stored in the queue according to the sequence of original image frames (ie, time sequence), and when the multiple cache locations of the cache are all full, The historical first image at the head position in the queue is deleted, the historical first image at the second position in the queue is taken as the head of the queue, and the historical first images at other positions are moved forward one by one, and The first image at the current moment is stored at the end of the queue as the first historical image of the next original image frame.

Step S106, obtaining a noise image frame;

Optionally, the noise image frame is a grayscale image. The noise image frame is from a noise image, and the noise image is a sequence of noise image frames, which can be played simultaneously with the playback of the original image, but its frame rate can be different from the original image. When the original image frame is acquired, Get the image frame of the noise image played at the current moment. The values of the pixels on the noise image are random values. Therefore, when the noise image frame changes, the final composite image will change accordingly, resulting in dynamic effects.

Step S107: Generate a composite image according to the original image frame, the first historical image, and the noise image frame.

In this step, the original image frame, the historical first image, and the noise image are mixed according to a predetermined rule to generate a composite image. Since the historical first image is one or more images generated from the image of the target object before the current moment, when the target object moves, an effect similar to ghost follow will be produced.

Optionally, the generating a composite image according to the original image frame, the historical first image, and the noise image frame includes:

Step S301: Multiply the value of the pixel in the noise image frame by the value of the pixel in the historical first image to obtain a second image;

Step S302: Perform first processing on the second image to obtain a third image;

Step S303: Generate a composite image according to the second image, the third image, and the original image frame.

In step S301, the value of the pixel in the noise image frame is multiplied by the value of the pixel at the position corresponding to the position of the pixel in the historical first image to obtain the value of the pixel in the second image, where The value of the pixel is a binarized value, even if the pixel value is normalized from [0,255] to [0,1]. In an optional embodiment, the multiplying the value of the pixel in the noise image frame by the value of the pixel in the historical first image to obtain the second image includes:

Step S401: Multiply the pixel value of the historical first image by the contrast coefficient and add the image offset value to obtain the historical first adjusted image;

Step S402: Multiply the value of the pixel in the noise image frame by the value of the pixel in the historical first adjusted image to obtain a second image.

The contrast coefficient and the image offset value in step S401 are preset parameters, and the purpose of which is to improve the pixel contrast of the first historical image. In step S402, the value of the pixel in the sound image frame is multiplied by the value of the pixel in the historical first adjusted image obtained after adjustment to obtain a second image. The second image thus obtained has a higher brightness than the second image that has not been processed by the above steps, making the second image more eye-catching. It is understandable that the pixel value in the historical first adjusted image is limited to [0,1]. If the pixel value of the historical first image is multiplied by the contrast coefficient and the image offset value is added to obtain If the pixel value is less than 0 or greater than 1, set its value to 0 or 1.

In the step S402, optionally, the multiplying the value of the pixel in the noise image frame by the value of the pixel in the historical first adjusted image to obtain the second image includes:

Step S501: Obtain the contour image of the historical first adjusted image according to the target object image and the historical first adjusted image;

Step S502: Multiply the value of the pixel in the noise image frame by the value of the pixel in the contour image to obtain a second image.

Optionally, in step S501, the historical first adjusted image is subtracted from the target object image to obtain the contour image of the historical first adjusted image. It is understandable that the contour image only includes the first adjusted historical image. Describe the part of the target object image. Therefore, in step S502, the second image also only includes the result of mixing the noise image frame and the contour image.

In step S302, a first process is further performed on the second image to obtain a third image. In order to achieve different image effects, the first process may be any process. Optionally, the first processing is mixing processing, and performing the first processing on the second image to obtain a third image includes:

Step S601: If the value of the pixel in the second image is greater than the first threshold, then the value of the pixel in the second image, the value of the pixel in the original image, and the first color parameter are mixed to obtain a third image The value of the pixel;

Typically, the first threshold value is 0.8, the first color parameter is color1, where 0<color1<1, the pixel value in the second image is set as flamemask, the pixel value in the original image is origincolor, and the value of the pixel in the original image is origincolor. The pixel value of is flamecolor, then the value of flamecolor is calculated according to the following formula:

flamecolor=origincolor*(1-color1)+flamemask*color1;

Step S602, if the value of the pixel in the second image is greater than the second threshold and less than or equal to the first threshold, then the value of the pixel in the second image, the value of the pixel in the original image, and the second color parameter are mixed , Get the value of the pixel in the third image;

Typically, the second threshold is 0.65, the second color parameter is color2, where 0<color2<1, and the pixel value in the second image is set as flamemask, the pixel value in the original image is origincolor, and the value of the pixel in the original image is origincolor. The pixel value of is flamecolor, then the value of flamecolor is calculated according to the following formula:

flamecolor=origincolor*(1-color2)+flamemask*color2;

Step S603, if the value of the pixel in the second image is greater than the third threshold and less than or equal to the second threshold, then the value of the pixel in the second image, the value of the pixel in the original image, and the third color parameter are mixed , Get the value of the pixel in the third image;

Typically, the third threshold is 0.4, and the third color parameter is color3, where 0<color3<1, and the pixel value in the second image is set as flamemask, the pixel value in the original image is origincolor, and the value of the pixel in the original image is origincolor. The pixel value of is flamecolor, then the value of flamecolor is calculated according to the following formula:

flamecolor=origincolor*(1-color3)+flamemask*color3;

Step S604: If the value of the pixel in the second image is less than or equal to the third threshold, then the value of the pixel in the contour image, the value of the pixel in the original image, and the fourth color parameter are mixed to obtain the third image The value of the pixel.

Typically, the fourth color parameter is color4, where 0<color4<1, set the value of the pixel in the contour image to mattingadjust, the value of the pixel in the original image is origincolor, and the value of the pixel in the third image is flamecolor, then Calculate the value of flamecolor according to the following formula:

flamecolor=origincolor*(1-color4)+mattingadjust)*0.5*color4;

It is understandable that the above-mentioned first color parameter, second color parameter, third color parameter, and fourth color parameter may be the same or different, and there is no limitation here. The above judgment and processing are performed on each pixel in the second image to obtain the third image.

Optionally, after the steps S201 to S205, the step S302 performs first processing on the second image to obtain a third image, which may further include:

Step S701: Perform a first sub-processing on the second image to obtain a fourth image;

Step S702, generating a first transparency coefficient according to the maximum adjusted transparency;

Step S703: Generate a third image according to the fourth image and the transparency coefficient.

In step S701, the first sub-processing can be any processing. Typically, the first sub-processing can be the first processing in step S601-step S604. At this time, the fourth image in step S701 is Step S601-The third image obtained in step 04. In step S702, a first transparency coefficient ttl is generated according to the maximum adjusted transparency obtained in step S204. Typically, the first transparency coefficient ttl=a.max*2, and 0≤ttl≤1. In step S703, a third image is generated according to the fourth image and the first transparency coefficient, and the value of the pixel in the fourth image is set as flamecolor, and the value of the pixel in the third image is flamecolorgrad. The first image can be calculated according to the following formula. The values of the pixels in the three images:

flamecolorgrad=flamecolor*vec3*(1-ttl)+flamecolor*ttl,

Where vec3 is a three-dimensional vector, wherein the value of each bit in the three-dimensional vector is between [0, 1], for example, vec3 can be (1, 0.8, 0.3).

In step 303, optionally, the generating a composite image according to the second image, the third image, and the original image frame includes calculating the composite image according to the following formula:

flamecolorgrad1=origincolor*(1-flamemask)+flamecolorgrad*flamemask

Among them, flamecolorgrad1 represents the value of the pixel in the composite image, origincolor represents the value of the pixel in the original image frame, flamecolorgrad represents the value of the pixel in the third image, and flamemask represents the value of the pixel in the second image.

The present disclosure discloses an image synthesis method, device, electronic equipment, and computer-readable storage medium. The image synthesis method includes: obtaining an original image frame from an image source, wherein the original image frame includes a target object; segmenting the target object image from the original image frame; and blurring the target object image to obtain the first An image; obtain a historical first image in the buffer queue, the historical first image is obtained by processing the original image frame before the original image frame; put the first image in the buffer queue; obtain a noise image Frame, the noise image frame is a grayscale image; a composite image is generated according to the original image frame, the historical first image, and the noise image frame. Through the above method, the technical problem that the video effect cannot be generated simply and quickly in the prior art is solved

In the above, although the steps in the above method embodiments are described in the above order, those skilled in the art should understand that the steps in the embodiments of the present disclosure are not necessarily executed in the above order, and they can also be reversed, parallel, or interleaved. Other steps are executed in other order, and, on the basis of the above steps, those skilled in the art can also add other steps. These obvious modifications or equivalent substitutions should also be included in the scope of protection of the present disclosure, and will not be repeated here. .

FIG. 8 is a schematic structural diagram of an embodiment of an image synthesis device provided by an embodiment of the disclosure. As shown in FIG. 8, the device 800 includes: an original image frame acquisition module 801, a target object image segmentation module 802, a blur processing module 803, and a historical image An image acquisition module 804, a first image buffer module 805, a noise image frame acquisition module 806, and a synthesis module 807. among them,

The original image frame obtaining module 801 is configured to obtain an original image frame from an image source, wherein the original image frame includes a target object;

The target object image segmentation module 802 is configured to segment the target object image from the original image frame;

The blur processing module 803 is configured to perform blur processing on the target object image to obtain a first image;

The historical first image obtaining module 804 is configured to obtain the historical first image in the buffer queue, the historical first image being obtained by processing the original image frame before the original image frame;

The first image buffer module 805 is configured to put the first image into the buffer queue;

The noise image frame acquisition module 806 is configured to acquire a noise image frame, where the noise image frame is a grayscale image;

The synthesis module 807 is configured to generate a synthesized image according to the original image frame, the first historical image, and the noise image frame.

Further, the blur processing module 803 further includes:

The fuzzy parameter acquisition module is used to acquire fuzzy parameters;

The Gaussian blur module is configured to perform Gaussian blur on the target object image according to the blur parameters to obtain a first image.

Further, the first image cache module 805 further includes:

The moving module is used to move the first historical image in the buffer queue one position in turn;

The first buffer module is used to put the first image into the end of the buffer queue.

Further, the synthesis module 807 further includes:

The second image generation module is configured to multiply the value of the pixel in the noise image frame by the value of the pixel in the historical first image to obtain a second image;

The first processing module is configured to perform first processing on the second image to obtain a third image;

The synthesis sub-module is used to generate a synthesized image according to the second image, the third image and the original image frame.

Further, the second image generation module further includes:

The historical first adjusted image generating module is configured to multiply the pixel value of the historical first image by the contrast coefficient and add the image offset value to obtain the historical first adjusted image;

The first generating module of the second image is used to multiply the value of the pixel in the noise image frame by the value of the pixel in the historical first adjusted image to obtain the second image.

Further, the first generating module of the second image further includes:

A contour image generating module, configured to obtain the contour image of the historical first adjustment image according to the target object image and the historical first adjustment image;

The second image generation module is configured to multiply the value of the pixel in the noise image frame and the value of the pixel in the contour image to obtain the second image.

Further, the first processing module further includes:

The first processing module is used to mix the value of the pixel in the second image, the value of the pixel in the original image, and the first color parameter if the value of the pixel in the second image is greater than the first threshold , Get the value of the pixel in the third image;

The first processing second module, if the value of the pixel in the second image is greater than the second threshold and less than or equal to the first threshold, then the value of the pixel in the second image, the value of the pixel in the original image and the second The color parameters are mixed to obtain the value of the pixel in the third image;

The first processing third module, if the value of the pixel in the second image is greater than the third threshold and less than or equal to the second threshold, then the value of the pixel in the second image, the value of the pixel in the original image and the third The color parameters are mixed to obtain the value of the pixel in the third image;

The fourth module of the first processing, if the value of the pixel in the second image is less than or equal to the third threshold, then the value of the pixel in the contour image, the value of the pixel in the original image and the fourth color parameter are mixed to obtain The value of the pixel in the third image.

Further, the historical first image acquisition module 804 further includes:

The original historical first image acquisition module, which is used to acquire multiple original historical first images in the cache queue;

A transparency coefficient obtaining module, configured to obtain the transparency coefficients of the multiple original historical first images;

An adjusted transparency generating module, configured to multiply the transparency of the multiple original historical first images by the corresponding transparency coefficients to obtain multiple adjusted transparency;

A maximum adjustment transparency calculation module, configured to calculate the maximum adjustment transparency among the plurality of adjustment transparency;

The historical first image acquisition sub-module is configured to generate the historical first image according to the maximum adjusted transparency.

Further, the first processing module further includes:

A fourth image generation module, configured to perform a first sub-processing on the second image to obtain a fourth image;

A first transparency coefficient generating module, configured to generate a first transparency coefficient according to the maximum adjusted transparency;

The third image generating module is configured to generate a third image according to the fourth image and the transparency coefficient.

Further, the blur processing module 803 further includes:

A size attribute obtaining module, configured to obtain the size attribute of the target object;

The blur processing sub-module is used to control the processing range of the blur processing according to the size attribute to perform the blur processing on the target object image to obtain the first image.

The device shown in Fig. 8 can execute the methods of the embodiments shown in Figs. For the implementation process and technical effects of this technical solution, please refer to the description in the embodiment shown in FIG. 1 to FIG. 7, which will not be repeated here.

Next, referring to FIG. 9, it shows a schematic structural diagram of an electronic device 900 suitable for implementing the embodiments of the present disclosure. Terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals (e.g. Mobile terminals such as car navigation terminals) and fixed terminals such as digital TVs, desktop computers, etc. The electronic device shown in FIG. 9 is only an example, and should not bring any limitation to the function and scope of use of the embodiments of the present disclosure.

As shown in FIG. 9, the electronic device 900 may include a processing device (such as a central processing unit, a graphics processor, etc.) 901, which may be loaded into a random access device according to a program stored in a read-only memory (ROM) 902 or from a storage device 906. The program in the memory (RAM) 903 executes various appropriate actions and processing. In the RAM 903, various programs and data required for the operation of the electronic device 900 are also stored. The processing device 901, the ROM 902, and the RAM 903 are connected to each other through a bus 904. An input/output (I/O) interface 905 is also connected to the bus 904.

Generally, the following devices can be connected to the I/O interface 905: including input devices 906 such as touch screens, touch pads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; including, for example, liquid crystal displays (LCD), speakers, vibration An output device 907 such as a device; a storage device 906 such as a magnetic tape, a hard disk, etc.; and a communication device 909. The communication device 909 may allow the electronic device 900 to perform wireless or wired communication with other devices to exchange data. Although FIG. 9 shows an electronic device 900 having various devices, it should be understood that it is not required to implement or have all of the illustrated devices. It may alternatively be implemented or provided with more or fewer devices.

In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments of the present disclosure include a computer program product, which includes a computer program carried on a non-transitory computer readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from the network through the communication device 909, or installed from the storage device 906, or installed from the ROM 902. When the computer program is executed by the processing device 901, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

It should be noted that the aforementioned computer-readable medium in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination of any of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable removable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as a part of a carrier wave, and a computer-readable program code is carried therein. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium. The computer-readable signal medium may send, propagate, or transmit the program for use by or in combination with the instruction execution system, apparatus, or device . The program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wire, optical cable, RF (Radio Frequency), etc., or any suitable combination of the above.

In some embodiments, the client and server can communicate with any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and can communicate with digital data in any form or medium. Communication (e.g., communication network) interconnects. Examples of communication networks include local area networks ("LAN"), wide area networks ("WAN"), the Internet (for example, the Internet), and end-to-end networks (for example, ad hoc end-to-end networks), as well as any currently known or future research and development network of.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist alone without being assembled into the electronic device.

The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: obtains an original image frame from an image source, wherein the original image frame includes a target object Segment the target object image from the original image frame; perform blur processing on the target object image to obtain the first image; obtain the historical first image in the buffer queue, the historical first image from the original image frame The previous original image frame is processed; the first image is put into the buffer queue; the noise image frame is obtained, and the noise image frame is a grayscale image; according to the original image frame, the historical first image, and the noise The image frame generates a composite image.

The computer program code used to perform the operations of the present disclosure can be written in one or more programming languages or a combination thereof. The above-mentioned programming languages include but are not limited to object-oriented programming languages such as Java, Smalltalk, C++, and Including conventional procedural programming languages-such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partly on the user's computer, executed as an independent software package, partly on the user's computer and partly executed on a remote computer, or entirely executed on the remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to pass Internet connection).

The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions, and operations of the system, method, and computer program product according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or part of code, and the module, program segment, or part of code contains one or more for realizing the specified logical function Executable instructions. It should also be noted that, in some alternative implementations, the functions marked in the block may also occur in a different order from the order marked in the drawings. For example, two blocks shown in succession can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and/or flowchart, and the combination of the blocks in the block diagram and/or flowchart, can be implemented by a dedicated hardware-based system that performs the specified functions or operations Or it can be realized by a combination of dedicated hardware and computer instructions.

The units involved in the embodiments described in the present disclosure can be implemented in software or hardware. Among them, the name of the unit does not constitute a limitation on the unit itself under certain circumstances.

The functions described hereinabove may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), Application Specific Standard Product (ASSP), System on Chip (SOC), Complex Programmable Logical device (CPLD) and so on.

In the context of the present disclosure, a machine-readable medium may be a tangible medium, which may contain or store a program for use by the instruction execution system, apparatus, or device or in combination with the instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the foregoing.

According to one or more embodiments of the present disclosure, there is provided an image synthesis method, including:

Obtain an original image frame from an image source, wherein the original image frame includes a target object; segment the target object image from the original image frame; perform blur processing on the target object image to obtain the first image; obtain the buffer queue The historical first image, the historical first image is obtained by processing the original image frame before the original image frame; the first image is put into the buffer queue; the noise image frame is acquired, the noise image frame It is a grayscale image; a composite image is generated according to the original image frame, the first historical image, and the noise image frame.

Further, the performing blur processing on the target object image to obtain the first image includes:

Obtain a blur parameter; perform Gaussian blur on the target object image according to the blur parameter to obtain a first image.

Further, the putting the first image into the cache queue includes: sequentially moving the historical first images in the cache queue one position forward; putting the first image into the cache queue The end of the team.

Further, the generating a composite image according to the original image frame, the historical first image, and the noise image frame includes: multiplying the value of the pixel in the noise image frame by the value of the pixel in the historical first image to obtain A second image; the second image is subjected to a first processing to obtain a third image; a composite image is generated according to the second image, the third image, and the original image frame.

Further, the multiplying the value of the pixel in the noise image frame with the value of the pixel in the historical first image to obtain the second image includes: multiplying the value of the pixel in the historical first image by a contrast coefficient And the image offset value is added to obtain the historical first adjusted image; the value of the pixel in the noise image frame is multiplied by the value of the pixel in the historical first adjusted image to obtain the second image.

Further, the multiplying the value of the pixel in the noise image frame by the value of the pixel in the historical first adjusted image to obtain the second image includes: obtaining the historical image according to the target object image and the historical first adjusted image The contour image of the first adjustment image; the second image is obtained by multiplying the value of the pixel in the noise image frame by the value of the pixel in the contour image.

Further, the first processing of the second image to obtain the third image includes: if the value of the pixel in the second image is greater than the first threshold, then the value of the pixel in the second image is compared with the original The value of the pixel in the image is mixed with the first color parameter to obtain the value of the pixel in the third image; if the value of the pixel in the second image is greater than the second threshold and less than or equal to the first threshold, then the second The value of the pixel in the image, the value of the pixel in the original image, and the second color parameter are mixed to obtain the value of the pixel in the third image; if the value of the pixel in the second image is greater than the third threshold and less than or equal to the second Threshold, the pixel value in the second image, the pixel value in the original image, and the third color parameter are mixed to obtain the value of the pixel in the third image; if the value of the pixel in the second image is less than If it is equal to the third threshold, the pixel value in the contour image, the pixel value in the original image, and the fourth color parameter are mixed to obtain the pixel value in the third image.

Further, the obtaining the first historical image in the cache queue includes: obtaining a plurality of original historical first images in the cache queue; obtaining the transparency coefficients of the plurality of original historical first images; The transparency of the historical first image is respectively multiplied by the corresponding transparency coefficient to obtain multiple adjusted transparency; the maximum adjusted transparency of the multiple adjusted transparency is calculated; and the historical first image is generated according to the maximum adjusted transparency.

Further, the performing the first processing on the second image to obtain the third image includes: performing the first sub-processing on the second image to obtain a fourth image; and generating a first transparency coefficient according to the maximum adjusted transparency ; Generate a third image according to the fourth image and the transparency coefficient.

Further, the performing blur processing on the target object image to obtain the first image includes: obtaining the size attribute of the target object; and controlling the processing range of the blur processing according to the size attribute to perform blur processing on the target object image Get the first image.

According to one or more embodiments of the present disclosure, there is provided an image synthesis device, including:

Further, the blur processing module further includes:

The fuzzy parameter acquisition module is used to acquire fuzzy parameters;

Further, the first image cache module further includes:

Further, the synthesis module further includes:

Further, the second image generation module further includes:

Further, the first generating module of the second image further includes:

Further, the first processing module further includes:

Further, the historical first image acquisition module further includes:

Further, the first processing module further includes:

Further, the blur processing module further includes:

According to one or more embodiments of the present disclosure, there is provided an electronic device including: at least one processor; and,

A memory that is communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processing The device can perform any of the aforementioned image synthesis methods.

According to one or more embodiments of the present disclosure, there is provided a non-transitory computer-readable storage medium, characterized in that the non-transitory computer-readable storage medium stores computer instructions, and the computer instructions are used to make a computer execute any of the foregoing The image synthesis method.

The above description is only a preferred embodiment of the present disclosure and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of disclosure involved in this disclosure is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover the above technical features or technical solutions without departing from the above disclosed concept. Other technical solutions formed by arbitrarily combining the equivalent features. For example, the above-mentioned features and the technical features disclosed in the present disclosure (but not limited to) having similar functions are replaced with each other to form a technical solution.

Claims

An image synthesis method, including:

Obtaining an original image frame from an image source, wherein the original image frame includes a target object;

Segmenting the target object image from the original image frame;

Performing blur processing on the target object image to obtain a first image;

Acquiring a historical first image in the buffer queue, where the historical first image is obtained by processing an original image frame before the original image frame;

Putting the first image in the buffer queue;

Acquiring a noise image frame, where the noise image frame is a grayscale image;

A composite image is generated according to the original image frame, the historical first image, and the noise image frame.
8. The image synthesis method according to claim 1, wherein said blurring said target object image to obtain the first image comprises:

Obtain fuzzy parameters;

Gaussian blurring is performed on the target object image according to the blurring parameter to obtain a first image.
5. The image synthesis method according to claim 1, wherein the putting the first image into the buffer queue comprises:

Moving the first historical image in the buffer queue one position in turn;

Put the first image at the end of the buffer queue.
8. The image synthesis method according to claim 1, wherein said generating a synthesized image according to said original image frame, historical first image and noise image frame comprises:

Multiplying the value of the pixel in the noise image frame with the value of the pixel in the historical first image to obtain a second image;

Performing first processing on the second image to obtain a third image;

A composite image is generated according to the second image, the third image, and the original image frame.
5. The image synthesis method according to claim 4, wherein said multiplying the value of the pixel in the noise image frame by the value of the pixel in the historical first image to obtain the second image comprises:

Multiplying the pixel value of the historical first image by the contrast coefficient and adding the image offset value to obtain the historical first adjusted image;

The second image is obtained by multiplying the value of the pixel in the noise image frame by the value of the pixel in the historical first adjusted image.
5. The image synthesis method according to claim 5, wherein said multiplying the value of the pixel in the noise image frame by the value of the pixel in the historical first adjusted image to obtain the second image comprises:

Obtaining the contour image of the historical first adjusted image according to the target object image and the historical first adjusted image;

The second image is obtained by multiplying the value of the pixel in the noise image frame by the value of the pixel in the contour image.
7. The image synthesis method according to claim 6, wherein the first processing of the second image to obtain the third image comprises:

If the value of the pixel in the second image is greater than the first threshold, then the value of the pixel in the second image, the value of the pixel in the original image, and the first color parameter are mixed to obtain the value of the pixel in the third image value;

If the value of the pixel in the second image is greater than the second threshold and less than or equal to the first threshold, then the value of the pixel in the second image, the value of the pixel in the original image, and the second color parameter are mixed to obtain the first The value of the pixels in the three images;

If the value of the pixel in the second image is greater than the third threshold and less than or equal to the second threshold, then the value of the pixel in the second image, the value of the pixel in the original image, and the third color parameter are mixed to obtain the first The value of the pixels in the three images;

If the value of the pixel in the second image is less than or equal to the third threshold, then the value of the pixel in the contour image, the value of the pixel in the original image, and the fourth color parameter are mixed to obtain the value of the pixel in the third image value.
5. The image synthesis method according to claim 4, wherein said obtaining the historical first image in the buffer queue comprises:

Acquire multiple original historical first images in the cache queue;

Acquiring the transparency coefficients of the multiple original historical first images;

Multiplying the transparency of the multiple original historical first images with the corresponding transparency coefficients to obtain multiple adjusted transparency;

The maximum adjustment transparency of the plurality of adjustment transparency is obtained by calculation;

The historical first image is generated according to the maximum adjusted transparency.
8. The image synthesis method according to claim 8, wherein the first processing of the second image to obtain the third image comprises:

Performing a first sub-processing on the second image to obtain a fourth image;

Generating a first transparency coefficient according to the maximum adjusted transparency;

A third image is generated according to the fourth image and the transparency coefficient.
8. The image synthesis method according to claim 1, wherein said blurring said target object image to obtain the first image comprises:

Acquiring the size attribute of the target object;

Control the processing range of the blur processing according to the size attribute to perform the blur processing on the target object image to obtain the first image.
An image synthesis device, including:

An original image frame obtaining module, configured to obtain an original image frame from an image source, wherein the original image frame includes a target object;

A target object image segmentation module, configured to segment the target object image from the original image frame;

The blur processing module is used to perform blur processing on the target object image to obtain the first image;

The historical first image acquisition module is configured to acquire the historical first image in the buffer queue, where the historical first image is obtained by processing the original image frame before the original image frame;

The first image buffer module is configured to put the first image into the buffer queue;

A noise image frame acquisition module, configured to acquire a noise image frame, where the noise image frame is a grayscale image;

The synthesis module is used to generate a synthesized image according to the original image frame, the first historical image, and the noise image frame.
An electronic device including:

Memory for storing computer readable instructions; and

The processor is configured to run the computer-readable instructions, so that the processor implements the image synthesis method according to any one of claims 1-10 when the processor is running.
A non-transitory computer-readable storage medium for storing computer-readable instructions. When the computer-readable instructions are executed by a computer, the computer can execute the image synthesis according to any one of claims 1-10 method.