WO2021254223A1

WO2021254223A1 - Video processing method, apparatus and device, and storage medium

Info

Publication number: WO2021254223A1
Application number: PCT/CN2021/099078
Authority: WO
Inventors: 杨洪刚; 舒娟媚
Original assignee: 中兴通讯股份有限公司
Priority date: 2020-06-19
Filing date: 2021-06-09
Publication date: 2021-12-23
Also published as: CN112118483A

Abstract

Disclosed are a video processing method, apparatus and device, and a storage medium. The video processing method comprises: detecting an object removal instruction with respect to a target video, and determining a target removal object corresponding to the object removal instruction; removing the target removal object from each original video frame among multiple original video frames of the target video, and acquiring multiple video frames to be restored, respectively corresponding to the multiple original video frames; and restoring the removal zone of each video frame to be restored, and obtaining an edited video corresponding to the target video.

Description

Video processing method, device, equipment and storage medium

Technical field

This application relates to the field of image processing technology, for example, to a video processing method, device, device, and storage medium.

Background technique

With the popularity of smart phones, people often use smart phones to shoot videos to record every bit of life. Moreover, before people save or share the video, they like to do some post-processing on the original video to achieve a better display effect. The video editing functions supported by smartphones are relatively basic, such as video editing, filters, area cropping, adding subtitles (or watermarks), adding music, adding transition animations, etc., but cannot support some deep editing functions, such as removing videos Target audience, etc.

Summary of the invention

This application provides a video processing method, device, equipment, and storage medium to implement the function of removing target objects in the video during video editing.

A video processing method is provided, including:

Detect an object cutout instruction for the target video, determine the target cutout object corresponding to the object cutout instruction; place the target cutout object in each original video of the multiple original video frames of the target video Cut out from the frame, obtain a plurality of to-be-restored video frames corresponding to the multiple original video frames; perform restoration processing on the cut-out area in each to-be-restored video frame to obtain the target video corresponding to the target video. Edit the video.

A video processing device is also provided, including:

The target removal object determination module is configured to detect the object removal instruction for the target video, and determine the target removal object corresponding to the object removal instruction; the to-be-restored video frame acquisition module is set to remove the target The object is cut out from each of the multiple original video frames of the target video, and multiple to-be-restored video frames corresponding to the multiple original video frames are obtained; the video frame restoration module is set to Perform restoration processing on the cut-out area in each video frame to be restored to obtain an edited video corresponding to the target video.

An electronic device is also provided, and the electronic device includes:

One or more processors; a memory, configured to store one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors realize the aforementioned video Approach.

A computer storage medium is also provided, on which a computer program is stored, and when the program is executed by a processor, the above-mentioned video processing method is realized.

Description of the drawings

FIG. 1 is a schematic flowchart of a video processing method provided by an embodiment of this application;

FIG. 2 is an example diagram of a cut-out object template mask corresponding to an original video frame provided by an embodiment of the application; FIG.

FIG. 3 is an example diagram of a regular pixel distribution provided by an embodiment of this application;

4 is an example diagram of an irregular pixel distribution provided by an embodiment of the application;

FIG. 5 is a schematic flowchart of another video processing method provided by an embodiment of this application;

FIG. 6 is an example diagram of a to-be-restored video frame provided by an embodiment of this application;

FIG. 7 is an example diagram of another to-be-restored video frame provided by an embodiment of the application;

FIG. 8 is an example diagram of another to-be-restored video frame provided by an embodiment of the application;

FIG. 9 is an example diagram of another to-be-restored video frame provided by an embodiment of the application;

FIG. 10 is a schematic diagram of system modules to which a video processing method according to an embodiment of the application is applicable;

FIG. 11 is a schematic structural diagram of a video processing device provided by an embodiment of this application;

FIG. 12 is a schematic structural diagram of an electronic device provided by an embodiment of this application.

detailed description

In the following description, the use of suffixes such as “module”, “component” or “unit” used to indicate elements is only for the description of the present application, and has no special meaning in itself. Therefore, "module", "part" or "unit" can be used in a mixed manner.

Hereinafter, the embodiments of the present application will be described with reference to the accompanying drawings.

In an exemplary embodiment, FIG. 1 is a schematic flowchart of a video processing method provided by an embodiment of this application. This method is suitable for the case of performing in-depth editing of the captured video in an electronic device (such as a mobile terminal, etc.), which may refer to removing some target objects from the video. The method can be executed by the video processing device provided in the present application, and the video processing device can be implemented by software and/or hardware and integrated in an electronic device, such as a mobile terminal.

As shown in Figure 1, the video processing method provided by this embodiment includes:

S110: Detect an object removal instruction for the target video, and determine a target removal object corresponding to the object removal instruction.

The target video refers to the video to be edited. Optionally, decrypt the target video to obtain all the original video frames corresponding to the target video. The user can select one or more of the original video frames for related editing operations as needed, such as adding animation, Add text or objects, or crop the original video frame, etc.

The object removal instruction refers to an instruction initiated in the edit page of the target video and used to instruct the removal of related objects in the target video. The object removal instruction can be performed by, for example, the user's click action and movement The action trigger can also be triggered by a user's sliding action, etc., which is not limited in this embodiment.

The target removal object corresponding to the object removal instruction is the object selected by the user through a set action and intended to be removed in the target video. For example, it can be a human body in the target video (such as a passerby who enters the video by mistake). ), objects (such as debris in the video), etc.

In an example, when the user initiates an object removal instruction in an original video frame selected by the user, the trigger position of the object removal instruction in the original video frame is obtained, and the object is identified near the trigger position, and the object is recognized The object of is the target removal object corresponding to the object removal instruction.

As an optional implementation manner, S110 includes:

The object cutout instruction for the target video is detected in the current display original video frame of the target video, and the object recognition is performed on the currently displayed original video frame; according to the object recognition result and the position information of the object cutout instruction in the currently displayed original video frame , To determine the target removal object corresponding to the object removal instruction.

In an example, the currently displayed original video frame is an original video frame displayed when an object removal instruction for the target video is detected. For example, it may be based on a pause operation triggered by the user during the playback of the original video frame of the target video. Positioned. If the object removal instruction is detected in the currently displayed original video frame, the object recognition is performed on the currently displayed original video frame. For example, OpenCV's artificial intelligence (AI) object recognition algorithm can be used to perform object recognition on the currently displayed original video frame. Recognize, and obtain one or more objects corresponding to the currently displayed original video frame. Label the recognized objects to obtain the location of the area corresponding to each object, and then save the location of the area of these objects. In an example, when storing the identified object information, the object identifier (ID), the object name, and the location of the object area can be stored correspondingly.

The position information of the object cut-out instruction in the currently displayed original video frame may refer to the position information triggered by the object cut-out instruction in the currently displayed original video frame, and the position information may refer to the object cut-out instruction in the currently displayed original video frame Coordinate location information in. After obtaining the location information corresponding to the object removal instruction, search for the object matching the location information in the stored object information, and the found object is the target removal object corresponding to the object removal instruction.

S120: Cut out the target cutout object in each original video frame of the target video, and obtain a to-be-restored video frame corresponding to each original video frame.

The video frame to be restored refers to the video frame corresponding to the original video frame obtained after the target keying object is cut out in an original video frame. There is a cut-out area in the video frame to be restored, and the cut-out area is the area where the target cut-out object is located in the original video frame. Correspondingly, a video frame that only includes the target keying object may be referred to as a keying object video frame.

Since the shape of the target cutout object is not necessarily the same in each original video frame, it is necessary to perform the cutout operation of the target cutout object with each original video frame as a unit. Optionally, in each original video frame, an area to be cut out matching the target cut object is determined, and the area to be cut out is cut out in the corresponding original video frame to obtain the to-be-restored video frame correspondingly.

If an original video frame does not contain the target cutout object, the to-be-restored video frame corresponding to the original video frame is still the original video frame.

In an optional implementation manner, S120 includes:

Perform edge detection on each original video frame, and determine the cutout object template mask corresponding to each original video frame; according to each cutout object template mask, put the target cutout object in each corresponding original video frame Cut out, and obtain the to-be-restored video frame corresponding to each original video frame.

For each original video frame, edge detection is first performed to obtain a cut-out object edge detection template corresponding to the target cut-out object, and a cut-out object template mask corresponding to the cut-out object edge detection template is generated.

When performing edge detection, you can use edge detection operators such as Sobel operator, Laplacian operator, and Canny operator. These operators have different advantages and disadvantages. For example, Laplacian operators are sensitive to noise, and Sobel operators are not accurate. Too high but better for image processing with gradual gradation and more noise. The Canny operator can detect weak and strong edges, and can also be controlled by setting a threshold. Therefore, an appropriate operator can be selected according to the actual situation of the cut-out target area of the video, which is not limited in this embodiment.

The pixel size of the cut-out object template mask is the same as the pixel size of the original video frame. As shown in Figure 2, in the cut-out object template mask, the pixel value in the area where the target cut-out object is located can be set to 1( It can also be set to 255), and the pixel value of the remaining area can be set to 0.

In an example, when obtaining the cut-out target video frame corresponding to the original video frame, the original video frame and the cut-out target template mask may be bit-wise ANDed to obtain the cut-out target video frame; correspondingly, When obtaining the to-be-restored video frame corresponding to the original video frame, you can first perform the bitwise "inversion" operation on the cutout object template mask, and then combine the original video frame with the cutout object template mask after the "inversion" operation Perform bitwise AND operation to get the video frame to be restored.

During the "AND" operation, if the target pixel in the original video frame is "ANDed" with the "1" in the cutout template mask, the pixel value of the target pixel remains unchanged. If the target pixel in the original video frame is The target pixel is ANDed with the "0" in the template mask of the cutout target, and the pixel value of the target pixel becomes 0. When the cut-out target template mask is "inverted" by bit, the area with the original pixel value of "1" becomes "0", and the area with the original pixel value of "0" becomes "1".

In one example, for the currently displayed original video frame, after object recognition is performed on the currently displayed original video frame and the target cutout object is determined, edge detection is performed on the currently displayed original video frame to obtain the corresponding target cutout object Cut out the object edge detection template, and generate a cut out object template mask corresponding to the cut out object edge detection template for the original video frame currently displayed, and then the target cut out object can be displayed in the current display according to the cut out object template mask Cut out the original video frame, and obtain the to-be-restored video frame corresponding to the currently displayed original video frame.

For any other original video frame except the currently displayed original video frame, first perform edge detection on the other original video frame, and perform edge detection detection results on the edge detection template based on the cutout object edge detection template determined by the currently displayed original video frame Match, the area that is successfully matched is the area to be cut out in the other original video frames (there is no need to perform object recognition on multiple other original video frames one by one to determine the area to be cut out that matches the target cutout object) , And then generate a cut-out object template mask for the other original video frames corresponding to the area to be cut out, and then cut out the target cut-out object in the other original video frames according to the cut-out object template mask to obtain To the to-be-restored video frame corresponding to the other original video frame.

Cut out the target cutout object in each original video frame of the target video, which can be: cut out the target cutout object in the current display original video frame of the target video, and cut out the target cutout object in the target video. Cut out from the original video frames other than the original video currently displayed.

In an optional implementation manner, after the target to be cut out is cut out in the currently displayed original video frame of the target video, the cut out target video frame corresponding to the currently displayed original video frame may also be obtained; in response to The object movement event associated with the object cutout instruction moves the target cutout object in the currently displayed original video frame according to the cutout target video frame corresponding to the currently displayed original video frame and the to-be-restored video frame. In an example, it is also possible to generate an animation that moves the target cutout object out of the currently displayed video frame editing interface according to the object cutout instruction direction.

Cut-out target video frame refers to the video frame that only includes the target cut-out object; the cut-out target video frame corresponding to the currently displayed original video frame includes only the target cut-out target video frame corresponding to the currently displayed original video frame , That is, the edge area of the target to be cut out is determined according to the original video frame currently displayed.

In an example, the object removal instruction may be implemented through an object click event and an object movement event. After the user triggers an object click event (e.g., triggered by a user's finger touch operation), if an object movement event (e.g., triggered by a user's finger movement operation) is detected, the operation of moving the target and removing the object in the currently displayed original video is started. Optionally, after the object movement event is detected, the change of the trigger position of the object movement event is detected in real time, and the trigger position of the object movement event is used as the display position of the target cutout object, and the target cutout object is drawn at the display position (For example, it can be realized by superimposing the cut-out target video frame at the display position of the to-be-restored video frame, etc.), to achieve the effect of moving the target cut-out object in the original video currently displayed.

In an example, it is also possible to generate an animation that moves the target and removes the object in the currently displayed original video according to the operation of moving the target to remove the object in the currently displayed original video.

Optionally, preset a moving position function, P(x,y)=f(t,l), where P(x,y) represents the display position of the target to be cut out in the video frame to be restored, and t represents The time period from the moment when the animation is generated to the current moment, l represents the distance from the original position of the target to be cut out to the set boundary of the video frame. After the object movement event is detected, the timing refresh notification process is started. After the timing refresh event is detected, the display position of the target cutout object in the video frame to be restored is obtained according to the movement position function, and the target cutout is drawn at the display position Objects, until the end of the animation display, the target removed objects are no longer displayed in the to-be-restored video frame.

In another optional implementation manner, the target cutout object can also be moved to a set position in the target video editing interface, for example, a position similar to the recycle bin mark, so as to delete the target cutout object in the currently displayed video frame. .

In yet another optional implementation manner, the target cutout object can also be deleted in the currently displayed video frame through the setting menu operation. For example, the target cutout instruction is a long-press operation, and the relevant key is popped up by long-pressing the target cutout object. Menu, click the delete button to realize the delete function.

S130: Perform restoration processing on the cut-out area in each video frame to be restored to obtain an edited video corresponding to the target video.

The edited video and the target video have the same number of video frames and the sequence of the video frames. The difference between the edited video and the target video is that each video frame does not include the target keying object, and the area where the target keying object is located has been removed Performed a background restoration.

After obtaining multiple to-be-restored video frames, perform restoration processing on the cutout area in each to-be-restored video frame to obtain multiple restored video frames corresponding to the multiple to-be-restored video frames, and combine the multiple restored video frames Arrange and save in order to get the edited video. Among them, performing restoration processing on the cut-out area is to fill the cut-out area with pixels that match the target video background.

In the technical solution provided by the embodiments of the present application, after detecting the object removal instruction for the target video, first determine the target removal object that needs to be removed in the target video, and then place the target removal object in each original video. Cut out in the frame, and restore the cut out area to obtain the edited video corresponding to the target video, thereby realizing the function of removing the target object in the video during video editing.

In an exemplary embodiment, this embodiment is described on the basis of the above-mentioned embodiment. Wherein, performing restoration processing on the cut-out area in each video frame to be restored includes: restoring the cut-out area in each video frame to be restored according to the difference between multiple video frames to be restored.

Since the position and shape of the target cutout object in different original video frames are not necessarily the same, after the target cutout object is cut out in each original video frame, the cutout area in multiple video frames to be restored is not necessarily the same. The location, form, etc. are not necessarily the same. Therefore, it is possible to perform restoration processing on the cut-out areas in the multiple to-be-restored video frames by using the difference in the cut-out areas between the multiple to-be-restored video frames, so as to realize partial or complete deletion of the cut-out areas.

For example, if the cut-out area in the video frame A to be restored is on the left side of the screen, and the cut-out area in the video frame B to be restored is on the right side of the screen, then the video frame B to be restored can be restored according to the area on the right side of the video frame A to be restored. Perform restoration processing on the cut-out area in the video frame B to be restored, and restore the cut-out area in the video frame A to be restored according to the area on the left side of the video frame B to be restored.

The area on the right side of the screen of the to-be-restored video frame A can be divided into the area corresponding to the cut-out area in the to-be-restored video frame B, and the area on the left side of the screen of the to-be-restored video frame B can be combined with the key in the to-be-restored video frame A. Except for the area corresponding to the area, it is called the reducible difference area.

The restorable difference area is used to indicate an area in a to-be-restored video frame that can be restored to another to-be-restored video frame. For example, the restorable difference area is an area in the to-be-restored video frame A that can be restored to the to-be-restored video frame B. For another example, the restorable difference area is an area in the video frame B to be restored that can be restored to the video frame A to be restored.

If there is a restorable difference area between any two video frames to be restored, the corresponding cut-out area can be restored according to the restorable difference area. There is a restoreable difference area between the two to-be-restored video frames. Taking the two to-be-restored video frames as the to-be-restored video frame A and the to-be-restored video frame B as an example, it may be that the to-be-restored video frame A is for the to-be-restored video frame B has a restorable difference area, and then restores the cutout area of the to-be restored video frame B according to the restorable difference area in the to-be-restored video frame A; it can also be that the to-be-restored video frame B exists for the to-be-restored video frame A The difference area can be restored, and then the cutout area of the to-be-restored video frame A can be restored according to the reducible-difference area in the to-be-restored video frame B; it can also be that the to-be-restored video frame A has a restoreable area for the to-be-restored video frame B In the difference area, the video frame B to be restored also has a difference area that can be restored for the video frame A to be restored, and the cutout areas of the video frame A to be restored and the video frame B to be restored can be restored.

As an optional implementation manner, after performing the restoration processing on the cut-out area in each video frame to be restored based on the difference between multiple video frames to be restored, if there is an uncompleted restoration in the video frame to be restored According to the distribution rule of pixels around the cut-out area, the cut-out area is filled with pixels.

The distribution law of pixels around the cutout area can be determined by statistical methods. For example, the distribution law is that the change of pixels is stable, and the distribution law is that the change of pixels is homogeneous, and the pixel changes of the regular pattern are homogeneous. . FIG. 3 is an example diagram of a regular pixel distribution provided by an embodiment of the application, and FIG. 4 is an example diagram of an irregular pixel distribution provided by an embodiment of the application.

In one example, when pixel-filling the cut-out area, the local growth method can be used, that is, while maintaining consistency with the surrounding pixels, a texture is grown on a pixel or pixel area; in one example, the cut-out area When performing pixel filling, the paradigm method can also be used, that is, the known block closest to the missing block is searched and copied to the missing area.

In an example, first, according to the difference between multiple video frames to be restored, the cut-out area in each video frame to be restored is restored; Remove the area, continue to fill the removed area according to the distribution of pixels around the removed area; again, continue to restore the removed area in each to-be-restored video frame based on the difference between multiple to-be-restored video frames deal with. If there are still uncompleted cutout areas in the to-be-restored video frames, repeat the above operation until the cut-out areas in each to-be-restored video frame have been completely restored.

As shown in FIG. 5, the video processing method provided by this embodiment includes:

S210. Detect an object removal instruction for the target video, and determine a target removal object corresponding to the object removal instruction.

S220: Perform edge detection on each original video frame, and determine a cut-out object template mask corresponding to each original video frame.

S230: Cut out the target cutout object in the original video frame according to each cutout object template mask, and obtain a to-be-restored video frame corresponding to each original video frame.

S240: Perform restoration processing on the cut-out area in each video frame to be restored according to the difference between the multiple video frames to be restored.

As an optional implementation manner, according to the difference between a plurality of to-be-restored video frames, performing restoration processing on the cut-out area in each to-be-restored video frame includes:

Compare the two adjacent video frames to be restored in the forward and/or reverse order; in the case that the first to-be-restored video frame has a restorable difference area for the second to-be-restored video frame, compare the second to be restored Restore the cut-out area in the restored video frame for restoration processing, and update the second to-be-restored video frame. Among them, the first to-be-restored video frame and the second to-be-restored video frame are the two adjacent to-be-restored video frames. The restored video frame can also be the next video frame to be restored.

For example, the two adjacent video frames to be restored are compared in a positive order. In the case that the first to-be restored video frame has a restorable difference area for the second to-be restored video frame, the second to be restored is compared according to the restorable difference area. The cut-out area in the restored video frame is restored, and the second to-be-restored video frame is updated; for example, it can also compare two adjacent to-be-restored video frames in reverse order, and the first to-be-restored video frame is for the second to be restored In the case that the video frame has a restorable difference area, the cut-out area in the second to-be-restored video frame is restored according to the restorable-difference area, and the second to-be-restored video frame is updated.

For example, it is also possible to first compare the two adjacent video frames to be restored in a positive order, and then compare the two adjacent video frames to be restored in reverse order. When there is a restoreable difference area in the video frame to be restored, restore the cutout area in the second to-be-restored video frame according to the restoreable difference area, and update the second to-be-restored video frame; for example, it can be reversed in order first Two adjacent video frames to be restored before and after, and then two adjacent video frames to be restored are compared in a positive order. During the comparison process, when the first to be restored video frame has a restoreable difference area for the second to be restored video frame At this time, the cut-out area in the second to-be-restored video frame is restored according to the restorable difference area, and the second to-be-restored video frame is updated.

After performing the restoration process on the cut-out area in each video frame to be restored based on the difference between the multiple video frames to be restored, it also includes:

In the case that there is a difference area between any two video frames to be restored that can be restored, continue to perform restoration processing on the cutout area in each video frame to be restored based on the difference between multiple video frames to be restored Until there is no reversible difference area between any two video frames to be reverted.

That is, in order to compare two adjacent video frames to be restored in the forward and/or reverse order, and in the case that the first to-be-restored video frame has a restorable difference area for the second to-be-restored video frame, according to the restorable difference The area restores the cut-out area in the second to-be-restored video frame, and after updating the second to-be-restored video frame, it also includes:

In the case that there is a difference area between any two video frames to be restored to be restored, continue to perform "Sequentially compare the two adjacent video frames to be restored in the forward and/or reverse order; in the first video frame to be restored, In the case that the second to-be-restored video frame has a restorable difference area, restore the cutout area in the second to-be-restored video frame according to the restorable difference area, and update the second to-be-restored video frame" operation, and so on , Until there is no restorable difference area between any two to-be-restored video frames that are compared, that is, until it is determined that any of the to-be-restored video frames cannot be restored.

Optionally, after performing "Sequentially compare two adjacent video frames to be restored in the forward and/or reverse order; in the case that the first to-be-restored video frame has a restorable difference area with respect to the second to-be-restored video frame, according to Recoverable difference area: After the operation of "recovering the cut-out area in the second to-be-reverted video frame, and update the second to-be-reverted video frame", determine whether there is no recoverable between the two to-be-reverted video frames to be compared. Difference area, if there is a reversible difference area between the two to-be-restored video frames to be compared, the “sequential positive and/or reverse-order comparison of two adjacent to-be-restored video frames is performed sequentially; In the case that the second to-be-restored video frame has a restorable difference area, restore the cutout area in the second to-be-restored video frame according to the restorable difference area, and update the second to-be-restored video frame" operation, if you compare If there is no reversible difference area between the two to-be-restored video frames, the loop process is exited.

Take the first comparison in positive order, and then the comparison in reverse order of two adjacent video frames to be restored as an example to illustrate:

Compare the two to-be-restored video frames before and after in positive order; in the case that the first to-be-restored video frame has a restorable difference area for the second to-be-restored video frame, compare the second to-be-restored video frame according to the restorable difference area Perform restoration processing on the cut-out area of, and update the second to-be-restored video frame; in the case that the second to-be-restored video frame has a restorable difference area for the first to-be-restored video frame, the first to be restored difference area is compared to the first The cut-out area in the video frame to be restored is restored, and the first video frame to be restored is updated; the two video frames to be restored before and after are compared in reverse order; the second video frame to be restored exists for the first to be restored video frame that can be restored In the case of a difference area, restore the cutout area in the first to-be-restored video frame according to the reversible difference area, and update the first to-be-restored video frame; the first to-be-restored video frame is for the second to be restored In the case that the video frame has a restorable difference area, restore the cut-out area in the second to-be-restored video frame according to the restorable difference area, and update the second to-be-restored video frame; in any two comparisons In the case where there is the reducible difference area between the video frames to be restored, the above process is repeated until the reducible difference area does not exist between any two of the compared video frames to be restored.

First, the first two video frames to be restored of the target video are obtained in a positive sequence, that is, the first video frame to be restored and the second video frame to be restored, and the difference between the two to be restored video frames is compared. If the first to-be-restored video frame has a restorable difference area for the second to-be-restored video frame, restore the cutout area of the second to-be-restored video frame according to the restorable difference area, and update the second The video frame to be restored; if the second video frame to be restored has a restorable difference area for the first video frame to be restored, the cutout area of the first video frame to be restored is restored according to the restorable difference area, And update the first video frame to be restored. After processing the first video frame to be restored and the second video frame to be restored, continue to obtain the next video frame to be restored for comparison, that is, compare the second video frame to be restored and the third video frame to be restored Whether there is a reversible difference area between frames. If there is no restorable difference area between the two video frames to be restored, the next video frame to be restored is directly obtained in the positive sequence for comparison, until the last video frame to be restored is processed in the positive sequence (that is, the Nth video to be restored) Frame, assuming that the target video has N video frames).

Then, obtain the last two video frames to be restored of the target video in reverse order, that is, the Nth video frame to be restored and the N-1th video frame to be restored, and compare whether there is a restoreable difference area between the two to be restored video frames . If the Nth video frame to be restored has a restorable difference area for the N-1th video frame to be restored, the cutout area of the N-1th video frame to be restored is restored according to the restorable difference area, and Update the N-1th to-be-restored video frame; if the N-1th to-be-restored video frame has a restorable difference area for the Nth to-be-restored video frame, then the Nth to-be-restored video frame is based on the restorable difference area Perform restoration processing on the cut-out area of, and update the Nth to-be-restored video frame. After the Nth video frame to be restored and the N-1th video frame to be restored are processed, the next video frame to be restored is obtained in reverse order for comparison, that is, the N-1th video frame to be restored is compared with the N-th video frame to be restored. Whether there is a reversible difference area between the 2 video frames to be reverted. If there is no restorable difference area between the two video frames to be restored, the next video frame to be restored is directly obtained in reverse order for comparison, until the last video frame to be restored is processed in reverse order (that is, the first video frame to be restored) .

The above process is executed in a loop until it is determined that the restoration processing cannot be performed on any one of the to-be-restored video frames, that is, there is no restorable difference area between the two to-be-restored video frames to be compared.

In an example, it is possible to record the number of reduction processes in one cycle process (including the positive sequence and the reverse sequence), and to clear the number of reduction processes before the start of the next cycle process. When one cycle of processing ends, if the number of reduction processing times is zero, the cycle is exited, and if the number of reduction processing times is greater than zero, the next cycle processing is entered.

In an example, when comparing whether there is a reducible difference area between two to-be-restored video frames, it may first be calculated whether there is an overlap area between the two to-be-restored video frames.

If there is no overlapping area between the two, it is determined that there is no reducible difference area between the two. Among them, the overlap area refers to the same area between the two to-be-restored video frames. Referring to the to-be-restored video frames shown in FIG. 6 and FIG. 7, the area marked by the box 60 on the left in FIG. The area identified by the box 70 is the overlap area between the to-be-restored video frames shown in FIG. 6 and FIG. 7. Since the video will move during shooting, the overlapped area can be used as a reference for restoration when restoring the video.

If there is an overlap area between the two, refer to the overlap area to determine whether there is a reducible difference area between the two. Referring to the to-be-restored video frames shown in Figures 8 and 9, the upper area in Figures 8 and 9 is the overlap area, and the lower area in Figures 8 and 9 can be used to cut out the area to restore, and the lower side in Figure 9 is used The area identified by the box 90 is the reducible difference area for the to-be-restored video frame shown in FIG. 8, which can be restored to the lower area in FIG. 8. In the case of overlapping areas between the two, if there is no cutout area in a to-be-restored video frame, you can directly check the area in the to-be-restored video frame that corresponds to the cut-out area of the other to-be-restored video frame. The cut-out area of another video frame to be restored is restored.

S250. In the case that there is a cutout area in the to-be-restored video frame that has not been restored, fill the cutout area with pixels according to the distribution law of pixels around the cut-out area to obtain a restored area corresponding to each to-be-restored video frame Video frame.

Optionally, after filling the cutout area according to the distribution of pixels around the cutout area, you can also perform image smoothness processing on the original cutout area edge in each restored video frame to improve the post-processing Video quality.

S260. Store multiple restored video frames sequentially to obtain an edited video corresponding to the target video.

In another exemplary embodiment, performing restoration processing on the cut-out area in each to-be-restored video frame includes: dividing each to-be-restored video frame into frame blocks to obtain the respective corresponding to the multiple to-be-restored video frames According to the distribution of the multiple target frame blocks in multiple consecutive video frames to be restored, the cutout area in each video frame to be restored is restored.

Each video frame to be restored is divided into frame blocks, and the target frame block corresponding to each video frame to be restored can be obtained. The distribution of the video frames to be restored can be used to obtain the motion trajectories of multiple target frame blocks in the video frame. Furthermore, when the cut-out area is restored, a target frame block corresponding to the cut-out area can be determined according to the motion trajectory of multiple target frame blocks in the video frame, and the cut-out area can be cut out according to the target frame block. The area is restored.

Optionally, a block index is established for each target frame block and the movement track is recorded. Among them, the block position information of each target frame block in each video frame is recorded, and the block position information of this target frame block is saved, so that when the cut-out area is restored, the block position information can be retrieved according to the block position information. Match the missing target frame block to realize the restoration processing of the cut-out area.

As an optional implementation manner, after performing restoration processing on the cut-out area in each to-be-restored video frame according to the distribution of multiple target frame blocks in the plurality of consecutive to-be-restored video frames, When there is a cutout area that has not been restored in the video frame to be restored, the cutout area is filled with pixels according to the distribution law of pixels around the cutout area.

In the above technical solution, multiple background restorations are performed on each video frame to be restored, which improves the accuracy of the restoration of the video frame and also guarantees the restoration effect of the video frame.

In an exemplary embodiment, the operating system of the electronic device is the android system, the object recognition algorithm is the AI object recognition algorithm of OpenCV, the target video is a video of people moving in a fixed scene (such as a park, a playground, etc.), and the target is removed. Is a character object, and the obtained edited video is a fixed scene.

Optionally, as shown in FIG. 10, the video editing method provided by this embodiment can be packaged into the following four functional modules: an object recognition function module, an object edge detection function module, a cut-out area restoration function module, and an object removal animation generation functional module.

In an example, the object recognition function module is mainly set to complete the positioning function of the currently displayed video frame and the object recognition function for the currently displayed video frame.

The positioning function of the currently displayed video frame needs to be combined with the user's operation. In the video editing interface, when the user finds that there is an object to be removed in the target video interface, the user executes the operation of pausing the video playback. When the target video is monitored to be paused, the currently displayed video frame is located and the current displayed video can be defined The frame is Current_Frame. Optionally, save the currently displayed video frame as bitmap (Bitmap) type data and pass it to the object recognition function interface to realize the object recognition of the current video display frame Current_Frame, and obtain the set of objects contained in the current video display frame Current_Frame.

Optionally, when implementing the object recognition function of the currently displayed video frame, the AI algorithm through the object recognition function interface recognizes the objects in the Bitmap according to the received Bitmap data, and labels multiple objects to determine multiple objects. For the object area, an object data can be defined as Identify_Object_Rect, and the Identify_Object_Rect can include, for example, the object ID, the object name, and the location of the object area. The multiple identified object information constitutes the object recognition result, which can be defined as Identify_Objects, that is, Identify_Objects is a collection containing one or more Identify_Object_Rect.

In an example, the processing flow of the object recognition function module may include:

S11. Start video browsing and playback, and detect an external positioning event for the target video. Among them, the external positioning event is an event triggered by the user and used to select a video frame.

S12. When an external positioning event is received, the playback is paused, and the current display video frame Current_Frame is determined.

S13. Save the currently displayed video frame Current_Frame as Bitmap data.

S14. Pass the Bitmap data to the object recognition function interface, and perform object recognition through the AI algorithm.

S15. Obtain an object recognition result corresponding to the currently displayed video frame, including the object name and the position of the object area.

S16. Save the object recognition result.

In an example, the object edge detection function module is mainly set to complete the determination of the target to be removed and the separation of the video frame to be restored and the video frame of the removed object.

Optionally, the user selects the target to be removed in the currently displayed video frame. Optionally, detects the coordinate information triggered by the user in the currently displayed video frame Current_Frame, such as detecting the coordinates of the user's finger touching the screen (Mx, My), in Identify_Objects, determine the target editing object Edit_Object corresponding to (Mx, My).

Optionally, after determining the target cutout object, edge detection is performed on the current display video frame Current_Frame using an edge detection algorithm to obtain the cutout object edge detection template Edit_Template corresponding to the target cutout object Edit_Object. Separate the currently displayed video frame Current_Frame, and separate a Bitmap (which can be defined as Edit_Crop_Object) that only contains the target cutout object Edit_Object, and a Bitmap (which can be defined as Edit_Save_Object) that does not contain the target cutout object Edit_Object.

Locate other video frames except the currently displayed video frame Current_Frame, perform edge detection on each other video frame, and use the cut-out object edge detection template Edit_Template to match each edge detection result. The area that is successfully matched is the corresponding The area of the object that needs to be cut out in other video frames. Separate each other corresponding video frame. The separation method is the same as the Current_Frame method. Correspondingly, a Bitmap (which can be defined as Edit_Crop_Object) containing only the target cut object Edit_Object and a Bitmap (which does not contain the target cut object Edit_Object) Can be defined as Edit_Save_Object).

Optionally, define a matching cutout object template mask Template_Mask for each video frame (including Current_Frame and other video frames), as shown in Figure 2, in the cutout template mask Template_Mask, the target cutout object The pixel value in the area can be set to 1 (or 255), and the pixel value of the remaining area can be set to 0.

Correspondingly, cut out the target video frame CropRect=Template_Mask&Edit_Frame; the to-be-restored video frame SaveRect=! Template_Mask&Edit_Frame; where "&" represents the data of the corresponding position to be multiplied, the value of the target pixel in Edit_Frame, if it is multiplied by the "0" of Template_Mask, it becomes 0, and if it is multiplied by the "1" of Template_Mask, then The value remains unchanged; "!" represents the inverse operation of Template_Mask, the pixel value in the area where the target cutout object is located becomes 0, and the pixel value of the remaining areas becomes 1.

In an example, the cut-out area restoration function module mainly completes the restoration function of the cut-out area in the video frame to be restored. Among them, the restoration function can be realized by inter-frame restoration or frame block restoration, and can also be realized by inter-frame restoration and speculative restoration, or by frame block restoration and speculative restoration. Among them, restoring between frames refers to restoring the cutout area in each video frame to be restored based on the difference between multiple video frames to be restored. Since the target cutout object may move in different video frames, the background behind the target cutout object can be revealed, and other video frames can be restored accordingly. Refer to Figure 8 and Figure 9, the bottom of Figure 9 The area marked by the box 90 on the side can be restored to the lower area in FIG. 8.

In an example, the processing flow of restoring between frames may include:

S21: Obtain the first two to-be-restored video frames SaveRect of the target video in a positive sequence for comparison, that is, compare Edit_Save_Object.

S22. If there is a restoreable difference area between the two to-be-restored video frames, perform corresponding restoration processing on the to-be-restored video frame according to the restoreable difference area, and update the restore times edit_count, and execute S23; if there is a difference between the two to-be-restored video frames If there is no reversible difference area, S23 is executed directly.

Optionally, the initial value of the number of restores edit_count is zero.

S23. Obtain the next video frame SaveRect to be restored of the target video, compare it with the previous video frame SaveRect to be restored, and execute S22 until the last video frame to be restored of the target video is processed.

S24. Obtain the first two to-be-restored video frames of the target video in reverse order, SaveRect, and perform S22, and when the processing of the last to-be-restored video frame of the target video is completed, perform S25.

S25: Determine whether edit_count is greater than 0, if edit_count is greater than 0, initialize edit_count, and return to execute S21, and if edit_count is equal to 0, exit the inter-frame restoration processing flow.

When comparing whether there is a reducible difference area between two video frames to be restored, it can first calculate whether there is an overlapping area between the two video frames to be restored. Referring to the to-be-restored video frames shown in FIGS. 6 and 7, the area marked by the box 60 on the left side of FIG. 6 and the area marked by the box 70 on the right side of FIG. The overlap area between video frames. If there is no overlapping area between the two, then it is determined that there is no reducible difference area between the two; if there is an overlapping area between the two, then refer to the overlapping area to determine whether there is a reducible difference area between the two. Referring to the to-be-restored video frames shown in Figures 8 and 9, the upper area in Figures 8 and 9 is the overlap area, and the lower area in Figures 8 and 9 can be used to cut out the area to restore, and the lower side in Figure 9 is used The area identified by the box 90 is the reducible difference area for the to-be-restored video frame shown in FIG. 8, which can be restored to the lower area in FIG. 8. In the case of overlapping areas between the two, if there is no cutout area in a to-be-restored video frame, you can directly check the area in the to-be-restored video frame that corresponds to the cut-out area of the other to-be-restored video frame. The cut-out area of another video frame to be restored is restored.

Frame block restoration refers to dividing each video frame to be restored into frame blocks to obtain multiple target frame blocks corresponding to the multiple video frames to be restored; according to the respective corresponding to the multiple video frames to be restored For the matching between multiple target frame blocks, the cutout area in each video frame to be restored is restored.

Each video frame to be restored is divided into frame blocks, and the target frame block RectBlock corresponding to each video frame to be restored can be obtained. Optionally, a block index BlockIndex is established for each target frame block and the movement track is recorded . Among them, the block position information BlockData of each target frame block RectBlock in each video frame is recorded, and the block position information BlockData of this target frame block is saved, so that when the cutout area is restored, the block data can be The location information BlockData is used to match the missing target frame block RectBlock to realize the restoration processing of the cutout area.

Optionally, after performing inter-frame restoration or frame block restoration processing on multiple to-be-restored video frames, if there is a cutout area in the to-be-restored video frame that has not been restored, continue speculative restoration of the to-be-restored video frame .

Speculative restoration refers to the pixel filling of the cut-out area according to the distribution law of the pixels around the cut-out area. Among them, the distribution law of pixels around the cutout area can be determined by statistical methods. For example, the distribution law is that the change of pixels is stable, and the distribution law is that the change of pixels is homogeneous. Among them, the change of pixels in a regular pattern is uniform. Times. Fig. 3 shows a regular pixel distribution, and Fig. 4 shows an irregular pixel distribution.

Optionally, a local growth method or a paradigm method is used to achieve speculative reduction. The local growth method is to grow a texture on a pixel or pixel area while maintaining consistency with the surrounding pixels; the paradigm method is to search for the known block closest to the missing block and copy it to the missing area.

In an example, the processing flow of speculative restoration may include:

S31. Obtain a video frame to be restored in sequence, and determine its cut-out area Lack_Rect.

S32. Analyze the pixel distribution law around the Lack_Rect of the cut-out area.

S33: Determine a corresponding repair method according to the pixel distribution law, for example, a local growth method or a paradigm method.

S34: Perform restoration processing on the cut-out area of the to-be-restored video frame according to the determined repair method.

S35. Return to S31 until the speculative restoration processing of other video frames to be restored is completed.

S36: Perform inter-frame restoration processing on the video frames to be restored.

S37. Detect whether there is a cutout area in the multiple to-be-restored video frames, if there is a cut-out area in the multiple to-be-restored video frames, execute S31, if there is no cutout area in the multiple to-be-restored video frames, exit Restore process.

In an example, the object-moving-out animation generating function module mainly completes the function of generating an animation of the target-cut-out object moving out of the video editing area. Wherein, the moving speed of the target editing object Edit_Object can be calculated according to the position of the target cutting object in the currently displayed video frame and the animation duration, and the position of the target editing object Edit_Object can also be determined by defining the speed change curve.

Optionally, a moving position function, P(x,y)=f(t,l), where P(x,y) represents the display position of the target to be removed in the video frame to be restored, and t represents the start of generation The time period from the animation moment to the current moment, l represents the distance from the original position of the target to be cut out to the set boundary of the video frame.

In one example, after the user's finger touches the screen, the animation starts with the detection of the finger movement event ACTION_MOVE as the starting point. The processing flow includes:

S41. The ACTION_MOVE event of the finger movement event is detected, and the animation is started.

S42. Start the regular refresh notification process.

S43. Detect a timing refresh event, and obtain the display position of the target cutout object in the to-be-restored video frame according to the moving position function P(x,y)=f(t,l).

S44. Draw the target cutout object at P(x, y).

S45: Determine whether the animation is over, if the animation is not over, execute S43, if the animation is over, exit the animation generating process.

In an exemplary embodiment, FIG. 11 is a schematic structural diagram of a video processing apparatus provided by an embodiment of the application. The apparatus may be implemented by software and/or hardware and integrated into an electronic device, such as a mobile terminal. As shown in FIG. 11, the video processing device includes: a target removal object determination module 410, a to-be-restored video frame acquisition module 420, and a video frame restoration module 430. Wherein, the target removal object determination module 410 is configured to detect the object removal instruction for the target video, and determine the target removal object corresponding to the object removal instruction; the to-be-restored video frame acquisition module 420 is configured to The target cutout object is cut out in each original video frame of the target video, and a video frame to be restored corresponding to each original video frame is obtained; the video frame restoration module 430 is configured to The cut-out area in the video frame is restored to obtain an edited video corresponding to the target video.

In the technical solution provided by the embodiments of the present application, after detecting the object cutout instruction for the target video, first determine the target cutout object that needs to be removed from the target video, and then place the target cutout object in each original video. Cut out in the frame, and restore the cut out area to obtain the edited video corresponding to the target video, thereby realizing the function of removing the target object in the video during video editing.

In one example, the to-be-restored video frame acquisition module 420 is configured to perform edge detection on each original video frame, and determine the cutout object template mask corresponding to each original video frame; An object template mask, which cuts out the target cutout object in each corresponding original video frame, and obtains a to-be-restored video frame corresponding to each original video frame.

In an example, the video frame restoration module 430 is configured to perform restoration processing on the cutout area in each to-be-restored video frame according to the difference between a plurality of to-be-restored video frames.

In one embodiment, the video frame restoring module 430 is configured to compare two adjacent video frames to be restored in positive and/or reverse order; in the first to-be-restored video frame, there is a restorable difference area for the second to-be-restored video frame In the case of, performing restoration processing on the cut-out area in the second to-be-restored video frame according to the restorable difference area, and update the second to-be-restored video frame.

The video frame restoration module 430 is further configured to perform restoration processing on the cutout area in each to-be-restored video frame according to the difference between a plurality of to-be-restored video frames, and then compare any two to-be-restored video frames. In the case that there is the restorable difference area between, continue to perform the operation of restoring the cutout area in each to be restored video frame according to the difference between the multiple to be restored video frames, until any of the comparisons There is no restorable difference area between the two video frames to be restored.

In another example, the video frame restoration module 430 is configured to divide each video frame to be restored into frame blocks to obtain multiple target frame blocks corresponding to the multiple video frames to be restored; The distribution of the blocks in a plurality of consecutive video frames to be restored is restored, and the cut-out area in each video frame to be restored is restored.

In another example, the video frame restoration module 430 is further configured to perform restoration processing on the cutout area in each to-be-restored video frame according to the difference between the different to-be-restored video frames, or, According to the distribution of multiple target frame blocks in a plurality of consecutive video frames to be restored, after performing restoration processing on the cutout area in each video frame to be restored, there are uncompleted video frames in the to-be restored video frames In the case of the restored cut-out area, pixel filling is performed on the cut-out area according to the distribution law of pixels around the cut-out area.

In an optional implementation manner, the to-be-restored video frame acquisition module 420 is configured to cut out the target cutout object in the current display original video frame of the target video; correspondingly, the above-mentioned device further includes: animation generation Module, configured to obtain the target video frame corresponding to the current display original video frame after the target cut-out object is cut out in the current display original video frame of the target video; In the object movement event associated with the object cutout instruction, the target cutout object is moved in the currently displayed original video frame according to the cutout target video frame corresponding to the currently displayed original video frame and the to-be-restored video frame.

In one embodiment, the target removal object determination module 410 is configured to detect an object removal instruction for the target video in the currently displayed original video frame of the target video, and perform object recognition on the currently displayed original video frame ; According to the object recognition result and the position information corresponding to the object cutout instruction, determine the target cutout object in the currently displayed video frame with the object cutout instruction.

The above-mentioned video processing device can execute the video processing method provided by the embodiment of the present application, and has the functional modules and effects corresponding to the execution method. For technical details that are not described in detail in this embodiment, please refer to the video processing method provided in the embodiment of this application.

In an exemplary embodiment, an embodiment of the present application further provides an electronic device. FIG. 12 is a schematic structural diagram of an electronic device provided by an embodiment of the application. As shown in FIG. 12, the electronic device provided by the present application, It includes: one or more processors 510 and a memory 520; the processor 510 of the electronic device may be one or more, and one processor 510 is taken as an example in FIG. 12; the memory 520 is configured to store one or more programs; The one or more programs are executed by the one or more processors 510, so that the one or more processors 510 implement the video processing method described in the embodiments of the present application.

The processor 510 and the memory 520 in the electronic device may be connected through a bus or in other ways. In FIG. 12, the connection through a bus is taken as an example.

The memory 520, as a computer-readable storage medium, can be configured to store software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the video processing method described in the embodiments of the present application (for example, in the video processing device). The target removal object determination module 410, the to-be-restored video frame acquisition module 420, and the video frame restoration module 430). The memory 520 may include a program storage area and a data storage area, where the program storage area may store an operating system and an application program required by at least one function; the data storage area may store data created according to the use of the device, and the like. In addition, the memory 520 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some examples, the memory 520 may include a memory remotely provided with respect to the processor 510, and these remote memories may be connected to a communication node through a network. Examples of the aforementioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof.

The embodiments of the present application also provide a storage medium, the storage medium stores a computer program, and when the computer program is executed by a processor, the video processing method in the embodiments of the present application is implemented, and the video processing method includes:

Detect the object cutout instruction for the target video, determine the target cutout object corresponding to the object cutout instruction; cut out the target cutout object in each original video frame of the target video, and obtain A to-be-restored video frame corresponding to each of the original video frames; performing restoration processing on the cut-out area in each to-be-restored video frame to obtain an edited video corresponding to the target video.

Optionally, when the computer executable instruction is executed by a computer processor, it can also be used to implement any of the video processing methods described in the embodiments of the present application.

Through the above description of the implementation manners, this application can be implemented by software and necessary general-purpose hardware, or can be implemented by hardware. The technical solution of this application can essentially be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as a computer floppy disk, a read-only memory (Read-Only Memory, ROM), and random access Random Access Memory (RAM), flash memory (FLASH), hard disk or optical disk, etc., including multiple instructions to make a communication device (which can be a personal computer, server, or network device, etc.) execute the embodiments described in this application Methods.

In the above embodiment of the video processing device, the multiple units and modules included are only divided according to the functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be realized; in addition, the multiple functional units The names are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application.

Generally speaking, the various embodiments of the present application can be implemented in hardware or dedicated circuits, software, logic or any combination thereof. For example, some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device, although the application is not limited thereto.

The embodiments of the present application may be implemented by executing computer program instructions by a data processor of a mobile device, for example, in a processor entity, or by hardware, or by a combination of software and hardware. Computer program instructions can be assembly instructions, instruction set architecture (Instruction Set Architecture, ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or written in any combination of one or more programming languages Source code or object code.

The block diagram of any logic flow in the drawings of the present application may represent program steps, or may represent interconnected logic circuits, modules, and functions, or may represent a combination of program steps and logic circuits, modules, and functions. The computer program can be stored on the memory. The memory can be of any type suitable for the local technical environment and can be implemented using any suitable data storage technology, such as but not limited to read-only memory (ROM), random access memory (RAM), optical storage devices and systems (digital multi-function discs) (Digital Video Disc, DVD) or Compact Disk (CD)), etc. Computer-readable media may include non-transitory storage media. The data processor can be any type suitable for the local technical environment, such as but not limited to general-purpose computers, special-purpose computers, microprocessors, digital signal processors (Digital Signal Processing, DSP), application specific integrated circuits (ASICs) ), programmable logic devices (Field-Programmable Gate Array, FPGA), and processors based on multi-core processor architecture.

Claims

A video processing method, including:

Detecting an object removal instruction for the target video, and determining a target removal object corresponding to the object removal instruction;

Cut out the target cutout object in each original video frame of the multiple original video frames of the target video, and obtain multiple to-be-restored video frames corresponding to the multiple original video frames respectively;

Perform restoration processing on the cut-out area in each video frame to be restored to obtain an edited video corresponding to the target video.
2. The method according to claim 1, wherein said removing the target object in each original video frame of a plurality of original video frames of the target video, and obtaining the same as the original Multiple video frames to be restored corresponding to the video frames, including:

Performing edge detection on each original video frame, and determining a cut-out object template mask corresponding to each original video frame;

According to the cut-out object template mask, cut out the target cut-out object in each original video frame, and obtain a to-be-restored video frame corresponding to each original video frame.
The method according to claim 1, wherein said performing restoration processing on the cut-out area in each video frame to be restored comprises:

According to the difference between the multiple video frames to be restored, restoration processing is performed on the cut-out area in each video frame to be restored.
The method according to claim 3, wherein said performing restoration processing on the cut-out area in each to-be-restored video frame according to the difference between said plurality of to-be-restored video frames comprises:

In turn, compare two adjacent video frames to be restored by at least one of the following methods: positive sequence, reverse sequence;

In the case that the first to-be-restored video frame of the two adjacent to-be-restored video frames has a restorable difference area for the second to-be-restored video frame, the second to-be-restored video frame is compared according to the restorable difference area. The cut-out area in the frame is restored, and the second video frame to be restored is updated.
The method according to claim 4, after the restoration process is performed on the cutout area in each to-be-restored video frame according to the difference between the plurality of to-be-restored video frames, the method further comprises:

In the case that there is the restoreable difference area between the two to-be-restored video frames to be compared, continue to execute the keying in each of the to-be-restored video frames according to the difference between the plurality of to-be-restored video frames. The operation of restoring the deleted area is performed until the restorable difference area does not exist between every two video frames to be restored that are compared.
The method according to claim 1, wherein the performing restoration processing on the cut-out area in each video frame to be restored includes:

Dividing each video frame to be restored into frame blocks to obtain multiple target frame blocks respectively corresponding to the multiple video frames to be restored;

According to the distribution of the multiple target frame blocks in the consecutive multiple to-be-restored video frames, performing restoration processing on the cutout area in each to-be-restored video frame.
The method according to claim 3 or 6, after the restoration process is performed on the cut-out area in each video frame to be restored, the method further comprises:

In the case where there is a cutout area that has not been restored in each video frame to be restored, pixel the cutout area that has not been restored is pixelated according to the distribution law of pixels around the cutout area that has not been restored. filling.
The method according to claim 1, wherein said removing the target to be removed from each original video frame of a plurality of original video frames of the target video comprises:

Cut out the target cutout object in the current display original video frame of the target video;

After the removing the target to be removed from the currently displayed original video frame of the target video, the method further includes:

Acquiring a cutout target video frame corresponding to the currently displayed original video frame;

In response to the object movement event associated with the object cutout instruction, according to the cutout target video frame corresponding to the currently displayed original video frame and the to-be-restored video frame, the target cutout object is displayed in the currently displayed original video frame. Move in the video frame.
The method according to claim 1, wherein the detecting the object removal instruction for the target video and determining the target removal object corresponding to the object removal instruction comprises:

Detecting an object removal instruction for the target video in the currently displayed original video frame of the target video, and performing object recognition on the currently displayed original video frame;

According to the object recognition result and the position information of the object removal instruction in the currently displayed video frame, a target removal object corresponding to the object removal instruction is determined.
A video processing device includes:

The target removal object determination module is configured to detect an object removal instruction for the target video, and determine the target removal object corresponding to the object removal instruction;

The to-be-restored video frame acquisition module is configured to cut out the target cutout object in each of the multiple original video frames of the target video, and obtain the respective corresponding to the multiple original video frames Multiple video frames to be restored;

The video frame restoration module is configured to restore the cut-out area in each video frame to be restored to obtain the edited video corresponding to the target video.
An electronic device including:

At least one processor;

Memory, set to store at least one program;

When the at least one program is executed by the at least one processor, the at least one processor implements the video processing method according to any one of claims 1-9.
A storage medium storing a computer program that, when executed by a processor, implements the video processing method according to any one of claims 1-9.