CN115529483A - A video processing method, device, electronic equipment and storage medium - Google Patents

Abstract

The present disclosure relates to a video processing method, device, electronic equipment, and storage medium. The method includes: segmenting and extracting key frames from the original video to obtain at least one video segment and key frames in each video segment. The same video segment The position information of the object to be processed contained in the image frame in the image frame to which the object belongs meets the preset position information, and determine the position information of the object to be processed in the key frame and the object to be processed in the key frame in each video segment , based on the object to be processed in each key frame and the position information of the object to be processed in each key frame, generate the mask of the object to be processed in each key frame, based on at least one video clip and the object to be processed in each key frame The mask performs object processing on at least one video segment to obtain a target video. The present application can reduce the hardware and software resources required for removing objects to be processed in subsequent videos, and is more universal.

Description

A video processing method, device, electronic equipment and storage medium

technical field

The present disclosure relates to the technical field of the Internet, and in particular to a video processing method, device, electronic equipment and storage medium.

Background technique

Videos, photos, etc. are increasingly being shared online for other users to obtain information and enjoy. However, before the video or photo is released, it may be defaced during the shooting or saving process, affecting the release of the video or photo.

The general decontamination method includes: collecting a large number of images with the same defacement, so as to estimate and align the defacement. Subsequently, the defacement is localized by detecting the gradients of all images in the entire image set to obtain a correct initial estimate, where the aligned detections are used to estimate the initial alpha mask and refine the estimated defacement layer, and then they Used as initial input to our multi-image decontamination algorithm to obtain the defaced image.

However, the above-mentioned decontamination method can only be aimed at the removal of a specific decontamination at a time, and has no universal applicability.

Contents of the invention

The present disclosure provides a video processing method, device, electronic equipment, and storage medium. The technical solutions of the present disclosure are as follows:

According to a first aspect of an embodiment of the present disclosure, a video processing method is provided, including:

Perform segmentation and key frame extraction on the original video to obtain at least one video segment and key frames in each video segment; wherein, the image frame in the same video segment contains the object to be processed on the image frame to which the object to be processed belongs The location information satisfies the preset location information; the preset location information is the information corresponding to the video segment to which the image frame belongs;

Determine the object to be processed in the key frame and the position information of the object to be processed in the key frame in each video segment;

Based on the object to be processed in each key frame and the position information of the object to be processed in each key frame, a mask of the object to be processed in each key frame is generated;

Object processing is performed on at least one video segment based on the at least one video segment and the mask of the object to be processed in each key frame to obtain a target video.

In some possible embodiments, the original video is segmented and key frame extracted to obtain at least one video segment and a key frame in each video segment, including:

Perform object recognition on the original video to obtain the type information of the object to be processed in the original video;

Determine the display rule of the object to be processed on the original video based on the type information of the object to be processed;

Segmenting the original video based on display rules to obtain at least one video segment;

A preset image frame in each video segment of the at least one video segment is determined as a key frame in each video segment.

In some possible embodiments, the display rule of the object to be processed on the original video is determined based on the type information of the object to be processed, and the original video is segmented based on the display rule to obtain at least one video segment, including:

Determine the display area and display duration of the object to be processed on the original video based on the type information of the object to be processed;

The original video is segmented based on the display area and display duration to obtain at least one video segment.

In some possible embodiments, the original video is segmented and key frame extracted to obtain at least one video segment and a key frame in each video segment, including:

Object recognition is carried out to the first image frame in the original video to obtain the object to be processed in the original video; the first image frame is the image frame in which the object to be processed appears for the first time in the original video;

determining first position information of the object to be processed on the first image frame;

performing image interception on the first image frame based on the first position information, to obtain a first sub-image corresponding to the first position information;

Determine the similarity data corresponding to each second image frame based on the first sub-image and each second image frame in the second image frame set; the second image frame set includes image frames other than the first image frame in the original video ;

Segmenting the original video based on the similarity data corresponding to each second image frame to obtain at least one video segment;

A preset image frame in each video segment of the at least one video segment is determined as a key frame in each video segment.

In some possible embodiments, the similarity data corresponding to each second image frame is determined based on the first sub-image and each second image frame, and the original video is classified based on the similarity data corresponding to each second image frame. segment, get at least one video segment, including:

acquiring a second sub-image corresponding to the first position information in each second image frame;

Based on the degree of similarity between each second sub-image and the first sub-image, determine similarity data corresponding to each second image frame;

If the similarity data corresponding to each second image frame satisfies the preset data, a video segment is obtained; the first image frame of the video segment is the first image frame.

In some possible embodiments, after determining the similarity data corresponding to each second image frame based on the degree of similarity between each second sub-image and the first sub-image, it further includes:

If there is a first target image frame set in the second image frame set, and the position of the first target image frame set is first, the first target image frame is adjacent to the first image frame, and is located after the first image frame in the original video , determining the first video segment based on the first image frame and the first target image frame set;

The first target image frame set includes a first target image frame or a plurality of continuous first target image frames; the first target image frame set is not equal to the second image frame set, and the first target image frame in the first target image frame set The corresponding similarity data satisfies the preset data.

In some possible embodiments, the method also includes:

Taking the difference between the second image frame set and the first target image frame set as the video to be segmented;

The first image frame of the video to be segmented is a new first image frame; in the video to be segmented, the image frames other than the new first image frame are a new second image frame set;

Determining new first position information of the object to be processed on the new first image frame;

From the new first image frame, obtain the new first sub-image corresponding to the new first position information, and obtain the new first sub-image corresponding to the new first position information from each second image frame in the new second image frame set The new second sub-image of ;

determining similarity data for each new second sub-image based on the degree of similarity between the new first sub-image and each new second sub-image;

If there is a new first target image frame set in the new second image frame set, and in the new first target image frame set, the first target image frame at the first position is adjacent to the new first image frame, and in After the new first image frame in the original video, the second video segment is determined based on the new first image frame and the new first target image frame set;

The new first target image frame set includes a new first target image frame or a plurality of continuous new first target image frames; the new first target image frame set is not equal to the new second image frame set, and the new The similarity data corresponding to the first target image frames in the first target image frame set satisfies the preset data.

In some possible embodiments, the method also includes:

Preset color proportion detection is carried out in the original video, and the proportion data of the preset color of each image frame in the original video is obtained;

If the ratio data of the preset colors of multiple image frames in the original video satisfies the third preset data, determine the start and end times of the multiple image frames; wherein, the multiple image frames are continuous image frames, and the multiple image frames The last image frame of is the video end image frame.

In some possible embodiments, the object to be processed includes the first type of object to be processed and the second type of object to be processed, then determine the position information of the object to be processed in the key frame and the object to be processed in the key frame in each video clip ,include:

Based on the first detection model in the object detection model, the first type of object detection is carried out to the key frame in each video clip, and the first type of object to be processed and the first type of object to be processed in the key frame in each video clip are determined. location information;

Based on the second detection model in the object detection model and the position information of the first type of object to be processed, the key frame in each video segment is detected for the second type of object, and the second type of the key frame in each video segment is determined to be processed. Location information of the processing object and the second type of object to be processed.

In some possible embodiments, based on the object to be processed in each key frame and the position information of the object to be processed in each key frame, generating a mask of the object to be processed in each key frame includes:

Based on the position information of the object to be processed in each key frame, the pixels of the object to be processed in each key frame are binarized to obtain the mask of the object to be processed in each key frame.

According to a second aspect of an embodiment of the present disclosure, a video processing device is provided, including:

The segmentation module is configured to perform segmentation and key frame extraction on the original video to obtain at least one video segment and key frames in each video segment; wherein, the image frames in the same video segment contain objects to be processed in The position information on the image frame to which the object to be processed belongs satisfies the preset position information; the preset position information is the information corresponding to the video segment to which the image frame belongs;

A determining module configured to determine the position information of the object to be processed in the key frame and the object to be processed in the key frame in each video clip;

The mask generating module is configured to generate a mask of the object to be processed in each key frame based on the object to be processed in each key frame and the position information of the object to be processed in each key frame;

The object removal module is configured to perform object processing on at least one video segment based on at least one video segment and the mask of the object to be processed in each key frame to obtain a target video.

In some possible embodiments, the segmentation module is configured to:

Perform object recognition on the original video to obtain the type information of the object to be processed in the original video;

Determine the display rule of the object to be processed on the original video based on the type information of the object to be processed;

Segmenting the original video based on display rules to obtain at least one video segment;

A preset image frame in each video segment of the at least one video segment is determined as a key frame in each video segment.

In some possible embodiments, the segmentation module is configured to:

Determine the display area and display duration of the object to be processed on the original video based on the type information of the object to be processed;

The original video is segmented based on the display area and display duration to obtain at least one video segment.

In some possible embodiments, the segmentation module is configured to:

Object recognition is carried out to the first image frame in the original video to obtain the object to be processed in the original video; the first image frame is the image frame in which the object to be processed appears for the first time in the original video;

determining first position information of the object to be processed on the first image frame;

performing image interception on the first image frame based on the first position information, to obtain a first sub-image corresponding to the first position information;

Determine the similarity data corresponding to each second image frame based on the first sub-image and each second image frame in the second image frame set; the second image frame set includes image frames other than the first image frame in the original video ;

Segmenting the original video based on the similarity data corresponding to each second image frame to obtain at least one video segment;

A preset image frame in each video segment of the at least one video segment is determined as a key frame in each video segment.

In some possible embodiments, the segmentation-based module is configured to perform:

acquiring a second sub-image corresponding to the first position information in each second image frame;

Based on the degree of similarity between each second sub-image and the first sub-image, determine similarity data corresponding to each second image frame;

If the similarity data corresponding to each second image frame satisfies the preset data, a video segment is obtained; the first image frame of the video segment is the first image frame.

In some possible embodiments, the segmentation module is configured to:

If there is a first target image frame set in the second image frame set, and the position of the first target image frame set is first, the first target image frame is adjacent to the first image frame, and is located after the first image frame in the original video , determining the first video segment based on the first image frame and the first target image frame set;

The first target image frame set includes a first target image frame or a plurality of continuous first target image frames; the first target image frame set is not equal to the second image frame set, and the first target image frame in the first target image frame set The corresponding similarity data satisfies the preset data.

In some possible embodiments, the segmentation module is configured to:

Taking the difference between the second image frame set and the first target image frame set as the video to be segmented;

The first image frame of the video to be segmented is a new first image frame; in the video to be segmented, the image frames other than the new first image frame are a new second image frame set;

Determining new first position information of the object to be processed on the new first image frame;

From the new first image frame, obtain the new first sub-image corresponding to the new first position information, and obtain the new first sub-image corresponding to the new first position information from each second image frame in the new second image frame set The new second sub-image of ;

determining similarity data for each new second sub-image based on the degree of similarity between the new first sub-image and each new second sub-image;

If there is a new first target image frame set in the new second image frame set, and in the new first target image frame set, the first target image frame at the first position is adjacent to the new first image frame, and in After the new first image frame in the original video, the second video segment is determined based on the new first image frame and the new first target image frame set;

The new first target image frame set includes a new first target image frame or a plurality of continuous new first target image frames; the new first target image frame set is not equal to the new second image frame set, and the new The similarity data corresponding to the first target image frames in the first target image frame set satisfies the preset data.

In some possible embodiments, the device further includes a time determination module configured to:

Preset color proportion detection is carried out in the original video, and the proportion data of the preset color of each image frame in the original video is obtained;

If the ratio data of the preset colors of multiple image frames in the original video satisfies the third preset data, determine the start and end times of the multiple image frames; wherein, the multiple image frames are continuous image frames, and the multiple image frames The last image frame of is the video end image frame.

In some possible embodiments, the determination module is configured to execute:

Based on the first detection model in the object detection model, the first type of object detection is carried out to the key frame in each video clip, and the first type of object to be processed and the first type of object to be processed in the key frame in each video clip are determined. location information;

Based on the second detection model in the object detection model and the position information of the first type of object to be processed, the key frame in each video segment is detected for the second type of object, and the second type of the key frame in each video segment is determined to be processed. Location information of the processing object and the second type of object to be processed.

In some possible embodiments, the mask generation module is configured to:

Based on the position information of the object to be processed in each key frame, the pixels of the object to be processed in each key frame are binarized to obtain the mask of the object to be processed in each key frame.

According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including: a processor; a memory for storing processor-executable instructions; wherein, the processor is configured to execute instructions to implement the above-mentioned first aspect. any method.

According to a fourth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, and when instructions in the computer-readable storage medium are executed by a processor of the electronic device, the electronic device can execute the first aspect of the embodiments of the present disclosure any of the methods.

According to a fifth aspect of the embodiments of the present disclosure, a computer program product is provided, the computer program product includes a computer program, the computer program is stored in a readable storage medium, and at least one processor of a computer device reads and executes the program from the readable storage medium A computer program that causes a computer device to execute the method of any one of the first aspects of the embodiments of the present disclosure.

The technical solutions provided by the embodiments of the present disclosure bring at least the following beneficial effects:

Perform segmentation and key frame extraction on the original video to obtain at least one video segment and key frames in each video segment, wherein the image frame in the same video segment contains the object to be processed on the image frame to which the object to be processed belongs The position information meets the preset position information, and the preset position information is the information corresponding to the video segment to which the image frame belongs, and the position information of the object to be processed in the key frame and the object to be processed in the key frame are determined in each video segment, based on each The position information of the object to be processed of each key frame and the object to be processed of each key frame, generate the mask of the object to be processed of each key frame, based on at least one video clip and the mask of the object to be processed of each key frame Object processing is performed on at least one video segment to obtain a target video. This application can accurately locate the location of the object to be processed in each video clip through video clips and key frames, reduce the computing power resources required to remove the object to be processed in subsequent videos, and can be applied to more objects to be processed video, more universal.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For those skilled in the art, other drawings can also be obtained based on these drawings without creative effort.

Fig. 1 is a schematic diagram showing an application environment according to an exemplary embodiment;

Fig. 2 is a flowchart of a video processing method shown according to an exemplary embodiment;

Fig. 3 is a flow chart of a method for video segmentation and key frame extraction according to an exemplary embodiment;

Fig. 4 is a flowchart of a method for video segmentation and key frame extraction according to an exemplary embodiment;

Fig. 5 is a flow chart showing a method for segmenting an original video based on similarity data according to an exemplary embodiment;

Fig. 6 is a flow chart showing a method for segmenting an original video based on similarity data according to an exemplary embodiment;

Fig. 7 is a flow chart showing an object to be processed in a key frame and a method for determining position information of the object to be processed in a key frame according to an exemplary embodiment;

Fig. 8 is a flow chart showing a method for deleting a credits trailer according to an exemplary embodiment;

Fig. 9 is a block diagram of a video processing device according to an exemplary embodiment;

Fig. 10 is a block diagram showing an electronic device for video processing according to an exemplary embodiment.

detailed description

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

It should be noted that the terms "first" and "second" in the specification and claims of the present disclosure and the above drawings are used to distinguish similar first objects, but not necessarily to describe a specific order or sequence order. It is to be understood that the data so used are interchangeable under appropriate circumstances such that the embodiments of the disclosure described herein can be practiced in sequences other than those illustrated or described herein. The implementations described in the following exemplary examples do not represent all implementations consistent with the present disclosure. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present disclosure as recited in the appended claims.

It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in this disclosure are authorized by the user. Or information and data fully authorized by the parties.

Please refer to FIG. 1 . FIG. 1 is a schematic diagram of an application environment of a video processing method according to an exemplary embodiment. As shown in FIG. 1 , the application environment may include a server 01 and a client 02 .

In some possible embodiments, the server 01 may include an independent physical server, or a server cluster or a distributed system composed of multiple physical servers, and may also provide cloud services, cloud databases, cloud computing, cloud functions, Cloud servers for basic cloud computing services such as cloud storage, network services, middleware services, domain name services, security services, CDN (Content Delivery Network, content distribution network), and big data and artificial intelligence platforms. The operating system running on the server may include but not limited to Android system, IOS system, linux, windows, Unix, etc.

In some possible embodiments, the above-mentioned client 02 may include but not limited to smartphones, desktop computers, tablet computers, notebook computers, smart speakers, digital assistants, augmented reality (augmented reality, AR)/virtual reality (virtual reality) , VR) devices, smart wearable devices and other types of clients. It may also be software running on the above-mentioned client, such as an application program, a small program, and the like. Optionally, the operating system running on the client may include but not limited to Android system, IOS system, linux, windows, Unix, etc.

In some possible embodiments, the server 01 or the client 02 can perform segmentation and key frame extraction on the original video to obtain at least one video segment and key frames in each video segment, wherein the images in the same video segment The position information of the object to be processed contained in the frame on the image frame to which the object to be processed belongs satisfies the preset position information, and the preset position information is the information corresponding to the video segment to which the image frame belongs, and the key frame to be processed in each video segment is determined. The position information of the processing object and the object to be processed in the key frame is based on the position information of the object to be processed in each key frame and the position information of the object to be processed in each key frame, and the mask of the object to be processed in each key frame is generated, based on at least A video segment and a mask of an object to be processed in each key frame perform object processing on at least one video segment to obtain a target video.

In some possible embodiments, the client 02 may be connected to the server 01 through a wired link, or may be connected through a wireless link.

In an exemplary embodiment, the client, the server, and the database corresponding to the server can all be node devices in the blockchain system, and can share the acquired and generated information with other node devices in the blockchain system to realize Information sharing between multiple node devices. Multiple node devices in the blockchain system can be configured with the same blockchain, which consists of multiple blocks, and the adjacent blocks have an association relationship, so that the data in any block is Any tampering can be detected through the next block, so that the data in the blockchain can be prevented from being tampered with, and the security and reliability of the data in the blockchain can be guaranteed.

Fig. 2 is a flowchart of a video processing method shown according to an exemplary embodiment. As shown in Fig. 2, the video processing method can be applied to a server or to other node devices, such as a client, and the server is used as an example below For example, the method includes the following steps:

In step S201, the original video is segmented and key frame extracted to obtain at least one video segment and a key frame in each video segment; wherein, the image frame in the same video segment contains the object to be processed in the object to be processed The position information on the associated image frame satisfies the preset position information; the preset position information is information corresponding to the video segment to which the image frame belongs.

In this embodiment of the present application, the server may segment and extract key frames from the original video to obtain at least one video segment and key frames in each video segment.

In order to enable the key frame in each video clip to better assist the removal of the object to be processed in the video clip where the key frame is located, the image frame in the same video clip contains the object to be processed in the image to which the object to be processed belongs The position information on the frame satisfies the preset position information, and the preset position information is the information corresponding to the video segment to which the image frame belongs.

Optionally, the position information of the object to be processed contained in the image frame in the same video segment on the image frame to which the object to be processed satisfies the preset position information may refer to: all image frames in the same video segment contain the position information to be processed The position information of the processing object on the image frame to which the object to be processed belongs satisfies the preset position information.

Optionally, the location information of the object to be processed contained in the image frame in the same video segment on the image frame to which the object to be processed satisfies the preset position information may refer to: the image frame to be processed contained in a part of the same video segment The position information of the processing object on the image frame to which the object belongs meets the preset position information, and another part of the image frames does not contain the object to be processed. For example, in a video clip, 80% of the image frames contain the object to be processed, and 80% of the The objects to be processed contained in the image frames satisfy the preset position information, and the remaining 20% of the image frames do not contain the objects to be processed.

In the embodiment of the present application, each video segment has its corresponding preset position information, and the position information of the object to be processed contained in the image frame belonging to the video segment on the image frame to which the object to be processed belongs is within the preset position information. within the location information.

Next, the first video clip is used as an example to illustrate what each video clip has its corresponding preset position information. Assume that the first video clip contains 60 image frames, and among the 60 image frames, the resolution of each image frame is 640*320, that is, the pixel value corresponding to the width of each image frame is 640, and each image frame The pixel value corresponding to the height of the frame is 320. Then, the preset position information corresponding to the first video clip can be a rectangular area defined by the following four vertices: the pixel value corresponding to the width is 0, and the pixel value corresponding to the height is 0 (0, 0), and the pixel value corresponding to the width is 0 (0, 0). The pixel value is 32, and the pixel value corresponding to the height is 0 (32,0), the pixel value corresponding to the width is 0, and the pixel value corresponding to the height is 16 (0,16) and the pixel value corresponding to the width is 32, and High corresponds to a pixel value of 16(32,16).

In an optional embodiment, the position information of the object to be processed included in the image frame belonging to the video segment on the image frame to which the object to be processed belongs is within the preset position information means: the first In the video clip, the position information of the object to be processed contained in each of the 60 image frames on the image frame to which the object to be processed belongs is the first rectangular area delimited by the following four vertices, and the four vertices are: (0,0), (0,16), (32,0) and (32,16).

In another optional embodiment, the position information of the object to be processed included in the image frame belonging to the video segment on the image frame to which the object to be processed belongs is within the preset position information means: first In a video segment, the position information of the object to be processed contained in a part of the image frames in the 60 image frames on the image frame to which the object to be processed belongs is the second rectangular area delimited by the following four vertices, and the four vertices are: (0,0), (0,15), (32,0) and (32,15), the position information of the object to be processed contained in another part of the image frame on the image frame to which the object to be processed belongs is the following four The third rectangular area defined by vertices, the four vertices are: (0,0), (0,16), (28,0) and (28,16). It can be seen that both the second rectangular area and the third rectangular area are included in the first rectangular area.

Optionally, there are many ways for the server to obtain at least one video clip and the key frames in each of the at least one video clip. The following describes how to obtain at least one video clip and the at least one video clip through multiple implementations. Keyframes in each video clip in the clip.

In an optional embodiment, the server may obtain a preset time interval, and obtain multiple key frames from the original video based on the preset time interval. Subsequently, the server may divide the original video into multiple video segments based on the positions of the multiple key frames in the original video.

For example, assuming that the frame rate of the original video is 30 frames per second, the duration of the original video is 10 seconds, and the preset time interval is 2 seconds, the server obtains multiple keys from the original video based on the preset time interval The frames are the first image frame, the 61st image frame, the 121st image frame, the 181st image frame, and the 241st image frame. Subsequently, the server may determine all image frames included in the first image frame to the 60th image frame as the first video segment, and determine all image frames included in the 61st image frame to the 120th image frame as the second video clip. video clips, determine all the image frames included in the 121st image frame to the 180th image frame as the third video clip... In this way, the server can divide the original video into 5 video clips, and divide each video clip into 5 video clips The first image frame is determined as the key frame in the video clip.

In another optional embodiment, the server may perform segmentation and key frame extraction on the original video based on the display rule of the object to be processed in the original video to obtain at least one video segment and a key frame in each video segment. Fig. 3 is a flowchart of a method for video segmentation and key frame extraction according to an exemplary embodiment, as shown in Fig. 3 , including:

In step S301, object recognition is performed on the original video to obtain type information of the object to be processed in the original video.

In this embodiment of the present application, the type information of the object to be processed may refer to company information or organization information to which the object to be processed belongs. Therefore, the server can perform object recognition on the original video to obtain company information or organization information of the object to be processed in the original video.

In step S303, the display rule of the object to be processed on the original video is determined based on the type information of the object to be processed.

In the embodiment of the present application, the display rule of the object to be processed on the original video refers to the display area and display duration of the object to be processed on the original video. For example, the display rule for the object A to be processed to be presented on the original video based on the display parameters set in advance is: the object A to be processed is always displayed in the upper left corner area of each image frame on the original video (with the upper left corner as a vertices, 5*5 regions). The display rule for the object B to be processed to be presented on the original video based on the display parameters set in advance is: the object B to be processed is displayed in the upper left corner area of the image frame between the 0-2 seconds of the original video (taking the upper left corner as a vertex, 5*5 area), displayed in the lower right corner area of the image frame between the 2nd and 4th seconds of the original video (with the lower right corner as a vertex, 5*5 area), in the 4th- Displayed in the upper right corner area of the image frame within 6 seconds (with the lower right corner as a vertex, 5*5 area)...

In step S305, the original video is segmented based on the display rule to obtain at least one video segment.

Continue to elaborate based on the above object A to be processed. Since the display area of the object A to be processed is the "upper left corner area" and the display duration is "always", the server may not segment the original video based on the display area and display duration. That is, the original video as a whole.

Based on the above-mentioned object B to be processed, the server can segment the original video according to the display area and display duration of the object B to be processed on the original video, and regard the video segment corresponding to the 0-2 second as the first video Segment, the video segment corresponding to the 2nd-4th second is regarded as the second video segment, and the video segment corresponding to the 4th-6th second is regarded as the third video segment...

In step S307, a preset image frame in each video segment of at least one video segment is determined as a key frame in each video segment.

Optionally, for the segmentation result corresponding to object A to be processed, the server may determine the first image frame as a key frame in the original video, or may determine any image frame in the original video as a key frame in the original video Keyframe.

Optionally, for the segmentation result corresponding to object B to be processed, the server may determine the first image frame in the first video segment as a key frame in the first video segment, or may determine the first video frame Any image frame in the clip is determined as a key frame in the first video clip, and in the second video clip, the first image frame is determined as a key frame in the second video clip, or the second Any image frame in the video clip is determined as a key frame in the second video clip...For the determination of key frames in other video clips, please refer to the determination of key frames in the first video clip, and will not be repeated here.

In summary, since the display rules are associated with the type information of the objects to be processed, the display of the objects to be processed on the image frame has certain regularity, so the server can follow the display rules of the objects to be processed, such as the display area and display Segment the original video accurately and determine the key frame of each video clip, paving the way for the subsequent use of the key frame in each video clip to better assist the video clip where the key frame is located for object removal.

In another optional embodiment, the server may perform segmentation and key frame extraction on the original video based on similarity data between image frames to obtain at least one video segment and a key frame in each video segment. Fig. 4 is a flow chart of a method for video segmentation and key frame extraction according to an exemplary embodiment, as shown in Fig. 4 , including:

In step S401, object recognition is performed on the first image frame in the original video to obtain the object to be processed in the original video; the first image frame is the image frame in which the object to be processed appears for the first time in the original video.

In the embodiment of the present application, the original video may be a video published on the video platform. Therefore, the objects to be processed in the original video may be unified in the original video, for example, they are all objects to be processed on the video platform, or the objects to be processed on the video platform Plus the publisher's pending object.

Optionally, the resolution of each image frame in the original video is consistent, such as 640*320.

In an optional embodiment, the object to be processed in the original video may start from the first image frame, therefore, the server may perform object recognition on the first image frame to obtain the object to be processed in the original video.

In another optional embodiment, if the object to be processed in the original video does not start from the first image frame, the server may perform object recognition on the image frames in sequence until the object to be processed is obtained. In this way, the server may determine the image frame in which the object to be processed exists as the first image frame.

In the embodiment of the present application, the server may perform object recognition on the first image frame in the original video by using the object detection model to obtain the object to be processed in the original video. Optionally, the object detection model may include, but is not limited to, a deep learning model such as a convolutional neural network, a recurrent neural network, or a recurrent neural network.

In step S403, the first position information of the object to be processed on the first image frame is determined.

In an optional embodiment, the server can not only perform object recognition on the first image frame in the original video through the object detection model to obtain the object to be processed in the original video, but also obtain the object to be processed on the first image frame The first location information of .

In another optional embodiment, after the server determines the object to be processed in the original video, it may also mark a rectangular frame around the object to be processed on the first image frame, and based on the rectangular frame in the first image frame The pixels contained in the frame determine the first position information of the object to be processed on the first image frame.

Optionally, the first position information refers to the four pixel pairs ((X1, Y1), (X1, Y2), (X2, Y1), (X2, Y2)) corresponding to the four corners of the rectangular frame, or the first The location information refers to the two pixel pairs ((X1, Y1), (X2, Y2)) corresponding to the two diagonal corners of the rectangular frame, and the two pixel pairs corresponding to the two diagonal corners ((X1, Y1), (X2, Y2)) can locate four pixel pairs ((X1, Y1), (X1, Y2), (X2, Y1), (X2, Y2)) corresponding to the four corners of the rectangular frame.

In step S405, image interception is performed on the first image frame based on the first position information to obtain a first sub-image corresponding to the first position information.

In the embodiment of the present application, the server may be based on the first position information, such as (four pixel pairs (X1, Y1), (X1, Y2), (X2, Y1), (X2, Y2) corresponding to the four corners of the rectangular frame ) performing image interception on the first image frame to obtain a first sub-image corresponding to the first position information.

In step S407, the similarity data corresponding to each second image frame is determined based on the first sub-image and each second image frame in the second image frame set; the second image frame set includes the original video, except for the first image Image frame out of frame.

Optionally, the server may combine image frames in the original video except the first image frame into a second image frame set. And based on the first sub-image and each second image frame in the second image frame set, the similarity data corresponding to each second image frame is determined.

In step S409, the original video is segmented based on the similarity data corresponding to each second image frame to obtain at least one video segment.

Optionally, the server may segment the original video based on the similarity data corresponding to each second image frame to obtain at least one video segment.

In step S411, a preset image frame in each video segment of at least one video segment is determined as a key frame in each video segment.

Optionally, the server may determine the first image frame in each video segment as a key frame in each video segment, or may determine any image frame in each video segment as a key frame in each video segment Keyframe.

As above, in the embodiment of the present application, according to the similarity data between the image frames, the image frames with the same object to be processed can be accurately divided into a video segment, so that in the final stage, it is necessary to use the key frame of the video segment to be processed. When removing the object to be processed by the mask of the processing object, the mask of the object to be processed in the key frame of the video clip can quickly locate the position of the object to be processed in the image frame in the video clip, and then unify Remove the object to be processed on the video clip.

Fig. 5 is a flow chart of a method for segmenting an original video based on similarity data according to an exemplary embodiment, as shown in Fig. 5 , including:

In step S501, a second sub-image corresponding to the first position information in each second image frame is acquired.

Based on the first position information above, that is, the four pixel pairs ((X1, Y1), (X1, Y2), (X2, Y1), (X2, Y2)) corresponding to the four corners of the rectangular frame continue to elaborate, the server A second sub-image corresponding to the first position information in each second image frame may be acquired. That is to say, the second sub-image corresponding to each second image frame is obtained by intercepting the rectangular frame corresponding to each second image frame.

In step S502, based on the degree of similarity between each second sub-image and the first sub-image, similarity data corresponding to each second image frame is determined.

Optionally, assuming that the first sub-image and the second sub-image are sub-images corresponding to 5*5 pixels, the server can determine the pixel pairs of the pixels on the first sub-image and the second sub-image to obtain 25 pixel pairs. For example, on the first sub-image, the pixel whose row position is 1 and whose column position is 1 and the pixel whose row position is 1 and whose column position is 1 on the second sub-image form a pixel pair, and on the first sub-image, The pixel with row position 5 and column position 5 and the pixel with row position 5 and column position 5 on the second sub-image form a pixel pair.

Subsequently, the server may compare similarity based on each pixel pair to obtain similarity data corresponding to each pixel pair. Optionally, the similarity data may be percentage data. Furthermore, the server may determine the similarity data corresponding to each second image frame based on the mean value of the similarity data of 25 pixel pairs.

In step S503, compare the similarity data corresponding to each second image frame with the preset data, if the similarity data corresponding to each second image frame meets the preset data, go to step S504; otherwise, go to Step S505.

In step S504, a video segment is obtained; the first image frame of the video segment is the first image frame.

Optionally, the server may compare the similarity data corresponding to each second image frame with the preset data, assuming that the preset data is 80%, if the similarity data corresponding to each second image frame is greater than or equal to the preset Set data, you can get a video clip. That is, the original video is regarded as a video clip. Wherein, the first image frame of the video segment is the first image frame.

In step S505, if there is a first target image frame set in the second image frame set, and the first target image frame in the first target image frame set is adjacent to the first image frame and located in the original video frame After the first image frame, determine the first video segment based on the first image frame and the first target image frame set; the first target image frame set includes a first target image frame or a plurality of continuous first target image frames; the second A target image frame set is not equal to the second image frame set, and the similarity data corresponding to the first target image frames in the first target image frame set satisfies preset data.

Optionally, the server can compare the similarity data corresponding to each second image frame with the preset data, assuming that the preset data is 80%, the first image frame is the first image frame in the original video, and the original video Contains 300 image frames. If the first target image frame set (including the 2-60th image frame) exists in the second image frame set (including the 2-300th image frame), and the position of the first target image frame set is located in the first first target The image frame (the second image frame) is adjacent to the first image frame and located after the first image frame in the original video, and the server may determine the first video segment based on the first image frame and the first target image frame set.

Wherein, the first target image frame set is not equal to the second image frame set. The similarity data (for example, all greater than or equal to 80%) corresponding to the first target image frames in the first target image frame set satisfy the preset data.

In this way, the server can determine the first video segment.

In step S506, the difference between the second image frame set and the first target image frame set is used as the video to be segmented.

Continuing to explain the above embodiment, the server can calculate the difference between the second image frame set (including the 2-300th image frame) and a target image frame set (including the 2-60th image frame) 61-300 image frames) are videos to be segmented.

In step S507, the first image frame of the video to be segmented is a new first image frame; in the video to be segmented, the image frames other than the new first image frame are a new second image frame set.

Continue to elaborate with the above embodiment, the server can use the first image frame (the 61st image frame) of the video to be segmented as the new first image frame; in the video to be segmented, remove the new first image frame The other image frames (the 62nd-300th image frames) are the new second image frame set.

In step S508, new first position information of the object to be processed on the new first image frame is determined.

Optionally, in this embodiment of the present application, the server may determine the new first position information of the object to be processed on the new first image frame through the object detection model, where the new first position information may be a rectangular box four The four pixel pairs corresponding to the corners ((X3, Y3), (X3, Y4), (X4, Y3), (X4, Y4)). Optionally, the object detection model may include, but is not limited to, a deep learning model such as a convolutional neural network, a recurrent neural network, or a recurrent neural network.

In step S509, from the new first image frame, a new first sub-image corresponding to the new first position information is obtained, and from each second image frame in the new second image frame set, the new sub-image corresponding to the new A new second sub-image corresponding to the first position information.

Optionally, the server may obtain the new first sub-image corresponding to the new first position information from the new first image frame, and obtain the new sub-image corresponding to the new sub-image from each second image frame in the new second image frame set. The new second sub-image corresponding to the first position information of .

In step S510, based on the degree of similarity between the new first sub-image and each new second sub-image, similarity data of each new second sub-image is determined.

Optionally, assuming that the new first sub-image and the new second sub-image are sub-images corresponding to 6*6 pixels, the server may compare the pixels on the new first sub-image and the new second sub-image Determination of pixel pairs yields 36 pixel pairs. For example, on the new first sub-image, the pixel whose row position is 1 and whose column position is 1 and the pixel whose row position is 1 and whose column position is 1 on the new second sub-image form a pixel pair, and the new The pixel with row position 6 and column position 6 on the first sub-image and the pixel with row position 6 and column position 6 on the new second sub-image form a pixel pair.

Subsequently, the server may compare similarity based on each pixel pair to obtain similarity data corresponding to each pixel pair. Optionally, the similarity data may be percentage data. Further, the server may determine the similarity data corresponding to each second image frame based on the mean value of the similarity data of the 36 pixel pairs.

In step S511, if there is a new first target image frame set in the new second image frame set, and in the new first target image frame set, the position is between the first first target image frame and the new first image frame Adjacent, and after the new first image frame in the original video, determine the second video segment based on the new first image frame and the new first target image frame set; the new first target image frame set includes a A new first target image frame or multiple consecutive new first target image frames; the new first target image frame set is not equal to the new second image frame set, the first target in the new first target image frame set The similarity data corresponding to the image frame satisfies the preset data.

Optionally, the server may compare the similarity data corresponding to each new second image frame with preset data. If there is a new first target image frame set (including the 62-120th image frame) in the new second image frame set (the 62nd-300th image frame), and the position of the new first target image frame set is at The new first target image frame (the 62nd image frame) is adjacent to the new first image frame (the 61st image frame), and after the new first image frame in the original video, the server can base on the new The first image frame and the new first target image frame set determine the second video segment (including image frames 61-120).

Wherein, the new first target image frame set is not equal to the new second image frame set. The similarity data (for example, all greater than or equal to 80%) corresponding to the first target image frames in the new first target image frame set satisfy the preset data.

In this way, the server can determine the second video segment. Next, the server can determine the third video segment, the fourth video segment... until all image frames in the original video are divided into video segments by referring to the determination method of the second video segment.

As above, through the similarity data between the first sub-image located in the first position information in the first image frame, and the second sub-image corresponding to the first position information in other image frames, the same situation will exist exactly. The image frames of the processing object are divided into one video segment, and then the original video is divided into multiple video segments. So that in the final stage, when the mask of the object to be processed in the key frame of the video clip needs to be used to remove the object to be processed in the video clip, the mask of the object to be processed in the key frame of the video clip can be quickly positioned to the The position of the object to be processed in the image frame in the video clip, and then remove the object to be processed for the video clip in a unified manner.

Fig. 6 is a flow chart of a method for segmenting an original video based on similarity data according to an exemplary embodiment, as shown in Fig. 6 , including:

In step S601, the first image frame is determined as the current image frame.

Assuming that the first image frame is the first image frame in the 300 frames of the original video, the server determines the first image frame as the current image frame.

In step S602, according to the sequence of video playback, the image frame in the original video that is at a preset distance from the current image frame is determined as the execution image frame.

In the embodiment of the present application, the server may use the image frame in the original video that is at a preset distance from the current image frame as the execution image frame according to the video playback sequence.

Optionally, in order to ensure the accuracy of subsequent video segmentation, the preset interval may be one image frame apart. That is to say, the server can take the image frame one image frame away from the current image frame in the original video as the execution image frame, that is, the second image frame of the original video as the execution image frame according to the order of video playback.

Optionally, in practical applications, the position of the object to be processed on the image frame in the original video may not change in a short period of time. Based on this, in order to ensure a certain software processing speed, the preset interval may be a few image frames apart, for example, five image frames apart, under the condition of sacrificing a certain segmentation accuracy. That is to say, the server may take the image frame five image frames away from the current image frame in the original video as the execution image frame, that is, the sixth image frame of the original video as the execution image frame according to the video playback sequence.

In step S603, the execution location information in the execution image frame is determined based on the first location information, and the execution image frame is intercepted based on the execution location information to obtain an execution sub-image corresponding to the execution location information.

As mentioned above, the resolution of each image frame in the original video is the same, such as 640*320. Assuming that the first position information is two pixel pairs ((X1, Y1), (X2, Y2)) corresponding to two opposite corners of the rectangular frame, the execution position information in the execution image frame can be the rectangle in the execution image frame Two pixel pairs ((X1, Y1), (X2, Y2)) corresponding to the two diagonal corners of the box. Next, the server may perform image interception on the execution image frame based on the execution location information to obtain an execution sub-image corresponding to the execution location information.

In step S604, the similarity data corresponding to the execution sub-image is determined based on the first sub-image and the execution sub-image.

Optionally, assuming that the first sub-image and the execution sub-image are sub-images corresponding to 5*5 pixels, the server can determine the pixel pairs of the pixels on the first sub-image and the execution sub-image to obtain 25 pixel pairs . For example, on the first sub-image, the pixel whose row position is 1 and whose column position is 1 and the pixel whose row position is 1 and whose column position is 1 on the execution sub-image form a pixel pair, and on the first sub-image, row The pixel with position 5 and column position 5 and the pixel with row position 5 and column position 5 on the execution sub-image form a pixel pair.

Subsequently, the server may compare similarity based on each pixel pair to obtain similarity data corresponding to each pixel pair. Optionally, the similarity data may be percentage data. Furthermore, the server may determine the similarity data corresponding to the execution image frame based on the mean value of the similarity data of 25 pixel pairs.

In step S605, compare the similarity data corresponding to the execution sub-image with the first preset data, if the similarity data corresponding to the execution sub-image satisfies the first preset data, determine the execution image frame as the current image frame, and turn to Go to step S602; otherwise, go to step S606.

In the embodiment of the present application, the first preset data can be preset, for example, the first preset data is greater than or equal to 95%, that is, if the similarity data satisfies greater than or equal to 95%, it is determined to execute the sub-image and the first sub-image. The similarity between images is very high, therefore, the execution sub-image includes the same object to be processed as that in the first sub-image.

Based on this, it can be considered that the first image frame and the execution image frame (the second image frame), or all image frames involved from the first image frame to the execution image frame (the sixth image frame) can be regarded as a video The image frame in the fragment. Subsequently, it is judged whether the image frame following the execution image frame also belongs to the image frame in the video segment.

Taking the execution image frame as the second image frame as an example, the server can regard the execution image frame as the current image frame, and then repeat steps S602-S605, that is, according to the order of the video, the image in the original video that is one image frame away from the current image frame frame (the third image frame in the original video) is regarded as the execution image frame. Obtain the execution sub-image corresponding to the first position information in the third image frame, determine the similarity data based on the first sub-image and the execution sub-image, and determine the execution sub-image and the second execution sub-image if the similarity data satisfies greater than or equal to 95%. The similarity between one sub-image is very high, therefore, the execution sub-image in the third image frame includes the same object to be processed as that in the first sub-image.

Based on this, the server can regard the execution image frame (the third image frame) as the current image frame, and continue to repeat steps S602-S605. For the process of repeating steps, please refer to the above, and will not repeat them here.

In step S606, until the similarity data corresponding to the execution sub-image satisfies the second preset data, the first image frame is determined as the starting image frame of the first video clip, and the previous image frame of the execution image frame is determined as The ending image frame of the first video clip to get the first video clip.

Optionally, corresponding to the above first preset data, the second preset data may be less than 95%. For example, if the server executes the image frame as the thirty-first image frame in the original video after several rounds of repeated steps, if the execution sub-image corresponding to the thirty-first image frame is similar to the first sub-image If the degree data is less than 95%, the server can regard the first image frame as the start image frame of the first video segment, and the previous image frame of the execution image frame as the end image frame of the first video segment, and obtain The first video segment, that is, the first video segment includes a first image frame to a thirtieth image frame, a total of 30 image frames. In this way, the server determines the first video clip, and the position of the object to be processed in each image frame in the first video clip is within the preset position information corresponding to the first video clip, such as Both are in the upper left corner (with the upper left corner as a vertex, a 5*5 area).

In step S607, the execution image frame is determined as the second image frame.

At this time, the execution image frame is the thirty-first image frame, and the server regards the thirty-first image frame as the second image frame.

In step S608, the second position information of the object to be processed on the second image frame is determined, and the second image frame is intercepted based on the second position information to obtain a second sub-image corresponding to the second position information.

Optionally, the server may determine the second position information of the object to be processed on the second image frame through the object detection model, where the second position information may be four pixel pairs corresponding to the four corners of the rectangular frame ((X3, Y3 ), (X3, Y4), (X4, Y3), (X4, Y4)). Optionally, the object detection model may include, but is not limited to, a deep learning model such as a convolutional neural network, a recurrent neural network, or a recurrent neural network. Also, the server may perform image interception on the second image frame based on the second position information to obtain a second sub-image corresponding to the second position information.

In step S609, the second image frame is determined as the current image frame.

Since the loop of determining the second video segment is about to be entered, the server may determine the second image frame as the current image frame.

In step S610 , in the original video, an image frame at a preset distance from the current image frame is determined as an execution image frame according to the video playing sequence.

In the embodiment of the present application, the server may use the image frame in the original video that is one image frame away from the current image frame as the execution image frame according to the video playback sequence. That is, the server may regard the thirty-second image frame as an execution image frame.

In step S611, the execution location information in the execution image frame is determined based on the second location information, and the execution image frame is intercepted based on the execution location information in the execution image frame to obtain an execution sub-image corresponding to the execution location information.

In this embodiment of the present application, the server may determine the execution location information in the execution image frame based on the second location information, and perform image capture on the execution image frame based on the execution location information in the execution image frame to obtain the execution sub-image corresponding to the execution location information.

In step S612, the similarity data corresponding to the execution sub-image is determined based on the second sub-image and the execution sub-image.

Optionally, assuming that the second subimage and the execution subimage are subimages corresponding to 5*5 pixels, the server can determine the pixel pairs of the pixels on the second subimage and the execution subimage to obtain 25 pixel pairs . For example, on the second sub-image, the pixel whose row position is 1 and whose column position is 1 and the pixel whose row position is 1 and whose column position is 1 on the execution sub-image form a pixel pair, and on the second sub-image, row The pixel with position 5 and column position 5 and the pixel with row position 5 and column position 5 on the execution sub-image form a pixel pair.

Subsequently, the server may compare similarity based on each pixel pair to obtain similarity data corresponding to each pixel pair. Optionally, the similarity data may be percentage data. Furthermore, the server may determine the similarity data corresponding to the execution image frame based on the mean value of the similarity data of 25 pixel pairs.

In step S613, the similarity data corresponding to the execution sub-image is compared with the first preset data, and if the similarity data corresponding to the execution sub-image satisfies the first preset data, the execution image frame is determined as the current image frame; Go to step S610; otherwise, step S613.

In the embodiment of the present application, if the similarity data satisfies greater than or equal to 95%, it is determined that the similarity between the execution sub-image and the first sub-image is very high. Therefore, the execution sub-image includes the same of pending objects.

Based on this, the second image frame (the 31st image frame) and the execution image frame (the 32nd image frame) can be regarded as image frames in a video segment. Subsequently, it is judged whether the image frame following the execution image frame also belongs to the image frame in the video segment.

Taking the execution image frame as the 32nd image frame as an example, the server can regard the execution image frame as the current image frame, and then repeat steps S610-S613, that is, according to the order of the video, the image in the original video that is one image frame away from the current image frame frame (the 33rd image frame in the original video) is regarded as the execution image frame. Obtain the execution sub-image corresponding to the second position information in the 33rd image frame, determine the similarity data based on the second sub-image and the execution sub-image, and determine the execution sub-image and the execution sub-image if the similarity data is greater than or equal to 95%. The similarity between the two sub-images is very high, therefore, the execution sub-image in the 33rd image frame includes the same object to be processed as that in the second sub-image.

Based on this, the server may regard the execution image frame (the 33rd image frame) as the current image frame, and continue to repeat steps S610-S613. For the process of repeating steps, please refer to the above, and will not repeat them here.

In step S614, until the similarity data corresponding to the execution sub-image satisfies the second preset data, the second image frame is determined as the start image frame of the second video segment, and the previous image frame of the execution image frame is determined as The ending image frame of the second video clip, resulting in the second video clip.

Optionally, corresponding to the above first preset data, the second preset data may be less than 95%. For example, if the server executes the image frame after several rounds of repeated steps, it is the 61st image frame in the original video, and if the similarity data between the execution sub-image and the second sub-image corresponding to the 61st image frame satisfies less than 95%, the server can regard the second image frame as the start image frame of the second video clip, and the previous image frame of the execution image frame as the end image frame of the second video clip, and obtain the second video segment, that is, the second video segment includes the 30th image frame to the 60th image frame, a total of 30 image frames, so that the server determines the second video segment, and each image frame in the second video segment The position of the object to be processed on the image frame is within the preset position information corresponding to the first video segment, such as the lower right corner (with the lower right corner as a vertex, a 5*5 area).

In step S615, until the original video is traversed, at least one video segment is obtained; the at least one video segment includes the first video segment and the second video segment.

Subsequently, the server may perform the above steps until each image frame in the original video is traversed to obtain at least one video segment. The first image frame in each video segment is determined as a key frame in each video segment, or any image frame in each video segment is determined as a key frame in each video segment.

As above, through the first image frame, the first sub-image located in the first position information, and the similarity data between the execution sub-images corresponding to the execution position information in other image frames, the same situation will exist exactly The image frame of the object to be processed is divided into one video segment, and then the original video is divided into multiple video segments through the implementation of a large loop surrounded by a small loop. So that in the final stage, when the mask of the object to be processed in the key frame of the video clip needs to be used to remove the object to be processed in the video clip, the mask of the object to be processed in the key frame of the video clip can be quickly positioned to the The position of the object to be processed in the image frame in the video clip, and then remove the object to be processed for the video clip in a unified manner. Moreover, since the implementation of a large loop enclosing a small loop can be realized through a loop statement in the code, this embodiment of the present application can realize this function with less code.

In step S203, the object to be processed in the key frame and the position information of the object to be processed in the key frame in each video segment are determined.

In the embodiment of the present application, the server may determine the object to be processed in the key frame and the position information of the object to be processed in the key frame in each video segment based on the object detection model.

Fig. 7 is a flow chart of a key frame object to be processed and a method for determining position information of the key frame object to be processed according to an exemplary embodiment, as shown in Fig. 7 , including:

In step S701, based on the first detection model in the object detection model, the first type of object detection is performed on the key frame in each video clip, and the first type of object to be processed and the first type of object to be processed in the key frame in each video clip are determined. The location information of the object to be processed.

In the embodiment of the present application, the object detection model may include two modules, namely a first detection model and a second detection model. Among them, the object detection model is a trained model structure. Optionally, the server may invoke the object detection model, and perform first-type object detection on the key frames in each video segment based on the first detection model in the object detection model, and determine the first-type object detection of the key frames in each video segment The location information of the object to be processed and the first type of object to be processed.

In step S703, based on the second detection model in the object detection model and the position information of the first type of object to be processed, the key frame in each video segment is detected for the second type of object, and the key frame in each video segment is determined The second type of object to be processed and the location information of the second type of object to be processed.

Optionally, the objects to be processed may include different types of objects to be processed, such as objects to be processed of a first type and objects to be processed of a second type.

Since the object to be processed can not only include the first type of object to be processed, but also the second type of object to be processed, the key frame in each video clip can also be performed for the first time based on the second detection model in the object detection model. The second-type object detection is to determine the second-type object to be processed and the position information of the second-type object to be processed in the key frame of each video segment.

However, if only the second detection model in the object detection model is used to detect the second type of object on the key frame in each video segment, the second type of object to be processed in the key frame that is not the object to be processed may be regarded as the object to be processed. The second type of pending object in the processing object. For example, when the second type of object to be processed is text, the subtitle in the key frame may be detected as the text in the object to be processed. Therefore, the server can perform second-type object detection on key frames in each video clip based on the second detection model in the object detection model and the position information of the first type of object to be processed, and determine the position of the key frame in each video clip The second type of object to be processed and the location information of the second type of object to be processed.

In the embodiment of the present application, the location information of the first type of object to be processed is used to define the location information of the second type of object to be processed. Specifically, in the process of the second detection model detecting the second type of object to be processed, if there are multiple second type of objects to be processed on a key frame, the distance from the first type of object to be processed within the preset distance can be The object to be processed of the second type is determined as the object to be processed of the second type among the objects to be processed to be removed later. Other objects to be processed of the second type do not belong to the objects to be processed to be removed later.

Optionally, the server may determine a collection of the first type of object to be processed and the second type of object to be processed as the object to be processed. Optionally, the server may determine the combination of the location information of the first type of object to be processed and the location information of the second type of object to be processed as the location information of the object to be processed.

Wherein, the first detection model may include, but is not limited to, deep learning models such as convolutional neural networks, recurrent neural networks, or recurrent neural networks. The deep learning model involves machine learning (Machine Learning, ML). Machine learning is a multi-field interdisciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. Specializes in the study of how computers simulate or implement human learning behaviors to acquire new knowledge or skills, and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its application pervades all fields of artificial intelligence. Machine learning and deep learning usually include techniques such as artificial neural network, belief network, reinforcement learning, transfer learning, inductive learning, and teaching learning. Machine learning can be divided into supervised machine learning, unsupervised machine learning and semi-supervised machine learning.

Through the above-mentioned embodiment, it is possible to avoid treating the object to be processed of the second type in the key frame that is not the object to be processed as the object to be processed of the second type in the object to be processed, and accurately locate the object containing the object to be processed of the first type and the object to be processed in the second type. The object to be processed of the second type of object to be processed reduces the possibility of misidentifying the object to be processed.

In step S205, based on the object to be processed in each key frame and the position information of the object to be processed in each key frame, a mask of the object to be processed in each key frame is generated.

In the embodiment of this application, in order to retain more background information, make the video after removing the object to be processed natural and reduce the problem of video information omission, the server can analyze the position information of each key frame based on the position information of the object to be processed in each key frame. The pixels of the object to be processed are binarized to obtain the mask of the object to be processed in each key frame.

Optionally, the position information of the object to be processed may not be a complete rectangular area in the key frame, but a partial area in a complete rectangular local area. Therefore, the server can determine the rectangular area where the object is located based on the position information of the object to be processed, and the rectangular area is the area in the key frame.

Next, the server may intercept the rectangular area from the key frame, and then determine the position information of the object to be processed on the rectangular area based on the position information of the object to be processed and the position information of the rectangular area on the key frame. The server can binarize the pixels of the rectangular area based on the position information of the object to be processed on the rectangular area, for example, set the gray value of the pixel of the object to be processed on the rectangular area to 0, and set the gray value of the pixel of the object not to be processed to The grayscale value of the pixel is set to 255. In this way, the server can obtain the processed rectangular area, that is, the mask of the object to be processed for each key frame.

In step S207, object removal is performed on at least one video clip based on the at least one video clip and the mask of the object to be processed in each key frame to obtain a target video.

In the embodiment of the present application, the server can use the object removal model to remove the object to be processed based on at least one video segment and the mask of the object to be processed in each key frame to obtain the target video; the target video does not contain the object to be processed. Process objects.

Optionally, the server inputs at least one video segment and a mask of the object to be processed in each key frame into the object removal model, wherein the mask of the object to be processed carries position information of the rectangular area in the key frame. Then, on the basis of the object removal model, use the mask of the object to be processed in the key frame of each video clip to remove the object to be processed for the video clip corresponding to the key frame, and obtain the target video that does not contain the object to be processed.

Wherein, the object removal model may include but not limited to deep learning models such as convolutional neural network, recurrent neural network or recurrent neural network. The deep learning model involves machine learning (Machine Learning, ML). Machine learning is a multi-field interdisciplinary subject, involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. Specializes in the study of how computers simulate or implement human learning behaviors to acquire new knowledge or skills, and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its application pervades all fields of artificial intelligence. Machine learning and deep learning usually include techniques such as artificial neural network, belief network, reinforcement learning, transfer learning, inductive learning, and teaching learning. Machine learning can be divided into supervised machine learning, unsupervised machine learning and semi-supervised machine learning.

In the embodiment of the present application, the original video may contain a video tail segment, and the video tail segment may not contain any content, but only a black image frame that lasts for nearly half a second, for example. If the server obtains the instruction to delete the credits, the server can identify the credits of the original video, and then delete them.

Fig. 8 is a flow chart of a method for deleting a credits trailer according to an exemplary embodiment, as shown in Fig. 8 , including:

In step S801, the preset color proportion detection is performed on the original video, and the proportion data of the preset color of each image frame in the original video is obtained.

In the embodiment of the present application, the server may detect the ratio of preset colors in the original video to obtain the ratio data of the preset color of each image frame in the original video. For example, assuming that the default color is black, the server can detect the proportion of black pixels in each image frame to obtain the proportion data of black pixels in each image frame.

In step S803, if the ratio data of the preset colors of multiple image frames in the original video satisfies the third preset data, determine the start and end times of the multiple image frames, wherein the multiple image frames are continuous image frames, And the last image frame of the plurality of image frames is the video end image frame.

In the embodiment of the present application, if the ratio data of preset colors of multiple image frames in the original video satisfies the third preset data (such as 80%), wherein, the multiple image frames are continuous image frames, and more The last image frame of the two image frames is the video end image frame, and the server can determine that multiple image frames form the video end segment. Then, the server can determine the start and end times of the video tail segment.

In step S805, a plurality of image frames are deleted from the original video based on the start and end time based on the end credit deletion instruction to obtain an updated original video.

In the embodiment of the present application, the server may delete multiple image frames from the original video based on the start and end time based on the end credit deletion instruction, to obtain an updated original video.

Optionally, deleting the end credits can be performed before segmenting and extracting key frames from the original video to obtain at least one video segment and key frames in each video segment, or before segmenting and extracting key frames from the original video, The method is implemented after obtaining at least one video segment and a key frame in each video segment.

As above, by locating the credits composed of multiple image frames, the credits can be deleted according to the credits deletion instruction, and it is no longer necessary to detect and remove objects to be processed for the credits, saving computing resources.

To sum up, this application can accurately locate the position of the object to be processed in each video clip through the video clips and key frames, and can accurately divide the image frames with the same object to be processed according to the similarity data between image frames. A video clip, so that in the final stage, when it is necessary to use the mask of the object to be processed in the key frame of the video clip to remove the object to be processed in the video clip, it can be because of the mask of the object to be processed in the key frame of the video clip Quickly locate the position of the object to be processed in the image frame in the video clip, and then uniformly remove the object to be processed in the video clip, reduce the computing resources required for the removal of the object to be processed in the subsequent video, and can be applied to more Videos with multiple objects to be processed are more universal.

Fig. 9 is a block diagram of a video processing device according to an exemplary embodiment. The device has the function of realizing the data processing method in the above method embodiment, and the function may be realized by hardware, or may be realized by hardware executing corresponding software. Referring to Figure 9, the device includes:

The segmentation module 901 is configured to perform segmentation and key frame extraction on the original video to obtain at least one video segment and key frames in each video segment; wherein, the image frames in the same video segment contain objects to be processed The position information on the image frame to which the object to be processed belongs satisfies the preset position information; the preset position information is the information corresponding to the video segment to which the image frame belongs;

The determination module 902 is configured to determine the object to be processed in the key frame and the position information of the object to be processed in the key frame in each video segment;

The mask generating module 903 is configured to generate a mask of the object to be processed in each key frame based on the object to be processed in each key frame and the position information of the object to be processed in each key frame;

The object removal module 904 is configured to perform object processing on at least one video clip based on the at least one video clip and the mask of the object to be processed in each key frame, to obtain a target video.

In some possible embodiments, the segmentation module is configured to:

Perform object recognition on the original video to obtain the type information of the object to be processed in the original video;

Determine the display rule of the object to be processed on the original video based on the type information of the object to be processed;

Segmenting the original video based on display rules to obtain at least one video segment;

A preset image frame in each video segment of the at least one video segment is determined as a key frame in each video segment.

In some possible embodiments, the segmentation module is configured to:

Determine the display area and display duration of the object to be processed on the original video based on the type information of the object to be processed;

The original video is segmented based on the display area and display duration to obtain at least one video segment.

In some possible embodiments, the segmentation module is configured to:

Object recognition is carried out to the first image frame in the original video to obtain the object to be processed in the original video; the first image frame is the image frame in which the object to be processed appears for the first time in the original video;

determining first position information of the object to be processed on the first image frame;

performing image interception on the first image frame based on the first position information, to obtain a first sub-image corresponding to the first position information;

Determine the similarity data corresponding to each second image frame based on the first sub-image and each second image frame in the second image frame set; the second image frame set includes image frames other than the first image frame in the original video ;

Segmenting the original video based on the similarity data corresponding to each second image frame to obtain at least one video segment;

A preset image frame in each video segment of the at least one video segment is determined as a key frame in each video segment.

In some possible embodiments, the segmentation-based module is configured to perform:

acquiring a second sub-image corresponding to the first position information in each second image frame;

Based on the degree of similarity between each second sub-image and the first sub-image, determine similarity data corresponding to each second image frame;

If the similarity data corresponding to each second image frame satisfies the preset data, a video segment is obtained; the first image frame of the video segment is the first image frame.

In some possible embodiments, the segmentation module is configured to:

If there is a first target image frame set in the second image frame set, and the position of the first target image frame set is first, the first target image frame is adjacent to the first image frame, and is located after the first image frame in the original video , determining the first video segment based on the first image frame and the first target image frame set;

The first target image frame set includes a first target image frame or a plurality of continuous first target image frames; the first target image frame set is not equal to the second image frame set, and the first target image frame in the first target image frame set The corresponding similarity data satisfies the preset data.

In some possible embodiments, the segmentation module is configured to:

Taking the difference between the second image frame set and the first target image frame set as the video to be segmented;

The first image frame of the video to be segmented is a new first image frame; in the video to be segmented, the image frames other than the new first image frame are a new second image frame set;

Determining new first position information of the object to be processed on the new first image frame;

From the new first image frame, obtain the new first sub-image corresponding to the new first position information, and obtain the new first sub-image corresponding to the new first position information from each second image frame in the new second image frame set The new second sub-image of ;

determining similarity data for each new second sub-image based on the degree of similarity between the new first sub-image and each new second sub-image;

If there is a new first target image frame set in the new second image frame set, and in the new first target image frame set, the first target image frame at the first position is adjacent to the new first image frame, and in After the new first image frame in the original video, the second video segment is determined based on the new first image frame and the new first target image frame set;

The new first target image frame set includes a new first target image frame or a plurality of continuous new first target image frames; the new first target image frame set is not equal to the new second image frame set, and the new The similarity data corresponding to the first target image frames in the first target image frame set satisfies the preset data.

In some possible embodiments, the device further includes a time determination module configured to:

Preset color proportion detection is carried out in the original video, and the proportion data of the preset color of each image frame in the original video is obtained;

If the ratio data of the preset colors of multiple image frames in the original video satisfies the third preset data, determine the start and end times of the multiple image frames; wherein, the multiple image frames are continuous image frames, and the multiple image frames The last image frame of is the video end image frame.

In some possible embodiments, the determination module is configured to execute:

Based on the first detection model in the object detection model, the first type of object detection is carried out to the key frame in each video clip, and the first type of object to be processed and the first type of object to be processed in the key frame in each video clip are determined. location information;

Based on the second detection model in the object detection model and the position information of the first type of object to be processed, the key frame in each video segment is detected for the second type of object, and the second type of the key frame in each video segment is determined to be processed. Location information of the processing object and the second type of object to be processed.

In some possible embodiments, the mask generation module is configured to:

Based on the position information of the object to be processed in each key frame, the pixels of the object to be processed in each key frame are binarized to obtain the mask of the object to be processed in each key frame.

It should be noted that, when realizing the functions of the device provided by the above-mentioned embodiments, the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional modules according to the needs. The internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and the method embodiment provided by the above embodiment belong to the same idea, and the specific implementation process thereof is detailed in the method embodiment, and will not be repeated here.

Fig. 10 is a block diagram of an apparatus 3000 for video processing according to an exemplary embodiment. For example, the apparatus 3000 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, and the like.

10, the device 3000 may include one or more of the following components: a processing component 3002, a memory 3004, a power component 3006, a multimedia component 3008, an audio component 3010, an input/output (I/O) interface 3012, a sensor component 3014, and a communication component 3016.

The processing component 3002 generally controls the overall operations of the device 3000, such as those associated with display, telephone calls, data communications, camera operations, and recording operations. The processing component 3002 may include one or more processors 3020 to execute instructions to complete all or part of the steps of the above method. Additionally, processing component 3002 may include one or more modules that facilitate interaction between processing component 3002 and other components. For example, processing component 3002 may include a multimedia module to facilitate interaction between multimedia component 3008 and processing component 3002 .

The memory 3004 is configured to store various types of data to support operations at the device 3000 . Examples of such data include instructions for any application or method operating on device 3000, contact data, phonebook data, messages, pictures, videos, and the like. The memory 3004 can be implemented by any type of volatile or non-volatile storage device or their combination, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable Programmable Read Only Memory (EPROM), Programmable Read Only Memory (PROM), Read Only Memory (ROM), Magnetic Memory, Flash Memory, Magnetic or Optical Disk.

Power component 3006 provides power to various components of device 3000 . Power components 3006 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for device 3000 .

The multimedia component 3008 includes a screen that provides an output interface between the device 3000 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensor may not only sense a boundary of a touch or swipe action, but also detect duration and pressure associated with the touch or swipe action. In some embodiments, the multimedia component 3008 includes a front camera and/or a rear camera. When the device 3000 is in an operation mode, such as a shooting mode or a video mode, the front camera and/or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capability.

The audio component 3010 is configured to output and/or input audio signals. For example, the audio component 3010 includes a microphone (MIC), which is configured to receive external audio signals when the device 3000 is in operation modes, such as call mode, recording mode and voice recognition mode. Received audio signals may be further stored in memory 3004 or sent via communication component 3016 . In some embodiments, the audio component 3010 also includes a speaker for outputting audio signals.

The I/O interface 3012 provides an interface between the processing component 3002 and a peripheral interface module, which may be a keyboard, a click wheel, a button, and the like. These buttons may include, but are not limited to: a home button, volume buttons, start button, and lock button.

Sensor assembly 3014 includes one or more sensors for providing status assessments of various aspects of device 3000 . For example, the sensor component 3014 can detect the open/closed state of the device 3000, the relative positioning of components, such as the display and keypad of the device 3000, and the sensor component 3014 can also detect a change in the position of the device 3000 or a component of the device 3000 , the presence or absence of user contact with the device 3000, the device 3000 orientation or acceleration/deceleration and the temperature change of the device 3000. Sensor assembly 3014 may include a proximity sensor configured to detect the presence of nearby objects in the absence of any physical contact. The sensor assembly 3014 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor component 3014 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

The communication component 3016 is configured to facilitate wired or wireless communication between the apparatus 3000 and other devices. The device 3000 can access wireless networks based on communication standards, such as WiFi, operator networks (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 3016 receives broadcast signals or broadcast related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 3016 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on Radio Frequency Identification (RFID) technology, Infrared Data Association (IrDA) technology, Ultra Wide Band (UWB) technology, Bluetooth (BT) technology and other technologies.

In an exemplary embodiment, apparatus 3000 may be programmed by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable A gate array (FPGA), controller, microcontroller, microprocessor or other electronic component implementation for performing the methods described above.

An embodiment of the present invention also provides a computer-readable storage medium, which can be set in an electronic device to store at least one instruction or at least one program related to implementing a video processing method, The at least one instruction or the at least one section of program is loaded and executed by the processor to implement the video processing method provided by the foregoing method embodiments.

Embodiments of the present invention also provide a computer program product, the computer program product includes a computer program, the computer program is stored in a readable storage medium, at least one processor of the computer device reads and executes the computer program from the readable storage medium , so that the computer device executes any one of the methods in the first aspect of the embodiments of the present disclosure.

It should be noted that: the order of the above embodiments of the present invention is only for description, and does not represent the advantages and disadvantages of the embodiments. And the above describes the specific embodiments of this specification. Other implementations are within the scope of the following claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve desirable results. In addition, the processes depicted in the accompanying figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. Multitasking and parallel processing are also possible or may be advantageous in certain embodiments.

Each embodiment in this specification is described in a progressive manner, the same and similar parts of each embodiment can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, as for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and for relevant parts, please refer to part of the description of the method embodiment.

Those of ordinary skill in the art can understand that all or part of the steps for implementing the above embodiments can be completed by hardware, and can also be completed by instructing related hardware through a program. The program can be stored in a computer-readable storage medium. The above-mentioned The storage medium mentioned may be a read-only memory, a magnetic disk or an optical disk, and the like.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within range.

Claims (14)

1. A video processing method, comprising:

segmenting an original video and extracting key frames to obtain at least one video clip and the key frames in each video clip; the position information of an object to be processed contained in image frames in the same video clip on the image frame to which the object to be processed belongs meets preset position information; the preset position information is information corresponding to a video clip to which the image frame belongs;

determining an object to be processed of a key frame in each video clip and position information of the object to be processed of the key frame;

generating a mask of the object to be processed of each key frame based on the object to be processed of each key frame and the position information of the object to be processed of each key frame;

and carrying out object processing on the at least one video clip based on the at least one video clip and the mask of the object to be processed of each key frame to obtain a target video.

2. The video processing method according to claim 1, wherein the segmenting and key frame extracting the original video to obtain at least one video segment and a key frame in each video segment comprises:

carrying out object identification on the original video to obtain the type information of an object to be processed of the original video;

determining a display rule of the object to be processed on the original video based on the type information of the object to be processed;

segmenting the original video based on the display rule to obtain at least one video segment;

determining a preset image frame in each of the at least one video clip as a key frame in each of the at least one video clip.

3. The video processing method according to claim 2, wherein the determining a display rule of the object to be processed on the original video based on the type information of the object to be processed, and segmenting the original video based on the display rule to obtain the at least one video segment comprises:

determining a display area and a display duration of the object to be processed on the original video based on the type information of the object to be processed;

and segmenting the original video based on the display area and the display duration to obtain the at least one video segment.

4. The video processing method according to claim 1, wherein the segmenting and key frame extracting the original video to obtain at least one video segment and a key frame in each video segment comprises:

carrying out object identification on a first image frame in the original video to obtain an object to be processed of the original video; the first image frame is an image frame of the original video where the object to be processed appears for the first time;

determining first position information of the object to be processed on the first image frame;

image interception is carried out on the first image frame based on the first position information, and a first sub-image corresponding to the first position information is obtained;

determining similarity data corresponding to each second image frame in the second image frame set based on the first sub-image and the each second image frame; the second image frame set comprises image frames except the first image frame in the original video;

segmenting the original video based on the similarity data corresponding to each second image frame to obtain at least one video segment;

determining a preset image frame in each of the at least one video clip as a key frame in each of the at least one video clip.

5. The video processing method according to claim 4, wherein determining similarity data corresponding to each second image frame based on the first sub-image and each second image frame, and segmenting the original video based on the similarity data corresponding to each second image frame to obtain the at least one video segment comprises:

acquiring a second sub-image corresponding to the first position information in each second image frame;

determining similarity data corresponding to each second image frame based on the similarity degree of each second sub-image and the first sub-image;

if the similarity data corresponding to each second image frame meets preset data, obtaining a video clip; the first image frame of the video clip is the first image frame.

6. The video processing method according to claim 5, wherein after determining the similarity data corresponding to each second image frame based on the similarity between each second sub-image and the first sub-image, the method further comprises:

if a first target image frame set exists in the second image frame set, a first target image frame positioned at a first position in the first target image frame set is adjacent to the first image frame, and the first target image frame set is positioned behind the first image frame in the original video, determining a first video segment based on the first image frame and the first target image frame set;

the first target image frame set comprises a first target image frame or a plurality of continuous first target image frames; the first target image frame set is not equal to the second image frame set, and the similarity data corresponding to a first target image frame in the first target image frame set meets the preset data.

7. The video processing method of claim 6, wherein the method further comprises:

taking a difference set between the second image frame set and the first target image frame set as a video to be segmented;

taking a first image frame of the video to be segmented as a new first image frame; taking image frames except the new first image frame in the video to be segmented as a new second image frame set;

determining new first position information of the object to be processed on the new first image frame;

obtaining a new first sub-image corresponding to the new first position information from the new first image frame, and obtaining a new second sub-image corresponding to the new first position information from each second image frame in the new second image frame set;

determining similarity data for each new second sub-image based on the similarity of the new first sub-image and each new second sub-image;

if a new first target image frame set exists in the new second image frame set and the new first target image frame set is positioned adjacent to the first image frame and is positioned after the new first image frame in the original video, determining a second video segment based on the new first image frame and the new first target image frame set;

the new first target image frame set comprises a new first target image frame or a plurality of consecutive new first target image frames; the new first target image frame set is not equal to the new second image frame set, and the similarity data corresponding to the first target image frame in the new first target image frame set meets the preset data.

8. The video processing method according to any of claims 1-7, wherein the method further comprises:

performing preset color proportion detection on the original video to obtain proportion data of preset colors of each image frame in the original video;

if the proportion data of the preset colors of a plurality of image frames in the original video meet third preset data, determining the starting and ending time of the plurality of image frames; wherein the plurality of image frames are consecutive image frames, and a last image frame of the plurality of image frames is a video end image frame.

9. The video processing method according to claim 1, wherein the objects to be processed include a first class of objects to be processed and a second class of objects to be processed, and the determining the objects to be processed of the key frames and the position information of the objects to be processed of the key frames in each video clip includes:

performing first-class object detection on key frames in each video clip based on a first detection model in object detection models, and determining first-class objects to be processed of the key frames in each video clip and position information of the first-class objects to be processed;

and performing second-class object detection on the key frame in each video clip based on a second detection model in the object detection model and the position information of the first-class object to be processed, and determining the second-class object to be processed and the position information of the second-class object to be processed of the key frame in each video clip.

10. The video processing method according to any one of claims 1 to 7 and 9, wherein the generating a mask of the object to be processed of each key frame based on the object to be processed of each key frame and the position information of the object to be processed of each key frame comprises:

and performing binarization processing on the pixels of the object to be processed of each key frame based on the position information of the object to be processed of each key frame to obtain a mask of the object to be processed of each key frame.

11. A video processing apparatus, comprising:

the segmentation module is configured to segment the original video and extract key frames to obtain at least one video segment and key frames in each video segment; the method comprises the steps that position information of an object to be processed contained in image frames in the same video clip on the image frame to which the object to be processed belongs meets preset position information; the preset position information is information corresponding to a video clip to which the image frame belongs;

a determining module configured to perform determining an object to be processed of a key frame and position information of the object to be processed of the key frame in each video clip;

a mask generation module configured to perform a mask generation of the object to be processed of each key frame based on the object to be processed of each key frame and the position information of the object to be processed of each key frame;

and the object removing module is configured to perform object processing on the at least one video segment based on the at least one video segment and the mask of the object to be processed of each key frame to obtain a target video.

12. An electronic device, comprising:

a processor;

a memory for storing the processor-executable instructions;

wherein the processor is configured to execute the instructions to implement the video processing method of any of claims 1 to 10.

13. A computer-readable storage medium, wherein instructions in the computer-readable storage medium, when executed by a processor of an electronic device, enable the electronic device to perform the video processing method of any of claims 1 to 10.

14. A computer program product, characterized in that the computer program product comprises a computer program, which is stored in a readable storage medium, from which at least one processor of a computer device reads and executes the computer program, causing the computer device to perform the video processing method or the video processing method according to any one of claims 1 to 10.

Images