CN108259781B

CN108259781B - Video synthesis method, terminal and computer-readable storage medium

Info

Publication number: CN108259781B
Application number: CN201711450874.7A
Authority: CN
Inventors: 朱艺师
Original assignee: Nubia Technology Co Ltd
Current assignee: Nubia Technology Co Ltd
Priority date: 2017-12-27
Filing date: 2017-12-27
Publication date: 2021-01-26
Anticipated expiration: 2037-12-27
Also published as: CN108259781A

Abstract

The invention discloses a video synthesis method, a terminal and a computer readable storage medium, wherein the terminal can synthesize contents in multiple sections of videos into one section of video so as to play or share the videos; the terminal respectively extracts a target object in each video frame of each video by aiming at each video to be synthesized; then, target objects of video frames corresponding to video frame numbers (representing positions of the video frames in the videos) among the videos are synthesized into a new video frame, and the obtained new video frames are combined in sequence to obtain a synthesized video; the invention also discloses a terminal and a computer readable storage medium, by implementing the scheme, the content in the multiple sections of videos is synthesized into one section of video, the processing modes of the terminal on the video and the display forms of the video content are enriched, the synthesized picture can be more vivid and interesting, and especially the video recording and sharing experience of users can be improved.

Description

Video synthesis method, terminal and computer-readable storage medium

Technical Field

The present invention relates to the field of communications technologies, and in particular, to a video synthesis method, a terminal, and a computer-readable storage medium.

Background

With the development of intelligent terminal technology, the current intelligent terminals all support the functions of recording, playing and sharing videos. At present, the intelligent terminal has a single video processing mode, and is generally limited to recording a video, playing the video on line or playing a local video. For example, one or more videos are recorded through a camera, and then are played or shared locally to other users, for example, a small video can be recorded through a friend circle of WeChat for sharing. At present, a single video processing mode of an intelligent terminal causes that a user cannot perform richer secondary processing on recorded and shared videos on the intelligent terminal, so that the video content forms recorded and shared by the user are uniform, and the satisfaction degree of user experience is poor.

Disclosure of Invention

The technical problem to be solved by the invention is as follows: the existing terminal cannot carry out secondary processing on video content, so that the user experience satisfaction is poor. In view of the technical problem, a video synthesis method, a terminal and a computer-readable storage medium are provided.

In order to solve the above technical problem, the present invention provides a video synthesis method, including the following steps:

for each video to be synthesized, respectively extracting a target object in each video frame of each video;

synthesizing target objects of video frames corresponding to the video frame numbers among the videos into a new video frame, and sequentially combining the obtained new video frames to obtain a synthesized video;

the video frame number is the position of the video frame within the video.

Optionally, the background images in the video frames of one video are the same, and the step of extracting the target object in each video frame of each video respectively includes:

for each video, extracting each video frame of the video;

and comparing the extracted image in each video frame with the background image of the video frame, and extracting the object which is not in the background image as the target object of each video frame.

Optionally, the background image in each video frame of each video is the same.

Optionally, the number of video frames included in each video to be synthesized is the same.

Optionally, the step of synthesizing target objects of video frames corresponding to the video frame numbers between the videos into a new video frame includes:

and synthesizing the target object of the video frame corresponding to the video frame number among the videos in a preset background image of the new video frame to obtain the new video frame.

Optionally, the new video frame background image is the same as the background image within each video frame of at least one of the videos.

Optionally, the step of synthesizing the target object of the video frame corresponding to the video frame number between the videos in a preset background image of the new video frame to obtain the new video frame includes:

determining the position of a target object of each video frame corresponding to the video frame number in the background image of the new video frame, wherein the positions of the target object of each video frame in the background image of the new video frame are not overlapped;

and synthesizing each target object in the background image of the new video frame according to the position of the target object of each video frame in the background image of the new video frame.

Optionally, the step of determining the position of the target object of each video frame corresponding to the video frame number in the background image of the new video frame includes:

acquiring the original positions of the target objects in the corresponding video frames;

judging whether the original positions of the target objects are overlapped, if not, taking the original positions of the target objects as the positions of the target objects in the background image of the new video frame;

if there is overlap, assigning a new position in the background image of the new video frame for at least one target object whose original positions are overlapped, and using the original position of the target object without overlap as its position in the background image of the new video frame.

Further, the invention also provides a terminal, which comprises a processor, a memory and a communication bus;

the communication bus is used for realizing communication connection between the processor and the memory;

the processor is configured to execute one or more programs stored in the memory to implement the steps of the video compositing method as described above.

Further, the present invention also provides a computer-readable storage medium storing one or more programs, which are executable by one or more processors to implement the steps of the video composition method as described above.

Advantageous effects

According to the video synthesis method, the terminal and the computer-readable storage medium, the terminal can synthesize the contents in the multiple sections of videos into one section of video, so that the contents can be played or shared; the terminal respectively extracts a target object in each video frame of each video by aiming at each video to be synthesized; then, target objects of video frames corresponding to video frame numbers (representing positions of the video frames in the videos) among the videos are synthesized into a new video frame, and the obtained new video frames are combined in sequence to obtain a synthesized video, namely, the content in a plurality of sections of videos is synthesized into one section of video, so that the processing mode of a terminal on the video and the showing mode of the video content are enriched, the synthesized picture can be more vivid and interesting, and especially the video recording and sharing experience of a user can be improved.

Drawings

The invention will be further described with reference to the accompanying drawings and examples, in which:

fig. 1 is a schematic diagram of a hardware structure of an alternative mobile terminal for implementing various embodiments of the present invention;

FIG. 2 is a diagram of a wireless communication system for the mobile terminal shown in FIG. 1;

fig. 3 is a schematic flow chart of a video synthesis method according to a first embodiment of the present invention;

FIG. 4 is a flowchart illustrating a process of extracting a target object in a video frame of each video according to a second embodiment of the present invention;

fig. 5 is a schematic flow chart of synthesizing a new video frame according to a second embodiment of the present invention;

fig. 6 is a schematic flowchart of a second embodiment of the present invention for confirming the position of a target object in a background image of a new video frame;

fig. 7 is a schematic structural diagram of a terminal according to a third embodiment of the present invention;

fig. 8 is a schematic view of an interface for a terminal to enter video shooting according to a third embodiment of the present invention;

fig. 9 is a schematic diagram of a terminal according to a third embodiment of the present invention after entering a video shooting composition mode;

fig. 10 is a schematic diagram of a terminal prompting a first video shooting interface according to a third embodiment of the present invention;

fig. 11 is a schematic diagram of a second video shooting interface prompted by a terminal according to a third embodiment of the present invention.

Detailed Description

It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

In the following description, suffixes such as "module", "component", or "unit" used to denote elements are used only for facilitating the explanation of the present invention, and have no specific meaning in itself. Thus, "module", "component" or "unit" may be used mixedly.

The terminal may be implemented in various forms. For example, the terminal described in the present invention may include mobile terminals such as a mobile phone, a tablet computer, a notebook computer, a palmtop computer, a Personal Digital Assistant (PDA), a Portable Media Player (PMP), a navigation device, a wearable device, a smart band, a pedometer, and fixed terminals such as a Digital TV, a desktop computer, and the like, and the terminal in the present invention may be various bendable flexible screen terminals made using a flexible screen.

While the bendable mobile terminal will be described as an example in the following description, those skilled in the art will appreciate that the configuration according to the embodiment of the present invention can be applied to a fixed type of bendable terminal, in addition to elements particularly used for moving purposes.

Referring to fig. 1, which is a schematic diagram of a hardware structure of a mobile terminal for implementing various embodiments of the present invention, the mobile terminal 100 may include: RF (Radio Frequency) unit 101, WiFi module 102, audio output unit 103, a/V (audio/video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110, and power supply 111. Those skilled in the art will appreciate that the mobile terminal structure shown in fig. 1 does not constitute a limitation of the mobile terminal, that the mobile terminal may include more or less components than those shown, or some components may be combined, or different component arrangements may be adopted, and that the mobile terminal may be adapted to change the positions, materials and functions of the components according to its own bending performance.

The following describes each component of the mobile terminal in detail with reference to fig. 1:

the radio frequency unit 101 may be configured to receive and transmit signals during information transmission and reception or during a call, and specifically, receive downlink information of a base station and then process the downlink information to the processor 110; in addition, the uplink data is transmitted to the base station. Typically, radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio frequency unit 101 can also communicate with a network and other devices through wireless communication. The wireless communication may use any communication standard or protocol, including but not limited to GSM (Global System for Mobile communications), GPRS (General Packet Radio Service), CDMA2000(Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division duplex Long Term Evolution), and TDD-LTE (Time Division duplex Long Term Evolution).

WiFi belongs to short-distance wireless transmission technology, and the mobile terminal can help a user to receive and send e-mails, browse webpages, access streaming media and the like through the WiFi module 102, and provides wireless broadband internet access for the user. Although fig. 1 shows the WiFi module 102, it is understood that it does not belong to the essential constitution of the mobile terminal, and may be omitted entirely as needed within the scope not changing the essence of the invention.

The audio output unit 103 may convert audio data received by the radio frequency unit 101 or the WiFi module 102 or stored in the memory 109 into an audio signal and output as sound when the mobile terminal 100 is in a call signal reception mode, a call mode, a recording mode, a voice recognition mode, a broadcast reception mode, or the like. Also, the audio output unit 103 may also provide audio output related to a specific function performed by the mobile terminal 100 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 103 may include a speaker, a buzzer, and the like.

The a/V input unit 104 is used to receive audio or video signals. The a/V input Unit 104 may include a Graphics Processing Unit (GPU) 1041 and a microphone 1042, the Graphics processor 1041 Processing image data of still pictures or video obtained by an image capturing device (e.g., a camera) in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 106. The image frames processed by the graphic processor 1041 may be stored in the memory 109 (or other storage medium) or transmitted via the radio frequency unit 101 or the WiFi module 102. The microphone 1042 may receive sounds (audio data) via the microphone 1042 in a phone call mode, a recording mode, a voice recognition mode, or the like, and may be capable of processing such sounds into audio data. The processed audio (voice) data may be converted into a format output transmittable to a mobile communication base station via the radio frequency unit 101 in case of a phone call mode. The microphone 1042 may implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated in the course of receiving and transmitting audio signals.

The mobile terminal 100 also includes at least one sensor 105, such as a light sensor, motion sensor, image capture sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor that can adjust the brightness of the display panel 1061 according to the brightness of ambient light, and a proximity sensor that can turn off the display panel 1061 and/or a backlight when the mobile terminal 100 is moved to the ear. As one of the motion sensors, the accelerometer sensor can detect the magnitude of acceleration in each direction (generally, three axes), can detect the magnitude and direction of gravity when stationary, and can be used for applications of recognizing the posture of a mobile phone (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), vibration recognition related functions (such as pedometer and tapping), and the like; as for other sensors such as a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor and the like which can be configured on the mobile phone, the description is omitted; the image acquisition sensor can be characterized as various cameras capable of realizing image shooting and video recording.

The display unit 106 is used to display information input by a user or information provided to the user. The display unit 106 may include a display panel 1061, and the display panel 1061 may be configured by using a common screen, or various flexible screens, such as an Organic Light-Emitting Diode (OLED), an Active-matrix Organic Light-Emitting Diode (AMOLED), and the like. In this embodiment, one terminal may have one display panel, or a plurality of display panels may be provided, for example, a flexible screen having a front surface and a back surface, respectively.

The user input unit 107 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the mobile terminal. Specifically, the user input unit 107 may include a bendable touch panel 1071 and other input devices 1072. The flexible touch panel 1071, also called a flexible touch screen, can collect touch operations of a user (for example, operations of the user on or near the touch panel 1071 using any suitable object or accessory such as a finger, a stylus pen, etc.) and drive the corresponding connection device according to a preset program. The touch panel 1071 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 110, and can receive and execute commands sent by the processor 110. In addition, the touch panel 1071 may be implemented in various types, such as a resistive type, a capacitive type, an infrared ray, and a surface acoustic wave. In addition to the touch panel 1071, the user input unit 107 may include other input devices 1072. In particular, other input devices 1072 may include, but are not limited to, one or more of a physical keyboard, function keys (e.g., volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like, and are not limited to these specific examples.

Further, the touch panel 1071 may cover the display panel 1061, and when the touch panel 1071 detects a touch operation thereon or nearby, the touch panel 1071 transmits the touch operation to the processor 110 to determine the type of the touch event, and then the processor 110 provides a corresponding visual output on the display panel 1061 according to the type of the touch event. Although the touch panel 1071 and the display panel 1061 are shown in fig. 1 as two separate components to implement the input and output functions of the mobile terminal, in some embodiments, the touch panel 1071 and the display panel 1061 may be integrated to implement the input and output functions of the mobile terminal, and is not limited herein.

The interface unit 108 serves as an interface through which at least one external device is connected to the mobile terminal 100. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 108 may be used to receive input (e.g., data information, power, etc.) from external devices and transmit the received input to one or more elements within the mobile terminal 100 or may be used to transmit data between the mobile terminal 100 and external devices.

The memory 109 may be used to store software programs as well as various data. The memory 109 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. Further, the memory 109 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The processor 110 is a control center of the mobile terminal, connects various parts of the entire mobile terminal using various interfaces and lines, and performs various functions of the mobile terminal and processes data by operating or executing software programs and/or modules stored in the memory 109 and calling data stored in the memory 109, thereby performing overall monitoring of the mobile terminal. Processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 110.

The mobile terminal 100 may further include a power supply 111 (e.g., a battery) for supplying power to various components, and preferably, the power supply 111 may be logically connected to the processor 110 via a power management system, so as to manage charging, discharging, and power consumption management functions via the power management system.

Although not shown in fig. 1, the mobile terminal 100 may further include a bluetooth module or the like, which is not described in detail herein.

In order to facilitate understanding of the embodiments of the present invention, a communication network system on which the mobile terminal of the present invention is based is described below.

Referring to fig. 2, fig. 2 is an architecture diagram of a communication Network system according to an embodiment of the present invention, where the communication Network system is an LTE system of a universal mobile telecommunications technology, and the LTE system includes a UE (User Equipment) 201, an E-UTRAN (Evolved UMTS Terrestrial Radio Access Network) 202, an EPC (Evolved Packet Core) 203, and an IP service 204 of an operator, which are in communication connection in sequence.

Specifically, the UE201 may be the terminal 100 described above, and is not described herein again.

The E-UTRAN202 includes eNodeB2021 and other eNodeBs 2022, among others. Among them, the eNodeB2021 may be connected with other eNodeB2022 through backhaul (e.g., X2 interface), the eNodeB2021 is connected to the EPC203, and the eNodeB2021 may provide the UE201 access to the EPC 203.

The EPC203 may include an MME (Mobility Management Entity) 2031, an HSS (Home Subscriber Server) 2032, other MMEs 2033, an SGW (Serving gateway) 2034, a PGW (PDN gateway) 2035, and a PCRF (Policy and Charging Rules Function) 2036, and the like. The MME2031 is a control node that handles signaling between the UE201 and the EPC203, and provides bearer and connection management. HSS2032 is used to provide registers to manage functions such as home location register (not shown) and holds subscriber specific information about service characteristics, data rates, etc. All user data may be sent through SGW2034, PGW2035 may provide IP address assignment for UE201 and other functions, and PCRF2036 is a policy and charging control policy decision point for traffic data flow and IP bearer resources, which selects and provides available policy and charging control decisions for a policy and charging enforcement function (not shown).

The IP services 204 may include the internet, intranets, IMS (IP Multimedia Subsystem), or other IP services, among others.

Although the LTE system is described as an example, it should be understood by those skilled in the art that the present invention is not limited to the LTE system, but may also be applied to other wireless communication systems, such as GSM, CDMA2000, WCDMA, TD-SCDMA, and future new network systems.

Based on the above mobile terminal hardware structure and communication network system, the present invention provides various embodiments of the method.

First embodiment

The video synthesis method provided by the embodiment is suitable for various intelligent terminals, particularly suitable for various mobile intelligent terminals, and also suitable for non-mobile intelligent terminals. The video synthesis method provided in this embodiment is shown in fig. 3, and includes:

s301: and respectively extracting the target object in each video frame of each video for each video to be synthesized.

In this embodiment, the video to be synthesized includes at least two segments, and may specifically be two segments, three segments, or more than three segments, and the specific number of the videos may support user-defined or may be set by default by the system.

In this embodiment, the video source of the video to be synthesized may also be flexibly set, for example, the video may be obtained by shooting through the terminal itself, or may be obtained from another terminal or a network side. For example, in one example, it is assumed that there are two pieces of video to be combined, where both pieces may be video obtained by shooting by the terminal itself, or both pieces may be video obtained from other terminals or networks, or one piece may be video obtained by shooting by the terminal and the other piece may be video obtained from other terminals or networks.

It should be understood that, in the present embodiment, the target object may support the user to customize and set the object of interest, and may also be set by the system. The object of interest may be at least one of a person, a scene, an animal, a thing. It should be understood that, in the present embodiment, when the target object is extracted from the video frame of each video, various image recognition processing manners may be adopted. When the target object is extracted, the target object can be extracted frame by frame in sequence; for example, assuming that a video has 100 video frames, a first video frame may be extracted in sequence, and after a target object is extracted from the first video frame, a second video frame may be extracted, and after a target object is extracted from the second video frame, the target objects may be extracted one by one in sequence according to the foregoing rules; or the 100 video frames can be extracted first, and then the target objects corresponding to the 100 video frames are extracted according to a flexible rule; for example, the extraction is performed sequentially from the 100 th frame, the extraction is performed sequentially from the 1 st frame, or the extraction is performed simultaneously on a plurality of video frames in a parallel manner, and so on.

S302: and synthesizing the target objects of the video frames corresponding to the video frame numbers among the videos into a new video frame, and sequentially combining the obtained new video frames to obtain the synthesized video.

In this embodiment, the video frame number is the position of each video frame within the respective video. For example, it is assumed that there are two sections of video to be synthesized currently, namely a first video and a second video, and the first video and the second video are both composed of 100 video frames. The video frames in the first video may be numbered sequentially from 1 to 100 and the video frames in the second video may also be numbered sequentially from 1 to 100. In this step, synthesizing the target objects of the video frames corresponding to the video frame numbers among the videos into a new video frame is as follows: and synthesizing a target object of the video frame with the video frame number i of the first video and the target object of the video frame with the video frame number i of the second video into a new video frame, wherein i is more than or equal to 1 and less than or equal to 100. Of course, it should be understood that the numbering rule of the video frames can be flexibly set, for example, the video frames in the first video can be numbered in sequence from 1 to 100, and the video frames in the second video can also be numbered in sequence from 101 to 200, and in this case, the target objects of the video frames corresponding to the video frame numbers between the videos are synthesized into a new video frame: and synthesizing a target object of a video frame with the video frame number i of the first video and a target object of a video frame with the video frame number 100+ i of the second video into a new video frame, wherein i is more than or equal to 1 and less than or equal to 100. In short, the video frames corresponding to the video frame numbers in this embodiment mean that the positions of the video frames in the respective videos are the same.

The above description is made only by taking two pieces of videos as an example of the video to be subjected to the synthesizing processing, and it should be understood that when three or more pieces of videos are taken, the processing manner may be analogized.

It should be understood that the number of video frames included in each video to be synthesized in the present embodiment may be the same, and as shown in the above example, each video includes 100 frames. However, the number of video frames included in each video to be synthesized may also be different in this embodiment. In this embodiment, when synthesizing a video to be synthesized, all video frames in the video may be synthesized, or only some video frames in the video may be synthesized. The present embodiment is briefly described in several examples.

Example one: as described above, the video to be subjected to the compositing processing includes the first video and the second video, and the number of video frames included in the first video and the second video is 100; in an application scenario, it is set that only the video frames of the first 80 frames are synthesized, and then only the target objects of the first 80 frames in the first video and the second video may be extracted in S301, and then the target objects of the video frames corresponding to the video frame numbers are synthesized into a new video frame in S302.

Example two: the video to be synthesized comprises a first video, a second video and a third video, wherein the video frame numbers of the first video and the second video are both 100, and the video frame number of the third video is 105; in an application scenario, it is still set that only the video frames of the first 80 frames are synthesized, in this case, only the target objects of the first 80 frames in the first video, the second video, and the third video may be extracted in S301, and then the target objects of the video frames corresponding to the video frame numbers are synthesized into a new video frame in S302. In another application scenario, all video frames of a composite video are set, at this time, 100 frame target objects in the first video and the second video are extracted in S301, 105 frame target objects in the third video are extracted, and then the target objects of the first 100 video frames corresponding to the video frame numbers are synthesized into a new video frame in S302; for 5 video frames with video frame numbers from 101 to 105 in the third video, the target object in the single video frame can be directly synthesized into a new video frame, that is, the target object of the video frame in the corresponding video frame number in the first video and the second video can be considered as empty; of course, the target objects corresponding to the 5 video frames with the video frame numbers from 101 to 105 in the third video may also be directly discarded without performing the synthesizing process.

In this embodiment, when the number of the plurality of video frames to be synthesized is different, and when video frame extraction is performed, the number of video frames included in the extracted shortest video may also be set, or the number of the extracted video frames may also be set to be less than or equal to the number of video frames included in the shortest video, and specifically, the number may be flexibly set according to a specific scene.

In this embodiment, the manner of combining the obtained new video frames in sequence to obtain the synthesized video can be flexibly set. The sequence in this embodiment may be from front to back, or from back to front; when combining new video frames in this embodiment, all new video frames may be obtained and then combined in sequence, or may be combined while generating new video frames. For ease of understanding, the following description is provided with several examples.

Example one: as shown above, the video to be synthesized includes the first video and the second video, the number of video frames included in the first video and the second video is 100, in S301, the target objects of 100 frames in the first video and the second video are extracted, and then in S302, the target objects of the first 100 video frames corresponding to the video frame number are synthesized into a new video frame. In this example, the synthesizing sequence may be from front to back, that is, the target object of the video frame with the video frame number of 1 in the first video and the target object of the video frame with the video frame number of 21 in the second video are sequentially synthesized, after each new video frame is obtained after synthesizing, whether a new video frame has been generated before is determined, and if yes, the new video frame is sequentially combined with the previous new video frame. The combination of the new video frames in the process and the generation process of the new video frames are performed in a crossed manner, and the sequence of the video frames in the finally obtained synthesized video is arranged according to a positive sequence of 1-100.

Example two: the video to be synthesized still includes the first video and the second video, the number of video frames included in the first video and the second video is 100, in S301, the target objects of 100 frames in the first video and the second video are extracted, and then in S302, the target objects of the first 100 video frames corresponding to the video frame numbers are synthesized into a new video frame. In this example, the generation process of all the new video frames may be completed first, and the new video frames that have been generated are not combined in the target object synthesis process of the corresponding video frames. After the new video frames are generated, 100 new video frames with the video frame numbers of 1-100 can be obtained, and then the 1-100 new video frames are combined in sequence, in the example, the new video frames can be combined in a positive sequence, namely, in a sequence from 1-100; it can also be combined in the reverse order, i.e., from 100-1. When reverse order combination is performed, the playing sequence of the obtained synthesized video is opposite to the playing sequence of each target object in the original video (such as the first video and the second video), so that the display form of the video content can be further enriched, and the interest degree and the satisfaction degree of the user can be improved.

By the video synthesis method provided by the embodiment, the terminal can synthesize the target object in multiple videos obtained by shooting or from other terminals or from a network into one video, and one video frame in the video obtained after the synthesis can simultaneously display the target object in the multiple videos before the synthesis, so that the dynamic cloning of the target object is realized, the synthesized video can be played in a forward or reverse order, and the processing, display and sharing modes of the videos are enriched, so that the satisfaction degree of user experience can be greatly improved.

Second embodiment

In order to facilitate understanding of the present invention, the present embodiment further exemplifies the present invention with reference to several specific application cases on the basis of the first embodiment.

In an example of the present embodiment, for each video to be subjected to the composition processing, the background image in each video frame in one video is the same, that is, the shooting background in each video frame in one video is the same. Of course, it should be understood that the graphical background of each video may be the same or different. For example, when the video to be subjected to the compositing process includes a first video and a second video, the graphical background in the first video and the graphical background in the second video may be the same or different. However, the graphical background of each video frame in the first video is the same, and the background image of each video frame in the second video is also the same. In this example, when extracting a target object within a video frame, contents other than the background image contents may be all the target objects. Preset contents (e.g., persons, scenes, animals, etc.) among other contents other than the background image may also be targeted. At this time, the step of extracting the target object in each video frame of each video, as shown in fig. 4, includes:

s401: for each video, video frames of the video are extracted.

S402: and comparing the extracted image in each video frame with the background image of the video frame, and extracting the object which is not in the background image as the target object of each video frame.

It should be understood that, in this embodiment, all video frames of a video may be extracted first, and then the target object may be extracted from each video frame, or the target object may be identified while extracting the video frames.

For example, for a first video, assuming the first video has 100 video frames, one way may be: firstly, 1 video frame is extracted, then the image content in the video frame is compared with the background image of the first video, and the content which is not possessed by the background image in the video frame is provided as a target object, wherein the target object can be a person, a scene, an animal or other things. In this embodiment, a specific type of the target object may also be defined, for example, a person is defined, and according to a specific application scenario, a specific person or types of persons may be further set. After the target object is extracted from the 1 st video frame, the 2 nd video frame is sequentially extracted to extract the target object until the 100 th video frame. Another way may be: 100 video frames of a first video are extracted, and then target objects in the 100 video frames are extracted according to a preset sequence. The preset sequence can be extracted one by one in sequence from the 1 st frame, or can be extracted in a mode of a plurality of parallel extractions.

It should be understood that the above-described manner of performing target object recognition extraction by the background image is merely an example manner. When the target object is identified and extracted, the target object is not limited to be extracted by the background image, and the target image may be extracted by any other pattern recognition technique. In addition, in the embodiment, for a plurality of video segments to be synthesized, the extraction manner of the target object in the video frame of each video segment may be the same or different.

In this embodiment, the background image of the video may be carried in the video, or may be sent to the terminal through a separate file, or may be captured and acquired by the terminal through a camera of the terminal.

In one example of the present embodiment, the background images in the respective video frames of the respective videos to be subjected to the composition processing are all the same. That is, each video is obtained by shooting in the same background image. At this time, each video frame of each video may share one background image.

As can be seen from the above analysis, the lengths of the videos to be synthesized (i.e., the number of included video frames) in the present embodiment may be the same or different. In an application scenario, the number of video frames included in each video to be synthesized may be set to be the same, that is, each video to be synthesized is a video with the same length.

In this embodiment, when synthesizing the target object of the video frame corresponding to the video frame number between the videos to be synthesized into a new video frame, the target object of the video frame corresponding to the video frame number may be synthesized in a background image of the new video frame, where the background image of the new video frame may be the same as the background image in the video frame of at least one video to be synthesized, or may be different from the background images in the video frames of the videos to be synthesized. In addition, in this embodiment, when the target objects of the video frames corresponding to different video frame numbers are synthesized into a new video frame, the background images of the adopted new video frame may be the same or different, and the user may be supported to select the background image by self-definition.

In this embodiment, the step of synthesizing, among videos to be synthesized, a target object of a video frame corresponding to a video frame number in a preset background image of a new video frame to obtain the new video frame is shown in fig. 5, and includes:

s501: and determining the position of the target object of each video frame corresponding to the video frame number in the background image of the new video frame.

In this example, in order to achieve a better video presentation effect, the positions of the target objects of the video frames in the background image of the new video frame are set to be non-overlapping, so that the target objects can be displayed completely and clearly in the synthesized video. Of course, when setting the position of each target object in the background image of the new video frame, whether there is a position overlap may also be disregarded.

In one example, the step of determining the position of the target object of each video frame corresponding to the video frame number in the background image of the new video frame is shown in fig. 6, and includes:

s601: and acquiring the original positions of the target objects in the video frames corresponding to the target objects respectively.

In this embodiment, the original position of each target object may be determined by the pixel point position of each target object in the corresponding video frame.

S602: judging whether the original positions of the target objects are overlapped, if not, turning to S603; otherwise, go to S604;

s603: taking the original position of each target object as the position of each target object in the background image of the new video frame; therefore, the display position of the synthesized target image in the new video is ensured to be the same as the position of the synthesized target image in the original video to the maximum extent, and the distortion is reduced; of course, when the factor is not considered, the position of each target object in the background image of the new video frame can be flexibly set and determined.

S604: and allocating a new position in the background image of the new video frame for at least one target object with the overlapped original position, and taking the original position of the part of the target object without the overlapped original position as the position of the target object in the background image of the new video frame.

In this step, the positions of the target objects in the background image of the new video frame may be directly reset, as long as it is ensured that the position areas corresponding to the target objects do not overlap.

S502: and synthesizing each target object in the background image of the new video frame according to the position of the target object of each video frame in the background image of the new video frame. It should be understood that the image synthesis method in the present embodiment may adopt any method capable of realizing image synthesis.

In the video synthesis method provided by the embodiment, the target object of each video frame in the video can be extracted based on the background image in each video, the extraction algorithm is simple and fast, and the extracted target object is comprehensive and accurate. In addition, in the embodiment, flexible setting is supported for the position of the target object to be synthesized in the new background image, and the satisfaction degree of user experience can be further improved.

Third embodiment

The embodiment provides a terminal, which may be various flexible screen terminals that can be bent, or a normal terminal that can not be bent, and which has the capability of shooting, recording or acquiring video from other terminals or networks, and the terminal in this embodiment may be the mobile terminal shown in fig. 1. Referring to fig. 7, the terminal in the present embodiment includes a processor 701, a memory 702, and a communication bus 703;

the communication bus 703 is used for realizing communication connection between the processor 701 and the memory 702;

the processor 701 is configured to execute one or more programs stored in the memory 702 to implement the steps of the video composition method as exemplified in the embodiments above.

The present embodiment also provides a computer-readable storage medium, which is applicable in various terminals, and which stores one or more programs executable by one or more processors to implement the steps of the video composition method as exemplified in the above embodiments.

In order to facilitate understanding of the present invention, the present embodiment further illustrates the present invention by taking a specific application scenario as an example in combination with the terminal.

In this example, it is assumed that the terminal has a camera through which pictures and videos can be taken. And the memory of the terminal is stored with steps for the processor to call to execute the video synthesis method. In this embodiment, the terminal may be set to have a video shooting composition mode, and when it is detected that the terminal starts the camera to shoot a video, the video shooting composition mode may be automatically started, or the user may select the video shooting composition mode, or the user is prompted whether to start the video composition mode, and when the video composition mode is started, the terminal executes the video composition process on the shot video. For example, in an example, the terminal is set to have a normal video shooting mode and a video shooting composition mode, and when the terminal starts a camera to shoot a video, a mode selection menu can pop up on a display interface for a user to select, for example, as shown in fig. 8, when the user selects the video shooting composition mode, the shot video is subjected to composition processing; and when the user selects the common video shooting mode, entering a common video shooting process.

In this example, when the terminal enters the video shooting composition mode, it may be required that the background image in the shooting process cannot be changed, that is, the background images of the videos are identical. Of course, the background images may not be required to be the same according to actual requirements.

In the present example, after the video shooting composition mode is selected, during the video shooting composition mode, the following video shooting process can be set.

In an example, the number of videos to be composited can be set by the system, for example, set to 2, in this case, the user can only perform compositing processing on two captured videos, and the video frame number (i.e., the video length) of each captured video is the same, and the background images of each video are the same. At this time, when the user selects the video capture composition mode on the basis of fig. 8, the user is prompted on the capture interface to capture a background image, as shown in fig. 9. After detecting that the background image is obtained by shooting at the terminal, the interface shown in fig. 10 is switched to, and the user is prompted to shoot the first video section at the interface, and in this embodiment, the length of the video to be synthesized can be set well, and the length can support user-defined setting. After detecting that the terminal shoots the first video, the interface shown in fig. 11 is switched to, and the user is prompted to shoot the second video on the interface, where the length of the second video is controlled to be the same as that of the first video in this embodiment. In this example, during the first and second video capturing processes, it may be monitored whether the captured background image is the same as the initially captured background image, and if not, the capturing is stopped or an alarm is given, and the camera may be controlled to automatically search for the previous background image.

In this example, after the second video is also captured, video composition can be performed. However, it should be understood that the first video and the second video are not limited to be obtained by shooting in this embodiment, for example, the first video may be obtained by the user locally at the terminal or on another terminal or network. In an example, after the first video segment is shot, in the second video segment shooting process, the video frame and the target image of the first video segment can be extracted by using the initially shot background image. Or after the first video and the second video are shot, respectively extracting the video frames and the target images of the first video and the second video. The method can also be used for extracting the video frames acquired by the camera and the target objects in the video frames in the shooting process of the first video and the second video, namely completing the extraction of the target objects in the recording process of the videos.

For example, in one example, the process of the terminal shooting method is as follows:

1. shooting a background picture P;

2. taking a first segment of small video A (about 2 seconds) to generate n continuous pictures (one picture corresponds to one video frame) of the first segment of small video, respectively marking the n pictures as A1, … and An, and respectively using the n pictures to compare the pictures P to identify a target object (a person, a scene or the like);

3. taking a second segment of the small video B (about 2 seconds) to also generate n continuous pictures of the second segment of the small video, marking the n pictures as B1, … and Bn, and identifying a target object (a person or a scene and the like) compared with the picture P after generating the series of pictures;

in other examples, if there is a third video segment and a fourth video segment in the video to be synthesized, the algorithm is the same as above, and is not described herein again;

4 after shooting is finished, sequentially synthesizing a target object in n photos generated by the first segment of the small video and a target object in n photos generated by the second segment of the small video, namely synthesizing the target object in A1 and the target object in B1 into P1 in the background picture P, synthesizing the target object in A2 and the target object in B2 into P2, … in the background picture P, synthesizing the target object in An and the target object in Bn into pn in the background picture P, then combining the obtained new pictures (new video frames) P1, … in a positive sequence or a reverse sequence to obtain a synthesized video, namely completing dynamic synthesis of two segments of the small videos, and simultaneously playing the two segments of the small videos in the same background. Dynamic presentation of two or more target objects (such as people, animals, scenes and the like) can be simultaneously presented in the video, namely, the dynamic cloning camera shooting is realized, the existing shooting mode is broken through, and the cloned target objects are dynamically presented, so that the picture is more vivid, interesting and creative, and more interesting shooting experience is brought.

In this example, the background images of the videos are the same, so that the video effect obtained after shooting and synthesizing by the terminal in combination with the tripod is better.

It should be understood that, in this embodiment, after the first segment of the small video and the second segment of the small video are synthesized to obtain the synthesized small video, the first segment of the small video and the second segment of the small video may be deleted, but after the first segment of the small video and the second segment of the small video are deleted, the synthesized small video may be previewed and played, so that the user may confirm the final synthesized effect, and if the user confirms that the synthesized small video is retained, the first segment of the small video and the second segment of the small video are deleted. Of course, in this example, after the synthesized small video is obtained, the first section of small video and the second section of small video may also be retained, and when the first section of small video and the second section of small video are locally stored, they may be stored in association with the synthesized small video, so as to facilitate searching.

In this embodiment, for the synthesized small video, the user may play the small video locally or share the small video on the network according to the needs of the user. In this embodiment, when the synthesized small video is stored locally in the terminal, a type flag for characterizing that the synthesized small video is obtained by video synthesis may be added to distinguish the synthesized small video from other general videos.

The video synthesis method provided by the embodiment can be applied to the video recording/shooting process of the terminal, and can extract the target object of each video frame in the video based on the background image in each video, the extraction algorithm is simple and rapid, and the extracted target object is comprehensive and accurate. In addition, in the embodiment, flexible setting is supported for the position of the target object to be synthesized in the new background image, and the satisfaction degree of user experience can be further improved.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.

While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A video compositing method, characterized in that the video compositing method comprises the steps of:

for each video to be synthesized, respectively extracting target objects in each video frame of each video in sequence frame by frame through an image recognition processing mode;

the video frame number is the position of the video frame within the video.

2. The video synthesis method according to claim 1, wherein the background images in the video frames of one video are the same, and the step of extracting the target object in each video frame of each video respectively comprises:

for each video, extracting each video frame of the video;

3. The video compositing method of claim 2, wherein the background image within each video frame of each video is the same.

4. The video synthesis method according to claim 1, wherein the number of video frames included in each video to be synthesized is the same.

5. The video synthesis method according to any one of claims 1 to 4, wherein the step of synthesizing the target objects of the video frames corresponding to the video frame numbers among the videos into a new video frame comprises:

6. The video compositing method of claim 5 wherein the new video frame background image is the same as the background image within each video frame of at least one of the videos.

7. The video synthesis method according to claim 5, wherein the step of synthesizing the target object of the video frame corresponding to the video frame number among the videos in the preset background image of the new video frame to obtain the new video frame comprises:

8. The video compositing method of claim 7, wherein the step of determining the position of the target object of each video frame corresponding to the video frame number in the background image of the new video frame comprises:

9. A terminal, characterized in that the terminal comprises a processor, a memory and a communication bus;

the processor is configured to execute one or more programs stored in the memory to implement the steps of the video composition method of any of claims 1-8.

10. A computer-readable storage medium, storing one or more programs, the one or more programs being executable by one or more processors to perform the steps of the video compositing method of any of claims 1-8.