CN114257775A

CN114257775A - Video special effect adding method and device and terminal equipment

Info

Publication number: CN114257775A
Application number: CN202011021330.0A
Authority: CN
Inventors: 吴霞; 张硕
Original assignee: Honor Device Co Ltd
Current assignee: Honor Device Co Ltd
Priority date: 2020-09-25
Filing date: 2020-09-25
Publication date: 2022-03-29
Anticipated expiration: 2040-09-25
Also published as: CN114257775B; CN116437034A; WO2022062985A1

Abstract

The embodiment of the application discloses a video special effect adding method and device and terminal equipment, wherein the original mode of starting the special effect through key clicking is replaced by gesture actions, the complexity of user operation can be reduced, and the special effect is quickly started when a video call is called. Regional portrait segmentation and background replacement or blurring can be performed according to the area or screen proportion of the area which the fingers of the user slide, so that the interestingness, the playability and the interactivity of the product are improved. And the original image can be downsampled before the portrait segmentation is performed, so that the calculation and power consumption expenses of the portrait segmentation are greatly reduced, a deep learning portrait segmentation model algorithm which needs a large calculation amount is allowed to run on a mobile terminal with limited calculation capacity and power consumption, and the frame rate requirement of video playing is met.

Description

Video special effect adding method and device and terminal equipment

[ technical field ] A method for producing a semiconductor device

The present application relates to the field of mobile communications, and in particular, to a method and an apparatus for adding a video special effect, and a terminal device.

[ background of the invention ]

At present, many video call products support the addition of special effects such as beauty, filter or background replacement and the like in the video call process, and the special effects are popular with users. In the video call products supporting the special effects, when a user wants to add a special effect, the user needs to call a special effect menu first and then click to select, and the operation steps are complicated. Especially in the incoming call scene, the user needs to simultaneously process two tasks of answering the call and adjusting the special effect menu, and the operation is very inconvenient. Therefore, a quick operation for adding special effects is needed.

In a video call product in the prior art, a special effect icon is usually displayed in an incoming call interface, and a user can expand a special effect menu or a beauty adjusting bar by clicking the special effect icon. When a user needs to apply a special effect, the user needs to click a special effect icon, expand a special effect editing menu, click to select a special effect thumbnail or slide in the menu to adjust the beauty level. In the user interaction process, the user needs to click at least twice to start the specified special effect, the interaction process is complicated, the time consumption of the whole process is long, the click area of the icon is small, and the operation is inconvenient and rapid. Therefore, the purpose of quickly achieving the opening special effect in the incoming call state cannot be met.

[ application contents ]

In view of this, embodiments of the present application provide a method, an apparatus, and a terminal device for adding a video special effect, so as to solve the technical problem in the prior art that adding a special effect is inconvenient and not fast when a user is in a video call.

In a first aspect, an embodiment of the present application provides a video special effect adding method, where the method includes the following steps: capturing gesture actions of a user, and acquiring sliding information of the gesture actions; and applying the special effect material to a screen according to the sliding information of the gesture action.

The scheme provided by the embodiment is different from the problem that the interaction process is complicated due to the fact that a user needs to click at least twice to start the special effect in the video call process in the prior art, and the effect that the special effect is added to where the fingers of the user slide can be achieved by capturing and processing the sliding information of the gesture actions of the user.

In a preferred embodiment, the step of capturing the gesture motion of the user and acquiring the sliding information of the gesture motion includes: capturing and recording start point coordinate information of a sliding start point and end point coordinate information of a sliding end point of a finger of a user in a two-dimensional xy coordinate system of the screen in real time; judging the sliding direction and the sliding speed of the finger according to the starting point coordinate information and the end point coordinate information of the finger; judging a sliding gesture adopted by a user according to the sliding direction of the finger; the sliding information comprises starting point coordinate information of a sliding starting point, end point coordinate information of a sliding end point, a sliding gesture, a sliding direction and a sliding speed.

According to the scheme provided by the embodiment, when a user slides, a finger slides through the screen, sliding information is left, the basis for judging the sliding gesture, the sliding direction and the sliding speed is provided by the starting point coordinate information of the sliding starting point and the end point coordinate information of the sliding end point, the sliding gesture provides a mode that the finger of the user slides on the screen, the sliding direction provides a path that the finger of the user slides on the screen, the sliding speed provides the speed of the finger of the user sliding on the screen, and the area, needing to replace special effect materials, on the screen can be accurately known by integrating the sliding information.

In a preferred embodiment, when the swipe gesture is a side swipe gesture, the following steps are performed: judging the coordinate displacement D of the finger on the screen and the preset minimum distance D according to the starting point coordinate information and the end point coordinate information of the finger_minAnd the slip velocity V is related to a preset minimum velocity V_minThe magnitude relationship of (1); when D > D_minAnd V > V_minAnd starting the special effect material.

Through the scheme provided by the embodiment, the minimum distance D is preset_minAnd minimum velocity V_minAs a standard for judging whether the fingers of the user effectively slide, whether the fingers of the user really perform gesture operation is judged by calculating the coordinate displacement D and the sliding speed V of the fingers on the screen, and the problem that a corresponding user mistakenly touches the screen to add a special effect material mistakenly is avoided.

In a preferred embodiment, when the slide is in useWhen the gesture is an angular sliding gesture, the following steps are executed: presetting a starting point area; wherein the maximum value of the abscissa of the starting point region is X_maxMaximum value of abscissa is Y_max(ii) a Judging whether the sliding starting point of the finger falls in the starting point area or not according to the starting point coordinate information of the finger; if the abscissa X of the starting point of the finger's sliding movement_s＜X_maxAnd the ordinate Y_s＜Y_maxJudging that the sliding starting point of the finger is located in a preset starting point area; judging the coordinate displacement D of the finger on the screen and the preset minimum distance D according to the starting point coordinate information and the end point coordinate information of the finger_minAnd the slip velocity V is related to a preset minimum velocity V_minThe magnitude relationship of (1); when D > D_min、V＞V_minAnd the range of the slope k of the connecting line of the sliding starting point and the sliding end point meets 0.268<|k|<3.73 or 0.268<|k^-1|<3.73, starting the special effect material.

According to the scheme provided by the embodiment, the starting point area and the sliding area are preset, the starting point area is used as a judgment condition for judging whether the finger of the user can trigger the sliding gesture recognition, the sliding area is used as a judgment condition for judging whether the finger of the user can trigger the application of the special effect material, two judgment conditions are set so as to accurately judge whether the finger of the user has the sliding gesture at the corner of the screen and the size of the area where the finger slides, the sliding areas are arranged at the four corners of the screen, the inclination angle of the edge of each sliding area is in the range of 15-75 degrees, when the finger of the user executes the angular sliding gesture, the sliding area covers part or all of the sliding area, the judgment requirement of the angular sliding gesture can be met, and the special effect material can be adaptively applied.

In a preferred embodiment, the step of applying the special effect material to the screen according to the sliding information of the gesture motion includes: dividing the screen into a first area and a second area according to the sliding information of the gesture action; carrying out portrait segmentation on the portrait displayed in the screen, and carrying out special effect material replacement on the part, located in the first area, of the background in the screen; the first area is an area of the screen which is perpendicular to a connecting line of a sliding starting point and a sliding end point of the gesture motion and passes through a dividing line of the sliding end point, and the second area is an area on the screen except the first area; the sliding information includes start point coordinate information of a sliding start point, end point coordinate information of a sliding end point, a sliding gesture, a sliding direction and a sliding speed.

According to the scheme provided by the embodiment, portrait segmentation and screen segmentation are carried out firstly, portrait is extracted, the screen is segmented into a first area through which fingers slide and a second area through which fingers do not slide, and only the background of the first area is replaced with special effect materials. The method can capture the position of the finger in the sliding process in real time, so as to determine a first area of the background needing to be replaced. When the background is replaced, the area occupied by the portrait still displays the portrait and cannot be replaced by the background, so that the effects of unchanged foreground portrait and changed background are formed, and the functions of capturing the sliding gesture of the user in real time, calculating the area which the fingers of the user slide through in real time and replacing the background of the area which the fingers of the user slide through in real time are realized.

In a preferred embodiment, the step of dividing the screen into a first area and a second area according to the sliding information of the gesture motion includes: creating the dividing line according to the coordinate information of the starting point of the sliding starting point and the coordinate information of the end point of the sliding end point of the gesture motion; dividing an area of the screen swept by the dividing line in the sliding direction into the first area according to the dividing line and the sliding direction.

With the scheme provided by the present embodiment, a function of a dividing line for dividing the first region and the second region in the xy coordinate system can be determined by a simple operation using the start point coordinate information of the slide start point and the end point coordinate information of the slide end point, so that the first region swept by the finger can be determined on the screen. The method occupies less computing resources, has higher processing speed, and can adapt to the special effect material addition in the scene with higher sliding speed of the gesture action.

In a preferred embodiment, the step of performing the portrait segmentation on the portrait displayed in the screen and performing the special effect material replacement on the part of the screen where the background is located in the first area includes: carrying out down-sampling on the original image of the portrait by using a bilinear interpolation algorithm to obtain a down-sampled image; calculating the figure outline in the down-sampling image, and up-sampling the figure outline by using a bilinear interpolation algorithm to obtain the outline of the original image; segmenting the portrait from the original image based on the outline of the original image to obtain a first image layer; carrying out special effect material replacement on the part of the background in the first area to obtain a second image; and overlapping the first image and the second image.

According to the scheme provided by the embodiment, the original image is subjected to down-sampling processing before the portrait segmentation is performed, so that the calculation and power consumption overhead of the portrait segmentation is greatly reduced, a deep learning portrait segmentation model algorithm requiring a large calculation amount is allowed to operate on a mobile terminal with limited calculation capacity and power consumption, and the frame rate requirement of video playing is met.

In a preferred embodiment, the addition level of the special effect material is selected according to the area of the first area or the screen ratio of the first area to the screen.

According to the scheme provided by the embodiment, in the process of adding the special effect materials, a user can select the amount of the added special effect materials, the adding level of the special effect materials can be selected according to the size of the area where the fingers of the user slide through the screen in the operation application, and the operation method can meet more and more complex special effect adding requirements of the user.

In a preferred embodiment, the adding level of the special effect material is correspondingly increased by one level every time the screen occupation ratio of the first area is increased by 10%.

Through the scheme provided by the embodiment, smoother video special effect adding experience can be provided for the user.

In a preferred embodiment, when the screen occupation ratio of the first area is more than 50% and the terminal coordinate information of the sliding terminal of the gesture motion is not changed any more, the whole background of the screen is replaced by the special effect material.

Through the scheme provided by the embodiment, the user often represents the requirement of replacing all backgrounds under the condition that the finger slides mostly, so the method adopting the preferred embodiment can facilitate the operation of the user and enhance the user experience.

In a preferred embodiment, before the step of dividing the screen into a first area and a second area according to the sliding information of the gesture motion, the method comprises: judging whether the sliding of the gesture action is effective or not according to the sliding speed; if the sliding speed is higher than the first speed, the background of the screen is subjected to overall special effect material replacement; if the sliding speed is less than the second speed, the step of applying the special effect material to the screen according to the sliding information of the gesture action is not executed; wherein the first speed is greater than the second speed.

According to the scheme provided by the embodiment, two judgment conditions for judging the sliding speed are preset, when the sliding speed is higher than the first speed, the user can be considered to need to replace the whole background, and when the sliding speed is lower than the second speed, the user can be considered not to execute the sliding gesture, so that the step of applying the special effect material is not needed to be started.

In a preferred embodiment, after the step of performing the portrait segmentation on the portrait displayed in the screen and performing the special effect material replacement on the part of the screen where the background is located in the first area, the method includes: judging the sliding direction of the gesture action; if the sliding direction is the forward direction far away from the sliding starting point of the finger on the screen, adding a new special effect material; and if the sliding direction is the reverse direction of the sliding starting point of the finger on the screen, recovering the previous special effect material.

Through the scheme provided by the embodiment, the backtracking operation is provided for the user through the identification of the sliding direction, so that the user can freely select a more favorable special effect material from a plurality of special effect materials, worry about sliding the favorable special effect material is avoided, and the user experience is enhanced.

In a preferred embodiment, the step of applying the special effect material to the screen according to the sliding information of the gesture motion includes: according to the sliding information of the gesture action, pulling out a control panel along with the sliding direction of the user at the sliding starting point of the gesture action on the screen, wherein a plurality of special effect materials are prevented on the control panel; applying the special effects material to the screen.

Through the scheme provided by the embodiment, the control panel with selectivity is provided for the user, when the special effect materials are replaced, the visual experience of the user is better, and the special effect materials which meet the requirements of the user better can be selected.

In a preferred embodiment, before the step of applying the special effect material to the screen according to the slide information of the gesture motion, the method further comprises: and detecting the use frequency of each special effect material, and sequencing the presentation sequence of the special effect materials from more to less according to the use frequency.

According to the scheme provided by the embodiment, the list sequence of the special effect materials commonly used by the user can be customized according to the habit of the user, so that the special effect materials can be added to the background of the screen more quickly when the user uses the special effect materials each time, the user can use the special effect materials more conveniently, and the experience is better.

In a preferred embodiment, the number of special effects material is less than 10.

According to the scheme provided by the embodiment, the upper limit of the material quantity is set for the standby library of the special effect materials, the special effect materials appear circularly when being stroked, and the situation that a user is difficult to restore to the original state after sliding due to too many special effect materials is avoided.

In a second aspect, an embodiment of the present application provides a video special effect adding apparatus, where the apparatus includes: the information acquisition module is used for capturing gesture actions of a user and acquiring sliding information of the gesture actions; and the special effect application module is used for applying the special effect material to the screen according to the sliding information of the gesture action.

The scheme provided by the embodiment is different from the problem that the interaction process is complicated due to the fact that a user needs to click at least twice to start a special effect in the video call process in the prior art, and the scheme provided by the embodiment captures and processes the sliding information of the gesture actions of the user by using the information acquisition module and the special effect application module, so that the effect that the finger of the user slides to where the special effect is added can be achieved.

In a preferred embodiment, the information acquisition module comprises: the recording unit is used for capturing and recording the start point coordinate information of a sliding start point and the end point coordinate information of a sliding end point of a finger of a user in a two-dimensional xy coordinate system of the screen in real time; the calculation unit is used for judging the sliding direction and the sliding speed of the finger according to the starting point coordinate information and the end point coordinate information of the finger; and the judging unit is used for judging the sliding gesture adopted by the user according to the sliding direction of the finger.

According to the scheme provided by the embodiment, the recording unit is used for recording sliding information generated when the fingers of the user slide on the screen, the calculating unit determines the area for adding the special effect material on the screen according to the coordinates of the sliding starting point and the sliding end point of the fingers when the user slides, and the judging unit judges the sliding tracks of the fingers of the user when the user uses different sliding gestures so as to meet the requirement for adding the special effect material in real time.

In a preferred embodiment, the special effects application module comprises: the segmentation unit is used for segmenting the screen into a first area and a second area according to the sliding information of the gesture action; and the application unit is used for carrying out portrait segmentation on the portrait displayed in the screen and carrying out special effect material replacement on the part, located in the first area, of the background in the screen.

According to the scheme provided by the embodiment, the segmentation unit calculates the size of the area where the fingers of the user slide, performs screen segmentation, divides the screen into the first area where the fingers slide and the second area where the fingers do not slide, performs image segmentation on the application unit first, and then performs special effect material replacement on the background of the first area, so that the functions of capturing the sliding gesture of the user in real time, calculating the area where the fingers of the user slide in real time, and replacing the background of the area where the fingers of the user slide in real time are realized.

In a third aspect, an embodiment of the present application provides a terminal device, which includes the video special effect adding apparatus according to the second aspect.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium, comprising a program or instructions, which when run on a computer, performs the method according to the first aspect.

Compared with the prior art, the technical scheme at least has the following beneficial effects:

according to the video special effect adding method and device and the terminal device, the gesture action is used for replacing the original mode of starting the special effect through key clicking, the complexity of user operation can be reduced, and the special effect can be started quickly when a video call is called. Regional portrait segmentation and background replacement or blurring can be performed according to the area or screen proportion of the area which the fingers of the user slide, so that the interestingness, the playability and the interactivity of the product are improved. And the original image can be downsampled before the portrait segmentation is performed, so that the calculation and power consumption expenses of the portrait segmentation are greatly reduced, a deep learning portrait segmentation model algorithm which needs a large calculation amount is allowed to run on a mobile terminal with limited calculation capacity and power consumption, and the frame rate requirement of video playing is met.

[ description of the drawings ]

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

Fig. 1 is a schematic structural diagram of a terminal device provided in embodiment 1 of the present application;

fig. 2 is a schematic step diagram of a video special effect adding method provided in embodiment 2 of the present application;

fig. 3 is a schematic diagram illustrating a Step100 in the video special effect adding method according to embodiment 2 of the present application;

fig. 4a is a rectangular coordinate system when a side sliding gesture is adopted in Step100 in the video special effect adding method provided in embodiment 2 of the present application, and fig. 4b is an operation schematic diagram when a side sliding gesture is adopted in Step100 in the video special effect adding method provided in embodiment 2 of the present application;

fig. 5a is a schematic view of a start area when an angular slide gesture is adopted in Step100 in the video special effect adding method provided in embodiment 2 of the present application, fig. 5b is an operation schematic diagram when an angular slide gesture is adopted in Step100 in the video special effect adding method provided in embodiment 2 of the present application, and fig. 5c is a diagram of an effective slide range when an angular slide gesture is adopted in Step100 in the video special effect adding method provided in embodiment 2 of the present application;

fig. 6 is a schematic diagram illustrating a Step200 in a video special effect adding method according to embodiment 2 of the present application;

fig. 7 is a schematic diagram illustrating a Step210 in the video special effect adding method according to embodiment 2 of the present application;

fig. 8 is a schematic diagram illustrating a Step220 in the video special effect adding method according to embodiment 2 of the present application;

fig. 9 is a schematic diagram of a contour image after a portrait segmentation nursing in Step220 in the video special effect adding method provided in embodiment 2 of the present application;

fig. 10 is a schematic diagram illustrating a Step before Step210 in the video special effect adding method according to embodiment 2 of the present application;

fig. 11 is an operation schematic diagram of switching of special effect materials when a finger of a user slides multiple times in the video special effect adding method provided in embodiment 2 of the present application;

fig. 12 is a schematic diagram illustrating a Step200 in another embodiment of a video special effect adding method provided in example 2 of the present application;

fig. 13a and 13b are schematic operation diagrams of another implementation of Step200 in the video special effect adding method provided in embodiment 2 of the present application;

fig. 14 is a schematic diagram illustrating a Step before Step200 in the video special effect adding method according to embodiment 2 of the present application;

fig. 15 is a block diagram of a video special effect adding apparatus according to embodiment 3 of the present application;

fig. 16 is a schematic block diagram of an information obtaining module in a video special effects adding apparatus according to embodiment 3 of the present application;

fig. 17 is a schematic block diagram of a special effect application module in the video special effect adding apparatus according to embodiment 3 of the present application.

Reference numerals:

1-an antenna;

2-an antenna;

100-a terminal device; 110-a processor; 120-external memory interface; 121-internal memory; 130-universal serial bus interface; 140-a charge management module; 141-power management module; 142-a battery; 150-a mobile communication module; 160-a wireless communication module; 170-an audio module; 170A-speaker; 170B-receiver; 170C-microphone; 170D-headset interface; 180-a sensor module; 180A-pressure sensor; 180B-a gyroscope sensor; 180C-air pressure sensor; 180D-magnetic sensor; 180E-acceleration sensor; 180F-distance sensor; 180G — low beam sensor; 180H-fingerprint sensor; 180J-temperature sensor; 180K-touch sensor; 180L-ambient light sensor; 180M-bone conduction sensor; 190-key press; 191-a motor; 192-an indicator; 193-camera; 194-a display screen; 195-a subscriber identity module card interface;

10-an information acquisition module; 11-a recording unit; 12-a calculation unit; 13-a determination unit;

20-special effect application module; 21-a segmentation unit; 22-application unit.

[ detailed description ] embodiments

For better understanding of the technical solutions of the present application, the following detailed descriptions of the embodiments of the present application are provided with reference to the accompanying drawings.

It should be understood that the embodiments described are only a few embodiments of the present application, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

Embodiments of a terminal device and a method for implementing the terminal device are described below, where the terminal device may be a mobile phone (also called an intelligent terminal device), a tablet personal computer (tablet personal computer), a personal digital assistant (personal digital assistant), an electronic book reader (e-book reader), or a virtual reality interactive device (virtual reality interactive device), and the terminal device may be accessed into various types of communication systems, for example: long Term Evolution (LTE) systems, future fifth Generation (5th Generation, 5G) systems, new radio access technology (NR), and future communication systems, such as 6G systems; but also Wireless Local Area Networks (WLANs) and the like.

For convenience of description, in the following embodiments, an intelligent terminal device is taken as an example for description.

Example 1

Fig. 1 shows a schematic structural diagram of a terminal device disclosed in embodiment 1 of the present application, where the terminal device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a Universal Serial Bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a key 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a Subscriber Identity Module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, and the like.

It is to be understood that the illustrated structure of the embodiment of the present invention does not specifically limit the terminal device 100. In other embodiments of the present application, terminal device 100 may include more or fewer components than shown, or some components may be combined, some components may be split, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

Processor 110 may include one or more processing units, such as: the processor 110 may include an Application Processor (AP), a modem processor, a Graphics Processor (GPU), an Image Signal Processor (ISP), a controller, a video codec, a Digital Signal Processor (DSP), a baseband processor, and/or a neural-Network Processing Unit (NPU), etc. The different processing units may be separate devices or may be integrated into one or more processors.

The controller can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of instruction fetching and instruction execution.

A memory may also be provided in processor 110 for storing instructions and data. In one embodiment, the memory in the processor 110 is a cache memory. The memory may hold instructions or data that have just been used or recycled by the processor 110. If the processor 110 needs to reuse the instruction or data, it can be called directly from the memory. Avoiding repeated accesses reduces the latency of the processor 110, thereby increasing the efficiency of the system.

In one embodiment, processor 110 may include one or more interfaces. The interface may include an integrated circuit (I2C) interface, an integrated circuit built-in audio (I2S) interface, a Pulse Code Modulation (PCM) interface, a universal asynchronous receiver/transmitter (UART) interface, a Mobile Industry Processor Interface (MIPI), a general-purpose input/output (GPIO) interface, a Subscriber Identity Module (SIM) interface, and/or a Universal Serial Bus (USB) interface, etc.

The I2C interface is a bi-directional synchronous serial bus that includes a serial data line (SDA) and a Serial Clock Line (SCL). In one embodiment, processor 110 may include multiple sets of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces, respectively. For example: the processor 110 may be coupled to the touch sensor 180K through an I2C interface, so that the processor 110 and the touch sensor 180K communicate through an I2C bus interface to implement the touch function of the terminal device 100.

The I2S interface may be used for audio communication. In one embodiment, processor 110 may include multiple sets of I2S buses. The processor 110 may be coupled to the audio module 170 via an I2S bus to enable communication between the processor 110 and the audio module 170. In one embodiment, the audio module 170 may transmit audio signals to the wireless communication module 160 through the I2S interface, so as to receive phone calls through a bluetooth headset.

The PCM interface may also be used for audio communication, sampling, quantizing and encoding analog signals. In one embodiment, the audio module 170 and the wireless communication module 160 may be coupled by a PCM bus interface. In one embodiment, the audio module 170 may also transmit audio signals to the wireless communication module 160 through the PCM interface, so as to implement a function of answering a call through a bluetooth headset. Both the I2S interface and the PCM interface may be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communications. The bus may be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In one embodiment, a UART interface is generally used to connect the processor 110 with the wireless communication module 160. For example: the processor 110 communicates with a bluetooth module in the wireless communication module 160 through a UART interface to implement a bluetooth function. In an embodiment, the audio module 170 may transmit the audio signal to the wireless communication module 160 through a UART interface, so as to realize the function of playing music through a bluetooth headset.

MIPI interfaces may be used to connect processor 110 with peripheral devices such as display screen 194, camera 193, and the like. The MIPI interface includes a Camera Serial Interface (CSI), a display screen serial interface (DSI), and the like. In one embodiment, the processor 110 and the camera 193 communicate through a CSI interface to implement the shooting function of the terminal device 100. The processor 110 and the display screen 194 communicate through the DSI interface to implement the display function of the terminal device 100.

The GPIO interface may be configured by software. The GPIO interface may be configured as a control signal and may also be configured as a data signal. In one embodiment, a GPIO interface may be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, and the like. The GPIO interface may also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, and the like.

The USB interface 130 is an interface conforming to the USB standard specification, and may specifically be a Mini USB interface, a Micro USB interface, a USB Type C interface, or the like. The USB interface 130 may be used to connect a charger to charge the terminal device 100, and may also be used to transmit data between the terminal device 100 and a peripheral device. And the earphone can also be used for connecting an earphone and playing audio through the earphone. The interface may also be used to connect other terminal devices, such as AR devices and the like.

It should be understood that the interface connection relationship between the modules according to the embodiment of the present invention is only an exemplary illustration, and does not limit the structure of the terminal device 100. In other embodiments of the present application, the terminal device 100 may also adopt different interface connection manners or a combination of multiple interface connection manners in the above embodiments.

The charging management module 140 is configured to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In one wired charging embodiment, the charging management module 140 may receive charging input from a wired charger via the USB interface 130. In one wireless charging embodiment, the charging management module 140 may receive a wireless charging input through a wireless charging coil of the terminal device 100. The charging management module 140 may also supply power to the terminal device through the power management module 141 while charging the battery 142.

The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and/or the charge management module 140, and supplies power to the processor 110, the internal memory 121, the display 194, the camera 193, the wireless communication module 160, and the like. The power management module 141 may also be used to monitor parameters such as battery capacity, battery cycle count, battery state of health (leakage, impedance), etc. In one embodiment, the power management module 141 may also be disposed in the processor 110. In another embodiment, the power management module 141 and the charging management module 140 may be disposed in the same device.

The wireless communication function of the terminal device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, a modem processor, a baseband processor, and the like.

The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in terminal device 100 may be used to cover a single or multiple communication bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 may be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 150 may provide a solution including 2G/3G/4G/5G wireless communication applied on the terminal device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a Low Noise Amplifier (LNA), and the like. The mobile communication module 150 may receive the electromagnetic wave from the antenna 1, filter, amplify, etc. the received electromagnetic wave, and transmit the electromagnetic wave to the modem processor for demodulation. The mobile communication module 150 may also amplify the signal modulated by the modem processor, and convert the signal into electromagnetic wave through the antenna 1 to radiate the electromagnetic wave. In one embodiment, at least some of the functional modules of the mobile communication module 150 may be disposed in the processor 110. In one embodiment, at least some of the functional modules of the mobile communication module 150 may be disposed in the same device as at least some of the modules of the processor 110.

The modem processor may include a modulator and a demodulator. The modulator is used for modulating a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used for demodulating the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then passes the demodulated low frequency baseband signal to a baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and then transferred to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, the receiver 170B, etc.) or displays an image or video through the display screen 194. In one embodiment, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be provided in the same device as the mobile communication module 150 or other functional modules, independent of the processor 110.

The wireless communication module 160 may provide a solution for wireless communication applied to the terminal device 100, including Wireless Local Area Networks (WLANs) (e.g., wireless fidelity (Wi-Fi) networks), bluetooth (bluetooth, BT), Global Navigation Satellite System (GNSS), Frequency Modulation (FM), Near Field Communication (NFC), Infrared (IR), and the like. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on electromagnetic wave signals, and transmits the processed signals to the processor 110. The wireless communication module 160 may also receive a signal to be transmitted from the processor 110, perform frequency modulation and amplification on the signal, and convert the signal into electromagnetic waves through the antenna 2 to radiate the electromagnetic waves.

In one embodiment, the antenna 1 of the terminal device 100 is coupled to the mobile communication module 150 and the antenna 2 is coupled to the wireless communication module 160, so that the terminal device 100 can communicate with a network and other devices through a wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), time-division code division multiple access (time-division code division multiple access, TD-SCDMA), Long Term Evolution (LTE), LTE, BT, GNSS, WLAN, NFC, FM, and/or IR technologies, etc. The GNSS may include a Global Positioning System (GPS), a global navigation satellite system (GLONASS), a beidou satellite navigation system (BDS), a quasi-zenith satellite system (QZSS), and/or a Satellite Based Augmentation System (SBAS).

The terminal device 100 implements a display function by the GPU, the display screen 194, and the application processor. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and an application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.

The display screen 194 is used for displaying images, videos, and the like, wherein the display screen 194 includes a display panel, the display screen may specifically include a folding screen, a special-shaped screen, and the display panel may adopt a Liquid Crystal Display (LCD), an organic light-emitting diode (OLED), an active matrix organic light-emitting diode (active-matrix organic light-emitting diode, AMOLED), a flexible light-emitting diode (flex-emitting diode, FLED), a miniature, a Micro-o led, a quantum dot light-emitting diode (QLED), and the like. In one embodiment, the terminal device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

The terminal device 100 may implement a shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194, the application processor, and the like.

The ISP is used to process the data fed back by the camera 193. For example, when a photo is taken, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, the optical signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converting into an image visible to naked eyes. The ISP can also carry out algorithm optimization on the noise, brightness and skin color of the image. The ISP can also optimize parameters such as exposure, color temperature and the like of a shooting scene. In one embodiment, the ISP may be provided in camera 193.

The camera 193 is used to capture still images or video. The object generates an optical image through the lens and projects the optical image to the photosensitive element. The photosensitive element may be a Charge Coupled Device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The light sensing element converts the optical signal into an electrical signal, which is then passed to the ISP where it is converted into a digital image signal. And the ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into image signal in standard RGB, YUV and other formats. In one embodiment, the terminal device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

The digital signal processor is used for processing digital signals, and can process digital image signals and other digital signals. For example, when the terminal device 100 selects a frequency point, the digital signal processor is used to perform fourier transform or the like on the frequency point energy.

Video codecs are used to compress or decompress digital video. The terminal device 100 may support one or more video codecs. In this way, the terminal device 100 can play or record video in a plurality of encoding formats, such as: moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, and the like.

The NPU is a neural-network (NN) computing processor that processes input information quickly by using a biological neural network structure, for example, by using a transfer mode between neurons of a human brain, and can also learn by itself continuously. The NPU can implement applications such as intelligent recognition of the terminal device 100, for example: image recognition, face recognition, speech recognition, text understanding, and the like.

The external memory interface 120 may be used to connect an external memory card, such as a Micro SD card, to extend the storage capability of the terminal device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function. For example, files such as music, video, etc. are saved in an external memory card.

The internal memory 121 may be used to store computer-executable program code, which includes instructions. The internal memory 121 may include a program storage area and a data storage area. The storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required by at least one function, and the like. The storage data area may store data (such as audio data, a phonebook, etc.) created during use of the terminal device 100, and the like. In addition, the internal memory 121 may include a high-speed random access memory, and may further include a nonvolatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (UFS), and the like. The processor 110 executes various functional applications of the terminal device 100 and data processing by executing instructions stored in the internal memory 121 and/or instructions stored in a memory provided in the processor.

The terminal device 100 may implement an audio function through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the earphone interface 170D, and the application processor. Such as music playing, recording, etc.

The audio module 170 is used to convert digital audio information into an analog audio signal output and also to convert an analog audio input into a digital audio signal. The audio module 170 may also be used to encode and decode audio signals. In one embodiment, the audio module 170 may be disposed in the processor 110, or some functional modules of the audio module 170 may be disposed in the processor 110.

The speaker 170A, also called a "horn", is used to convert the audio electrical signal into an acoustic signal. The terminal device 100 can listen to music through the speaker 170A, or listen to a handsfree call.

The receiver 170B, also called "earpiece", is used to convert the electrical audio signal into an acoustic signal. When the terminal device 100 answers a call or voice information, it is possible to answer a voice by bringing the receiver 170B close to the human ear.

The microphone 170C, also referred to as a "microphone," is used to convert sound signals into electrical signals. When making a call or transmitting voice information, the user can input a voice signal to the microphone 170C by speaking the user's mouth near the microphone 170C. The terminal device 100 may be provided with at least one microphone 170C. In other embodiments, the terminal device 100 may be provided with two microphones 170C, which may implement a noise reduction function in addition to collecting sound signals. In other embodiments, the terminal device 100 may further include three, four or more microphones 170C to collect sound signals, reduce noise, identify sound sources, and implement directional recording functions.

The headphone interface 170D is used to connect a wired headphone. The headset interface 170D may be the USB interface 130, or may be an Open Mobile Terminal Platform (OMTP) standard interface of 3.5mm, or a cellular telecommunications industry association (cellular telecommunications industry association of the USA, CTIA) standard interface.

The pressure sensor 180A is used for sensing a pressure signal, and converting the pressure signal into an electrical signal. In one embodiment, the pressure sensor 180A may be disposed on the display screen 194. The pressure sensor 180A can be of a wide variety, such as a resistive pressure sensor, an inductive pressure sensor, a capacitive pressure sensor, and the like. The capacitive pressure sensor may be a sensor comprising at least two parallel plates having an electrically conductive material. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The terminal device 100 determines the intensity of the pressure from the change in the capacitance. When a touch operation is applied to the display screen 194, the terminal device 100 detects the intensity of the touch operation based on the pressure sensor 180A. The terminal device 100 may also calculate the touched position from the detection signal of the pressure sensor 180A. In one embodiment, the touch operations that are applied to the same touch position but different touch operation intensities may correspond to different operation instructions. For example: and when the touch operation with the touch operation intensity smaller than the first pressure threshold value acts on the short message application icon, executing an instruction for viewing the short message. And when the touch operation with the touch operation intensity larger than or equal to the first pressure threshold value acts on the short message application icon, executing an instruction of newly building the short message.

The gyro sensor 180B may be used to determine the motion attitude of the terminal device 100. In one embodiment, the angular velocity of the terminal device 100 about three axes (i.e., x, y, and z axes) may be determined by the gyro sensor 180B. The gyro sensor 180B may be used for photographing anti-shake. Illustratively, when the shutter is pressed, the gyro sensor 180B detects the shake angle of the terminal device 100, calculates the distance to be compensated for by the lens module according to the shake angle, and allows the lens to counteract the shake of the terminal device 100 through a reverse movement, thereby achieving anti-shake. The gyroscope sensor 180B may also be used for navigation, somatosensory gaming scenes.

The air pressure sensor 180C is used to measure air pressure. In one embodiment, the terminal device 100 calculates an altitude from the barometric pressure measured by the barometric pressure sensor 180C, and assists in positioning and navigation.

The magnetic sensor 180D includes a hall sensor. The terminal device 100 may detect the opening and closing of the flip holster using the magnetic sensor 180D. In one embodiment, when the terminal device 100 is a folder, the terminal device 100 may detect the opening and closing of the folder according to the magnetic sensor 180D. And then according to the opening and closing state of the leather sheath or the opening and closing state of the flip cover, the automatic unlocking of the flip cover is set.

The acceleration sensor 180E can detect the magnitude of acceleration of the terminal device 100 in various directions (generally, three axes). The magnitude and direction of gravity can be detected when the terminal device 100 is stationary. The method can also be used for recognizing the posture of the terminal equipment, and is applied to horizontal and vertical screen switching, pedometers and other applications.

A distance sensor 180F for measuring a distance. The terminal device 100 may measure the distance by infrared or laser. In one embodiment, the terminal device 100 may take a picture of a scene and may range using the distance sensor 180F to achieve fast focus.

The proximity light sensor 180G may include, for example, a Light Emitting Diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The terminal device 100 emits infrared light to the outside through the light emitting diode. The terminal device 100 detects infrared reflected light from a nearby object using a photodiode. When sufficient reflected light is detected, it can be determined that there is an object near the terminal device 100. When insufficient reflected light is detected, the terminal device 100 can determine that there is no object near the terminal device 100. The terminal device 100 can utilize the proximity light sensor 180G to detect that the user holds the terminal device 100 close to the ear for talking, so as to automatically turn off the screen to achieve the purpose of saving power. The proximity light sensor 180G may also be used in a holster mode, a pocket mode automatically unlocks and locks the screen.

The ambient light sensor 180L is used to sense the ambient light level. The terminal device 100 may adaptively adjust the brightness of the display screen 194 according to the perceived ambient light level. The ambient light sensor 180L may also be used to automatically adjust the white balance when taking a picture. The ambient light sensor 180L may also cooperate with the proximity light sensor 180G to detect whether the terminal device 100 is in a pocket, in order to prevent accidental touches.

The fingerprint sensor 180H is used to collect a fingerprint. The terminal device 100 can utilize the collected fingerprint characteristics to realize fingerprint unlocking, access to an application lock, fingerprint photographing, fingerprint incoming call answering and the like.

The temperature sensor 180J is used to detect temperature. In one embodiment, the terminal device 100 executes a temperature processing strategy using the temperature detected by the temperature sensor 180J. For example, when the temperature reported by the temperature sensor 180J exceeds the threshold, the terminal device 100 performs a reduction in performance of a processor located near the temperature sensor 180J, so as to reduce power consumption and implement thermal protection. In other embodiments, the terminal device 100 heats the battery 142 when the temperature is below another threshold to avoid the terminal device 100 being abnormally shut down due to low temperature. In other embodiments, when the temperature is lower than a further threshold, the terminal device 100 performs boosting on the output voltage of the battery 142 to avoid abnormal shutdown due to low temperature.

The touch sensor 180K is also called a "touch device". The touch sensor 180K may be disposed on the display screen 194, and the touch sensor 180K and the display screen 194 form a touch screen, which is also called a "touch screen". The touch sensor 180K is used to detect a touch operation applied thereto or nearby. The touch sensor can communicate the detected touch operation to the application processor to determine the touch event type. Visual output associated with the touch operation may be provided through the display screen 194. In other embodiments, the touch sensor 180K may be disposed on the surface of the terminal device 100, different from the position of the display screen 194.

In one embodiment, the touch screen composed of the touch sensor 180K and the display screen 194 may be located in a side area or a folding area of the terminal device 100, and is used for determining a position touched by a user and a gesture touched by the user when the user touches the touch screen with a hand; for example, when the user holds the terminal device, the user can click any position on the touch screen with a thumb, the touch sensor 180K can detect the click operation of the user and transmit the click operation to the processor, and the processor determines the click operation according to the click operation to wake up the screen.

The bone conduction sensor 180M may acquire a vibration signal. In one embodiment, the bone conduction sensor 180M may acquire a vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M may also contact the human pulse to receive the blood pressure pulsation signal. In one embodiment, the bone conduction sensor 180M may also be disposed in a headset, integrated into a bone conduction headset. The audio module 170 may analyze a voice signal based on the vibration signal of the bone mass vibrated by the sound part acquired by the bone conduction sensor 180M, so as to implement a voice function. The application processor can analyze heart rate information based on the blood pressure beating signal acquired by the bone conduction sensor 180M, so as to realize the heart rate detection function.

The keys 190 include a power-on key, a volume key, and the like. The keys 190 may be mechanical keys. Or may be touch keys. The terminal device 100 may receive a key input, and generate a key signal input related to user setting and function control of the terminal device 100.

The motor 191 may generate a vibration cue. The motor 191 may be used for incoming call vibration cues, as well as for touch vibration feedback. For example, touch operations applied to different applications (e.g., photographing, audio playing, etc.) may correspond to different vibration feedback effects. The motor 191 may also respond to different vibration feedback effects for touch operations applied to different areas of the display screen 194. Different application scenes (such as time reminding, receiving information, alarm clock, game and the like) can also correspond to different vibration feedback effects. The touch vibration feedback effect may also support customization.

Indicator 192 may be an indicator light that may be used to indicate a state of charge, a change in charge, or a message, missed call, notification, etc.

The SIM card interface 195 is used to connect a SIM card. The SIM card can be brought into and out of contact with the terminal device 100 by being inserted into the SIM card interface 195 or being pulled out of the SIM card interface 195. The terminal device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 may support a Nano SIM card, a Micro SIM card, a SIM card, etc. The same SIM card interface 195 can be inserted with multiple cards at the same time. The types of the plurality of cards may be the same or different. The SIM card interface 195 may also be compatible with different types of SIM cards. The SIM card interface 195 may also be compatible with external memory cards. The terminal device 100 interacts with the network through the SIM card to implement functions such as communication and data communication. In one embodiment, the terminal device 100 employs eSIM, namely: an embedded SIM card. The eSIM card may be embedded in the terminal device 100 and cannot be separated from the terminal device 100.

When the terminal device adopts the special-shaped screen or the folding screen, the touch display screen of the terminal device may include a plurality of touch display areas, for example, the folding screen of the terminal device includes a folding area in a folded state, and the folding area may also implement touch response. However, in the prior art, the operation of the terminal device on a specific touch display area is limited to a relatively large extent, and no relevant operation is specifically performed on the specific touch display area, and based on this, an embodiment of the present application provides a gesture interaction method, where a touch response area exists in a side area or a folding area of the terminal device in the gesture interaction method, and the terminal device may obtain an input event of the touch response area, and in response to the input event, trigger the terminal device to execute an operation instruction corresponding to the input event, so as to implement a gesture operation on the side area or the folding area of the terminal device, and improve the operation experience of the terminal device.

In the terminal device disclosed in embodiment 1 of the present application, the memory is configured to store a computer program, and the processor is configured to execute the computer program stored in the memory, so that the terminal device executes the method described in embodiment 2 of the present application.

Example 2

As shown in fig. 2, embodiment 2 of the present application discloses a video special effect adding method, which can be applied to an incoming call state, and the method includes the following steps:

step 100: capturing gesture actions of a user, and acquiring sliding information of the gesture actions;

step 200: and applying the special effect material to the screen according to the sliding information of the gesture action.

The video special effect adding method provided by the embodiment is different from the problem that in the prior art, the interaction process is complicated due to the fact that a user needs to click at least twice to start a special effect in the video call process.

As shown in fig. 3, in the video special effect adding method of the present embodiment, in Step 100: the method for capturing the gesture action of the user and acquiring the sliding information of the gesture action comprises the following steps:

step 110: capturing and recording start point coordinate information of a sliding start point and end point coordinate information of a sliding end point of a finger of a user in a two-dimensional xy coordinate system of a screen in real time;

step 120: judging the sliding direction and the sliding speed of the finger according to the starting point coordinate information and the end point coordinate information of the finger;

step 130: and judging the sliding gesture adopted by the user according to the sliding direction of the finger.

The sliding information comprises starting point coordinate information of a sliding starting point, end point coordinate information of a sliding end point, a sliding gesture, a sliding direction and a sliding speed.

According to the video special effect adding method, when a user slides, a finger slides through a screen, sliding information is left, the starting point coordinate information of the sliding starting point and the end point coordinate information of the sliding end point provide a basis for judging a sliding gesture, a sliding direction and a sliding speed, the sliding gesture provides a mode that the finger of the user slides on the screen, the sliding direction provides a path that the finger of the user slides on the screen, the sliding speed provides the speed of the finger of the user sliding on the screen, and the area, needing to replace special effect materials, on the screen can be accurately known by integrating the sliding information.

In the video special effect adding method of the embodiment, the sliding gesture of the user is divided into a side sliding gesture and an angle sliding gesture, and the side sliding gesture starts to slide from four sides of the mobile phone to the center of the screen transversely or longitudinally; the angular swipe gesture is dominated by an angular swipe towards the center of the screen.

When the sliding gesture is a side sliding gesture, executing the following steps:

step 140: judging the coordinate displacement D of the finger on the screen and the preset minimum distance D according to the starting point coordinate information and the end point coordinate information of the finger_minAnd sliding speed V and preset minimum speed V_minThe magnitude relationship of (1);

when D > D_minAnd V > V_minThen, Step180 is executed.

Take the example of sliding right from the side of the screen to the center. Judging the coordinate displacement | Xs-Xe | of the finger on the screen and the preset minimum distance D_minAnd the magnitude relation of the sliding speed Vx and the preset minimum speed Vmin;

when | Xs-Xe | > D_minAnd V is_x＞V_minIf so, the sliding is judged to be effective.

The core thought is as follows: the motion track and speed of the coordinate position (x, y) of the user's finger on the two-dimensional coordinate axis of the screen are captured and recorded in real time (as shown in fig. 4a to 4 b).

It judgesThe off condition is as follows: if the sliding origin abscissa X_sIs 0 (preferably, a fault-tolerant space of 10 pixels can be reserved, the specific value of the fault-tolerant space is determined according to the actual requirement), and the ordinate Y_sIs the screen middle point range (the length of the vertical axis is maximum Y)_max+200 pixels); abscissa X of sliding end point_eIs greater than X_sCoordinate displacement | Xs-Xe | greater than D_min(preferably, the reference value is 100 pixels, which can be specifically determined according to application scenarios, user habits, screen sensitivity, etc.), and the moving speed V_xGreater than V_min(reference value 200 pixels/sec).

The above determination process is to determine the lateral pulling motion of the finger shown in fig. 4a to 4 b; based on the thought, the area of the area transversely pulled by the user can be expanded and obtained, so that the subsequent step of applying the special effect material is conveniently executed.

It should be noted that the swipe gesture motion on the android phone may be the default Home key (Home key) function of the system, so the system level gesture listening capture is to be noticed in this scenario.

Similarly, if the finger slides downwards from the top of the screen, the coordinate displacement is | Ys-Ye |, and the sliding speed is V_yThe remaining steps may be substituted into the conditions and steps described above.

In the video special effect adding method of the embodiment, the minimum distance D is preset_minAnd minimum velocity V_minAs a standard for judging whether the fingers of the user effectively slide, whether the fingers of the user really perform gesture operation is judged by calculating the coordinate displacement D and the sliding speed V of the fingers on the screen, and the problem that a corresponding user mistakenly touches the screen to add a special effect material mistakenly is avoided.

Take the upper right corner of the screen moving towards the center area of the screen as an example. When the sliding gesture is a side sliding gesture, executing the following steps:

step 150: presetting a starting point area; wherein the maximum value of the abscissa of the starting point region is X_maxMaximum value of abscissa is Y_max；

Step 160: judging whether the sliding starting point of the finger falls in the starting point area or not according to the starting point coordinate information of the finger;

if the abscissa X of the starting point of the finger's sliding movement_s＜X_maxAnd the ordinate Y_s＜Y_maxJudging that the sliding starting point of the finger is located in a preset starting point area; thereby performing Step 170.

Step 170: judging the coordinate displacement D of the finger on the screen and the preset minimum distance D according to the starting point coordinate information and the end point coordinate information of the finger_minAnd sliding speed V and preset minimum speed V_minThe magnitude relationship of (1);

when D > D_min、V＞V_minAnd the range of the slope k of the connecting line of the sliding starting point and the sliding end point meets 0.268<|k|<3.73 or 0.268<|k-1|<At 3.73, Step180 is executed.

At this time, the coordinates are displaced

Step 180: and starting the special effect materials.

As shown in fig. 5a to 5c, when the user's finger performs the angular sliding gesture, the triggering condition for recognizing the angular center movement motion is that the starting point O must be within the starting point area of the box shape in fig. 5b, and 50 is a reference value, and the size of the starting point area can be adjusted according to actual needs or screen sensitivity.

The angular centripetal sliding motion recognition is a judgment rule obtained by appropriately optimizing the judgment conditions of the side sliding gesture shown in fig. 4a to 4b, taking point a in fig. 5c as an end point, coordinates a are (x (a), y (a)), and starting points are (x (o), y (o)). The judgment rule is as follows:

(ii) coordinate displacement exceeds D_min(reference value 100 pixels) i.e

② the coordinate moving speed exceeds V_min(reference value 200 pixels/sec);

③ the slope k of the line AO ranges from (0.268, 3.73), i.e. the angle corresponding to the shift excursion is between 15 ° and 75 °, as shown by the triangle of OAB in fig. 5 c. (the triangular area range can be adjusted according to actual requirements).

Sliding region identification (i.e., isosceles right triangle region identification): the following processing is performed based on the sliding end point of the finger sliding: taking point a in fig. 5c as an example, let a be the sliding end point of the user sliding, and make a straight line with a slope of-1 at the passing point a, then the triangular region enclosed by the straight line and the coordinate axis is the sliding region, because the slope of the straight line is-1, the internal angles with the coordinate axis are both 45 °, as shown in fig. 5c, an isosceles right triangle is ready for realizing the background replacement of the triangular region in the figure.

Fig. 5c only illustrates the situation of one corner of the terminal device, and the determination rules of the other three corners can be deduced according to the situation illustrated in fig. 5c, and finally the determination rule is 0.268<|k|<3.73 or 0.268<|k^-1|<3.73。

The video special effect adding method of the embodiment presets a starting point area and a sliding area, wherein the starting point area is used as a judgment condition for judging whether a finger of a user can trigger sliding gesture recognition, the sliding area is used as a judgment condition for judging whether the finger of the user can trigger application of special effect materials, two judgment conditions are set so as to accurately judge whether the finger of the user has a sliding gesture at the corner of a screen and the size of the area where the finger slides, the sliding areas are arranged at the four corners of the screen, the inclination angle of the edge of each sliding area is in the range of 15-75 degrees, when the finger of the user executes an angular sliding gesture, the finger of the user covers part or all of the sliding area, the judgment requirement of the angular sliding gesture can be met, and then the special effect materials can be adaptively applied.

As shown in fig. 6, in the video special effect adding method of the present embodiment, at Step 200: according to the sliding information of the gesture action, the special effect material is applied to the screen, and the method comprises the following steps:

step210: according to the sliding information of the gesture action, dividing the screen into a first area and a second area;

step 220: and carrying out portrait segmentation on the portrait displayed in the screen, and carrying out special effect material replacement on the part of the background in the first area in the screen.

The first area is an area of the screen which is perpendicular to a connecting line of a sliding starting point and a sliding end point of the gesture motion and passes through a dividing line of the sliding end point, and the second area is an area on the screen except the first area; the slide information includes start point coordinate information of a slide start point, end point coordinate information of a slide end point, a slide gesture, a slide direction, and a slide speed.

The video special effect adding method of the embodiment comprises the steps of firstly carrying out portrait segmentation and screen segmentation, extracting the portrait, segmenting the screen into a first area which is slid by fingers and a second area which is not slid, and only replacing a special effect material for the background of the first area. The method can capture the position of the finger in the sliding process in real time, so as to determine a first area of the background needing to be replaced. When the background is replaced, the area occupied by the portrait still displays the portrait and cannot be replaced by the background, and the effects that the portrait in front is unchanged and the background in back is changed are achieved, so that the functions of capturing the sliding gesture of the user in real time, calculating the area where the fingers of the user slide through in real time and replacing the background of the area where the fingers of the user slide through in real time are achieved.

As shown in fig. 7, in the video special effect adding method of the present embodiment, Step210: according to the sliding information of the gesture action, the screen is divided into a first area and a second area, and the method comprises the following steps:

step 211: creating a dividing line according to the coordinate information of the starting point of the sliding starting point and the coordinate information of the end point of the sliding end point of the gesture motion;

step 212: an area of the screen swept by the dividing line in the sliding direction is divided into first areas according to the dividing line and the sliding direction.

According to the video special effect adding method, the size of the area where the fingers of the user slide is calculated firstly, then the screen is divided into the first area where the fingers slide and the second area where the fingers do not slide, and only the background of the first area is replaced with special effect materials, so that the functions of capturing the sliding gesture of the user in real time, calculating the area where the fingers of the user slide in real time and replacing the background of the area where the fingers of the user slide in real time are achieved.

As shown in fig. 8, in the video special effect adding method of the present embodiment, Step 220: the method comprises the following steps of carrying out portrait segmentation on a portrait displayed in a screen, and carrying out special effect material replacement on a part, located in a first area, of a background in the screen, wherein the method comprises the following steps:

step 221: carrying out down-sampling on an original image of the portrait by using a bilinear interpolation algorithm to obtain a down-sampled image;

step 222: calculating the figure contour (figure Mask) in the down-sampled image, and up-sampling the figure contour by using a bilinear interpolation algorithm to obtain the contour of the original image;

step 223: segmenting a portrait from an original image based on the outline of the original image to obtain a first image layer;

step 224: carrying out special effect material replacement on the part of the background in the first area to obtain a second image layer;

step 225: and superposing the first image layer and the second image layer.

Specifically, in the method disclosed in this embodiment, taking the google android standard camera API2.0 engine as an example, other camera engines may be used to output continuous image frame data, and the frame data format may be YUV-NV21 or a universal format such as RGB. Firstly, downsampling an original image by using a bilinear interpolation algorithm, for example, downsampling an original 1920x1080 resolution image into a 320x180 resolution image, thereby obtaining a downsampled image; in the method, a depth learning model used for face segmentation is not limited in the embodiment of the present invention, and common models such as CNN, FCN/FCN +/UNet, and the like may be used. After the portrait segmentation process, data with the contour (Mask) of the person is output, which is essentially a frame image, as shown in fig. 9, but it should be noted that the resolution of the contour is temporarily 320 × 180, and here, the computational complexity and power consumption overhead of the portrait segmentation are greatly reduced by processing the contour with low resolution. The output contour is up-sampled to the resolution of the original image (1920 × 1080 in the embodiment) by using a bilinear interpolation algorithm, and the black-and-white image is up-sampled, so that the image loss rate of up-sampling interpolation can be reduced to the lowest. Based on the data of the outline, the portrait is 'scratched out' from the original image to obtain a first image layer, and then the first image layer and a second image layer which is replaced by the special effect material of the background are subjected to image layer superposition rendering on a GPU to finally obtain an effect image after the background is replaced.

In the video special effect adding method of the embodiment, the original image is downsampled frame by adopting a bilinear interpolation method, so that the resolution is reduced in an equal proportion, the computational power loss is reduced, then the portrait is divided, the contour of the portrait is output, and then the contour is upsampled into the resolution of the original image by using the bilinear interpolation method. Therefore, the power consumption and the processing time delay of the portrait segmentation on the mobile terminal equipment are greatly reduced, and the frame rate requirement of the 30FPS video call is met. According to the method, the down-sampling processing is carried out on the original image before the portrait segmentation is carried out, so that the calculation and power consumption overhead of the portrait segmentation is greatly reduced, a deep learning portrait segmentation model algorithm which needs a large calculation amount is allowed to be operated on a mobile terminal with limited calculation capacity and power consumption, and the frame rate requirement of video playing is met.

In the video effect adding method of the embodiment, the adding level of the effect material is selected according to the area of the first area or the screen occupation ratio of the first area to the screen.

According to the video special effect adding method, in the process of adding the special effect materials, a user can select the amount of the added special effect materials, the adding level of the special effect materials can be selected according to the size of the area where the fingers of the user slide through the screen in the operation application, and the operation method can meet more and more complex special effect adding requirements of the user.

In the video special effect adding method of the embodiment, each time the gesture movement crosses 10%, the adding level of the special effect material is correspondingly increased by one level.

The video special effect adding method can provide smoother video special effect adding experience for users.

In practical applications, the face-beautifying algorithm may be triggered by the capturing method of the edge-sliding gesture described in fig. 4a to 4b along the transverse/longitudinal direction of the screen, and the proportion of the sliding area of the face-beautifying algorithm in the whole screen is used as the face-beautifying level. Take ten-grade beauty as an example: for example, when the sliding area ratio is 10%, starting one-level beauty when the sliding area ratio is from the left side or the top, starting ten-level beauty when the sliding area ratio is all to the right side, and the like.

Because the beauty is the special effect of level realization class, the edge sliding gesture capture triggering method is suggested to be used, but if factors such as business requirements or product design and the like exist, the angle sliding gesture capture method can be used for triggering, and the realization ideas of the edge sliding gesture capture method and the angle sliding gesture capture method are similar.

In the video special effect adding method of the embodiment, when the screen occupancy of the first area is greater than 50% and the end point coordinate information of the sliding end point of the gesture motion is not changed any more, the whole background of the screen is replaced with the special effect material.

According to the video special effect adding method, the user often has a requirement for replacing all backgrounds under the condition that the fingers of the user slide for the most part, so that the method adopting the preferred embodiment can facilitate the user operation and enhance the user experience.

As shown in fig. 10, in the video special effect adding method of the present embodiment, before the Step of dividing the screen into the first area and the second area according to the sliding information of the gesture motion in Step210, the method includes:

step 201: judging whether the sliding of the gesture action is effective or not according to the sliding speed;

if the sliding speed is greater than the first speed, Step202 is executed: carrying out integral special effect material replacement on the background of the screen;

if the sliding speed is less than the second speed, the Step200 is not executed; wherein the first speed is greater than the second speed.

The video special effect adding method of the embodiment presets two judgment conditions for judging the sliding speed, judges whether the sliding is effective or not by combining the sliding speed of the user, if the sliding speed is high, and when the sliding speed is higher than the first speed, the user can consider that the user needs to replace the whole background, the complete replacement or virtualization of the background is started, and when the sliding speed is lower than the second speed, the user can consider that the user does not execute the sliding gesture, namely, the step of applying the special effect material is not needed to be started.

Referring to fig. 6, in the video special effect adding method of the present embodiment, at Step 220: the method comprises the following steps of carrying out portrait segmentation on a portrait displayed in a screen, and carrying out special effect material replacement on a part, located in a first area, of a background in the screen, and comprises the following steps:

step 230: judging the sliding direction of the gesture action;

if the sliding direction is a positive direction away from the starting point of the finger sliding on the screen, Step240 is executed: adding new special effect materials;

if the sliding direction is the reverse direction towards the starting point of the finger sliding on the screen, Step250 is executed: and recovering the last special effect material.

As shown in fig. 11, when the finger of the user performs the gesture motion, the special effect material can be switched by sliding for a plurality of times, that is, the finger is slid again on the basis of the rule described above, and then the next special effect material can be replaced.

According to the video special effect adding method, backtracking operation is provided for a user through identification of the sliding direction, background replacement/blurring of a screen can be cancelled, for example, fingers slide to the left, and background replacement/blurring is started; sliding the finger to the left, and changing another background material; the finger slides to the right (i.e., reverse operation), restoring the last background material/real background. Therefore, the user can freely select the preferred special effect material from the plurality of special effect materials, worry about sliding the preferred special effect material is avoided, and the user experience is enhanced.

As shown in fig. 12, 13a and 13b, in the video special effect adding method of the present embodiment, the Step200 of applying a special effect material to a screen according to the slide information of the gesture motion includes:

step 210': according to the sliding information of the gesture action, pulling out a control panel along with the sliding direction of the user at the sliding starting point of the gesture action on the screen, and preventing a plurality of special effect materials on the control panel;

step 220': and applying the special effect material to a screen.

The video special effect adding method aims at achieving regional background replacement/blurring, is not limited to being triggered by the capturing method of the edge sliding gesture and the angle sliding gesture, and is characterized in that the special effect is quickly started through the sliding gesture, a selective control panel is provided for a user according to the area of a sliding region of the edge sliding/angle sliding as the range of background replacement/blurring, when special effect materials are replaced, the visual experience of the user is better, and the special effect materials which meet the requirements of the user better can be selected.

As shown in fig. 14, in the video special effect adding method of the present embodiment, at Step 200: before the special effect material is applied to the screen according to the sliding information of the gesture motion, the method further comprises the following steps:

step 200': and detecting the use frequency of each special effect material, and sequencing the presentation sequence of the special effect materials from more to less according to the use frequency.

According to the video special effect adding method, the list sequence of the special effect materials commonly used by the user can be customized according to the habit of the user, for example, the special effect materials used last time are preferentially displayed, or the special effect materials are sequenced from a plurality of times according to the use frequency, so that the special effect materials can be more rapidly added to the background of a screen when the user uses the video special effect adding method every time, the use of the user is more convenient, and the experience is better.

In the video special effect adding method of the present embodiment, the material quantity of the special effect material is less than 10.

The video special effect adding method of the embodiment sets the upper limit of the material quantity for the standby library of the special effect materials, the special effect materials appear circularly when being scratched, and the situation that a user is difficult to restore to the original state after sliding due to too many special effect materials is avoided.

According to the video special effect adding method disclosed by the embodiment, the original interactive design of starting the special effect in a key clicking mode is replaced by the sliding gesture. And performing regional portrait segmentation and background replacement/blurring according to the area or screen proportion of the region which the finger of the user slides on the screen, so as to realize regional background replacement/blurring. Therefore, when the user slides to replace the regional background in the video call, the region can be divided according to the position of the finger of the user on the screen and the angle of the sliding gesture, and the partial video stream of the replaced region is not transmitted, so that the data transmission of the video stream is saved. The area/screen ratio over which the user's finger slides may be used as an adjustment to the cosmetic level. Or the material, the beauty and make-up material, the 3D-Animoji or the sticker material can be switched by one key by taking the complete finger sliding action of the user as the background. In addition, the method also uses a capturing and analyzing method of the angular sliding gesture.

Example 3

Fig. 15 shows a video special effect adding apparatus provided in embodiment 3 of the present application, where the apparatus includes:

the information acquisition module 10 is configured to capture a gesture motion of a user and acquire sliding information of the gesture motion;

and the special effect application module 20 is configured to apply the special effect material to the area where the gesture action slides on the screen according to the sliding information of the gesture action.

The video special effect adding device of the embodiment is different from the problem that in the prior art, the interaction process is complicated due to the fact that a user needs to click at least twice to start a special effect in the video call process, and the sliding information of the gesture action of the user is captured and processed by the information acquisition module 10 and the special effect application module 20, so that the effect that the finger of the user slides to where the special effect is added can be achieved.

As shown in fig. 16, in the video special effects addition apparatus of the present embodiment, the information acquisition module 10 includes:

the recording unit 11 is used for capturing and recording the start point coordinate information of the sliding start point and the end point coordinate information of the sliding end point of the finger of the user in the two-dimensional xy coordinate system of the screen in real time;

the calculating unit 12 is used for judging the sliding direction and the sliding speed of the finger according to the starting point coordinate information and the end point coordinate information of the finger;

and the judging unit 13 is configured to judge a sliding gesture adopted by the user according to the sliding direction of the finger.

In the video special effect adding device of the embodiment, the recording unit 11 is used for recording sliding information generated when the fingers of the user slide on the screen, the calculating unit 12 determines the area for adding the special effect material on the screen according to the coordinates of the sliding starting point and the sliding ending point of the fingers when the user slides, and the judging unit 13 judges the sliding tracks of the fingers of the user when the user uses different sliding gestures, so as to meet the requirement of adding the special effect material in real time.

As shown in fig. 17, in the video special effects addition apparatus of the present embodiment, the special effects application module 2 includes:

a dividing unit 21 configured to divide the screen into a first area and a second area according to the sliding information of the gesture motion;

and the application unit 22 is used for performing portrait segmentation on the portrait displayed in the screen and performing special effect material replacement on the part, located in the first area, of the background in the screen.

In the video special effect adding device of the embodiment, the dividing unit 21 calculates the size of the area where the finger of the user slides, performs screen division, divides the screen into the first area where the finger slides and the second area where the finger does not slide, and the application unit 22 performs image division first and then performs special effect material replacement on the background of the first area, thereby realizing the functions of capturing the sliding gesture of the user in real time, calculating the area where the finger of the user slides in real time, and replacing the background of the area where the finger of the user slides in real time.

Example 4

Embodiment 4 of the present application provides a terminal device, which includes the video special effect adding apparatus according to embodiment 2 of the present application.

Example 5

Embodiment 5 of the present application provides a computer-readable storage medium, which includes a program or instructions, and when the program or instructions are run on a computer, the method according to embodiment 1 of the present application is executed.

In the above embodiments, the implementation may be wholly or partially realized by software, hardware, firmware, or any combination thereof. When implemented in software, may be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When loaded and executed on a computer, cause the processes or functions described in accordance with the embodiments of the application to occur, in whole or in part. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored on a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that incorporates one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., Digital Video Disk (DVD)), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.

Those of ordinary skill in the art will appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be embodied in electronic hardware, computer software, or combinations of both, and that the components and steps of the examples have been described in a functional general in the foregoing description for the purpose of illustrating clearly the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

The present invention is not intended to be limited to the particular embodiments shown and described, but is to be accorded the widest scope consistent with the principles and novel features herein disclosed.

Claims

1. A video special effect adding method, characterized in that the method comprises the steps of:

capturing gesture actions of a user, and acquiring sliding information of the gesture actions;

and applying the special effect material to a screen according to the sliding information of the gesture action.

2. The video special effect addition method according to claim 1,

the step of capturing the gesture action of the user and acquiring the sliding information of the gesture action comprises the following steps:

capturing and recording start point coordinate information of a sliding start point and end point coordinate information of a sliding end point of a finger of a user in a two-dimensional xy coordinate system of the screen in real time;

judging the sliding direction and the sliding speed of the finger according to the starting point coordinate information and the end point coordinate information of the finger;

judging a sliding gesture adopted by a user according to the sliding direction of the finger;

3. The video special effect addition method according to claim 2,

judging the coordinate displacement D of the finger on the screen and the preset minimum distance D according to the starting point coordinate information and the end point coordinate information of the finger_minAnd the slip velocity V is related to a preset minimum velocity V_minThe magnitude relationship of (1);

when D > D_minAnd V > V_minAnd starting the special effect material.

4. The video special effect addition method according to claim 2,

when the slide gesture is an angular slide gesture, executing the following steps:

presetting a starting point area; wherein the maximum value of the abscissa of the starting point region is X_maxMaximum value of abscissa is Y_max；

Judging whether the sliding starting point of the finger falls in the starting point area or not according to the starting point coordinate information of the finger;

if the abscissa X of the starting point of the finger's sliding movement_s＜X_maxAnd the ordinate Y_s＜Y_maxJudging that the sliding starting point of the finger is located in a preset starting point area;

when D > D_min、V＞V_minAnd the connection between the sliding starting point and the sliding end pointThe range of the slope k of the line satisfies 0.268<|k|<3.73 or 0.268<|k^-1|<3.73, starting the special effect material.

5. The video special effect addition method according to claim 1,

the step of applying the special effect material to the screen according to the sliding information of the gesture action comprises the following steps:

dividing the screen into a first area and a second area according to the sliding information of the gesture action;

carrying out portrait segmentation on the portrait displayed in the screen, and carrying out special effect material replacement on the part, located in the first area, of the background in the screen;

wherein the content of the first and second substances,

the first area is an area of the screen which is perpendicular to a connecting line of a sliding starting point and a sliding end point of the gesture motion and passes through a dividing line of the sliding end point, and the second area is an area on the screen except the first area;

the sliding information includes start point coordinate information of a sliding start point, end point coordinate information of a sliding end point, a sliding gesture, a sliding direction and a sliding speed.

6. The video effect addition method according to claim 5, wherein the step of dividing the screen into a first region and a second region according to the sliding information of the gesture motion includes:

creating the dividing line according to the coordinate information of the starting point of the sliding starting point and the coordinate information of the end point of the sliding end point of the gesture motion;

dividing an area of the screen swept by the dividing line in the sliding direction into the first area according to the dividing line and the sliding direction.

7. The video effect adding method according to claim 5, wherein the step of performing the portrait segmentation on the portrait displayed in the screen and performing the effect material replacement on the part of the screen where the background is located in the first area comprises:

carrying out down-sampling on the original image of the portrait by using a bilinear interpolation algorithm to obtain a down-sampled image;

calculating the figure outline in the down-sampling image, and up-sampling the figure outline by using a bilinear interpolation algorithm to obtain the outline of the original image;

segmenting the portrait from the original image based on the outline of the original image to obtain a first image layer;

carrying out special effect material replacement on the part of the background in the first area to obtain a second image;

and overlapping the first image and the second image.

8. The video effect addition method according to claim 5, wherein an addition level of the effect material is selected according to a region area of the first region or a screen ratio of the first region to the screen.

9. The video effect adding method according to claim 8, wherein the adding level of the effect material is raised by one level for every 10% increase in the screen occupation ratio of the first area.

10. The video effect addition method according to claim 5, wherein when the screen occupancy of the first area is greater than 50% and the end point coordinate information of the sliding end point of the gesture motion is no longer changed, the entire background of the screen is replaced with the effect material.

11. The video special effect addition method according to claim 5,

before the step of dividing the screen into a first area and a second area according to the sliding information of the gesture action, the method comprises the following steps:

judging whether the sliding of the gesture action is effective or not according to the sliding speed;

if the sliding speed is higher than the first speed, the background of the screen is subjected to overall special effect material replacement;

if the sliding speed is less than the second speed, the step of applying the special effect material to the screen according to the sliding information of the gesture action is not executed;

wherein the first speed is greater than the second speed.

12. The video special effect addition method according to claim 5,

after the step of performing portrait segmentation on the portrait displayed in the screen and performing special effect material replacement on the part, located in the first area, of the background in the screen, the method includes:

judging the sliding direction of the gesture action;

if the sliding direction is the forward direction far away from the sliding starting point of the finger on the screen, adding a new special effect material;

and if the sliding direction is the reverse direction of the sliding starting point of the finger on the screen, recovering the previous special effect material.

13. The video effect adding method according to claim 1, wherein the step of applying effect material to the screen according to the slide information of the gesture motion includes:

according to the sliding information of the gesture action, pulling out a control panel along with the sliding direction of the user at the sliding starting point of the gesture action on the screen, wherein a plurality of special effect materials are prevented on the control panel;

applying the special effects material to the screen.

14. The video effect adding method according to claim 1, wherein, before the step of applying effect material to the screen according to the slide information of the gesture motion, the method comprises:

and detecting the use frequency of each special effect material, and sequencing the presentation sequence of the special effect materials from more to less according to the use frequency.

15. The video effect addition method according to claim 1, characterized in that the material quantity of the effect material is less than 10.

16. A video special effect adding apparatus, characterized in that the apparatus comprises:

the information acquisition module is used for capturing gesture actions of a user and acquiring sliding information of the gesture actions;

and the special effect application module is used for applying the special effect material to the screen according to the sliding information of the gesture action.

17. The video special effects adding apparatus according to claim 16, wherein the information obtaining module includes:

the recording unit is used for capturing and recording the start point coordinate information of a sliding start point and the end point coordinate information of a sliding end point of a finger of a user in a two-dimensional xy coordinate system of the screen in real time;

the calculation unit is used for judging the sliding direction and the sliding speed of the finger according to the starting point coordinate information and the end point coordinate information of the finger;

and the judging unit is used for judging the sliding gesture adopted by the user according to the sliding direction of the finger.

18. The video effect adding apparatus according to claim 16, wherein the effect application module includes:

the segmentation unit is used for segmenting the screen into a first area and a second area according to the sliding information of the gesture action;

and the application unit is used for carrying out portrait segmentation on the portrait displayed in the screen and carrying out special effect material replacement on the part, located in the first area, of the background in the screen.

19. A terminal device, characterized in that it comprises a video special effects adding apparatus according to any one of claims 16 to 18.

20. A computer-readable storage medium comprising a program or instructions for performing the method of any one of claims 1 to 15 when the program or instructions are run on a computer.