CN115550671A

CN115550671A - Video playback method and terminal equipment in live broadcast process

Info

Publication number: CN115550671A
Application number: CN202110732459.0A
Authority: CN
Inventors: 郭站伟; 王文见; 刘晓鹏; 黄明超; 赵斌
Original assignee: Petal Cloud Technology Co Ltd
Current assignee: Petal Cloud Technology Co Ltd
Priority date: 2021-06-29
Filing date: 2021-06-29
Publication date: 2022-12-30

Abstract

The application relates to the technical field of terminals, in particular to a video playback method and terminal equipment in a live broadcast process, which can solve the problems that video playback in the live broadcast process depends on control of a director, and the terminal equipment cannot timely play back wonderful sections or easily omits to play back wonderful sections. The technical scheme provided by the embodiment includes that the terminal equipment identifies a target event in a live video in the process of playing the live video, determines a video clip related to the target event in the live video as a playback video, and automatically plays the playback video.

Description

Video playback method and terminal device in live broadcast process

Technical Field

The application relates to the technical field of terminals, in particular to a video playback method and terminal equipment in a live broadcast process.

Background

With the development of communication technology, various activities (such as sports events, product releases, and artistic nights) can be live broadcast for off-site viewers to watch online.

During the course of an activity, there are often some highlights, such as goals, penalties, etc. in a sporting event. After some highlights are live, the live party can also pause the live video and play the highlights again (i.e., playback). However, during playback, the playback strategy (including whether to playback, when to playback, the length of the playback video, etc.) of the highlight is dependent on the operation of the director. Therefore, in the process of playing the live video, the terminal device may have the problem that the highlight clip cannot be played back in time or is easily missed, and the user experience is not good.

Disclosure of Invention

The application provides a video playback method and terminal equipment in a live broadcast process, which can solve the problem that a playback video in the live broadcast process depends on the control of a director and cannot timely playback a highlight or easily omits the playback of the highlight.

In order to achieve the purpose, the following technical scheme is adopted in the application:

in a first aspect, an embodiment of the present application provides a video playback method in a live broadcast process, which is applied to a terminal device, where the terminal device identifies a target event in a live broadcast video in a process of playing the live broadcast video; determining a video clip related to a target event in a live video as a playback video; and playing the playback video.

In this embodiment, different types of live video typically correspond to different target events. For example, for a football or basketball game, the target event may be a goal, a foul, a penalty, a spectator cheering, etc. For track and field events, the target event may be a player fall, a player break, etc. For a product release meeting, the target event may be a product function demonstration process or the like. The embodiment of the present application does not limit the specific target event.

By the video playback method provided by the embodiment of the application, the terminal equipment can locally identify the target event (such as a goal event) without depending on the control of a live broadcast guide person in the process of playing the live broadcast video, plays back the video clip related to the target event, can timely play back the wonderful clip (such as a goal clip), and reduces the omission of the wonderful clip during playback.

In this embodiment, the video playback method provided in this embodiment of the present application further includes: when detecting that the first control (for example, the automatic playback control) is triggered, controlling the terminal device to enter an automatic playback mode, and executing the video playback method in the live broadcast process provided by the above embodiment.

In some embodiments, during the playing of the live video, a target event in the live video is identified; and determining a video segment related to the target event in the live video as a playback video, wherein the determining comprises the following steps: in the process of playing live video, storing video clips played in preset time as a video clip at intervals of preset time; identifying whether a target event is included in each video slice; if the Nth video fragment comprises the target event, determining the related video fragment of the Nth video fragment; determining a playback video from the nth video slice and the associated video slice.

In this embodiment, for identifying a live video frame by frame to determine a playback video corresponding to a target event, a terminal device divides the live video into a plurality of video segments, and determines a video segment including the target event and a video segment related to the video segment, so that the operation efficiency of the terminal device can be improved under the condition that the playback video is accurate as much as possible.

In some embodiments, identifying whether a target event is included in each video slice comprises: the terminal equipment locally identifies at least one frame of image in each video fragment and determines whether each video fragment comprises a target event. Compared with the method for identifying whether each video fragment comprises the target event or not through the application server, the method provided by the embodiment reduces the interaction process of the terminal equipment and the application server.

In some embodiments, identifying whether a target event is included in each video slice comprises: the terminal equipment sends at least one frame of image in each video fragment to the application server; the at least one frame of image is used for determining an identification result by the application server, and the identification result is used for judging whether the corresponding video fragment comprises a target event or not; receiving the identification result of each video fragment sent by the application server; and determining whether each video segment comprises the target event according to the identification result.

Compared with the method that the terminal equipment locally identifies whether the video fragment comprises the target event or not, the method provided by the embodiment reduces the local operation pressure of the terminal equipment and can improve the operation rate of the terminal equipment.

In some embodiments, if the target event is included in the nth video slice, determining an associated video slice of the nth video slice includes: and if the Nth video fragment comprises the target event, determining the associated video fragment of the Nth video fragment from the live video according to the storage time.

In some embodiments, determining an associated video clip of the nth video clip from the live video comprises: determining M continuous video slices with storage time before the Nth video slice as the associated video slice of the Nth video slice; the M consecutive video slices are stored adjacent to the nth video slice.

By the method provided by the embodiment, after the terminal device determines that the target event occurs in the Nth video fragment, image recognition is not needed, and the associated video fragment of the Nth video fragment can be determined quickly.

In some embodiments, determining an associated video clip of the nth video clip from the live video comprises: sequentially carrying out image recognition on video fragments with the storage time before the Nth video fragment in the live video according to the sequence of the storage time from back to first; and determining the associated video fragment of the Nth video fragment according to the identification result. By the method provided by the embodiment, the correlation degree between the associated video fragment and the target event determined by the terminal equipment is higher.

In some embodiments, playing the playback video comprises: the method comprises the steps that a live video is played in a first area of a display interface of the terminal equipment, and a playback video is played in a second area of the display interface of the terminal equipment; wherein the first region and the second region are different.

Wherein the first area (which may also be referred to as a main screen in the display area) and the second area (which may also be referred to as a sub-screen in the display area) may not overlap; for example, the terminal device plays live video and plays back video in a split screen manner. Or the second area is located inside the first area, for example, the terminal device plays the live video in a full screen mode, displays the floating window on the picture of the live video, and plays the playback video in the floating window.

By the method provided by the embodiment, the terminal equipment can play the live video and the playback video at the same time, and better user experience is achieved.

In some embodiments, playing the playback video comprises: the playback video is played, but the picture content of the live video is not displayed. Namely, the display interface of the terminal equipment only displays the picture content of the playback video. By the method provided by the embodiment, the terminal equipment plays the playback video in a larger area, so that a user can clearly see the detailed content of the playback video.

In some embodiments, the method further comprises: in some embodiments, the method further comprises: and deleting the video fragments according to the sequence of the storage time of the video fragments under the condition that the storage capacity of the video fragments reaches a preset upper storage limit. By the method provided by the embodiment, the occupation of the video fragment on the memory of the terminal equipment can be reduced.

In some embodiments, the method further comprises: in response to an editing operation on the playback video, determining K video slices with adjacent storage time; and regenerating the playback video according to the K video slices with adjacent storage time. By the method provided by the embodiment, the user can edit the playback video by himself, for example, add a video segment to the playback video, or remove a video segment from the playback video.

In a second aspect, an embodiment of the present application further provides a terminal device, where the terminal device is configured to execute the video playback method in the live broadcast process shown in the first aspect.

In a third aspect, an embodiment of the present application further provides a chip system, where the chip system includes a processor, and the processor executes a computer program stored in a memory to implement the video playback method in the live broadcast process shown in the first aspect.

In a fourth aspect, the present application further provides a computer-readable storage medium, where a computer program is stored, and when executed by a processor, the computer program implements the video playback method in the live broadcast process shown in the first aspect.

In a fifth aspect, the present application further provides a computer program product, where the program product includes a program, and when the program is executed by an electronic device, the electronic device is caused to perform the video playback method in a live broadcast process as shown in the above first aspect.

It is to be understood that, for the beneficial effects of the second aspect to the fifth aspect, reference may be made to the relevant description in the first aspect, and details are not described herein again.

Drawings

Fig. 1 is an exemplary architecture diagram of a live broadcast system provided in an embodiment of the present application;

fig. 2 is a schematic structural diagram of a terminal device according to an embodiment of the present application;

FIG. 3 is a schematic view of a user interface provided by an embodiment of the present application;

fig. 4 is a schematic flow chart of a video playback method provided by an embodiment of the present application;

fig. 5 is a schematic diagram of a video slice provided by an embodiment of the present application;

FIG. 6 is a flow chart of video playback regarding a goal event provided by an embodiment of the present application;

FIG. 7 is a schematic diagram of a process for determining a ball right provided by an embodiment of the present application;

FIG. 8 is a schematic diagram of an image to be detected in a video slice according to an embodiment of the present application;

fig. 9 is a schematic diagram of a process for determining associated video slices according to an embodiment of the present application;

FIG. 10A is a schematic view of a video playback interface provided by an embodiment of the present application;

FIG. 10B is a schematic view of a video playback interface provided by another embodiment of the present application;

fig. 11 is a schematic diagram of a playback video list provided by an embodiment of the present application;

fig. 12 is a schematic control diagram of playing back a video according to an embodiment of the present application;

fig. 13 is a schematic diagram of an editing interface for playing back a video according to an embodiment of the present application;

fig. 14 is a schematic architecture diagram of a video playback system provided in an embodiment of the present application;

fig. 15 is a schematic structural diagram of a chip provided in an embodiment of the present application.

Detailed Description

The technical solutions provided by the embodiments of the present application are described below with reference to the accompanying drawings.

It should be understood that in the description of the embodiments herein, "/" indicates an OR meaning unless otherwise noted, for example, A/B may indicate A or B; "and/or" herein is merely an association describing an associated object, and means that there may be three relationships, e.g., a and/or B, which may mean: a exists alone, A and B exist simultaneously, and B exists alone.

In the present embodiment, the terms "first", "second" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of the present embodiment, the meaning of "a plurality" is two or more unless otherwise specified.

With the development of communication technology, various activities (such as sports events, literature evenings, product releases, knowledge lectures, product sales, and the like) can be live broadcast. And receiving and playing the live broadcast video by the off-site audience through the terminal equipment, so that the progress of the activity can be watched on line.

Fig. 1 is an exemplary architecture diagram of a live broadcast system provided in an embodiment of the present application. The live broadcast system comprises a camera, an application server, a terminal device and the like.

In this embodiment, the terminal device includes a desktop computer, a mobile phone, a tablet computer, a wearable device, an in-vehicle device, an Augmented Reality (AR)/Virtual Reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a Personal Digital Assistant (PDA), and the like, and the specific type of the terminal device is not limited in this embodiment.

Referring to fig. 1, during the course of an event, the live party of the event will typically set up one or more cameras at the event site. Taking a football game as an example, the live broadcasting party usually sets cameras respectively for different directions of a court, a goal, a spectator seat and the like to acquire videos of an activity scene from different directions. For videos collected by each camera, a video is selected as a live video from the videos collected by the cameras according to activity progress conditions, and the live video is stored in an application server of a live party. For example, during the course of a game, the director will have the video about the court as live video, while during a pause or midcourt break, the video of the auditorium or the substitution auditorium will be live video. The terminal equipment can acquire and play the live broadcast video by accessing the application server, so that the user can watch the activity progress status in real time on line.

During the course of an activity, there often occur some wonderful pieces, such as the processes of goal entering, foul, penalty, break, audience cheering, etc. in a sports game, the process of demonstrating the function of a product in a product release meeting, etc. For these highlights, the user typically wants to review the highlight again after watching his live video. Thus, after some highlight occurs, the director can pause the live broadcast of the event at the appropriate time and reinsert the relevant video for the highlight into the live video to replay (i.e., playback) the highlight, which the present embodiment refers to as playback video. For example, the director may play back the highlight during a pause in the game, a half break.

In the video live broadcasting process, each frame of image in the live video corresponds to one playing time point. Based on this, after the director discovers that a highlight appears in the live video, the director can manually determine the start time T1 and the end time T2 of the highlight and clip and combine the video information of one or more cameras between T1 and T2 to form the playback video. Alternatively, the director may manually confirm the end time T2 of the highlight and automatically determine a video of a preset duration (e.g., 5S or 10S) before T2 as the playback video by the live management module of the application server.

It follows that in the above live broadcast process, the playback strategy of the highlight (including whether to playback, when to playback, playback video length, etc.) is highly dependent on the operation of the director. Therefore, in the process of playing the live video, the terminal device may have a problem that the highlight clip cannot be played back in time or is omitted to be played back, and the user experience is not good.

Therefore, the video playback method provided by the embodiment of the application enables the terminal device to play back the highlight in time without being limited by the director in the process of playing the live video, reduces omission of the highlight in the playback process, and improves user experience.

Fig. 2 shows a schematic structural diagram of a terminal device. The terminal device 200 may include a processor 210, an external memory interface 220, an internal memory 221, a Universal Serial Bus (USB) interface 230, a charging management module 240, a power management module 241, a battery 242, an antenna 1, an antenna 2, a mobile communication module 250, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, an earphone interface 270D, a sensor module 280, a key 290, a motor 291, an indicator 292, a camera 293, a display screen 294, a Subscriber Identity Module (SIM) card interface 295, and the like.

It is to be understood that the illustrated structure of the embodiment of the present application does not constitute a specific limitation to the terminal device 200. In other embodiments of the present application, terminal device 200 may include more or fewer components than shown, or some components may be combined, some components may be split, or a different arrangement of components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

For example, when the terminal device 200 is a mobile phone or a tablet computer, all components in the drawing may be included, or only some components in the drawing may be included.

When the terminal device 200 is a large-screen device such as a television, a smart screen, etc., the terminal device may include a processor 210, an external memory interface 220, an internal memory 221, a Universal Serial Bus (USB) interface 230, a charging management module 240, a power management module 241, a wireless communication module 260, an audio module 270, a speaker 270A, a receiver 270B, a microphone 270C, a camera 293, and a display 294.

Processor 210 may include one or more processing units, such as: the processor 210 may include an Application Processor (AP), a modem processor, a Graphics Processor (GPU), an Image Signal Processor (ISP), a controller, a memory, a video codec, a Digital Signal Processor (DSP), a baseband processor, and/or a neural-Network Processing Unit (NPU), among others. Wherein, the different processing units may be independent devices or may be integrated in one or more processors.

The controller may be a neural center and a command center of the terminal device 200, among others. The controller can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of instruction fetching and instruction execution.

A memory may also be provided in processor 210 for storing instructions and data. In some embodiments, the memory in processor 210 is a cache memory. The memory may hold instructions or data that have just been used or recycled by processor 210. If the processor 210 needs to use the instruction or data again, it can be called directly from memory. Avoiding repeated accesses reduces the latency of the processor 210, thereby increasing the efficiency of the system.

The charge management module 240 is configured to receive a charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 240 may receive charging input from a wired charger via the USB interface 230. In some wireless charging embodiments, the charging management module 240 may receive a wireless charging input through a wireless charging coil of the terminal device 200. The charging management module 240 may also supply power to the terminal device through the power management module 241 while charging the battery 242.

The power management module 241 is used to connect the battery 242, the charging management module 240 and the processor 210. The power management module 241 receives input from the battery 242 and/or the charging management module 240, and provides power to the processor 210, the internal memory 221, the external memory, the display 294, the camera 293, and the wireless communication module 260. The power management module 241 may also be used to monitor parameters such as battery capacity, battery cycle number, battery state of health (leakage, impedance), etc.

In some other embodiments, the power management module 241 may also be disposed in the processor 210. In other embodiments, the power management module 241 and the charging management module 240 may be disposed in the same device.

The wireless communication function of the terminal device 200 may be implemented by the antenna 1, the antenna 2, the mobile communication module 250, the wireless communication module 260, a modem processor, a baseband processor, and the like.

The

antennas

1 and 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in terminal device 200 may be used to cover a single or multiple communication bands. Different antennas can also be multiplexed to improve the utilization of the antennas. For example: the antenna 1 may be multiplexed as a diversity antenna of a wireless local area network. In other embodiments, the antenna may be used in conjunction with a tuning switch.

The mobile communication module 250 may provide a solution including wireless communication of 2G/3G/4G/5G, etc. applied on the terminal device 200. The mobile communication module 250 may include at least one filter, a switch, a power amplifier, a Low Noise Amplifier (LNA), and the like. The mobile communication module 250 can receive the electromagnetic wave from the antenna 1, filter, amplify, etc. the received electromagnetic wave, and transmit the electromagnetic wave to the modem processor for demodulation. The mobile communication module 250 may also amplify the signal modulated by the modem processor, and convert the signal into electromagnetic wave through the antenna 1 to radiate the electromagnetic wave.

In some embodiments, at least some of the functional modules of the mobile communication module 250 may be disposed in the processor 210. In some embodiments, at least some of the functional modules of the mobile communication module 250 may be disposed in the same device as at least some of the modules of the processor 210.

The modem processor may include a modulator and a demodulator. The modulator is used for modulating a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used for demodulating the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then passes the demodulated low frequency baseband signal to a baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and then transferred to the application processor. The application processor outputs sound signals through an audio device (not limited to the speaker 270A, the receiver 270B, etc.) or displays images or video through the display screen 294. In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be separate from the processor 210, and may be disposed in the same device as the mobile communication module 250 or other functional modules.

The wireless communication module 260 may provide solutions for wireless communication applied on the terminal device 200, including Wireless Local Area Networks (WLANs) (e.g., wireless fidelity (Wi-Fi) networks), bluetooth (bluetooth, BT), global Navigation Satellite System (GNSS), frequency Modulation (FM), near Field Communication (NFC), infrared (IR), and the like. The wireless communication module 260 may be one or more devices integrating at least one communication processing module. The wireless communication module 260 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signal, and transmits the processed signal to the processor 210. The wireless communication module 260 may also receive a signal to be transmitted from the processor 210, frequency-modulate and amplify the signal, and convert the signal into electromagnetic waves via the antenna 2 to radiate the electromagnetic waves.

The terminal device 200 implements a display function through the GPU, the display screen 294, and the application processor. The GPU is a microprocessor for image processing, coupled to a display screen 294 and an application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 210 may include one or more GPUs that execute program instructions to generate or alter display information.

The display screen 294 is used to display images, video, and the like. Such as instructional video and user action screen video in the embodiments of the present application, the display screen 294 includes a display panel. The display panel may be a Liquid Crystal Display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (active-matrix organic light-emitting diode, AMOLED), a flexible light-emitting diode (FLED), a miniature, a Micro-oeld, a quantum dot light-emitting diode (QLED), or the like. In some embodiments, terminal device 200 may include 1 or N display screens 294, N being a positive integer greater than 1.

The terminal device 200 may implement a shooting function through the ISP, the camera 293, the video codec, the GPU, the display screen 294, the application processor, and the like.

The ISP is used to process the data fed back by the camera 293. For example, when a user takes a picture, the shutter is opened, light is transmitted to the camera photosensitive element through the lens, an optical signal is converted into an electric signal, and the camera photosensitive element transmits the electric signal to the ISP for processing and converting the electric signal into an image visible to the naked eye. The ISP can also carry out algorithm optimization on the noise, brightness and skin color of the image. The ISP can also optimize parameters such as exposure, color temperature and the like of a shooting scene. In some embodiments, the ISP may be provided in camera 293.

The camera 293 is used to capture still images or video. The object generates an optical image through the lens and projects the optical image to the photosensitive element. The photosensitive element may be a Charge Coupled Device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to be converted into a digital image signal. And the ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into image signal in standard RGB, YUV and other formats. In some embodiments, terminal device 200 may include 1 or N cameras 293, N being a positive integer greater than 1.

The digital signal processor is used for processing digital signals, and can process other digital signals besides digital image signals. For example, when the terminal device 200 selects a frequency bin, the digital signal processor is used to perform fourier transform or the like on the frequency bin energy.

Video codecs are used to compress or decompress digital video. The terminal device 200 may support one or more video codecs. In this way, the terminal device 200 can play or record video in a plurality of encoding formats, such as: moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, and the like.

The NPU is a neural-network (NN) computing processor, which processes input information quickly by referring to a biological neural network structure, for example, by referring to a transfer mode between neurons of a human brain, and can also learn by itself continuously. The NPU can implement applications such as intelligent recognition of the terminal device 200, for example: image recognition, face recognition, speech recognition, text understanding, and the like.

In the embodiment of the present application, the NPU or other processor may be configured to perform operations such as analysis processing on images in the video stored in the terminal device 200.

The external memory interface 220 may be used to connect an external memory card, such as a Micro SD card, to extend the storage capability of the terminal device 200. The external memory card communicates with the processor 210 through the external memory interface 220 to implement a data storage function. For example, files such as music, video, etc. are saved in an external memory card.

The internal memory 221 may be used to store computer-executable program code, which includes instructions. The processor 210 executes various functional applications of the terminal device 200 and data processing by executing instructions stored in the internal memory 221. The internal memory 221 may include a program storage area and a data storage area. The storage program area may store an operating system, and an application program (such as a sound playing function, an image playing function, etc.) required by at least one function. The storage data area may store data (such as audio data, a phonebook, etc.) created during use of the terminal device 200.

In addition, the internal memory 221 may include a high-speed random access memory, and may further include a nonvolatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (UFS), and the like.

The terminal device 200 may implement an audio function through the audio module 270, the speaker 270A, the receiver 270B, the microphone 270C, the headphone interface 270D, and the application processor.

Audio module 270 is used to convert digital audio signals to analog audio signals for output and also to convert analog audio inputs to digital audio signals. Audio module 270 may also be used to encode and decode audio signals. In some embodiments, the audio module 270 may be disposed in the processor 210, or some functional modules of the audio module 270 may be disposed in the processor 210.

The speaker 270A, also called a "horn", is used to convert electrical audio signals into sound signals. The terminal device 200 can listen to music through the speaker 270A or listen to a handsfree call.

The receiver 270B, also called "earpiece", is used to convert the electrical audio signal into a sound signal. When the terminal apparatus 200 receives a call or voice information, it is possible to receive voice by bringing the receiver 270B close to the human ear.

The microphone 270C, also referred to as a "microphone," is used to convert acoustic signals into electrical signals. When making a call or transmitting voice information, the user can input a voice signal to the microphone 270C by speaking near the microphone 270C through the mouth. The terminal device 200 may be provided with at least one microphone 270C. In other embodiments, the terminal device 200 may be provided with two microphones 270C, so as to implement a noise reduction function in addition to collecting sound signals. In other embodiments, the terminal device 200 may further include three, four, or more microphones 270C to collect sound signals, reduce noise, identify sound sources, and implement directional recording functions.

The headphone interface 270D is used to connect wired headphones. The headset interface 270D may be the USB interface 230, or may be an Open Mobile Terminal Platform (OMTP) standard interface of 3.5mm, or a cellular telecommunications industry association (cellular telecommunications industry association of the USA, CTIA) standard interface.

The sensor module 280 may include a pressure sensor 280A, a gyroscope sensor 280B, an air pressure sensor 280C, a magnetic sensor 280D, an acceleration sensor 280E, a distance sensor 280F, a proximity light sensor 280G, a fingerprint sensor 280H, a temperature sensor 280J, a touch sensor 280K, an ambient light sensor 280L, a bone conduction sensor 280M, and the like.

The keys 290 include a power-on key, a volume key, and the like. The keys 290 may be mechanical keys. Or may be touch keys. The terminal device 200 may receive a key input, and generate a key signal input related to user setting and function control of the terminal device 200.

The motor 291 may generate a vibration cue. The motor 291 can be used for both incoming call vibration prompting and touch vibration feedback. For example, touch operations applied to different applications (e.g., photographing, audio playing, etc.) may correspond to different vibration feedback effects. The motor 291 may also respond to different vibration feedback effects for touch operations on different areas of the display 294. Different application scenes (such as time reminding, receiving information, alarm clock, game and the like) can also correspond to different vibration feedback effects. The touch vibration feedback effect may also support customization.

Indicator 292 may be an indicator light that may be used to indicate a state of charge, a change in charge, or may be used to indicate a message, missed call, notification, etc.

The SIM card interface 295 is used to connect a SIM card. The SIM card can be attached to and detached from the terminal device 200 by being inserted into the SIM card interface 295 or being pulled out of the SIM card interface 295. The terminal device 200 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 295 may support a Nano SIM card, a Micro SIM card, a SIM card, etc. Multiple cards can be inserted into the same SIM card interface 295 at the same time. The types of the plurality of cards may be the same or different. The SIM card interface 295 may also be compatible with different types of SIM cards. The SIM card interface 295 may also be compatible with external memory cards. The terminal device 200 interacts with the network through the SIM card to implement functions such as communication and data communication. In some embodiments, the terminal device 200 employs eSIM, namely: an embedded SIM card. The eSIM card may be embedded in the terminal apparatus 200 and cannot be separated from the terminal apparatus 200.

FIG. 3 is a schematic view of a user interface provided by an embodiment of the present application. Optionally, referring to fig. 3, in the process of playing the live video, the terminal device may set an automatic playback control on the playing interface, so as to control whether the terminal device enters an automatic playback mode. Therefore, optionally, in the automatic playback mode, the terminal device executes the video playback method provided in the embodiment of the present application, and locally controls playback of the highlight. After the user opens the automatic playback control, the terminal device enters an automatic playback mode.

Optionally, the terminal device may also enter the automatic playback mode automatically in the process of playing the live video, or enter the automatic playback mode according to a gesture instruction or a voice instruction of the user. The present embodiment does not limit the manner in which the terminal device enters the automatic playback mode.

Optionally, in the process of playing the live video, the terminal device may execute the video playback method provided by the embodiment of the present application without executing an operation of entering an automatic playback mode.

Fig. 4 is an exemplary flowchart of a video playback method provided in an embodiment of the present application, and relates to a playback process of a highlight by a terminal device. The method specifically comprises the following steps.

S401, in the process of playing the live video, the terminal device takes the video segment played in the preset time as a video segment and stores the video segment locally at preset time intervals.

In this embodiment, the preset time Δ T may be 2S, 3S, 5S, and the like, and may be specifically determined according to the type of the live video. For example, for a faster-paced event such as basketball, hectometer track and field, the terminal device may set its preset time shorter, for example 2S. For slower activities such as football, 5 km track and field events, product distribution meetings and the like, the terminal device can set the preset time for a little longer time, for example, 5S and the like.

In this embodiment, the types of live video include a soccer game, a basketball game, a track and field game, a product release, a knowledge lecture, and the like. For each live event, the live broadcaster typically provides the event name, such as football (team a VS team B), cell phone announcements, female 100 m track and field events, historical knowledge lectures, etc. The terminal equipment can determine the type of the live video according to the activity project name provided by the live party.

For a live video played by a terminal device, the terminal device needs to determine a live Identification (ID) thereof, where the live ID is used to identify a live party of the video. Illustratively, the live ID may be 1234 or the like.

Taking Δ T =5S as an example, in the process of playing a live video, the terminal device stores one video segment at an interval of 5S. Further, as shown in fig. 5, for each video clip stored locally, the terminal device needs to mark a Sequence Number (SN) in the order of its stored sequence. For example, SN =1, SN =2, SN =3, \8230 \ 8230;, SN = N-2, SN = N-1, SN = N +1, SN = N +2, etc. When the SN = N of one video fragment, the video fragment is the Nth video fragment stored by the terminal device for the current live video.

The terminal device may store the video slices in the form of a video slice list. Taking the preset time Δ T =5S as an example, and the live ID =1234 as an example, the video slicing list may be as shown in table 1.

Table 1 video clip list

SN	Live broadcast ID	Video time period
			1	1234	00:00:00-00:00:05
2	1234	00:00:06-00:00:10
			3	1234	00:00:11-00:00:15
4	1234	00:00:16-00:00:20
			5	1234	00:00:21-00:00:25
…	…	…

S402, the terminal device detects whether each video fragment includes a target event.

Different types of live video typically correspond to different target events. For example, for a football or basketball game, the target event may be a goal, a foul, a penalty, a spectator cheering, etc. For track and field events, the target event may be a player fall, a player break, etc. For a product release, the target event may be a product function demonstration process or the like. The embodiment of the present application does not limit the specific target event.

After storing each video segment, the terminal device needs to identify whether the video segment includes a corresponding target event. For example, in the process of playing a live video about a football game, for each stored video segment, the terminal device needs to identify whether target events such as goal, foul, penalty, audience cheering are included in the video segment.

Taking the target event as a goal event in a soccer game as an example, game information of both teams, such as team name, score, and uniform color, is usually displayed in a picture of a live video (e.g., the upper left corner of the picture). Based on this, the terminal device may identify the last frame image (i.e., the last frame image) of each video slice, and determine the score in the last frame image (referred to as the last frame score). And if the ratio of the last frame of the Nth video fragment is the same as that of the last frame of the (N-1) th video fragment, determining that the Nth video fragment does not include the goal event. And if the ratio of the last frame of the Nth video fragment is increased compared with the ratio of the last frame of the (N-1) th video fragment, determining that the Nth video fragment comprises the goal event.

Taking a football match between the team a and the team B as an example, if the terminal device detects that the score of the team a and the team B in the last frame image of the nth video segment is 1. If the terminal device detects that the score of the team A and the team B in the last frame image of the Nth video fragment is 1.

It should be noted that some pictures without match information are usually included in the live video, for example, the process of playing back the highlight by the live party. Thus, in some embodiments, the last frame image of a video segment stored by a terminal device may not include a score. And if the end frame score is not detected in the Nth video fragment, the terminal equipment determines that the video fragment does not comprise the goal event. If the nth video detects a last frame score (referred to as a first last frame score in this embodiment), but the nth-1 frame image does not detect a last frame score, the terminal device needs to determine a last frame score (referred to as a second last frame score in this embodiment) of a video slice with the largest SN (for example, a video slice with SN = N-2) among video slices with detected last frame scores, and compare the sizes of the first last frame score and the second last frame score. And if the first last frame score is greater than the second last frame score, the terminal equipment determines that the Nth video fragment comprises the goal event. And if the first last frame score is less than or equal to the second last frame score, the terminal equipment determines that the Nth video fragment does not comprise the goal event.

For example, referring to the end frame score identification results shown in table 2, for the video segments of SN =103, SN =104, and SN =105, since no terminal device identifies its end frame score, it is determined that no goal event is included in these video segments.

For a video segment with SN =106, since the score is 1. As can be seen by comparison, 1.

TABLE 2 end frame score identification results

SN	Last frame score (A team: B team)
		101	0:2
102	0:2
		103	Is not identified to
104	Is not identified to
		105	Is not recognized
106	1:2

In some embodiments, after storing each video segment, the terminal device sends the last frame image of the video segment to the application server, so that the application server identifies the last frame image of the video segment, obtains an identification result, and returns the identification result to the terminal device. The application server is a server corresponding to the live ID.

When the types of the live videos are different, the recognition results are also different. For example, for a ball game, the recognition result may be a score; for track and field events, the recognition result may be whether the athlete has exceeded the finish line; for a product release meeting, the identification result may be whether a product display screen is included in the image or not. The method can reduce the local operation pressure of the terminal equipment and improve the operation speed of the terminal equipment.

In other embodiments, after storing each video segment, the terminal device locally identifies the image of the video segment to obtain an identification result. Compared with the previous embodiment, in the method provided by the embodiment, the terminal device reduces the interaction process with the application server.

S403, if the nth video segment includes the target event, the terminal device determines a video segment associated with the nth video segment.

Each highlight (e.g. in a football game, a player starts holding a ball and goes into the goal) usually has a certain duration, e.g. 8S, 10S, etc. Thus, the highlight may occur in one video clip, or may occur in multiple video clips. For example, for a highlight with a duration of 8S, if the terminal device stores one video slice every 5S intervals, the duration of the highlight may occur in two consecutive video slices, or in three consecutive video slices. Therefore, after determining that the target event is included in the nth video slice, the terminal device needs to determine an associated video slice of the nth video slice.

It should be noted that the associated video segment is a part of the target event, and does not include a process unrelated to the target event. Taking the example that the target event is the goal of the team A, the associated video slices should all belong to the process of the goal of the team A, but not include the process of the ball holding of the team B. Alternatively, the back segment of the associated video slice must include a process related to the target event, while the front segment may include a process unrelated to the target event. Taking the target event as the goal event of the team a, the later segment of the associated video segment should include the goal process of the team a, and the earlier segment may include the goal process of the team B.

In some embodiments, after determining that the target event is included in the nth video segment, the terminal device may automatically determine, for the type of the current live video, M consecutive video segments before the nth video segment as associated video segments of the nth video segment. Wherein M is determined according to the type of the current live video, and M is not less than 0 and is an integer. For fast paced activities, such as basketball games, M may be set to 1, 2, etc. For a slow-paced event, such as a basketball game, M may be set to 2, 3, etc. In addition, when M ≧ 1, at least one of the M consecutive video slices is adjacent to the Nth video slice, i.e., at least includes the (N-1) th video slice.

For example, when M =2, the terminal device may automatically determine the video slices with SN = N-2 and SN = N-1 as the associated video slice of the nth video slice after determining that the target event is included in the nth video slice.

In other embodiments, after determining that a target event occurs in the nth video segment, the terminal device needs to sequentially perform image recognition on the video segments before the nth video segment according to the sequence of the sequence numbers from large to small, and determine the associated video segment of the nth video segment according to the recognition result. That is, the terminal device sequentially determines whether or not the video segments are related video segments of the nth video segment in the order of SN = N-1, SN = N-2, SN = N-3 \ 8230and \ 8230.

For example, taking the target event as a goal event as an example, referring to fig. 6, after detecting that a goal event occurs in the nth video clip (i.e., the video clip with SN = N), the terminal device needs to further determine, through image recognition, an associated video clip of the video clip. Specifically, the terminal device may sequentially perform image recognition on the video fragments with SN < N according to the sequence from the largest to the smallest of the sequence numbers, and according to whether the video fragment corresponding to the recognition result is the associated video fragment of the nth video fragment, until a video fragment that is not in the current goal process (for example, the video fragment with SN = N-3) is recognized, stop the detection of the associated video fragment in this round. By the method provided by the embodiment, the correlation degree between the associated video fragment and the target event determined by the terminal device is higher.

The following describes a process of determining whether a certain video slice is an associated video slice.

Taking the target event as a goal event as an example, the terminal device can determine whether the goal event occurs in the nth video segment, and also determine the team of the goal and the uniform color of the team of the goal by identifying the game information in the image. Based on this, the terminal equipment can judge the team to which the player belongs according to the color of the team clothes of the player. For example, if the uniform color of the a team is red and the uniform color of the B team is white, the terminal device identifies the player wearing the red uniform as the player of the a team and the player wearing the white uniform as the player of the B team after recognizing the uniform colors of the players in the image.

The team to which the player belongs can be identified based on the terminal equipment, and the terminal equipment can also determine the ball power according to the position relation between the ball and the player. Taking a football game as an example, the terminal device determines the team where the player contacting the ball physically is as the ball power party. In addition, if the ball is not in physical contact with a player (e.g., on the ground or flying in the air) and is less than or equal to a predetermined distance X (e.g., 5 meters) from the nearest player, the team in which the player is located is determined to be the ball player. If the ball is not in physical contact with the player (e.g., on the ground or flying in the air) and the distance between the two players is greater than a predetermined distance X (e.g., 5 meters), the default ball power is the goal.

For example, referring to fig. 7, taking an example that a team plays a soccer game with a team B, and the goal party is team a, the terminal device needs to identify the distance of each player in each image with respect to the ball when identifying the ball right in the image. If the player in the B team is closest to the ball and the distance does not exceed X, the ball weight of the image is determined as the B team. If the player in team A is closest to the ball and the distance does not exceed X, the ball weight of the image is determined to be team A. And if the distances between the players of the A pair of players and the B team and the ball exceed X, determining that the team A is the ball power.

Taking the process that the associated video segment does not include the process unrelated to the target event as an example, if the ball right in a certain video segment is always the goal party, the video segment is determined to be the associated video segment of the nth video segment. And if the ball right party in the video fragment is the goal party in a period of time and is not the goal party in another period of time, determining that the video fragment is not the associated video fragment of the Nth video fragment.

Optionally, referring to fig. 8, the terminal device may select some images to be detected from the (N-1) th video slice, for example, the first frame image, S random intermediate frame images, the last frame image, and identify the sphere weights in these images to be detected. Wherein S ≧ 1, e.g., S =3, S =5, and the like.

If the sphere right of each image to be detected in the (N-1) th video fragment is a goal party, the (N-1) th video fragment is part of the goal event, and the terminal equipment determines that the (N-1) th video fragment is the associated video fragment of the (N) th video fragment. And then, the terminal equipment continuously determines whether the N-2 th video fragment is the associated video fragment of the Nth video fragment, and the like until the N-i th video fragment is detected not to be the associated video fragment, and then the current round of detection is stopped.

If the ball right in any image to be detected in the (N-1) th video fragment is not the goal party, the process that the non-goal party holds the ball in the video fragment is shown, and a scene irrelevant to the goal exists, so that the (N-1) th video fragment is determined not to be the relevant video fragment of the (N) th video fragment, and the detection in the round is stopped.

In a possible implementation manner, referring to fig. 9, when determining whether a certain video segment is an associated video segment of an nth video segment, the terminal device may first identify whether a ball right (called a last frame ball right for short) in a last frame image of the video segment belongs to a goal party. If the last frame of the ball right does not belong to the goal party, the fact that the whole video fragment is irrelevant to the goal is indicated, and therefore the terminal device determines that the video frequency division is not the related video fragment of the Nth video fragment. If the last frame ball right belongs to the goal party, the ball right in the first frame image of the video fragment (called the first frame ball right for short) is further judged, and if the first frame ball right does not belong to the goal party, the fact that the front section of the whole video fragment is irrelevant to the goal of the time is indicated, so that the video frequency division is determined not to be the associated video fragment of the Nth video fragment. If the first frame ball weight belongs to the goal party, the ball weights of S intermediate frame images (referred to as intermediate frame ball weights for short) of the video fragment are further judged. Wherein S is a preset numerical value, and S is not less than 1 and is an integer. By detecting the ball weight of the intermediate frame, whether the ball weight exchange exists in the intermediate process of the video fragment can be judged. If any intermediate frame ball right does not belong to the goal party, it is indicated that ball right exchange occurs in the intermediate segment of the video segment, and the terminal device determines that the video segment does not belong to the associated video segment of the nth video segment. If each intermediate frame ball weight belongs to the goal party, the intermediate part of the video fragment belongs to the goal process, and the terminal device determines that the video fragment belongs to the associated video fragment of the Nth video fragment.

In summary, taking the process that the associated video segment does not include a process unrelated to the target event as an example, when the first frame ball weight, the S middle frame ball weights, and the last frame ball weight of the video segment all belong to the goal party, it is determined that the associated video segment of the nth video segment of the video segment is associated with the target event. The video fragment identification method provided by the embodiment of the application can reduce the number of images identified by the terminal equipment and improve the identification efficiency of the associated video fragments.

S404, the terminal equipment plays the live broadcast video and plays back the Nth video fragment and the related video fragment thereof.

Optionally, after determining the nth video segment and the associated video segments thereof, the terminal device merges the video segments into a playback video according to the sequence of the sequence numbers from small to large. Take the video slices with SN = N-1 and SN = N-2 associated with the nth video slice as an example. And the terminal equipment merges the three video fragments into a playback video according to the sequence of SN = N-2, SN = N-1 and SN = N, and puts the playback video into a playback video list. The terminal device can play the playback video while playing the live video.

It should be understood that the terminal device may not merge the nth video slice and its associated video slice. And when the terminal equipment plays back the Nth video fragment and the related video fragment thereof, the terminal equipment plays back the Nth video fragment and the related video fragment thereof according to the sequence of the sequence numbers from small to large.

In some embodiments, referring to fig. 10A, the terminal device may set a main screen and a sub screen at different positions of the display, respectively, without overlapping between the main screen and the sub screen.

In other embodiments, referring to fig. 10B, the terminal device may use the entire screen of the display as a main screen and display one sub-screen in a floating manner on the main screen.

By the method provided by the embodiment, the terminal equipment can play the live video and the playback video in different areas of the display interface at the same time, and better user experience is achieved. The display interface in the embodiment of the application may include a first area and a second area, where the first area may be regarded as a main screen and the second area may be regarded as a sub-screen.

For either of the embodiments of fig. 10A or 10B, the primary screen is used to play live video and the secondary screen is used to play playback video. And aiming at the secondary screen, the terminal equipment can set an exit control and a play control. The exit control is used for controlling the terminal equipment to exit the playback of the wonderful segment. The playback control is used to control the playback process. Illustratively, the play controls may include a replay control, a slow play control, a fast play control, a next control, a previous control, a pause/play control, a selection control, and the like.

And the replay control is used for controlling the terminal equipment to play the replay video again so as to be watched by the user again. The fast-play control is used to play the playback video at a speed lower than 1 times speed (e.g., 0.5 times speed) so that the user carefully views the video content. The fast play control is used to play the playback video at a speed higher than 1 times speed (e.g., 1.2 times speed, 1.5 times speed, 2 times speed, 3 times speed, etc.) for fast browsing by the user. And the next control is used for controlling the terminal equipment to play the next playback video. The last control is used for controlling the terminal device to play the last playback video. And the pause/play control is used for controlling the terminal equipment to pause or start playing the playback video. The selection control is used for selecting any one playback video which is supported by the terminal equipment to be played back in the current live broadcast process.

Illustratively, referring to fig. 11 (a) and fig. 11 (b), after the terminal device detects the operation of the user on the selection control, a playback video list may be displayed, where the playback video list includes videos that the terminal device supports playback during the current live broadcast, such as playback video 1, playback video 2, playback video 3, and so on. The user can select any one of the played back videos according to the requirement so as to be played in the secondary screen by the terminal equipment. It can be understood that, by the method provided by the embodiment, the user can play back the interesting highlight for multiple times.

Optionally, when the terminal device stores the playback video, some description information may be added to the playback video according to the highlight corresponding to the playback video. Taking the product release party as an example, the description information can be 'product function demonstration'; taking track and field games as an example, the description information may be "player line-punching"; taking the ball games of team a and team B as an example, the description information may be "team a goal score 1. In the process of displaying the playback video list or playing the playback video, the terminal device can display the corresponding description information, so that the user can know the video content and the user experience is improved.

In some embodiments, the terminal device may automatically play the playback video at a speed less than 1 times speed (e.g., 0.5 times speed). For example, when the duration of the playback video is less than a preset duration (e.g., 2S or 3S), the terminal device automatically plays the playback video at 0.5 × speed so that the user carefully watches the video content.

In other embodiments, the terminal device may automatically play the playback video at a speed greater than 1 times speed (e.g., 2 times speed). For example, when the duration of the playback video is greater than a preset duration (e.g., 10S or 20S), the terminal device automatically plays the playback video at 2 × speed, so that the user can quickly browse the video content.

In other embodiments, the terminal device may only play the playback video while automatically playing back the highlight, and does not display the picture content of the live video, so that the user can clearly see the detailed content of the playback video.

Because the terminal device usually only has one set of audio resources, in the process of simultaneously playing the live video and playing back the video, if the sound of the live video and the sound of the played back video are simultaneously played, the accent will appear, and the user experience is not good. Therefore, optionally, when the terminal device simultaneously plays the live video and the image information of the playback video, only the sound of the live video may be played, or only the sound of the playback video may be played.

Optionally, the terminal device may play a playback prompt picture, for example, a flashing match logo (identification) picture, before each playback video is played on the sub-screen and after each playback video is played, so as to prompt the user that the playback starts or ends, so as to improve the user experience.

It can be understood that as the live broadcast progresses, more and more video segments are stored in the terminal device, and in order to reduce the occupation of useless video segments on the memory of the terminal device, the terminal device may set an upper storage limit of one video segment, for example, at most 100 video segments are stored, or at most 500Kb video segments are stored. When the number of video fragments or the size of the occupied memory exceeds the upper limit of the storage, the terminal device may delete the redundant video fragments according to the sequence of the video fragment sequence numbers from small to large.

Alternatively, in other embodiments, after the plurality of video slices are merged into the playback video, the terminal device may determine a minimum sequence number of the merged plurality of video slices after storing the playback video in the playback video list, and delete other video slices having sequence numbers smaller than the minimum sequence number, so as to reduce memory usage of the terminal device. For example, after the nth-2, nth-1 and nth video slices are merged into playback video, the terminal device may delete all locally stored video slices with sequence numbers less than N-2.

Optionally, an automatic video fragment cleaning switch may be preset on the terminal device, and is used to control whether the terminal device automatically cleans redundant video fragments in the process of automatically playing back the highlight fragments. The video fragment automatic cleaning switch can be in an on state by default, and in addition, a user can also close or open the switch according to requirements.

When the user wants to quit playing back the highlight, the terminal device can be controlled to quit the automatic playback mode through the automatic playback control shown in fig. 3, or the quit control shown in fig. 10A and 10B, or a voice quit instruction, or a quit gesture, etc. After exiting the automatic playback mode, the terminal device continues to keep displaying the main screen to continue playing the live video, but does not display the sub-screen.

It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

It should also be understood that the present embodiment may not only be applicable to highlight playback in a live broadcast process, but also the terminal device may automatically play back a highlight in a recorded or replayed video, and the present embodiment does not limit the type of the video played by the main screen.

After the terminal device automatically plays the playback video, the playback video can be edited again, shared, or downloaded according to a user instruction. For example, referring to fig. 12, the terminal device may set an edit control, a share control, and a download control on the secondary screen. The editing control is used for the user to manually edit the playback video, for example, adding a video segment stored locally by the terminal device to the playback video, or removing a video segment in the playback video. The sharing control is used for controlling the terminal device to send the playback video to other electronic devices. In some embodiments, when the terminal device sends the playback video, the terminal device may also send a live link of the current live video to other electronic devices together, so that the opposite user can watch the live video. And the download control is used for downloading the corresponding playback video in the local terminal equipment.

In some embodiments, the terminal device displays an editing interface for playing back the video after detecting the user operation of the editing control. Illustratively, referring to fig. 13, the editing interface includes all video slices stored locally by the terminal device, or video slices close to the sequence number of the video slice in the currently played back video.

Taking the example that the video slices in the current playback video are video slices of SN = N-2, SN = N-1, and SN = N, video slices of SN = N-5, SN = N-4, SN = N-3, SN = N-2, SN = N-1, SN = N +1, and SN = N +2 may be displayed in the editing interface. The user may control the terminal device to add a video clip before SN = N-2, e.g. SN = N-4 and SN = N-3 video clips, to the playback video; alternatively, the control terminal device adds a video clip after the nth video clip, for example, a video clip with SN = N +1, to the playback video.

In addition, the user may also control the cancellation of video slices currently included in the playback video, for example, the cancellation of video slices with SN = N-2, or the cancellation of video slices with SN = N, etc. Typically, all video slices in the playback video should be continuous to avoid situations where the playback picture is discontinuous.

By the video playback method provided by the embodiment of the application, the terminal equipment can play back the highlight in time without depending on the broadcasting guide operation of a video provider after detecting the highlight, the omission of the highlight in the playback process is reduced, and better user experience is achieved.

In addition, in the process of playing back the wonderful segment, the terminal equipment does not need to pause the playing of the live video, but simultaneously plays the live video and the playback video so as to facilitate the user to watch the live video and the playback video at the same time.

Referring to fig. 14, the present embodiment also provides a video playback system including the application server shown in (a) of fig. 14, and the terminal device shown in (b) of fig. 14. The system is configured to manage video live activities and execute the video playback methods provided by the various embodiments described above. The details are as follows.

Referring to (a) of fig. 14, the application server includes a live management module 1401, a content management module 1402, a user management module 1403, and a first recognition module 1404.

And a live broadcast management module 1401, configured to manage a live broadcast room, for example, managing creation, running, ending, and a full life cycle of a dubbing broadcast of the live broadcast room.

The content management module 1402 is configured to select live videos collected by the cameras according to a director operation, form a live video to be played, and play back a played video according to the director operation.

A user management module 1403, configured to manage user information, such as an account number, a password, a history play record, and the like of the user. And the system is also used for managing the contents of authentication, comment, bullet screen management and the like when the member and the tourist participate in live broadcasting and watching the film.

The first identifying module 1404 is configured to perform image identification according to a request of the terminal device to obtain an identification result. The identification result is used for judging whether a target event occurs in a certain video fragment or whether a certain video fragment is a related video fragment of other video fragments. Taking a football or basketball game as an example, the first identifying module 1404 can be used to identify game information (such as team name, uniform color and score), the power of the game, etc. in the image. Taking a product distribution conference as an example, the first identifying module 1404 may be configured to identify whether an image of a target product of the current distribution conference is included in the image.

Referring to fig. 14 (b), the terminal device includes a slice management module 1405, a second identification module 1406, a slice playback judgment module 1407, and a playback module 1408.

The fragment management module 1405 is configured to store, as a video fragment, a video fragment within a preset time at a preset interval in the process of playing the live video by the terminal device, in the local area of the terminal device.

A second identifying module 1406 is configured to identify whether a target event occurs in each video segment and an associated video segment of the video segments that includes the target event. In some embodiments, the image recognition model is preset in the second recognition module 1406, and can recognize the video slices by itself, determine whether a target event occurs in each video slice, and determine its associated video slice. In some embodiments, the second recognition module 1406 does not have the capability of recognizing the video clip locally at the terminal device, but sends the image to be detected in the video clip to the first recognition module 1404 of the application server to be recognized by the first recognition module 1404, and sends the recognition result to the second recognition module 1406.

The segment playback judging module 1407 is configured to determine, according to the identification result of the second identifying module 1406, the video segment in which the target event occurs and the associated video segment of the video segment.

A playback module 1408, configured to merge multiple consecutive video segments into one playback video according to the sequence number from small to large. For example, the video slices are combined in the order of SN = N-2, SN = N-1, and SN = N to form one playback video. In addition, the playback module 1408 is further configured to play the video segments while playing the live video in a split screen or floating window mode.

Through the video playback system provided by the embodiment of the application, the terminal equipment can play back the highlight in time without depending on the broadcasting guide operation of a video provider after detecting the highlight, the missing of the highlight in the playback process is reduced, and better user experience is achieved.

Based on the video playback method provided by each of the above embodiments, the embodiments of the present application further provide the following:

the present embodiment provides a computer program product including a program that, when executed by an electronic apparatus, causes the electronic apparatus to execute the video playback method shown in the above-described embodiments.

Embodiments of the present application provide a computer-readable storage medium, which stores a computer program that, when executed by a processor, implements the video playback method shown in the above embodiments.

The present embodiment provides a chip, as shown in fig. 15, the chip includes a memory and a processor, and the processor executes a computer program stored in the memory to implement controlling the electronic device to execute video playback shown in the above embodiments.

It should be understood that the processor referred to in the embodiments of the present application may be a Central Processing Unit (CPU), and may also be other general purpose processors, digital Signal Processors (DSPs), application Specific Integrated Circuits (ASICs), field Programmable Gate Arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

It will also be appreciated that the memory referred to in the embodiments of the application may be either volatile memory or nonvolatile memory, or may include both volatile and nonvolatile memory. The non-volatile memory may be a read-only memory (ROM), a Programmable ROM (PROM), an Erasable PROM (EPROM), an Electrically Erasable PROM (EEPROM), or a flash memory. Volatile memory can be Random Access Memory (RAM), which acts as external cache memory. By way of example, and not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous Dynamic Random Access Memory (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), SLDRAM (synchronous DRAM), and direct rambus RAM (DR RAM).

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above-described systems, apparatuses and units may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless otherwise specifically stated.

The above-mentioned embodiments are only used to illustrate the technical solutions of the present application, and not to limit the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.

Claims

1. A video playback method in a live broadcast process is applied to terminal equipment, and is characterized by comprising the following steps:

in the process of playing a live video, identifying a target event in the live video;

determining a video segment related to the target event in the live video as a playback video;

and playing the playback video.

2. The method of claim 1, wherein during playing of a live video, a target event in the live video is identified; and determining a video segment related to the target event in the live video as a playback video, wherein the determining comprises the following steps:

in the process of playing the live video, storing the video clips played in the preset time as a video clip at intervals of preset time;

identifying whether the target event is included in each of the video slices;

if the Nth video fragment comprises the target event, determining a related video fragment of the Nth video fragment;

and determining the playback video according to the Nth video slice and the associated video slice.

3. The method of claim 2, wherein the identifying whether the target event is included in each of the video slices comprises:

locally identifying at least one image in each of the video slices, determining whether each of the video slices includes the target event;

alternatively, the first and second electrodes may be,

sending at least one frame of image in each video fragment to an application server; the at least one frame of image is used for determining an identification result corresponding to the video fragment, and the identification result is used for judging whether the video fragment comprises the target event;

receiving the identification result of each video fragment sent by the application server;

determining whether each video segment respectively comprises the target event according to the identification result of each video segment.

4. The method of claim 2 or 3, wherein determining the associated video slice of the Nth video slice if the target event is included in the Nth video slice comprises:

and if the Nth video fragment comprises the target event, determining a related video fragment of the Nth video fragment from the live video according to the storage time.

5. The method of claim 4, wherein the determining an associated video slice of an Nth video slice from the live video comprises:

determining M continuous video slices with storage time before the Nth video slice as associated video slices of the Nth video slice;

the M consecutive video slices are stored adjacent to the nth video slice.

6. The method of claim 4, wherein the determining the associated video slice of the Nth video slice from the live video comprises:

sequentially carrying out image recognition on video fragments with the storage time before the Nth video fragment in the live video according to the sequence of the storage time from the back to the first;

and determining the associated video fragment of the Nth video fragment according to the identification result.

7. The method of any of claims 1-6, wherein playing the playback video comprises:

the live video is played in a first area of a display interface of the terminal equipment, and the playback video is played in a second area of the display interface; the first region and the second region are different.

8. The method of claim 7,

the first region and the second region do not overlap; alternatively, the first and second electrodes may be,

the second region is located inside the first region.

9. The method of any of claims 1-6, wherein playing the playback video comprises:

playing the playback video but not displaying the picture content of the live video.

10. The method according to any one of claims 2-9, further comprising:

and deleting the video fragments according to the sequence of the video fragment storage time under the condition that the storage capacity of the video fragments reaches a preset upper storage limit.

11. The method according to any one of claims 2-10, further comprising:

in response to an editing operation on the playback video, determining K video slices with adjacent storage time;

and regenerating the playback video according to the K video fragments with adjacent storage time.

12. A terminal device, characterized in that the terminal device is configured to perform a video playback method in a live broadcast process according to any one of claims 1-11.

13. A system-on-chip comprising a processor executing a computer program stored in a memory to implement a method of video playback during live broadcast as claimed in any one of claims 1 to 11.

14. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out a method of video playback during live broadcast according to any one of claims 1-11.