CN114025185A

CN114025185A - Video playback method and device, electronic equipment and storage medium

Info

Publication number: CN114025185A
Application number: CN202111260144.7A
Authority: CN
Inventors: 郑锦昂; 孙有军; 刘启钧; 陈丽; 徐杭生
Original assignee: Hangzhou Netease Zhiqi Technology Co Ltd
Current assignee: Hangzhou Netease Zhiqi Technology Co Ltd
Priority date: 2021-10-28
Filing date: 2021-10-28
Publication date: 2022-02-08

Abstract

The present disclosure relates to the field of video processing technologies, and in particular, to a video playback method, an apparatus, an electronic device, and a storage medium, where in response to a playback request for a target video, each multimedia resource data associated with the target video and at least one play event data corresponding to each multimedia resource data are obtained from a server, where the play event data are used to control a play state of the multimedia resource data, and each multimedia resource data is obtained by storing data streams corresponding to different multimedia elements; respectively determining target playing areas of the multimedia resource data according to the playing area layout information of the client; and playing the multimedia resource data based on the target playing areas and the playing event data, so that a live broadcast scene is restored based on the multimedia resource data and the playing event data, and the user experience can be improved.

Description

Video playback method and device, electronic equipment and storage medium

Technical Field

The present disclosure relates to the field of video processing technologies, and in particular, to a video playback method and apparatus, an electronic device, and a storage medium.

Background

With the development of the internet, the webcast technology is widely applied to more and more scenes. For example, the method can be applied to teaching, finance, meeting, entertainment and other scenes. When the audience cannot watch the real-time live broadcast, the live broadcast content can be known in a mode of watching the playback video in the client.

In the related technology, a live broadcast client sends an obtained live broadcast video stream to a server for storage in a live broadcast process, and when video playback is needed, a playback client can obtain the live broadcast video stream from the server for playing, so that live broadcast video playback is realized.

However, in the method in the related art, because the live video stream is a standardized audio/video file that is sent from the live client to the server and is generated by the server through processing, the live video stream generated in this way can only ensure that the video frame is not distorted when the size of the playing interface of the other client is the same as that of the playing interface of the live client. However, when video playback is required in a play interface with different sizes, a situation that video pictures are distorted may occur, thereby degrading user experience.

Disclosure of Invention

The embodiment of the disclosure provides a video playback method and device, electronic equipment and a storage medium, so as to avoid distortion of a played back live video and improve user experience.

The specific technical scheme provided by the embodiment of the disclosure is as follows:

a video playback method is applied to a client and comprises the following steps:

responding to a playback request aiming at a target video, acquiring multimedia resource data related to the target video and at least one piece of play event data corresponding to the multimedia resource data from a server, wherein the play event data is used for controlling the play state of the multimedia resource data, and the multimedia resource data are acquired by respectively storing data streams corresponding to different multimedia elements;

respectively determining target playing areas of the multimedia resource data according to the playing area layout information of the client;

and playing the multimedia resource data based on the target playing areas and the playing event data.

Optionally, the method further includes:

respectively obtaining multimedia resource data and at least one play event data corresponding to each multimedia element of a video to be processed;

and sending the multimedia resource data and the playing event data to a server for associated storage.

Optionally, the obtaining multimedia resource data and at least one play event data corresponding to each multimedia element of the video to be processed respectively specifically includes:

respectively storing data streams corresponding to multimedia elements of a video to be processed to obtain multimedia resource data;

and respectively acquiring at least one playing event corresponding to each multimedia element according to a preset data format to obtain at least one playing event data.

Optionally, the obtaining, from a server, each multimedia resource data associated with the target video and at least one play event data corresponding to each multimedia resource data specifically includes:

sending the playback request to a server so that the server responds to the playback request and returns the searched multimedia resource data and at least one play event data corresponding to the multimedia data to a client;

the multimedia resource data and the playing event data are obtained by searching the multimedia resource data and the playing event data associated with the target video identifier based on the target video identifier in the playback request;

and receiving the multimedia resource data and the playing event data sent by the server.

Optionally, the determining the target playing area of each multimedia resource data according to the playing area layout information of the client includes:

acquiring a size parameter of a playing interface of the client;

determining playing area layout information of a playing interface of the client based on the size parameter, wherein the playing area layout information comprises playing areas of multimedia elements in the playing interface;

and respectively determining target playing areas corresponding to the multimedia resource data according to the playing area layout information.

Optionally, playing the multimedia resource data based on each target playing area and each playing event data specifically includes:

sequencing the playback sequence of the multimedia resource data and the playing event data according to a preset sequencing mode to obtain sequenced multimedia resource data and playing event data;

and sequentially playing the sequenced multimedia resource data based on the target playing areas, and sequentially executing the playing event data.

Optionally, the sequencing, according to a preset sequencing manner, the sequencing of the multimedia resource data and the playback sequence of the play event data specifically includes:

respectively determining the time stamp corresponding to each multimedia resource data and the time stamp corresponding to each playing event data;

and sequencing the playback sequence of the multimedia resource data and the playing event data based on the determined timestamps.

Optionally, playing the sequenced multimedia resource data in sequence, and executing the playing event data in sequence, specifically including:

determining multimedia resource data or playing event data meeting the condition of the target timestamp from the sequenced multimedia resource data and playing event data based on the target timestamp in the obtained playing progress control instruction;

and taking the determined multimedia resource data or playing event data as initial data, sequentially playing the sequenced multimedia resource data, and sequentially executing the playing event data.

Optionally, the method further includes:

responding to a playing control request, and determining a playing control type based on a control type identifier in the playing control request, wherein the playing control request at least comprises the control type identifier;

generating a control instruction aiming at the playing control type;

and controlling the playing states of the sequenced multimedia resource data and playing event data based on the control instruction.

A video playback device applied to a client comprises:

an obtaining module, configured to, in response to a playback request for a target video, obtain, from a server, each multimedia resource data associated with the target video and at least one play event data corresponding to each multimedia resource data, where the play event data is used to control a play state of the multimedia resource data, and each multimedia resource data is obtained by storing data streams corresponding to different multimedia elements;

a determining module, configured to determine target playing areas of the multimedia resource data according to the playing area layout information of the client;

and the playing module is used for playing the multimedia resource data based on the target playing areas and the playing event data.

Optionally, the system further includes an association storage module, where the association storage module is configured to:

Optionally, when the multimedia resource data and the at least one play event data corresponding to each multimedia element of the video to be processed are obtained respectively, the associated storage module is specifically configured to:

Optionally, when acquiring each multimedia resource data associated with the target video and at least one play event data corresponding to each multimedia resource data from the server, the acquiring module is specifically configured to:

Optionally, the determining module is specifically configured to:

acquiring a size parameter of a playing interface of the client;

Optionally, the playing module is specifically configured to:

Optionally, when the multimedia resource data and the playback sequence of the play event data are sorted according to a preset sorting manner, the play module is specifically configured to:

Optionally, when playing the sequenced multimedia resource data in sequence and executing the playing event data in sequence, the playing module is specifically configured to:

Optionally, the system further comprises a control module, wherein the control module is configured to:

generating a control instruction aiming at the playing control type;

An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of the video playback method described above when executing the program.

A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the above-mentioned video playback method.

The beneficial effects of the disclosed embodiment are as follows:

in response to a playback request for a target video, the video playback method, the video playback device, the electronic device, and the storage medium provided by the embodiments of the present disclosure acquire, from a server, each multimedia resource data associated with the target video and at least one play event data corresponding to each multimedia resource data, respectively determine a target play area of each multimedia resource data according to play area layout information of a client, and play each multimedia resource data based on each target play area and each play event data.

Therefore, the acquired multimedia resource data and the acquired playing event data are played in the target playing area of the multimedia resource data, and the playing size requirements of different playing interfaces can be met in the playback process, so that the user experience is improved. And when video playback is carried out, the multimedia resource data related to the target video and at least one play event data corresponding to the multimedia resource data are obtained from the server, and the play of the multimedia resource data is controlled through the play event data, so that the recording playback scene can be restored more abundantly, and the user experience is improved.

Drawings

The above and other objects, features and advantages of exemplary embodiments of the present disclosure will become readily apparent from the following detailed description read in conjunction with the accompanying drawings. Several embodiments of the present disclosure are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which:

fig. 1 is a schematic view of an application scenario of a video playback method in an embodiment of the present disclosure;

FIG. 2 is a flow chart of a method for generating multimedia asset data and playing event data in an embodiment of the present disclosure;

FIG. 3 is a flowchart of a method for obtaining multimedia resource data and playing event data according to an embodiment of the disclosure;

FIG. 4 is a flow chart of a video playback method in an embodiment of the present disclosure;

FIG. 5 is a flowchart of a method for searching multimedia resource data and playing event data according to an embodiment of the disclosure;

fig. 6 is a schematic flow chart illustrating the process of determining a target playing area in the embodiment of the present disclosure;

FIG. 7 is a first diagram of a playback interface in an embodiment of the present disclosure;

FIG. 8 is a second schematic diagram of a playback interface in an embodiment of the present disclosure;

FIG. 9 is a flowchart illustrating a method for playing multimedia asset data according to an embodiment of the disclosure;

FIG. 10 is a flowchart of a method for ordering multimedia asset data and play event data according to an embodiment of the disclosure;

FIG. 11 is a third diagram of a playback interface in an embodiment of the disclosure;

FIG. 12 is a flow chart of a method of modifying playback progress in an embodiment of the present disclosure;

FIG. 13 is a flow chart of a method of controlling a play state in an embodiment of the present disclosure;

FIG. 14 is another flow chart of a video playback method in an embodiment of the present disclosure;

fig. 15 is a schematic structural diagram of a video playback apparatus in an embodiment of the present disclosure;

fig. 16 is a schematic structural diagram of an electronic device in an embodiment of the disclosure.

Detailed Description

The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the drawings in the embodiments of the present disclosure, and it is obvious that the described embodiments are only some embodiments of the present disclosure, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments disclosed herein without making any creative effort, shall fall within the protection scope of the present disclosure.

embodiments of the present disclosure may be implemented as a system, apparatus, device, method, or computer program product. Accordingly, the present disclosure may be embodied in the form of: entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or a combination of hardware and software.

In this document, it is to be understood that any number of elements in the figures are provided by way of illustration and not limitation, and any nomenclature is used for differentiation only and not in any limiting sense.

For convenience of understanding, technical terms involved in the embodiments of the present disclosure are explained:

multimedia resource data: data associated with multimedia elements stored during a live broadcast.

The multimedia resource data may be audio, an interactive whiteboard, video, animation, document, and the like, which is not limited in the embodiment of the disclosure.

And playing event data: characterizing event data for controlling a play state of the multimedia asset data.

The playing event data may be animation start, animation stop, control restore visible, audio open, and the like, which is not limited in the embodiment of the present disclosure.

Playing area layout information: the layout information of each playing area contained in the representation playing interface refers to the size and the position of a user window which is displayed on the electronic equipment and the platform by setting multimedia contents such as audio and video, an interactive whiteboard, screen sharing video and the like through parameters appointed by a protocol.

It should be noted that each playing area is used for playing the multimedia resource data corresponding to one multimedia element.

Interactive white board: the interactive whiteboard is mainly applied to online classroom teaching scenes, and users can draw points while demonstrating contents by means of functional tools such as graffiti, texts, lines, laser pens and the like.

An interactable component: the interactive component refers to a self-defined component which can be operated by a user, the user can receive interactive feedback after operation, for example, a praise icon can be displayed when the user clicks, the praise amount is displayed and accumulated, for example, a questionnaire or a test question is displayed, and the user can interact through options and filling text.

Playback speed magnification: the playback speed magnification with respect to the natural time flow is generally 0.5 times, 1 time, 1.5 times, 2 times, and other playback magnifications, for example, the magnification is 2 times, the progress of playing for 1 second is taken, the progress of playback of a recorded playback is 2 times, and the 1-time speed indicates the speed matching the natural time.

Data: the data stream comprises information such as a starting timestamp, a duration, an ID, a user UID and a data stream type. The audio and video data stream comprises audio and video addresses or audio and video data, the interactive whiteboard data stream comprises interactive whiteboard playback historical track data, and the playing event data comprises information such as event types, data contents for executing playing events, event description and the like.

The principles and spirit of the present disclosure are explained in detail below with reference to several representative embodiments of the present disclosure.

Summary of The Invention

The present inventors have found that live broadcasting can be applied in a variety of different scenarios, such as teaching, finance, conferencing, entertainment, etc. In electronic equipment, a video is recorded in a current online live broadcast scene, and a server generates a standardized audio/video file for quality inspection, playback, archiving and evidence keeping. In the related technology, a live client sends acquired video streams, audio streams and the like to a server for storage in a live process, and the server generates a standardized audio and video file, so that the audio and video file is acquired from the server in a playback process, and then video playback can be performed. However, in the method in the related art, since the audio/video file is generated by the server, it is only ensured that the video element is prevented from being lost and the video picture is ensured not to be distorted under the condition that the size parameter of the play interface of the playback client is the same as the size parameter of the play interface of the live broadcast client. If video playback needs to be performed in different playing interfaces, the picture may be distorted due to the change of the size parameter (e.g., aspect ratio), which reduces the user experience.

In order to solve the above problem, the present disclosure provides a video playback method, an apparatus, an electronic device, and a storage medium, where in response to a playback request for a target video, each multimedia resource data associated with the target video and at least one play event data corresponding to each multimedia resource data are obtained from a server, a target play area of each multimedia resource data is determined according to play area layout information of a client, and each multimedia resource data is played based on each target play area and each play event data. In this way, in the embodiment of the present disclosure, in each target playing area, each multimedia resource data is played, and each playing event data is executed, so that in the process of video playback, the playing size requirements and customized viewing requirements of different playing interfaces can be adapted, thereby improving user experience. In addition, by the method in the embodiment of the disclosure, the playing state of the multimedia resource data is controlled by playing the event data, and a live scene can be restored, so that the user experience is improved.

Having described the general principles of the present disclosure, various non-limiting embodiments of the present disclosure are described in detail below.

Application scene overview

Reference is made to fig. 1, which is a schematic view of an application scenario of a video playback method according to an embodiment of the present disclosure. The application scenario may include a client 101 and a server 102, and the client 101 and the server 102 are connected through a communication network. The server 102 sends each multimedia resource data associated with the target video and at least one play event data corresponding to each multimedia resource data to the client 101, so that the client 101 determines a target play area of each multimedia resource data according to the layout information of the play area, and plays each multimedia resource data based on each target play area and each play event data, thereby realizing video playback.

The communication network may be a local area network, a wide area network, a mobile internet, etc., and the client 101 may be, but is not limited to: the server 102 may be an independent physical server, or may be a cloud server that provides basic cloud computing services such as a cloud server, a cloud database, and cloud storage, which is not limited in this disclosure.

Exemplary method

A video playback method according to an exemplary embodiment of the present disclosure is described below in conjunction with the above application scenarios. It should be noted that the above application scenarios are merely illustrated for the convenience of understanding the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in this respect. Rather, embodiments of the present disclosure may be applied to any scenario where applicable.

First, a detailed description is given to a process of generating multimedia resource data and playing event data in a live broadcast process in the embodiment of the present disclosure, and referring to fig. 2, a flowchart of a method for generating multimedia resource data and playing event data in the embodiment of the present disclosure is specifically included:

s20: and respectively obtaining multimedia resource data and at least one play event data corresponding to each multimedia element of the video to be processed.

In the embodiment of the disclosure, multimedia resource data and at least one play event data corresponding to each multimedia element of a video to be processed are obtained respectively.

Optionally, in the embodiment of the present disclosure, a possible implementation manner is provided for obtaining each multimedia resource data and each playing event data, and data streams of each multimedia element are stored, and a playing event of each multimedia element is collected to obtain each multimedia resource data and each playing event data, referring to fig. 3, which is a flowchart of a method for obtaining each multimedia resource data and each playing event data in the embodiment of the present disclosure, and the following describes in detail a process of obtaining each multimedia resource data and each playing event data in the embodiment of the present disclosure with reference to fig. 3, and specifically includes:

s201: and respectively storing the data streams corresponding to the multimedia elements of the video to be processed to obtain the multimedia resource data.

In the embodiment of the disclosure, in the live broadcasting process, since the video to be processed is composed of a plurality of multimedia elements, and each multimedia element can continuously generate data, data streams corresponding to each multimedia element of the video to be processed are respectively stored, so as to obtain each multimedia resource data generated in the live broadcasting process.

The video to be processed is a live video in a live broadcasting process, and the multimedia resource data comprises various data generated by multimedia elements in the live broadcasting process.

For example, assuming that a video to be processed is composed of an audio, a video, an interactive whiteboard and an animation, in a live broadcasting process, a data stream corresponding to the audio is stored to obtain audio data, the data stream corresponding to the video is stored to obtain video data, the data stream corresponding to the interactive whiteboard is stored to obtain interactive whiteboard data, and the data stream corresponding to the animation is stored to obtain animation data, wherein the audio data, the video data, the interactive whiteboard data and the animation data are multimedia resource data.

It should be noted that, in the embodiment of the present disclosure, when data streams corresponding to different multimedia elements are stored, the data streams may be stored simultaneously; the data stream corresponding to the animation may also be stored according to the presentation time of the multimedia element, for example, when the animation starts to be presented in the playing interface, the data stream corresponding to the animation is stored, which is not limited in the embodiment of the present disclosure.

Therefore, the multimedia resource data in the embodiment of the disclosure comprise animation, interface elements which can be operated by the user and the like, so that the requirements of the user on the animation and the interaction can be met, and the user experience is improved.

S202: and respectively collecting at least one playing event corresponding to each multimedia element according to a preset data format to obtain at least one piece of playing event data.

In the embodiment of the present disclosure, in a live broadcast process, at least one play event corresponding to each multimedia element data is obtained according to an agreed data format, so as to generate at least one play event data including a play event field.

It should be noted that, in the embodiment of the present disclosure, each piece of play event data is composed of play event fields, and the play event fields may be basic information such as a unique user identification code, an event type, other data, and the like, and may also be auxiliary attribute fields such as a timestamp, which is not limited in the embodiment of the present disclosure.

For example, see table 1 for a table of meaning of the play event field in the embodiment of the present disclosure.

Table 1.

Name of field	Meaning of a field
		userUid	User unique identification code
type	Event type
		Timetamp	Time stamp
data	Other data

The user unique identification code is used for uniquely identifying identity information of a live object; the event type represents the type of the playing event, such as animation playing, invisible control and the like; the timestamp represents the occurrence time of the playing event and is used for aligning the playing time of the multimedia resource data so as to synchronize the playing progress; other data is used to supplement additional data required for the execution of the play event, such as the animation content and the option operations of the interactable component.

Further, in the embodiment of the present disclosure, the playing events may be further distinguished according to event types, and refer to table 2, which is a playing event type table in the embodiment of the present disclosure.

Table 2.

It should be noted that the play event in the embodiment of the present disclosure is not limited to the above types, and is not limited thereto.

S21: and sending each multimedia resource data and each playing event data to a server for associated storage.

In the embodiment of the disclosure, each piece of multimedia resource data and each piece of play event data are sent to a server, so that the server generates a historical playback record based on the multimedia resource data and each piece of play event data, generates a video identifier for the historical playback record, associates and stores the video identifier with each piece of multimedia resource data and each piece of play event data in a video database, and the video database stores a plurality of historical playback records.

Therefore, each multimedia resource data and each playing event data correspond to one to-be-processed video, each to-be-processed video corresponds to a video identifier, and each multimedia resource data and each playing event data associated with the video identifier can be determined to be played back according to the video identifier in the playback process.

Further, in the embodiment of the present disclosure, the history playback record further includes a record ID, live broadcast room information, a live broadcast layout parameter of a live broadcast interface, and the like, which is not limited in the embodiment of the present disclosure.

The live broadcast layout parameter represents live broadcast area information of each multimedia element of a live broadcast interface in the live broadcast process.

In the embodiment of the disclosure, when a video to be processed is stored, the data streams of the multimedia elements are stored respectively. Thus, when playing back video, redundant content is not contained, and the storage volume and transmission bandwidth can be reduced.

Based on the foregoing embodiments, an embodiment of the present disclosure provides a video playback method, as shown in fig. 4, which is a flowchart of a video playback method in an embodiment of the present disclosure, and specifically includes:

s40: and responding to the playback request aiming at the target video, and acquiring each multimedia resource data associated with the target video and at least one play event data corresponding to each multimedia resource data from the server.

The playing event data is used for controlling the playing state of the multimedia resource data, and each multimedia resource data is obtained by storing the data stream corresponding to each different multimedia element.

In the embodiment of the disclosure, when a user needs to play back a video, a target video needing to be played back is selected in an operation interface of a client, a playback request for the target video is triggered in the client, the client responds to the triggered playback request and sends the triggered playback request to a server, so that the server finds each multimedia resource data associated with the target video and at least one play event data corresponding to each multimedia resource data, and then the server returns the found multimedia resource data and the at least one play event data to the client.

Optionally, in this embodiment of the present disclosure, when acquiring each multimedia resource data and each play event data from the server, each multimedia resource data and each play event data may be searched in a database set in the server based on a target video identifier corresponding to a target video, and the following describes in detail the manner of acquiring each multimedia resource data and each play event data, referring to fig. 5, which is a flowchart of a method for searching each multimedia resource data and each play event data in this embodiment of the present disclosure, specifically including:

s401: and sending the playback request to the server so that the server responds to the playback request and returns the searched multimedia resource data and at least one play event data corresponding to the multimedia data to the client.

And searching the multimedia resource data and the playing event data associated with the target video identifier based on the target video identifier in the playback request to obtain the multimedia resource data and the playing event data.

In the embodiment of the disclosure, a playback request including a target video identifier is sent to a server, so that the server responds to the received playback request after receiving the playback request to obtain the target video identifier, searches multimedia resource data and playing event data associated with the target video identifier in a database based on the target video identifier, and then returns the searched multimedia resource data and playing event data to a client.

For example, referring to table 3, a table of association relations between video identifiers and multimedia asset data and playing event data in the embodiment of the present disclosure includes video identifiers a1, a2, and A3, and multimedia asset data and playing event data associated with the video identifiers in table 3.

Table 3.

Assuming that the video identifier included in the playback request is a2, based on the association relationship between the video identifier and the multimedia resource data and each play event data, the multimedia resource data Y1 and the play event data Y2 can be found, and the multimedia resource data Y1 includes play control data and interactive whiteboard data; the play event data Y2 includes play control visible, interactive whiteboard start, interactive whiteboard stop, so that the play state of the multimedia resource data Y1 can be controlled using the play event data Y2 during video playback.

Optionally, in this disclosure, each associated multimedia resource data and each associated play event data may also be queried based on the live broadcast room identifier, which is not limited in this disclosure.

Further, in the embodiment of the present disclosure, when searching for each multimedia resource data and each play event data, the search may also be performed by using a video name of the target video, which is not limited in the embodiment of the present disclosure.

S402: and receiving each multimedia resource data and each playing event data sent by the server.

In the embodiment of the disclosure, after the server sends the searched multimedia resource data and playing event data to the client, the client receives the multimedia resource data and the playing event data sent by the server.

S41: and respectively determining the target playing area of each multimedia resource data according to the playing area layout information of the client.

In the embodiment of the present disclosure, the playing area layout information of the client is determined, and since the playing area layout information is used to indicate the playing area of each multimedia resource data in the playing interface, the target playing area of each multimedia resource data is determined based on the playing area layout information.

It should be noted that, in the embodiment of the present disclosure, the playing area layout information is different between different types of clients, for example, the playing area layout information is different between a mobile terminal and a portable computer, so that, since the devices and platforms used by a user during live broadcast and recording playback generally include a tablet computer, a mobile phone, a smart television, a PC, a browser, and the like, a suitable playing area layout can be combined based on the characteristics of the platform and the devices, so as to enhance the playback viewing experience of the user.

Optionally, in this embodiment of the present disclosure, the corresponding playing layout information may be determined according to the device identifier of the client, specifically, the device identifier of the client is determined, and the corresponding playing area layout information is determined according to the device identifier of the client and an association relationship between the device identifier and the playing area layout information.

Of course, when determining the corresponding layout information, the user may set the layout information of the playing area by himself, which is not limited in the embodiment of the present disclosure.

Optionally, in this embodiment of the present disclosure, since the sizes of the playing interfaces of the clients of the same type may be different, the playing area layout information of the client may also be determined according to the size parameter of the playing interface of the client, and the following describes in detail the process of determining the target playing area in the embodiment of the present disclosure by taking the determination of the playing area layout information based on the size parameter as an example, and referring to fig. 6, a schematic flow diagram of determining the target playing area in the embodiment of the present disclosure specifically includes:

s411: and acquiring the size parameter of the playing interface of the client.

In the embodiment of the disclosure, because the playing interface of each client corresponds to the size parameter, the interface size of the playing interface is identified, so as to obtain the size parameter of the playing interface.

For example, referring to fig. 7, the size parameter of the playing interface may be an aspect ratio, which is a first schematic diagram of the playing interface in the embodiment of the present disclosure, as can be seen from fig. 7, the aspect ratio of the playing interface is 9: 16; the size parameters of the playing interface may also be a width and a height, for example, the width is 9 and the height is 16, which is not limited in the embodiments of the present disclosure.

S412: and determining the layout information of the playing area of the playing interface of the client based on the size parameter.

The playing area layout information comprises the playing area of each multimedia element in the playing interface.

In the embodiment of the disclosure, the playing area layout information associated with the size parameter of the playing interface is determined based on the size parameter and the association relationship between the size parameter and the playing area layout information.

It should be noted that the layout information of the playing area includes the playing area of each multimedia element in the playing interface.

S413: and respectively determining target playing areas corresponding to the multimedia resource data according to the playing area layout information.

In the embodiment of the present disclosure, since the layout information of the playing area includes the position information of each target playing area on the playing interface, the target playing area corresponding to each multimedia resource data is determined according to the layout information of the playing area.

For example, referring to fig. 8, which is a second schematic diagram of a playing interface in the embodiment of the present disclosure, the multimedia elements are an interactive whiteboard, a playing control, a rate control, a progress bar control, an a user video, and a B user video, respectively, as can be seen from fig. 8, a playing area of the interactive whiteboard is area 1, a playing area of the playing control is area 2, a playing area of the rate control is area 3, a playing area of the progress bar control is area 4, a playing area of the a user video is area 5, and a playing area of the B user video is area 6.

Because the playing area of the interactive whiteboard is area 1, interactive whiteboard data is played in area 1 in the playback process; because the playing area of the playing control is area 2, in the playback process, the playing control data is played in area 2; since the playing area of the rate control is area 3, the rate control is played in area 3 during the playback process; because the playing area of the progress bar control is the area 4, in the playback process, the progress bar control data is played in the area 4; since the playing area of the user a video is area 5, the user a video data is played in area 5 during playback; since the playback area of the B-user video is area 6, the B-user video data is played in area 6 during playback.

S42: and playing each multimedia resource data based on each target playing area and each playing event data.

In the embodiment of the present disclosure, the multimedia resource data are respectively played in the playing regions corresponding to the multimedia resource data, and meanwhile, the playing event data are executed to control the playing state of the multimedia resource data, thereby implementing video playback.

Specifically, in the embodiment of the present disclosure, a possible implementation manner is provided for playing each multimedia resource data, and referring to fig. 9, a flowchart of a method for playing each multimedia resource data in the embodiment of the present disclosure is specifically shown, and includes:

s421: and sequencing the playback sequence of each multimedia resource data and each playing event data according to a preset sequencing mode to obtain the sequenced multimedia resource data and playing event data.

In the embodiment of the disclosure, a preset sorting mode is determined, and the playback sequence of each multimedia resource data and each play event data is sorted based on the preset sorting mode, so as to obtain each sorted multimedia resource data and each play event data.

The preset sorting manner may be, for example, sorting according to the size of the timestamp, or sorting according to the type priority of the multimedia resource data and each play event data, which is not limited in the embodiment of the present disclosure.

In the following, taking the sorting manner as an example of sorting according to the size of the timestamp, S421 in the embodiment of the present disclosure is described in detail, and as shown in fig. 10, a flowchart of a method for sorting each multimedia resource data and each play event data in the embodiment of the present disclosure is specifically included:

s4211: and respectively determining the time stamp corresponding to each multimedia resource data and the time stamp corresponding to each playing event data.

In the embodiment of the present disclosure, in the live broadcasting process, each stored multimedia resource data has a timestamp corresponding to each stored multimedia resource data, and each stored play event data also has a timestamp corresponding to each stored play event data, so that a timestamp corresponding to each multimedia resource data and a timestamp corresponding to each play event data are obtained.

For example, the multimedia resource data is animation data, the time stamp corresponding to the animation is 00:00, and the playing event data corresponding to the animation data is animation start and animation stop, respectively, where the time stamp corresponding to the animation start is 00:00 and the time stamp corresponding to the animation stop is 00: 15.

It should be noted that each multimedia resource data corresponds to a time stamp, and each play event data also corresponds to a time stamp.

S4212: and sequencing the playback sequence of the multimedia resource data and the playing event data based on the determined timestamps.

In the embodiment of the present disclosure, based on the determined size of the timestamp, the playback order of each multimedia resource data and each play event data is sorted, so as to obtain each sorted multimedia resource data and each play event data.

A specific example is adopted below to describe in detail a process of sequencing each multimedia resource data and each play event data in the embodiment of the present disclosure, for example, it is assumed that each multimedia resource data is animation data and interactive whiteboard data, a timestamp corresponding to the animation data is 00:00, play event data corresponding to the animation data is animation start and animation stop, a timestamp corresponding to the animation start is 00:00, a timestamp corresponding to the animation stop is 00:15, a timestamp corresponding to the interactive whiteboard data is 00:14, play event data corresponding to the interactive whiteboard data is interactive whiteboard start play and interactive whiteboard stop play, a timestamp corresponding to the interactive whiteboard start play is 00:14, and a timestamp corresponding to the interactive whiteboard stop play is 00: 30. Therefore, based on the size of the timestamp, the multimedia resource data and the playing event data are sequenced, and the data are animation data, animation start, interactive whiteboard data, interactive whiteboard start playing, animation stop and interactive whiteboard stop playing.

Since the animation data and the time stamp of the animation start are the same, the multimedia resource data animation data and the playing event data animation start are played and executed at the same time.

S422: and sequentially playing the sequenced multimedia resource data based on the target playing areas, and sequentially executing the playing event data.

In the embodiment of the present disclosure, in each target playing area, the sequenced multimedia resource data are sequentially played, and in each target playing area, each playing event data is sequentially executed, so as to control the playing state of each multimedia resource data.

For example, referring to fig. 11, a third schematic diagram of a playing interface in the embodiment of the present disclosure is shown, as shown in fig. 11, in the playing area of the interactive whiteboard, the interactive whiteboard data is played, and the playing state of the interactive whiteboard data is controlled by the playing event data, the audio/video call data of the user a is played in the audio/video call area of the user a, and the playing state of the audio/video call data of the user a is controlled by the playing event data, the audio/video call data of the user B is played in the audio/video call area of the user a, and the audio/video call data of the user C is controlled by the playing event data, and the playing state of the audio/video call data of the user C is controlled by the playing event data, the playing control data is played in the playing area, and the playing control data is controlled by the playing event data to start playing or pause playing, and playing the playing rate control data in the playing rate control area, and controlling the display and the size of the playing rate control by the playing event data.

Optionally, in the embodiment of the present disclosure, when playing multimedia resource data and executing playing event data, the playing progress may also be modified, and the process of modifying the playing progress in the embodiment of the present disclosure is described in detail below, referring to fig. 12, which is a flowchart of a method for modifying the playing progress in the embodiment of the present disclosure, specifically including:

s4221: and determining the multimedia resource data or the playing event data meeting the condition of the target timestamp from the sequenced multimedia resource data and playing event data based on the target timestamp in the acquired playing progress control instruction.

In the embodiment of the disclosure, after the play progress control instruction is acquired, in response to the play progress control instruction, since the play progress control instruction includes the target timestamp, each multimedia resource data and each play event data meeting the target timestamp condition are determined from each sequenced multimedia resource data and each play event data.

Each of the multimedia data and the play event data that satisfy the condition of the target event stamp may be multimedia resource data or play event data corresponding to a timestamp having a smallest difference from the target timestamp.

For example, assuming that the target timestamp is 00:15, two timestamps adjacent to the target timestamp are determined, namely, the timestamp 00:17 corresponding to the stop of the animation and the timestamp 00:14 corresponding to the interactive whiteboard data, so that the timestamp with the smallest difference value with the target timestamp is determined to be the timestamp 00:14 corresponding to the interactive whiteboard data, and the timestamp corresponds to the multimedia resource data, so that the determined multimedia resource data is obtained, the multimedia resource data is used as the starting data, and the subsequent multimedia resource data and the playing event data are sequentially played from the multimedia resource data.

S4222: and taking the determined multimedia resource data or playing event data as initial data, sequentially playing the sequenced multimedia resource data, and sequentially executing the playing event data.

In the embodiment of the present disclosure, the determined multimedia resource data or playing event data is used as the start data, and the sequenced multimedia resource data is sequentially played from the start data, and each playing event data is sequentially executed.

Optionally, in this embodiment of the present disclosure, during the process of playing each multimedia resource data and each playing event data, other playing states of each multimedia resource data and each playing event data may also be controlled, for example, a playing rate, a starting operation, a pausing operation, and the like, and the following explains the process of controlling the playing states of each multimedia resource data and each playing event data in this embodiment of the present disclosure, referring to fig. 13, which is a flowchart of a method for controlling the playing states in this embodiment of the present disclosure, and specifically includes:

s43: in response to the play control request, the play control type is determined based on the control type identifier in the play control request.

Wherein, the playing control request at least comprises a control type identifier.

In the embodiment of the present disclosure, a play control request is generated, and in response to the play control request, a corresponding play control type is determined based on a control type identifier in the play control request.

The control type may be, for example, start playing, pause, drag progress, stop, change the playing speed and magnification, which is not limited in the embodiment of the present disclosure.

S44: and generating a control instruction for the playing control type.

In the embodiment of the present disclosure, the determined playback control type is encapsulated, so as to generate a control instruction including the playback control type.

It should be noted that, in the embodiment of the present disclosure, the purpose of generating the control command is to control the playing status of each sequenced multimedia resource data and each playing event data, and what kind of control is specifically performed is determined by the playing control type in the control command.

S45: and controlling the playing states of the sequenced multimedia resource data and playing event data based on the control instruction.

In the embodiment of the present disclosure, based on the control instruction, the playing states of the sequenced multimedia resource data and playing event data are controlled.

For example, each multimedia asset data and each play event data can synchronously play and execute the start, pause and change the progress, stop and change the play speed multiplying factor.

In addition, it should be noted that the control operation may affect one or more multimedia asset data and play event data at the same time, for example, start, pause, stop, change progress, play speed and magnification at the same time, which is not limited in the embodiment of the present disclosure.

In the embodiment of the disclosure, in the recording process, multimedia material elements including audio, video, animation, document, interactive whiteboard trajectory, screen sharing recording and other data are stored, and playing event data is synchronously recorded, and during video playback, playing event data different from audio and video is adopted, and through the matching use of each playing event data and each multimedia resource data, the multimedia resource data and the playing event data are synchronized, a live broadcast recording scene is restored, and through executing the playing event data, the content can be controlled and displayed in an interface area including a multimedia window and other interface areas, so that the recorded playback scene is abundantly restored.

Based on the foregoing embodiment, referring to fig. 14, another flowchart of a video playback method in an embodiment of the present disclosure is shown, which specifically includes:

s1400: and starting.

S1401: and acquiring the video time length, the video address and the video timestamp corresponding to the video data.

In the embodiment of the disclosure, in response to a playback request for a target video, the playback request is sent to a server, and the server may determine, based on a video identifier in the playback request, a video duration, a video address, and a video timestamp associated with the video identifier, and send the determined video duration, video address, and video timestamp to a client, where the video address is used to obtain video data.

S1402: and acquiring interactive whiteboard data, interactive whiteboard duration and interactive whiteboard timestamp.

In the embodiment of the disclosure, the server can determine interactive whiteboard data, interactive whiteboard duration and interactive whiteboard timestamp associated with the video identifier based on the video identifier, and send the interactive whiteboard data, the interactive whiteboard duration and the interactive whiteboard timestamp to the client.

S1403: and acquiring the data of each playing event, and the corresponding timestamp and the corresponding event type.

In the embodiment of the present disclosure, the video data and the interactive whiteboard data correspond to the playing event data respectively, and the number and the type of the playing event data are not limited.

S1404: and sequencing the video data, the interactive whiteboard data and the playing event data according to the size of the timestamp to generate a playing queue.

In the embodiment of the present disclosure, based on the timestamp, the video data, the interactive whiteboard data, and each play event data are arranged from small to large, so as to generate a play queue.

The playing queue comprises sequenced video data, interactive whiteboard data and playing event data.

S1405: and determining the timestamp with the minimum value.

In the embodiment of the disclosure, the numerical value of each timestamp is determined respectively, and the timestamp with the minimum numerical value is determined.

S1406: and starting timing by taking the determined time stamp as a starting point.

In the embodiment of the present disclosure, the video data, the interactive whiteboard data, or the play event data corresponding to the timestamp with the smallest value is used as the starting point, and the timing is started.

S1407: and executing the progress dragging operation.

In the embodiment of the disclosure, the dragging progress operation corresponds to a target timestamp.

S1408: and searching corresponding data according to the target timestamp.

In the embodiment of the disclosure, the corresponding video data playing progress, interactive whiteboard data playing progress or playing event data is found according to the target timestamp.

S1409: and judging the data type, if the data type is video data, executing S1410, if the data type is interactive whiteboard data, executing S1411, and if the data type is playing event data, executing S1412.

In the embodiment of the disclosure, data corresponding to the dragged timestamp can be determined based on the data type, if the data type is video data, the playing progress of the video data is modified, if the data type is interactive whiteboard data, the playing progress of the interactive whiteboard data is modified, and if the data type is playing event data, the corresponding playing event data is skipped to and played.

S1410: and modifying the video data playing progress.

In the embodiment of the present disclosure, if the data type is video data, the playing progress of the video data is modified, that is, the video data corresponding to the target timestamp starts to be played.

S1411: and modifying the interactive whiteboard data playing progress.

In the embodiment of the present disclosure, if the data type is the interactive whiteboard data, the playing progress of the interactive whiteboard data is modified, that is, the interactive whiteboard data is played from the target timestamp.

S1412: skipping to the corresponding play event data.

In the embodiment of the present disclosure, if the data type is the play event data, the corresponding play event data is skipped to, and the play event data is executed.

S1413: and continuously playing the video data, the interactive whiteboard data and the playing event data.

In the embodiment of the present disclosure, after the corresponding video data, interactive whiteboard data, or play event data is determined, play and execution are continued according to the sorted video data, interactive whiteboard data, and play event data.

S1414: and executing normal play operation.

In the embodiment of the present disclosure, when the playing is started, each multimedia resource data and each playing event data are played at the same playing rate.

It should be noted that, if a normal play operation is executed, the video data, the interactive whiteboard data, and each play event data are sorted based on the timestamp.

S1415: and playing the video data, the interactive whiteboard data and each playing event data according to the timestamp.

In the embodiment of the disclosure, the video data, the interactive whiteboard data and the playing event data are sorted according to the size of the timestamp to obtain the sorted video data, the interactive whiteboard data and the playing event data, and the sorted video data, the interactive whiteboard data and the playing event data are sequentially played according to the timestamp.

S1416: and judging whether all the data are played completely, if so, executing S1417, and if not, executing S1415.

In the embodiment of the present disclosure, it is determined whether all data are played completely, if yes, this video playback is ended, and if not, video data, interactive whiteboard data, and each playing event data are continuously played according to the timestamp.

S1417: and (6) ending.

In the embodiment of the disclosure, in the video playback process, the playback scene is generated by matching the multimedia resource data and the play event data, so that the size requirements and layout customization requirements of different clients on the play interface can be met, and the user experience is improved.

Exemplary device

Based on the same inventive concept, an embodiment of the present disclosure further provides a video playback apparatus, where the video playback apparatus may be a hardware structure, a software module, or a hardware structure plus a software module, and the video playback apparatus embodiment may inherit the content described in the foregoing method embodiment, and based on the foregoing embodiment, as shown in fig. 15, is a schematic structural diagram of the video playback apparatus 150 in the embodiment of the present disclosure. In one embodiment, video playback device 150 includes: an acquisition module 1501, a determination module 1502, a play module 1503, an association storage module 1504, and a control module 1505.

An obtaining module 1501, configured to, in response to a playback request for a target video, obtain, from a server, each multimedia resource data associated with the target video and at least one play event data corresponding to each multimedia resource data, where the play event data is used to control a play state of the multimedia resource data, and each multimedia resource data is obtained by storing data streams corresponding to different multimedia elements;

a determining module 1502, configured to determine target playing areas of the multimedia resource data according to the playing area layout information of the client;

the playing module 1503 is configured to play the multimedia resource data based on each target playing area and each playing event data.

Optionally, the system further includes an association storage module 1504, where the association storage module 1504 is configured to:

Optionally, when obtaining multimedia resource data and at least one play event data corresponding to each multimedia element of the video to be processed, the associated storage module 1504 is specifically configured to:

Optionally, when acquiring each multimedia resource data associated with the target video and at least one play event data corresponding to each multimedia resource data from the server, the acquiring module 1501 is specifically configured to:

Optionally, the determining module 1502 is specifically configured to:

acquiring a size parameter of a playing interface of the client;

Optionally, the playing module 1503 is specifically configured to:

Optionally, when the multimedia resource data and the playback sequence of the play event data are sorted according to a preset sorting manner, the play module 1503 is specifically configured to:

Optionally, when playing the sequenced multimedia resource data in sequence and executing the playing event data in sequence, the playing module 1503 is specifically configured to:

Optionally, the system further comprises a control module 1505, the control module 1505 being configured to:

generating a control instruction aiming at the playing control type;

Based on the above embodiments, fig. 16 is a schematic structural diagram of an electronic device in an embodiment of the disclosure.

The disclosed embodiment provides an electronic device, which may include a processor 1610 (CPU), a memory 1620, an input device 1630, an output device 1640, and the like, wherein the input device 1630 may include a keyboard, a mouse, a touch screen, and the like, and the output device 1640 may include a Display device, such as a Liquid Crystal Display (LCD), a Cathode Ray Tube (CRT), and the like.

Memory 1620 may include Read Only Memory (ROM) and Random Access Memory (RAM), and provides processor 1610 with program instructions and data stored in memory 1620. In the embodiment of the present disclosure, the memory 1620 may be used to store a program of any one of the video playback methods in the embodiment of the present disclosure.

The processor 1610 is configured to execute any one of the video playback methods according to the present disclosure by calling the program instructions stored in the memory 1620.

Based on the above embodiments, in the embodiments of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, the computer program, when executed by a processor, implementing the video playback method in any of the above method embodiments.

As will be appreciated by one skilled in the art, embodiments of the present disclosure may be provided as a method, system, or computer program product. Accordingly, the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present disclosure may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and so forth) having computer-usable program code embodied therein.

The present disclosure is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to the present disclosure. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.

These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.

It will be apparent to those skilled in the art that various changes and modifications can be made in the present disclosure without departing from the spirit and scope of the disclosure. Thus, if such modifications and variations of the present disclosure fall within the scope of the claims of the present disclosure and their equivalents, the present disclosure is intended to include such modifications and variations as well.

Claims

1. A video playback method is applied to a client and comprises the following steps:

2. The method of claim 1, further comprising:

3. The method of claim 2, wherein the obtaining multimedia resource data and at least one play event data corresponding to each multimedia element of the video to be processed respectively comprises:

4. The method of claim 1, wherein obtaining, from a server, each multimedia resource data associated with the target video and at least one play event data corresponding to each multimedia resource data specifically comprises:

5. The method of claim 1, wherein the determining the target playing area of each multimedia resource data according to the playing area layout information of the client comprises:

acquiring a size parameter of a playing interface of the client;

6. The method of claim 1, wherein playing the multimedia asset data based on the target playing areas and the playing event data comprises:

7. The method as claimed in claim 6, wherein the step of sequencing the playback sequence of the multimedia resource data and the play event data according to a preset sequencing manner comprises:

8. A video playback apparatus applied to a client, comprising:

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the steps of the method of any of claims 1-7 are implemented when the program is executed by the processor.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that: the computer program when executed by a processor implements the steps of the method of any one of claims 1 to 7.