WO2021012665A1

WO2021012665A1 - Interactive audio and video playing method and apparatus

Info

Publication number: WO2021012665A1
Application number: PCT/CN2020/074964
Authority: WO
Inventors: 陈燕鹏
Original assignee: 网宿科技股份有限公司
Priority date: 2019-07-25
Filing date: 2020-02-12
Publication date: 2021-01-28
Also published as: US20210160577A1; CN111787406A; CN111787406B; CN110381384A; GB202101362D0; GB2590018A

Abstract

Disclosed are an interactive audio and video playing method and apparatus. The method comprises: a mobile device acquires directory information of audios and videos to be played (201); when determining that the playing mode is interactive preview playing, the mobile device determines an interactive tree relationship and an audio and video list corresponding thereto according to path information (202); the mobile device performs interactive audio and video playing according to the interactive tree relationship and the audio and video list corresponding thereto as well as an audio and video clip selected by a user (203).

Description

Method and device for interactive audio and video playback

cross reference

This application refers to the Chinese patent application No. 201910676150.7 entitled "A method and device for interactive audio and video playback" filed on July 25, 2019, which is fully incorporated into this application by reference.

Technical field

The embodiments of the present application relate to the technical field of video playback, and in particular, to a method and device for interactive audio and video playback.

Background technique

With the rapid development of mobile Internet and the continuous improvement of terminal equipment performance, the behavior of using mobile devices to watch live and on-demand videos online anytime and anywhere is becoming more and more popular, and because of the real-time nature of live broadcasting, good interaction between the host and the audience can be achieved. There is no real-time interaction.

The audience in the traditional streaming media on-demand playback method is still a single receiver, and the introduction of interactive video allows traditional on-demand viewers to participate in the interaction related to the video content when watching the video content, enhance the interactive interest, and achieve more Value video on demand.

The current audio and video playback scene on the mobile terminal only performs a series of steps of reading from start to end, decoding, audio and video rendering, and audio and video synchronization for a single media stream or file. The current audio and video playback scene does not yet support interactive mode Play.

Summary of the invention

The embodiments of the present application provide a method and device for interactive audio and video playback, which are used to implement interactive audio and video playback on mobile devices.

An embodiment of the application provides an interactive audio and video playback method, including:

The mobile device obtains directory information of the audio and video to be played, where the directory information includes path information and play mode;

When the mobile device determines that the play mode is interactive preview play, determine the interactive tree relationship and its corresponding audio and video list according to the path information;

The mobile device performs interactive audio and video playback according to the interactive tree relationship and its corresponding audio and video list and audio and video clips selected by the user.

In the above technical solution, after the mobile device parses out the interactive tree relationship and its corresponding audio and video list, it can perform interactive audio and video playback based on the interactive tree relationship, the audio and video list, and the audio and video clips selected by the user. , Since the interactive tree relationship can reflect the association relationship of each audio and video clip, during audio and video playback, it can show the user other audio and video clips associated with the currently playing audio and video clip, so that interactive audio and video can be realized on mobile devices The playback of fragments solves the problem that the current audio and video playback scenes of mobile devices are only played against a single audio and video stream.

In an embodiment, the method further includes:

When the mobile device determines that the playback mode is the mainline preview playback, it determines the mainline tree relationship and its corresponding audio and video list according to the path information;

The mobile device performs mainline audio and video playback according to the mainline tree relationship and its corresponding audio and video list.

In the above technical solution, when the playback mode is determined to be the mainline preview playback, the mainline audio and video playback can be performed according to the mainline tree relationship and the corresponding audio and video list, thereby realizing two playback modes.

In one embodiment, the mobile device performs mainline audio and video playback according to the mainline tree relationship and its corresponding audio and video list, including:

Acquiring, by the mobile device, the segment identifier of the first audio and video segment in the mainline tree relationship;

The mobile device determines the URL of the first audio and video segment from the audio and video list corresponding to the main-line tree relationship according to the segment identifier of the first audio and video segment;

The mobile device creates a player according to the URL of the first audio and video segment to perform mainline audio and video playback, and adds the player to the pre-loaded player queue.

In the above technical solution, the preview of audio and video clips can be realized by preloading, which saves system resources.

In an embodiment, after the mobile device performs mainline audio and video playback, the method further includes:

The mobile device obtains the segment identifier of the next audio and video segment of the currently played audio and video segment, and determines the URL of the next audio and video segment from the audio and video list corresponding to the mainline tree relationship;

The mobile device determines, according to the URL of the next audio and video segment, whether there is a player in the preloaded player queue that plays the URL of the next audio and video segment, and if not, creates and plays the next audio and video segment The URL of the player and the player added to the pre-loaded player queue, and continue to obtain the segment identifier of the audio-video segment after the next audio-video segment; until the number of players in the pre-loaded player queue It is greater than the preloading threshold and/or the number of audio and video loaded by the players in the preloading player queue is greater than the number threshold.

In the above technical solution, by setting the preload threshold and the quantity threshold, system resources can be saved.

After the mobile device determines that the playback of the currently played audio and video fragment is over, it determines the next audio and video fragment of the currently played audio and video fragment as the currently played audio and video fragment to perform the main-line audio and video playback, Until the audio and video in the main-line tree relationship is completed or the user's instruction to stop playing is received.

In the above technical solution, the main-line audio and video playback can be realized by preloading, which further saves system resources.

In one embodiment, the mobile device performs interactive audio and video playback according to the interactive tree relationship and its corresponding audio and video list and audio and video clips selected by the user, including:

Acquiring, by the mobile device, the segment identifier of the audio and video segment corresponding to the root node of the interactive tree relationship;

The mobile device determines the URL of the audio and video segment corresponding to the root node from the audio and video list corresponding to the interactive tree relationship according to the segment identifier of the audio and video segment corresponding to the root node, and according to the The URL of the audio and video segment corresponding to the root node performs interactive video playback;

The mobile device parses all sub-nodes of the root node to obtain the segment identifiers of the audio and video segments corresponding to all the sub-nodes; according to the segment identifiers of the audio and video segments corresponding to all the sub-nodes from the interactive tree The URLs of the audio and video clips corresponding to all the child nodes are determined in the audio and video list corresponding to the relationship, and are added to the list of play addresses to be interactively selected for preloading, where the number of players in the preloaded player queue is greater than The preloading threshold and/or the number of audio and video loaded by the players in the preloading player queue is greater than the number threshold.

In the above technical solution, by analyzing the interactive tree relationship, all the child nodes of the root node are obtained and preloaded, which can save system resources.

In an embodiment, after the mobile device performs interactive video playback according to the URL of the audio and video segment corresponding to the root node, the method further includes:

When the mobile device determines that the playback of the currently played audio and video fragment is over, it shows the user the problem of the currently played audio and video fragment and the audio and video fragments corresponding to each sub-node in the play address list to be interactively selected URL options to enable the user to select the corresponding audio and video clips;

The mobile device determines the URL of the audio and video fragment selected by the user from the list of playback addresses to be interactively selected according to the fragment identifier of the audio and video fragment selected by the user to continue the interactive video playback, and parses The child node of the audio/video segment selected by the user obtains the segment identifier of the audio/video segment corresponding to the child node of the audio/video segment selected by the user, according to the audio/video segment corresponding to the child node of the audio/video segment selected by the user The segment identifier determines the URL of the audio and video segment corresponding to the sub node of the audio and video segment selected by the user from the audio and video categories corresponding to the interactive tree relationship, and adds it to the list of play addresses to be interactively selected Pre-loading is performed in the interactive tree until the audio and video clips in the interactive tree relationship are all played or the user's instruction to stop playing is received.

In the above technical solution, interactive audio and video playback is performed through interaction with the user, which can solve the problem that the current audio and video playback scenes of mobile devices are only played on a single audio and video stream.

In one embodiment, the audio/video segment selected by the user is determined from the audio/video category corresponding to the interactive tree relationship based on the segment identifier of the audio/video segment corresponding to the child node of the audio/video segment selected by the user. The URL of the audio and video clip corresponding to the child node of the video clip is added to the list of playback addresses to be interactively selected for preloading, including:

Determine whether the URL of the audio and video segment corresponding to the child node of the audio and video segment selected by the user exists in the play address list to be interactively selected, if it exists, do not add it, if it does not exist, remove the non-existent child node The URL of the corresponding audio and video segment is added to the list of play addresses to be interactively selected for preloading; and the play address list to be interactively selected is corresponding to the child node that does not belong to the audio and video segment selected by the user The URL of the audio and video clip is deleted.

In the above technical solution, the URL of the audio and video segment corresponding to the child node of the audio and video segment selected by the user is determined in the play address list to be interactively selected, and the URL of the audio and video segment in the play address list to be interactively selected can be realized Perform deduplication and removal, so that there is no repeated removal and preloading, reducing bandwidth and system resource waste.

In the second aspect, an embodiment of the present application provides an interactive audio and video playback device, including:

An obtaining unit, configured to obtain directory information of the audio and video to be played, the directory information including path information and playing mode;

The processing unit is configured to determine an interactive tree relationship and its corresponding audio and video list according to the path information when it is determined that the playback mode is interactive preview playback; and according to the interactive tree relationship and its corresponding The audio and video list and the audio and video clips selected by the user are played interactively.

In an embodiment, the processing unit is further configured to:

When determining the play mode as main-line preview playback, determine the main-line tree relationship and its corresponding audio and video list according to the path information;

Perform main-line audio and video playback according to the main-line tree relationship and its corresponding audio and video list.

In an embodiment, the processing unit is specifically configured to:

Acquiring the segment identifier of the first audio and video segment in the mainline tree relationship;

Determine the URL of the first audio and video segment from the audio and video list corresponding to the mainline tree relationship according to the segment identifier of the first audio and video segment;

Create a player according to the URL of the first audio and video segment to perform mainline audio and video playback, and add the player to the pre-loaded player queue.

In an embodiment, the processing unit is further configured to:

After the main-line audio and video playback is performed, the segment identifier of the next audio-video segment of the currently-played audio-video segment is obtained, and the next audio-video list is determined from the audio-video list corresponding to the main-line tree relationship. The URL of the video clip;

According to the URL of the next audio and video segment, determine whether there is a player in the preloaded player queue that plays the URL of the next audio and video segment; if not, create a playback that plays the URL of the next audio and video segment And add to the preloaded player queue, and continue to obtain the segment identifiers of the audio and video clips after the next audio and video clip; until the number of players in the preloaded player queue is greater than the preload threshold And/or until the number of audio and video loaded by the players in the preloaded player queue is greater than the number threshold.

In an embodiment, the processing unit is further configured to:

After the main-line audio and video playback is performed, after it is determined that the playback of the currently played audio and video fragment is over, the next audio and video fragment of the currently played audio and video fragment is determined to be the currently played audio and video fragment. The main-line audio and video are played until the audio and video in the main-line tree relationship is finished or the user's instruction to stop playing is received.

In an embodiment, the processing unit is specifically configured to:

Acquiring the segment identifier of the audio and video segment corresponding to the root node of the interactive tree relationship;

According to the segment identifier of the audio and video segment corresponding to the root node, the URL of the audio and video segment corresponding to the root node is determined from the audio and video list corresponding to the interactive tree relationship, and according to the corresponding root node URL of audio and video clips for interactive video playback;

Analyze all the sub-nodes of the root node to obtain the segment identifiers of the audio and video segments corresponding to all the sub-nodes; according to the segment identifiers of the audio and video segments corresponding to all the sub-nodes, obtain the audio and video corresponding to the interactive tree relationship The URLs of the audio and video clips corresponding to all the child nodes are determined in the list and added to the list of play addresses to be interactively selected for preloading, where the number of players in the preloaded player queue is greater than the preloading threshold and/ Or the number of audio and video loaded by the players in the preloaded player queue is greater than the number threshold.

In an embodiment, the processing unit is further configured to:

After the interactive video playback is performed according to the URL of the audio and video fragment corresponding to the root node, when it is determined that the playback of the currently played audio and video fragment is over, the problem of the currently played audio and video fragment is shown to the user and The option of the URL of the audio and video segment corresponding to each sub-node in the play address list to be interactively selected, so that the user can select the corresponding audio and video segment;

According to the segment identifier of the audio and video segment selected by the user, the URL of the audio and video segment selected by the user is determined from the play address list to be interactively selected to continue the interactive video playback, and the user selection is analyzed The sub-node of the audio-visual segment of the user selects the audio-video segment corresponding to the sub-node of the audio-video segment selected by the user, and the audio-video segment corresponding to the sub-node of the audio-video segment selected by the user is obtained from The URL of the audio and video segment corresponding to the sub-node of the audio and video segment selected by the user is determined from the audio and video categories corresponding to the interactive tree relationship, and is added to the list of play addresses to be interactively selected for preloading , Until the audio and video clips in the interactive tree relationship are all played or the user's instruction to stop playing is received.

In an embodiment, the processing unit is further configured to:

In the third aspect, an embodiment of the present application also provides a computing device, including:

Memory, used to store program instructions;

The processor is configured to call the program instructions stored in the memory and execute the above interactive audio and video playback method according to the obtained program.

In a fourth aspect, the embodiments of the present application also provide a computer-readable non-volatile storage medium, including computer-readable instructions. When the computer reads and executes the computer-readable instructions, the computer executes the interactive audio The method of video playback.

Description of the drawings

In order to more clearly describe the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative work, other drawings can be obtained from these drawings.

FIG. 1 is a schematic diagram of a system architecture provided by an embodiment of the application;

2 is a schematic flowchart of an interactive audio and video playback method provided by an embodiment of the application;

FIG. 3 is a schematic structural diagram of an interactive audio and video playback device provided by an embodiment of the application.

Detailed ways

In order to make the purpose, technical solutions and advantages of the application more clear, the application will be described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the application, rather than all of them. Based on the embodiments in this application, all other embodiments obtained by a person of ordinary skill in the art without creative work shall fall within the protection scope of this application.

Figure 1 is a system architecture to which an embodiment of the application is applicable. As shown in FIG. 1, the system architecture may be a mobile device 100, including a radio frequency (RF) circuit 110, a memory 120, an input unit 130, a display unit 140, a sensor 150, an audio circuit 160, and wireless fidelity (wireless fidelity). , WiFi) module 170 and processor 180. The mobile device 100 can be used to generate interactive audio and video.

Those skilled in the art can understand that the structure of the mobile device 100 shown in FIG. 1 is only an example and not a limitation. The mobile device 100 may also include more or less components than those shown in the figure, or combine certain components, or Different component arrangements.

The RF circuit 110 can be used to receive and send signals during the process of sending and receiving data or during a call. In particular, the received audio and video streams are sent to the processor 180 for processing.

The memory 120 may be used to store software programs and modules. The processor 180 executes various functional applications and data processing of the mobile device 100 by running the software programs and modules stored in the memory 120. The memory 120 may mainly include a program storage area and a data storage area. The program storage area may store an operating system, an application program required by at least one function (such as a sound playback function, an image playback function, etc.), etc.; Data (such as audio data, phone book, etc.) created by the use of the mobile device 100, etc. In addition, the memory 120 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other volatile solid-state storage devices.

The input unit 130 may be used to receive input numeric or character information, and generate key signals related to user settings and function control of the mobile device 100. Specifically, the input unit 130 may include a touch panel 131, a camera 132, and other input devices 133.

Among them, the display unit 140 can be used to display information input by the user or information provided to the user and various menus of the mobile device 100. The display unit 140 may include a display panel 141. Optionally, the display panel 141 may be configured in the form of a liquid crystal display unit (LCD, Liquid Crystal Display), an organic light emitting diode (OLED, Organic Light-Emitting Diode), etc. Further, the touch panel 131 can cover the display panel 141. When the touch panel 131 detects a touch operation on or near it, it transmits it to the processor 180 to determine the type of the touch event, and then the processor 180 responds to the touch event. The type provides corresponding visual output on the display panel 141.

In addition, the mobile device 100 may further include at least one sensor 150, such as an attitude sensor, a distance sensor, and other sensors.

The audio circuit 160, the speaker 161, and the microphone 162 can provide an audio interface between the user and the mobile device 100. The audio circuit 160 may transmit the converted electrical signal of the received audio data to the processor 180 for language recognition.

WiFi is a short-distance wireless transmission technology. Through the WiFi module 170, the mobile device 100 can help users send and receive emails, browse webpages, and access streaming media (such as live video), etc. It provides users with wireless broadband Internet access.

The processor 180 is the control center of the mobile device 100. It uses various interfaces and lines to connect the various parts of the entire mobile device 100, runs or executes software programs and/or modules stored in the memory 120, and calls and stores them in the memory 120. , Execute various functions of the mobile device 100 and process data, so as to monitor the mobile device 100 as a whole.

Based on the foregoing description, FIG. 2 exemplarily shows the flow of a method for interactive audio and video playback provided by an embodiment of the present application. The flow can be executed by an interactive audio and video playback device, which can be located in FIG. 1 The mobile device 100 shown may also be the mobile device 100.

As shown in Figure 2, the process specifically includes:

Step 201: The mobile device obtains directory information of the audio and video to be played.

In the embodiment of the present application, the directory information may include path information and a play mode, where the path information may be the path of the directory of the audio and video to be played, or may be a URL. Wherein, if the source of the audio and video clip is locally produced, the path can be a local path, and if the source of the audio and video clip is remote, it is the URL where the current audio and video clip is stored on the remote mobile device. The play mode can be divided into two modes: interactive preview play and mainline preview play. Among them, the interactive preview mode can interact with the user. In the specific implementation process, the play mode can be identified by setting the flag. For example, when the flag is 1, the play mode is interactive preview play, and when the flag is 0, the play mode is mainline preview play.

Step 202: When the mobile device determines that the play mode is interactive preview play, it determines the interactive tree relationship and its corresponding audio and video list according to the path information.

When it is determined that the play mode is interactive preview play, the mobile device can obtain the interactive tree relationship and its corresponding audio and video list according to the path information. For example, to obtain the path of JSON, and parse the JSON to obtain 2 objects: tree and videos, corresponding respectively: interactive tree relationship and audio and video list.

Step 203: The mobile device performs interactive audio and video playback according to the interactive tree relationship and its corresponding audio and video list and audio and video clips selected by the user.

After the interactive tree relationship and its audio and video list are obtained, interactive audio and video playback can be carried out. Specifically, the segment identifier of the audio and video fragment corresponding to the root node of the interactive tree relationship can be obtained, and then based on the corresponding root node The segment identifier of the audio and video segment determines the URL of the audio and video segment corresponding to the root node from the audio and video list corresponding to the interactive tree relationship, and performs interactive video playback according to the URL of the audio and video segment corresponding to the root node. Then parse all the sub-nodes of the root node to obtain the segment identifiers of the audio and video segments corresponding to all sub-nodes, and determine all sub-nodes from the audio and video list corresponding to the interactive tree relationship according to the segment identifiers of the audio and video segments corresponding to all sub-nodes The URL of the corresponding audio and video clip is added to the list of play addresses to be interactively selected for preloading. Wherein the number of players in the preloaded player queue is greater than the preload threshold and/or the number of audio and video loaded by the players in the preloaded player queue is greater than the number threshold, and the preload threshold and the number threshold may be based on Experience setting.

In addition, when the mobile device determines that the currently playing audio and video clip is over, it can show the user the problem of the currently playing audio and video clip and the option of the URL of the audio and video clip corresponding to each sub-node in the play address list to be interactively selected. , So that the user can select the corresponding audio and video clips. Then, according to the segment identifier of the audio and video segment selected by the user, the URL of the audio and video segment selected by the user can be determined from the play address list to be interactively selected to continue the interactive video playback, and the child nodes of the audio and video segment selected by the user can be parsed , To obtain the segment identifier of the audio and video segment corresponding to the sub node of the audio and video segment selected by the user, and from the audio and video categories corresponding to the interactive tree relationship according to the segment identifier of the audio and video segment corresponding to the sub node of the audio and video segment selected by the user Determine the URL of the audio and video clip corresponding to the child node of the audio and video clip selected by the user, and add it to the list of play addresses to be interactively selected for preloading until the audio and video clips in the interactive tree relationship are all played or received Until the user's instruction to stop playing.

Among them, when adding to the list of play addresses to be interactively selected for preloading, you can first determine whether the URL of the audio and video segment corresponding to the child node of the audio and video segment selected by the user exists in the list of play addresses to be interactively selected. If it exists, Do not add, if it does not exist, add the URL of the audio and video clip corresponding to the child node that does not exist to the list of play addresses to be interactively selected for preloading; and select the play address list to be interactively selected that does not belong to the user selection The URL of the audio and video clip corresponding to the child node of the audio and video clip is deleted.

For example, the root node of the tree (interactive tree relationship) is parsed, video_id (fragment identifier) is obtained, videos (audio and video list) are traversed, and the player address of the object with the same video_id is obtained for "single fragment preview" to play. Among them, "single clip preview" can be to get the clip selected by the user to be previewed, traverse JSON to get the player path of the current clip, if the interactive video source is made locally, it is the local path, if the interactive video source is remote, it is the remote terminal Store the URL of the current video. At this time, you can call the path or URL of the player to play the segment.

Analyze the children (child node) field of the currently playing node, if it exists, get the video_id of n child nodes, and traverse videos to determine the list of player addresses to be interactively selected after the playback of the currently playing node ends. Create players one by one, initialize the list of player addresses to be interactively selected, and store them in the preloaded player array, and configure the amount of preloaded data. The amount of preloaded data can be set based on experience. The amount of preloaded data is the aforementioned preload threshold.

When the currently playing node has finished playing, the question corresponding to the currently playing node will pop up, and the answer of each child node in the player address list to be interactively selected will be displayed. After the user's selection is completed, the corresponding player is taken out of the preloaded player array and played directly. If it does not exist, the player is recreated and the current playback address is played.

After the traversal selection is completed, the player addresses contained in all child nodes of the currently playing node are compared with the playback addresses of the players contained in the pre-loaded player array. If they match, they will be retained, otherwise the player will be removed from the array and combined Release (de-duplication).

Repeat the above steps until the player address list to be interactively selected is empty, or the user issues an instruction to stop playing.

For example, the child nodes of node 1 are 2 and 3; the child nodes of node 2 are 4 and 5. When node 1 is playing, the child nodes 2 and 3 of node 1 have been added to the player address list to be interactively selected for preloading. At this time, when interacting with the user, the user selects node 2, then the audio and video clips of node 2 are played. Since there is node 3 in the player address list to be interactively selected, node 3 needs to be removed at this time, and then node 4 and Node 5 is put into the list of player addresses to be interactively selected for preloading.

If the child nodes of node 1 are 2, 3, and 4, the child nodes of node 2 are 3 and 5. When node 1 is playing, the child nodes 2, 3, and 4 of node 1 have been added to the list of player addresses to be interactively selected for preloading. At this time, when interacting with the user, the user selects node 2, and the audio and video clips of node 2 are played. Since there are nodes 3 and 4 in the list of player addresses to be interactively selected, node 3 needs to be removed at this time, and then Node 5 is put into the list of player addresses to be interactively selected for preloading. And node 4 does not need to be removed, only node 5 needs to be preloaded, which can save system resources.

It should be noted that when the mobile device determines the play mode as the mainline preview, it can determine the mainline tree relationship and its corresponding audio and video list according to the path information, and finally according to the mainline tree relationship and its corresponding audio and video List for main-line audio and video playback.

Specifically, the segment identifier of the first audio and video segment in the main-line tree relationship can be obtained, and the first audio-video segment is determined from the audio and video list corresponding to the main-line tree relationship according to the segment identifier of the first audio and video segment. A URL of an audio and video segment, and then create a player based on the URL of the first audio and video segment to perform mainline audio and video playback, and add the player to the pre-loaded player queue.

During main-line playback, the mobile device can obtain the segment identifier of the next audio-video segment of the currently playing audio-video segment, and determine the URL of the next audio-video segment from the audio-video list corresponding to the main-line tree relationship. According to the URL of the next audio and video clip, it is determined whether there is a player in the preloaded player queue that plays the URL of the next audio and video clip. If not, a player that plays the URL of the next audio and video clip is created and added to the pre-loaded player queue. Load the player queue, and continue to obtain the segment identifiers of the audio and video clips located after the next audio and video clip, until the number of players in the preloaded player queue is greater than the preloaded threshold and/or is in the preloaded player queue The number of audio and video loaded by the player is greater than the number threshold. The preload threshold and the quantity threshold can be set based on experience, for example, the preload threshold can be the amount of preloaded data.

When the mobile device determines that the currently playing audio and video clip is over, it can determine the next audio and video clip of the currently playing audio and video clip as the currently playing audio and video clip for main-line audio and video playback until the main-line tree relationship The audio and video in the video are played until the end or the user's stop playing instruction is received.

For example, after obtaining the JSON path, parse the JSON to obtain 2 objects: main_thread and videos, which correspond to the main thread tree relationship and the audio and video list respectively.

Obtain the first index object of main_thread, traverse videos through the video_id of the object to obtain the player address of the object with the same video_id to create a player, and add it to the preloaded player queue. Traverse the player objects in the pre-loaded player queue through the current playback address, and find a matching player for preview playback.

Get the next index object of the currently playing audio and video clip, traverse videos through the video_id of the object to obtain the player address of the object with the same video_id. If the player's playback address matches with the pre-loaded player queue, you don't need to create it, otherwise create a playback And initialize the playback address of the audio and video clip of the next index object, store it in the preload player queue, and configure the amount of preload data.

After the playback of the current audio and video clip ends, if the address of the currently played audio and video clip matches the playback address of the audio and video clip after the current index of main_thread, it is added to the queue, and if it does not match, it is released (reused). After the playback of the current audio and video clip ends, the player to be added to the pre-loaded player queue determines whether there is a player with the same playback address in the queue. If it exists, it will not be added and released. If it does not exist, it will be added to the queue.

After the playback of the current audio and video clip ends, take the next index of main_thread and set it as the currently playing audio and video clip. Repeat the above steps until the audio and video in the main-line tree relationship is finished or the user's instruction to stop playing is received.

The above embodiment shows that the mobile device obtains the directory information of the audio and video to be played. The directory information includes path information and play mode. When the play mode is determined to be interactive preview play, the interactive tree relationship and its corresponding relationship are determined according to the path information. The audio and video list, based on the interactive tree relationship and its corresponding audio and video list, and the audio and video clips selected by the user for interactive audio and video playback. After analyzing the interactive tree relationship and its corresponding audio and video list, interactive audio and video playback can be performed according to the interactive tree relationship, the audio and video list, and the audio and video clips selected by the user. Due to the interactive tree relationship It can reflect the association relationship of various audio and video clips. During audio and video playback, it can show the user other audio and video clips associated with the currently playing audio and video clips, so that interactive audio and video clips can be played on mobile devices, which solves the current problem. The audio and video playback scenes of mobile devices are only for a single audio and video stream.

Based on the same technical concept, FIG. 3 exemplarily shows the structure of an interactive audio and video playback device provided by an embodiment of the present application. The device can execute the interactive audio and video playback process. The device can be located in FIG. 1 The mobile device 100 shown may also be the mobile device 100.

As shown in Figure 3, the device specifically includes:

The obtaining unit 301 is configured to obtain directory information of audio and video to be played, where the directory information includes path information and playing mode;

The processing unit 302 is configured to determine an interactive tree relationship and its corresponding audio and video list according to the path information when it is determined that the play mode is interactive preview playback; according to the interactive tree relationship and its corresponding The audio and video list and the audio and video clips selected by the user can be played interactively.

In an embodiment, the processing unit 302 is further configured to:

When determining the play mode as the mainline preview playback, determine the mainline tree relationship and its corresponding audio and video list according to the path information;

In an embodiment, the processing unit 302 is specifically configured to:

In an embodiment, the processing unit 302 is further configured to:

In an embodiment, the processing unit 302 is specifically configured to:

Parse all the sub-nodes of the root node to obtain the segment identifiers of the audio and video segments corresponding to all the sub-nodes; according to the segment identifiers of the audio and video segments corresponding to all the sub-nodes from the audio and video segments corresponding to the interactive tree relationship The URLs of the audio and video clips corresponding to all the child nodes are determined in the video list and added to the list of play addresses to be interactively selected for preloading, where the number of players in the preloaded player queue is greater than the preloading threshold and /Or the number of audio and video loaded by the players in the preloaded player queue is greater than the number threshold.

In an embodiment, the processing unit 302 is further configured to:

Based on the same technical conception, an embodiment of the present application also provides a computing device, including:

Memory, used to store program instructions;

Based on the same technical concept, the embodiments of the present application also provide a computer-readable non-volatile storage medium, including computer-readable instructions. When the computer reads and executes the computer-readable instructions, the computer is caused to perform the aforementioned interactions. Method of audio and video playback.

This application is described with reference to flowcharts and/or block diagrams of methods, equipment (systems), and computer program products according to the embodiments of this application. It should be understood that each process and/or block in the flowchart and/or block diagram, and the combination of processes and/or blocks in the flowchart and/or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing equipment to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing equipment are generated It is a device that realizes the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device. The device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.

These computer program instructions can also be loaded on a computer or other programmable data processing equipment, so that a series of operation steps are executed on the computer or other programmable equipment to produce computer-implemented processing, so as to execute on the computer or other programmable equipment. The instructions provide steps for implementing functions specified in a flow or multiple flows in the flowchart and/or a block or multiple blocks in the block diagram.

Although some embodiments of the present application have been described, those skilled in the art can make additional changes and modifications to these embodiments once they learn the basic creative concept. Therefore, the appended claims are intended to be interpreted as including some of the above-mentioned embodiments and all changes and modifications falling within the scope of this application.

Obviously, those skilled in the art can make various changes and modifications to the application without departing from the spirit and scope of the application. In this way, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application also intends to include these modifications and variations.

Claims

An interactive audio and video playback method, including:

The mobile device obtains directory information of the audio and video to be played, where the directory information includes path information and play mode;

When the mobile device determines that the play mode is interactive preview play, determine the interactive tree relationship and its corresponding audio and video list according to the path information;

The mobile device performs interactive audio and video playback according to the interactive tree relationship and its corresponding audio and video list and audio and video clips selected by the user.
The method of claim 1, wherein the method further comprises:

When the mobile device determines that the playback mode is the mainline preview playback, it determines the mainline tree relationship and its corresponding audio and video list according to the path information;

The mobile device performs mainline audio and video playback according to the mainline tree relationship and its corresponding audio and video list.
The method according to claim 2, wherein the mobile device performs mainline audio and video playback according to the mainline tree relationship and its corresponding audio and video list, comprising:

Acquiring, by the mobile device, the segment identifier of the first audio and video segment in the mainline tree relationship;

The mobile device determines the URL of the first audio and video segment from the audio and video list corresponding to the main-line tree relationship according to the segment identifier of the first audio and video segment;

The mobile device creates a player according to the URL of the first audio and video segment to perform mainline audio and video playback, and adds the player to the pre-loaded player queue.
The method according to claim 3, wherein after the mobile device performs mainline audio and video playback, the method further comprises:

The mobile device obtains the segment identifier of the next audio and video segment of the currently played audio and video segment, and determines the URL of the next audio and video segment from the audio and video list corresponding to the mainline tree relationship;

The mobile device determines, according to the URL of the next audio and video segment, whether there is a player in the preloaded player queue that plays the URL of the next audio and video segment, and if not, creates and plays the next audio and video segment The URL of the player and the player added to the pre-loaded player queue, and continue to obtain the segment identifier of the audio-video segment after the next audio-video segment; until the number of players in the pre-loaded player queue It is greater than the preloading threshold and/or the number of audio and video loaded by the players in the preloading player queue is greater than the number threshold.
The method according to claim 3, wherein after the mobile device performs mainline audio and video playback, the method further comprises:

After the mobile device determines that the playback of the currently played audio and video fragment is over, it determines the next audio and video fragment of the currently played audio and video fragment as the currently played audio and video fragment to perform the main-line audio and video playback, Until the audio and video clips in the main-line tree relationship are played out or the user's instruction to stop playing is received.
The method according to any one of claims 1 to 5, wherein the mobile device performs interactive audio and video playback according to the interactive tree relationship and its corresponding audio and video list and audio and video clips selected by the user, comprising :

Acquiring, by the mobile device, the segment identifier of the audio and video segment corresponding to the root node of the interactive tree relationship;

The mobile device determines the URL of the audio and video segment corresponding to the root node from the audio and video list corresponding to the interactive tree relationship according to the segment identifier of the audio and video segment corresponding to the root node, and according to the The URL of the audio and video segment corresponding to the root node performs interactive video playback;

The mobile device parses all sub-nodes of the root node to obtain the segment identifiers of the audio and video segments corresponding to all the sub-nodes; according to the segment identifiers of the audio and video segments corresponding to all the sub-nodes from the interactive tree The URLs of the audio and video clips corresponding to all the child nodes are determined in the audio and video list corresponding to the relationship, and are added to the list of play addresses to be interactively selected for preloading, where the number of players in the preloaded player queue is greater than The preloading threshold and/or the number of audio and video loaded by the players in the preloading player queue is greater than the number threshold.
The method according to claim 6, wherein after the mobile device performs interactive video playback according to the URL of the audio and video segment corresponding to the root node, the method further comprises:

When the mobile device determines that the playback of the currently played audio and video fragment is over, it shows the user the problem of the currently played audio and video fragment and the audio and video fragments corresponding to each sub-node in the play address list to be interactively selected URL options to enable the user to select the corresponding audio and video clips;

The mobile device determines the URL of the audio and video fragment selected by the user from the list of playback addresses to be interactively selected according to the fragment identifier of the audio and video fragment selected by the user to continue the interactive video playback, and parses The child node of the audio/video segment selected by the user obtains the segment identifier of the audio/video segment corresponding to the child node of the audio/video segment selected by the user, according to the audio/video segment corresponding to the child node of the audio/video segment selected by the user The segment identifier determines the URL of the audio and video segment corresponding to the sub node of the audio and video segment selected by the user from the audio and video categories corresponding to the interactive tree relationship, and adds it to the list of play addresses to be interactively selected Preloading is performed in the interactive tree relationship until the audio and video clips in the interactive tree relationship are all played or the user's instruction to stop playing is received.
The method of claim 7, wherein the segment identifier of the audio and video segment corresponding to the child node of the audio and video segment selected by the user is determined from the audio and video category corresponding to the interactive tree relationship. The URL of the audio and video clip corresponding to the child node of the audio and video clip selected by the user is added to the list of playback addresses to be interactively selected for preloading, including:

Determine whether the URL of the audio and video segment corresponding to the child node of the audio and video segment selected by the user exists in the play address list to be interactively selected, if it exists, do not add it, if it does not exist, remove the non-existent child node The URL of the corresponding audio and video segment is added to the list of play addresses to be interactively selected for preloading; and the play address list to be interactively selected is corresponding to the child node that does not belong to the audio and video segment selected by the user The URL of the audio and video clip is deleted.
An interactive audio and video playback device, including:

An obtaining unit, configured to obtain directory information of the audio and video to be played, the directory information including path information and playing mode;

The processing unit is configured to determine an interactive tree relationship and its corresponding audio and video list according to the path information when it is determined that the playback mode is interactive preview playback; and according to the interactive tree relationship and its corresponding The audio and video list and the audio and video clips selected by the user are played interactively.
The device according to claim 9, wherein the processing unit is further configured to:

When determining the play mode as the mainline preview playback, determine the mainline tree relationship and its corresponding audio and video list according to the path information;

Perform main-line audio and video playback according to the main-line tree relationship and its corresponding audio and video list.
The device according to claim 10, wherein the processing unit is specifically configured to:

Acquiring the segment identifier of the first audio and video segment in the mainline tree relationship;

Determine the URL of the first audio and video segment from the audio and video list corresponding to the mainline tree relationship according to the segment identifier of the first audio and video segment;

Create a player according to the URL of the first audio and video segment to perform mainline audio and video playback, and add the player to the pre-loaded player queue.
The apparatus of claim 11, wherein the processing unit is further configured to:

After the main-line audio and video playback is performed, the segment identifier of the next audio-video segment of the currently-played audio-video segment is obtained, and the next audio-video list is determined from the audio-video list corresponding to the main-line tree relationship. The URL of the video clip;

According to the URL of the next audio and video segment, determine whether there is a player in the preloaded player queue that plays the URL of the next audio and video segment; if not, create a playback that plays the URL of the next audio and video segment And add to the preloaded player queue, and continue to obtain the segment identifiers of the audio and video clips after the next audio and video clip; until the number of players in the preloaded player queue is greater than the preload threshold And/or until the number of audio and video loaded by the players in the preloaded player queue is greater than the number threshold.
The apparatus of claim 11, wherein the processing unit is further configured to:

After the main-line audio and video playback is performed, after it is determined that the playback of the currently played audio and video fragment is over, the next audio and video fragment of the currently played audio and video fragment is determined to be the currently played audio and video fragment. The main-line audio and video are played until the audio and video clips in the main-line tree relationship are played completely or the user's instruction to stop playing is received.
The device according to any one of claims 9 to 13, wherein the processing unit is specifically configured to:

Acquiring the segment identifier of the audio and video segment corresponding to the root node of the interactive tree relationship;

According to the segment identifier of the audio and video segment corresponding to the root node, the URL of the audio and video segment corresponding to the root node is determined from the audio and video list corresponding to the interactive tree relationship, and according to the audio and video segment corresponding to the root node The URL of the video clip is used for interactive video playback;

Parse all the sub-nodes of the root node to obtain the segment identifiers of the audio and video segments corresponding to all the sub-nodes; according to the segment identifiers of the audio and video segments corresponding to all the sub-nodes from the audio and video segments corresponding to the interactive tree relationship The URLs of the audio and video clips corresponding to all the child nodes are determined in the video list and added to the list of play addresses to be interactively selected for preloading, where the number of players in the preloaded player queue is greater than the preloading threshold and /Or the number of audio and video loaded by the players in the preloaded player queue is greater than the number threshold.
The apparatus of claim 14, wherein the processing unit is further configured to:

After the interactive video playback is performed according to the URL of the audio and video fragment corresponding to the root node, when it is determined that the playback of the currently played audio and video fragment is over, the problem of the currently played audio and video fragment is shown to the user and The option of the URL of the audio and video segment corresponding to each sub-node in the play address list to be interactively selected, so that the user can select the corresponding audio and video segment;

According to the segment identifier of the audio and video segment selected by the user, the URL of the audio and video segment selected by the user is determined from the play address list to be interactively selected to continue the interactive video playback, and the user selection is analyzed The sub-node of the audio-visual segment of the user selects the audio-video segment corresponding to the sub-node of the audio-video segment selected by the user, and the audio-video segment corresponding to the sub-node of the audio-video segment selected by the user is obtained from The URL of the audio and video segment corresponding to the sub-node of the audio and video segment selected by the user is determined from the audio and video categories corresponding to the interactive tree relationship, and is added to the list of play addresses to be interactively selected for preloading , Until the audio and video clips in the interactive tree relationship are all played or the user's instruction to stop playing is received.
The device according to claim 15, wherein the processing unit is further configured to:

Determine whether the URL of the audio and video segment corresponding to the child node of the audio and video segment selected by the user exists in the play address list to be interactively selected, if it exists, do not add it, if it does not exist, remove the non-existent child node The URL of the corresponding audio and video segment is added to the list of play addresses to be interactively selected for preloading; and the play address list to be interactively selected is corresponding to the child node that does not belong to the audio and video segment selected by the user The URL of the audio and video clip is deleted.
A computing device including:

Memory, used to store program instructions;

The processor is configured to call the program instructions stored in the memory, and execute the method according to any one of claims 1 to 8 according to the obtained program.
A computer-readable non-volatile storage medium, comprising computer-readable instructions, when the computer reads and executes the computer-readable instructions, the computer executes the method according to any one of claims 1 to 8.