CN113225309A

CN113225309A - Multimedia file online playing method, device, server and storage medium

Info

Publication number: CN113225309A
Application number: CN202110298255.0A
Authority: CN
Inventors: 李志鹏
Original assignee: Shenzhen Xunlei Network Technology Co Ltd
Current assignee: Shenzhen Xunlei Network Technology Co Ltd
Priority date: 2021-03-19
Filing date: 2021-03-19
Publication date: 2021-08-06
Anticipated expiration: 2041-03-19
Also published as: CN113225309B

Abstract

The application discloses a multimedia file online playing method, equipment, a server and a storage medium, wherein the method comprises the following steps: establishing a first link and a second link with a server; correspondingly reading the audio data and the video data of the multimedia file in the server through the first link and the second link respectively, wherein the first link is used for reading the audio data, and the second link is used for reading the video data; and playing the read audio data and the read video data after preset audio processing. Through the mode, the fluency of video playing can be improved.

Description

Multimedia file online playing method, device, server and storage medium

Technical Field

The invention belongs to the technical field of multimedia files, and particularly relates to a multimedia file online playing method, multimedia file online playing equipment, a multimedia file online playing server and a multimedia file online playing medium.

Background

With the rapid development of the spiritual civilization of the modern society, videos become an important way for cultural transmission. However, the original audio and video for direct playing has data with high redundancy, which occupies a large amount of space, and the audio and video data is usually compressed during production so as to be stored conveniently.

There are various ways for storing the original audio and video data after being compressed. For multimedia files in which audio and video data are alternately stored according to the time stamp sequence, that is, interleaved multimedia files, data reading and playing are generally performed according to the sequence. However, for some multimedia files in which audio and video data are not stored in the order of the time stamps, that is, non-interleaved multimedia files, when the multimedia files are played in the order of data storage, the problems that the video playing of the multimedia files is abnormal and not smooth easily occur.

Disclosure of Invention

The invention mainly solves the technical problem of providing a multimedia file online playing method, equipment, a server and a computer storage medium to solve the problem that video playing is not smooth when multimedia files which are not stored according to a timestamp sequence are played online.

In order to solve the technical problems, the invention adopts a technical scheme that: a multimedia file online playing method is provided, and the method comprises the following steps: establishing a first link and a second link with a server; correspondingly reading the audio data and the video data of the multimedia file in the server through the first link and the second link respectively, wherein the first link is used for reading the audio data, and the second link is used for reading the video data; and playing the read audio data and the read video data after preset audio processing.

In order to solve the technical problem, the invention adopts another technical scheme that: a media file online playing method is provided, and the method comprises the following steps: establishing a first link and a second link in connection with an electronic terminal; and correspondingly sending the audio data and the video data of the multimedia file to the electronic terminal through the first link and the second link respectively, so that the electronic terminal can play the read audio data and the read video data after preset audio processing.

In order to solve the technical problem, the invention adopts another technical scheme that: there is provided an electronic device, the device comprising: the system comprises a communication circuit, a memory, a processor and a display, wherein the memory and the display are coupled with the processor, the processor performs data interaction with a server through the communication circuit, the memory stores a computer program, and the processor can read the computer program and execute the computer program to realize the online playing method.

In order to solve the technical problem, the invention adopts another technical scheme that: providing a server, the server comprising: the device comprises a communication circuit, a memory and a processor, wherein the memory is coupled with the processor, the processor performs data interaction with the electronic terminal through the communication circuit, the memory stores a computer program, and the processor can read the computer program and execute the computer program to realize the online playing method.

In order to solve the technical problem, the invention adopts another technical scheme that: there is provided a computer-readable storage medium storing a computer program executable by a processor to implement the online playback method described above.

The beneficial effect of this application is: the first link and the second link for linking the multimedia file are established between the server and the electronic terminal, the audio data and the video data of the multimedia file are respectively and correspondingly read in the server through the first link and the second link, when the multimedia file with audio and video not stored according to the time stamp sequence is played on line, the audio data can be read through the first link and the video data can be read through the second link, the video data and the audio data can be obtained without frequent disconnection and reconnection of a single link, the problems of video playing blockage, large resource consumption and the like caused by frequent disconnection and request of the single link can be effectively solved, the problems of abnormal playing and playing blockage of the non-interlaced multimedia file can be solved, and the speed of video online playing can be effectively improved. In addition, the problem of data flow waste can be solved by respectively reading the audio data and the video data of the multimedia file through the two links, the bandwidth of the server can be more fully utilized, and the waste of the bandwidth can be further reduced.

Drawings

FIG. 1 is a schematic view of a multimedia file storage structure according to an embodiment of the present application;

FIG. 2 is a timing diagram illustrating an embodiment of an online playing method for multimedia files according to the present application;

FIG. 3 is a flowchart illustrating an embodiment of a method for playing a multimedia file online according to the present application;

FIG. 4 is a flowchart illustrating an embodiment of step S13 in FIG. 3;

FIG. 5 is a schematic flowchart illustrating a multimedia file online playing method according to another embodiment of the present application;

FIG. 6 is a schematic diagram of an embodiment of an electronic device;

FIG. 7 is a schematic structural diagram of an embodiment of a server according to the present application;

fig. 8 is a schematic structural diagram of a computer-readable storage medium of the present application.

Detailed Description

The technical solution in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. It is to be understood that the embodiments described are only a few embodiments of the present application and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

The diversification of network application leads the application of audio and video to be more and more extensive, and the video media also becomes the most popular and most influential media form at present. The popularity of various cultures in video makes the requirements of people for video playing higher and higher. Therefore, improving the fluency of video playing and improving the viewing experience of video users is an urgent problem to be solved in the multimedia field.

Generally, a multimedia file will contain video data and audio data, and may be formed by compressing original video data and original audio data into data packets and storing the data packets according to a certain data storage format. The method comprises the steps of compressing original image data to form at least one video data packet, compressing original sound data to form at least one audio data packet, and combining and storing the at least one video data packet and the at least one audio data packet in a multimedia file.

Specifically, in a multimedia file, there are a plurality of storage modes of video data packets and audio data packets in combination. The video data packet and the audio data packet can be interleaved and stored according to a video playing time stamp sequence, the video data packet and the audio data packet are respectively and completely stacked and then stored without interleaving, and the video data packet and the audio data packet can be partially stacked and then stored in an interleaved manner. The former is a normal multimedia file, i.e., an interleaved type multimedia file, and the latter two are abnormal multimedia files, i.e., non-interleaved type multimedia files.

Fig. 1 exemplarily shows a data storage format of an interleaving type multimedia file and a data storage format of a non-interleaving type. In the interleaving type multimedia file shown in fig. 1, the video data packets and the audio data packets are stored in an interleaving manner according to the playing time stamps, and when playing, the video data packets and the audio data packets are sequentially read and played according to the time stamp sequence, that is, sequentially read and played from the sequence 1 to 18. In the non-interleaved multimedia file shown in fig. 1, the storage format is not interleaved according to the playing time stamps, but non-interleaved, and the storage order becomes 1-4-7- … …, which obviously cannot be read and played according to the storage order. The playing process is roughly as follows: after reading the corresponding video data "1", the following read audio data packet "2" is skipped, then the audio data packet "3" is read, and the process returns to read the video data packet "4", and so on.

The embodiment of the method for playing the multimedia file on line can be applied to scenes needing on line playing, such as video platforms, video software, network cloud disks and the like of electronic terminals (such as smart phones, tablet computers, notebook computers and the like). The video platform, the video software, the network cloud disk and the like applied to the online playing method can improve the smoothness of online playing of the non-interlaced multimedia file and reduce the data flow and the waste of bandwidth.

Referring to fig. 2 and fig. 3, fig. 2 is a timing diagram of an embodiment of a method for playing a multimedia file online, and fig. 3 is a flowchart of an embodiment of a method for playing a multimedia file online. The embodiment is applied to an electronic terminal, and specifically comprises the following steps:

s11: a first link and a second link are established with a server.

The electronic terminal establishes two links, namely a first link and a second link, with the server through the two IO ports, the two links are both connected to the multimedia file in the server, the two links are respectively connected to the video data and the audio data of the multimedia file, and the electronic terminal can respectively read the audio data and the video data of the multimedia file through the links of the two IO ports.

When the electronic terminal needs to play the video online, a link needs to be established with the server. After the link is established between the electronic terminal and the server, data interaction can be carried out by sending a request and responding to the request. For example, the electronic terminal may send a video playing request to the server through the link, and the server, upon receiving the video playing request initiated by the electronic terminal, responds to the request and sends audio data and video data of the multimedia file to the electronic terminal through the link.

In this embodiment, after the first link and the second link are established between the electronic terminal and the server, data interaction and communication are performed in the HTTP protocol, so that the first link and the second link may be TCP links. Of course, the first link and the second link may also be other types of links, and the electronic terminal and the server may also communicate and interact with each other through other types of protocols. For example, data exchange is performed between the electronic terminal and the server through an HTTP protocol, the electronic terminal and the server may establish the first link and the second link through multiple handshakes, and specific principles and detailed processes for establishing the links between the electronic terminal and the server may refer to the prior art, which is not described herein again.

As mentioned above, if the existing technology is used to play the non-interleaved multimedia file online, the skipping back and forth and reading requires continuous disconnection and reconnection of the link, which results in playing pause, waste of bandwidth resources, and the like. The present embodiment is responsible for reading different data by establishing the first link and the second link, respectively, and specifically, a first link and a second link are established through two ports of the electronic terminal, the first link and the second link are respectively connected to the audio data and the video data of the multimedia file at the server terminal, during the online playing process of the multimedia file, when the electronic terminal needs the audio data, the electronic terminal selects to switch to the first link to read the audio data, and when the electronic terminal needs the video data, selects to switch to the second link to read the video data, by the method for serially reading the audio data and the video data of the multimedia file by switching the link, the electronic terminal can read the multimedia data according to the time stamp without continuously disconnecting the reconnection link, and further, the problem that the link is frequently disconnected and reconnected when the single link is used for playing the multimedia file on line is solved.

With continued reference to fig. 2, in the process of establishing the first link and the second link with the server, a link corresponding to the initial packet format is established, specifically, the following steps included in step S11 are included:

s111: and establishing one of the first link and the second link which is matched with the initial data packet format with the server.

In this embodiment, the initial data packet is a data packet that needs to be read first in the data storage format of the multimedia file, that is, a data packet that is played first in the playing time stamp. The initial data packet may be a video data packet or an audio data packet. The initial packet format corresponds to either a video format or an audio format.

When the electronic terminal needs to play a video, a link is established with the server, the electronic terminal initiates a multimedia file data acquisition request to the server through the link, the server responds to the request after receiving the request initiated by the electronic terminal, and then the multimedia file data is sent to the electronic terminal through the link according to the time sequence, so that one of the first link and the second link with the same format as the initial data packet is formed. If the initial data packet is in an audio data packet format, a first link is established, and if the initial data packet is in a video data packet format, a second link is established. In the non-interleaved type multimedia file as shown in fig. 1, the data packet played first is a video data packet, i.e., a video data packet "1".

S112: an initial data packet is received and read.

After the link matched with the format of the initial data packet is established, the electronic terminal receives the initial data packet sent by the server through the link.

For the multimedia file of the interweaving type, video data and corresponding audio data are sequentially stored according to the time stamp, and the electronic terminal sequentially reads the subsequent audio/video data packets of the initial data packet to realize synchronous acquisition and playing of audio/video data. However, if the multimedia file is of a non-interlaced type, and the data are not stored adjacently according to the time stamp sequence, the electronic terminal cannot directly read the initial data packet and then sequentially read the initial data packet according to the time stamp sequence to obtain the audio/video data packet corresponding to the initial data packet, so as to implement synchronous playing.

In order to realize the audio/video synchronous online playing of the multimedia file, for the non-interlaced type multimedia file, if the next data packet cannot be read according to the time stamp sequence after the initial data packet is read, the read position of the data packet needs to be adjusted, and then the data packet is jumped to the corresponding position. Specifically, the file pointer offset instruction may be used to jump to the position pointed by the file pointer offset instruction to reestablish the link, thereby implementing reading and synchronous playing of the corresponding audio/video data packet.

Therefore, after the electronic terminal reads the initial data packet, if the first file pointer offset instruction occurs, it means that it needs to adjust the reading position to obtain the subsequent audio/video data packet. The details are as follows:

s113: a first file pointer offset instruction is detected to occur after reading the initial data packet.

The first file offset instruction is, for example, a file.

After the initial data packet is read, a first file pointer offset instruction occurs, which means that the multimedia file is a non-interleaved type multimedia file and another data packet with a format different from that of the initial data packet needs to be read.

Taking the non-interleaved multimedia file shown in fig. 1 as an example, the initial data packet is a video data packet "1", the data packet to be read next time is an audio data packet "2", but the audio data packet "2" is not adjacent to the video data packet "1", so the audio data packet "2" and the video data packet "1" cannot be read according to normal storage, and the file pointer needs to be shifted to the position corresponding to the audio data packet "2", at this time, the electronic terminal may detect that the first file pointer shift instruction occurs.

S114: and responding to the first file pointer offset instruction, establishing one of the first link and the second link which does not match with the initial data packet format with the server, and maintaining the connection of the one of the first link and the second link which matches with the initial data packet format with the server.

Specifically, a first file pointer offset command occurs, which indicates that the format of the initial data packet is different from that of the next data packet, and the data packets are far apart in the data storage format. At this time, the electronic terminal temporarily does not read data from the established link, and establishes another link which does not match the format of the initial data packet with the server at the position pointed by the first file pointer offset instruction. For example, the initial data packet is a video data packet, and if a first file pointer offset instruction occurs after the initial data packet is read through the second link, the first link is established between the electronic terminal and the server and is connected to a position indicated by the first file pointer offset instruction, so that the corresponding audio data packet is read at the position.

In step S114, the electronic terminal reads the initial data packet through one of the first link and the second link, which is matched with the format of the initial data packet, and if the first file pointer offset instruction is continuously executed through the single link, the electronic terminal needs to reestablish a new connection by disconnecting the link, so that the reading speed of the multimedia file is slow, and the smoothness of online playing of the multimedia file is further affected. In this embodiment, after the first file pointer offset instruction occurs, another link is selected to be established, that is, one of the first link and the second link, which is not matched with the format of the initial data packet, is established with the server, and the connection between the one matched with the format of the initial data packet and the server is not disconnected, so that the establishment of the first link and the second link connected with the server is completed, and the subsequent corresponding reading of the audio data and the video data is facilitated.

S12: and correspondingly reading the audio data and the video data of the multimedia file in the server through the first link and the second link respectively.

In the process of establishing the first link and the second link, the first link and the second link respectively correspond to the audio data and the video data of the read multimedia file. Taking the initial data packet as the video data packet as an example, after the second link is established to read the initial data packet, if the first file pointer offset instruction is not detected, the subsequent video data packet is continuously read through the second link, and after the first file pointer offset instruction is detected, the first link is established to read the corresponding audio data packet. And after the first link and the second link are established, continuously reading the audio data packet through the first link and reading the data packet through the second link.

In this embodiment, since at least some audio data packets of the non-interleaved multimedia file are stored in a stacked manner, and at least some video data packets are stored in a stacked manner, the first link connected to the server is responsible for reading audio data, the first link may not be required to be disconnected and reconnected too much to implement a jump, the second link is responsible for reading video data, and the second link may also not be required to be disconnected and reconnected too much to implement a jump, which may effectively reduce the play jam and extra bandwidth waste caused by the single link being disconnected and reconnected too much to implement a jump, and may further improve the play fluency, reduce the waste of bandwidth resources, and also reduce the occupation of hardware processing resources of the electronic terminal.

As described above, in playing the multimedia file online, the electronic terminal may switch the first link and the second link to read the corresponding audio data and video data in an alternate reading manner without disconnecting and reconnecting the first link and the second link too much. Specifically, the following steps included in step S12 can be referred to:

s121: and correspondingly reading the audio data and the video data in the server through the first link and the second link in an alternate reading mode.

In the data storage format of the multimedia file, reading is alternately performed through the first link and the second link in the order required for playing. For example, after audio data is read through the first link, the link is switched to the second link for reading.

Taking the non-interleaved multimedia file shown in fig. 1 as an example, after the video data packet "1" is read through the second link, the audio data packet "2" and "3" are read through the second link, and then the video data packet "4" is read through the first link, so that the audio data and the video data are alternately read through the first link and the second link according to the data storage format of the multimedia file.

The multimedia file is read in an alternate mode through the first link and the second link, so that the problem of playing pause caused by frequent disconnection and restarting of the traditional single link reading is solved, the data acquisition speed can be further improved in the alternate reading mode, the playing pause is reduced, and the bandwidth waste can be reduced.

Specifically, after the electronic terminal finishes reading the audio/video data packet once through one of the first link and the second link, the position of the next audio/video data packet to be read in the data storage format is determined according to the playing time stamp.

Optionally, if the next audio/video data packet to be read and the audio/video data packet read this time are both in the same data packet format, the next audio/video data packet can be read directly through the link for reading the audio/video data packet this time. For example, the audio data packets "2" and "3" of the non-interleaved type multimedia file shown in fig. 1 may continue to be read through the first link.

When the next audio/video data packet to be read and the audio/video data packet read this time are in different data packet formats, the data is not read from the link of the audio/video data packet read this time, and the data is read by switching to another link. For example, if an audio data packet is currently read through the first link, and a video data packet needs to be read next time, the second link needs to be switched to read the video data packet, so that alternate reading can be realized according to a data storage format, and the data reading speed is increased.

For a non-interleaved multimedia file, in the process of alternately reading through a first link and a second link, the format of a currently read data packet is different from that of a data packet read next time, and due to non-interleaved storage, the positions are often not adjacent, and at this time, it is generally required to jump from the position corresponding to the currently read data packet to the position corresponding to the data packet read next time. For a specific procedure, see the following steps included in step S12:

s122: detecting a second file pointer offset instruction which occurs after one-time reading is completed through one of the first link and the second link in the alternate reading process;

specifically, in the process of alternate reading, each time any one of the first link and the second link is switched back to the local link, the link updates the system clock used by the local link according to the time of being switched back. The second file pointer offset instruction is, for example, a file.seek () instruction, which is also used to move the file read pointer to a specified location. In the alternate reading process after the first link and the second link are established, a second file pointer offset instruction occurs after one reading is completed through one of the first link and the second link, which indicates that the other one needs to be used for reading the data packet corresponding to the second file pointer offset instruction.

S123: responding to a second file pointer offset instruction, and judging whether the other one of the first link and the second link is in a connection state or a disconnection state with a server;

the audio data packet and the video data packet of the multimedia file are stored in a non-interleaving manner, so that each time the electronic terminal generates the second file offset instruction, the electronic terminal can switch to the corresponding link to read the corresponding audio/video data packet from the server.

For example, if the second file offset instruction occurs after the video data packet is read through the second link, the second file pointer offset instruction is used to offset the file acquisition pointer to acquire the corresponding audio data packet, and at this time, the server temporarily does not transmit the video data to the electronic terminal through the second link, but transmits the audio data to the electronic terminal through the first link. However, since the electronic terminal reads the current video data packet through the second link and does not continuously use the first link to read data, the state of the first link needs to be determined before executing the second file pointer offset instruction. The same applies if the second file shifting command occurs after the audio data packet is read through the first link.

If the determination result in step S123 is that the switch is in the off state, the following step S124 is executed.

S124: and reestablishing the connection between the other of the first link and the second link and the server, and reading the data packet corresponding to the second file pointer offset instruction.

In particular, based on a second file pointer offset instruction, a connection of the other of the first link and the second link to the server is reestablished at a location of the second file pointer offset instruction. For example, if the second file offset instruction occurs after the video data packet is read through the second link, and it is determined that the first link and the server are in the disconnected state, the connection between the first link and the server is reestablished at the position pointed by the second file pointer offset instruction, so as to be in the connected state again, and then the audio data packet corresponding to the second file pointer offset instruction, that is, the audio data packet corresponding to the position pointed by the second file pointer offset instruction is read.

If the determination result in step S122 is that the connection state is established, the following step S125 is executed.

S125: it is determined whether the other of the first link and the second link currently satisfies the requirements of the second file pointer offset instruction.

Specifically, whether a packet connected to the other of the first link and the second link currently satisfies a packet corresponding to the position pointed to by the second file pointer offset instruction is determined.

Following the example in step S123, if it is determined that the first link is in the connection state, it needs to be further determined whether the first link meets the requirement of the second file pointer offset instruction.

Whether the corresponding link in the first link and the second link meets the requirement of the second file pointer offset instruction currently or not is judged, whether the corresponding link needs to be disconnected or not can be clearly known, different processing is carried out according to different conditions, and the disconnection and reconnection can be simply carried out without the need of the prior art.

For determining whether the other of the first link and the second link currently satisfies the requirement of the second file pointer offset instruction, the following steps included in step S125 may be specifically implemented:

s1251: and judging whether the current read position of the other one of the first link and the second link is the same as the position of the data packet corresponding to the second file pointer offset instruction.

If the data packet corresponding to the second file pointer offset instruction needs to be read through the other of the first link and the second link, it needs to be determined whether the corresponding positions of the first link and the second link are the same or coincident.

If the determination result in the step S1251 is yes, the requirement that the other of the first link and the second link currently satisfies the second file pointer offset instruction is satisfied.

If the determination result in the step S1251 is negative, the other of the first link and the second link does not currently satisfy the requirement of the second file pointer offset instruction.

Following the example in step S124, if the read position of the first link currently in the data storage format coincides with the position pointed by the second file pointer offset instruction, the current connection state of the first link may satisfy the requirement of the second file pointer offset instruction. If the position read by the first link in the data storage format is not coincident with the position pointed by the second file pointer offset instruction, the current connection state of the first link does not meet the requirement of the second file pointer offset instruction.

With the non-interleaved type multimedia file shown in fig. 1, assuming that a second file pointer offset instruction occurs after the second link reads the video data packet "4", the second file pointer offset instruction points to the audio data packet "5", and if the currently read position of the first link is in the audio data packet "5", the first link currently satisfies the requirement of the second file pointer offset instruction. If the currently read position of the first link is not in the audio data packet "5", then the first link does not currently satisfy the requirement of the second file pointer offset instruction.

If the determination result in step S125 is yes, the following step S126 is performed. If the determination result in step S125 is no, the following step S127 is executed.

S126: and if so, reading the data packet corresponding to the second file pointer offset instruction through the other one of the first link and the second link.

That is, if it is determined in step S125 that the other of the first link and the second link currently satisfies the requirement of the second file pointer offset instruction, the data packet corresponding to the second file pointer offset instruction is continuously read through the other of the first link and the second link. In other words, the other of the first link and the second link meets the requirement of the second file pointer offset instruction, and then the current reading position is the same as or coincides with the position pointed by the second file pointer offset instruction, so that the data packet corresponding to the position pointed by the second file pointer offset instruction can be smoothly read.

S127: if not, disconnecting the other one of the first link and the second link from the server, and reestablishing the connection between the other one of the first link and the second link and the server so as to read the data packet corresponding to the second file pointer offset instruction.

That is, if it is determined in step S125 that the other of the first link and the second link does not currently satisfy the requirement of the second file pointer offset instruction, based on the system clocks of the first link and the second link in step S122, it is determined that the one of the first link and the second link with the earlier system clock value is the other of the first link and the second link, and the other of the first link and the second link is disconnected from the server. In other words, the other of the first link and the second link does not satisfy the requirement of the second file pointer offset instruction, and the current reading position is not the same as or coincident with the position pointed by the second file pointer offset instruction, so that the data packet corresponding to the position pointed by the second file pointer offset instruction cannot be read continuously. In this case, the embodiment disconnects the other of the first link and the second link from the server, and further, the link between the other of the first link and the second link and the server is reestablished in accordance with the requirement of the second file pointer offset instruction, so as to be able to continue to read the data packet corresponding to the position pointed by the second file offset instruction.

And after the connection state of the corresponding link in the first link and the second link meets the requirement of the second file pointer offset instruction, the server can send the audio/video data packet corresponding to the second file pointer offset instruction to the electronic terminal through the link. Through the steps, the times that the link is required to be disconnected and reestablished when the data packet with a long reading interval is read when the non-interlaced multimedia file is read by a single link can be reduced, and the waiting time of the online playing of the multimedia file is further reduced.

In this embodiment, the read audio data and video data are both stored in corresponding formats after being compressed, generally, the audio data and video data cannot be directly played, and the audio data and video data need to be preprocessed and converted into playable formats.

S13: and playing the read audio data and video data after preset audio processing.

In order to realize the online playing of the multimedia file, the electronic terminal can synchronously perform preset audio processing on the video data packet and the audio data packet which are read last time in the process of reading the video data packet and the audio data packet of the multimedia file so as to obtain and play the original image data and the original sound data which can be normally played.

After the audio data packet and the video data packet of the multimedia file are correspondingly acquired, the synchronous playing of the corresponding audio and video data packet can be played, so that the reading and playing of the multimedia file data are realized, and the utilization rate of data flow resources and bandwidth resources is improved.

By the method for reading the video data and the audio data separately stored in the multimedia file by jumping to different links, the sequential acquisition of the audio data and the video data can be realized in the process of alternately reading the multimedia audio data and the video data through the first link and the second link, and the frequent disconnection and reconnection of the first link and the second link are not needed, so that the problem of frequent disconnection and reconnection of the links during the online playing of the non-interlaced multimedia file can be solved, and the occupied time of the buffer area during the online playing of the multimedia file can be shortened.

Regarding the preset audio processing of the video data packets and the audio data packets, specifically, referring to fig. 4, fig. 4 is a schematic flowchart of an embodiment of step S13 in fig. 3 of the present application, and specifically, the preset audio processing may include the following steps:

s131: decapsulating the audio data and the video data to obtain a separated video code stream and an audio code stream;

the audio data and the video data are put into a file according to a certain format to form a multimedia file, such as an MP4 format, an AVI format, an MPEG-TS format, an FLV format, an MKV format, an RM/RMVB format, and the like. When a multimedia file needs to be played, video data and audio data need to be opened from the file according to the format, that is, the audio data and the video data are unpacked to obtain a video code stream and an audio code stream.

S132: decoding the video code stream and the audio code stream to obtain original image data and original sound data;

before compression, a multimedia file needs to compress original image data (such as RGB \ YUV and the like) into a video code stream, compress audio original data (such as PCM) into an audio code stream to reduce the data volume of the original image data and the original audio data, and then put the video code stream and the audio code stream into the file according to a certain format to form the multimedia file. Namely, after the video data and the audio data of the multimedia file are unpacked and wait for the video code stream and the audio code stream, the video code stream and the audio code stream are decoded, so that the original image data and the original sound data can be obtained.

S133: rendering the original image data;

in the process of compressing and processing original image data, complex special effects and effects are involved, and after the processes of de-encapsulation and decoding, the special effects and effects hardly achieve a good display effect in terms of the current computer operation capacity, so that after decoding a video code stream and an audio code stream, the obtained original image data can be rendered to achieve the required final playing effect.

S134: and playing the original sound data and the rendered original image data.

After rendering the original image data, the original sound data and the rendered original image data can be respectively sent to the corresponding sound card and the corresponding display card for playing.

The above-mentioned embodiment of the method for playing a multimedia file online only takes the data packet storage manner as that the video data packet and the audio data packet are stored in a non-interleaved manner after being respectively and integrally stacked, the audio data packet is stacked and stored at the front end of the multimedia file, and the video data packet is stacked and stored at the rear end of the multimedia file as an example, and introduces the implementation process of the method for playing a multimedia file online so as to be convenient for understanding. If the audio and video data packets in the multimedia file are stored in other non-interlaced manners, the scheme can also realize the online playing of the corresponding multimedia file by establishing two links to alternately read the audio and video data, and the details are not repeated here.

For a multimedia file stored in a non-interleaving manner, by the method for playing the multimedia file on line by respectively reading the audio data packet and the video data packet by creating the first link and the second link, and by the method for reading the audio data and the video data of the non-interleaving type multimedia file by switching the links, the problems that in the process of reading the audio data and the video data of the non-interleaving type multimedia file by a single link, a file pointer offset instruction needs to be frequently executed, and further the single link needs to frequently disconnect the connection at the position of the data packet transmitted at this time to reestablish the connection at the position pointed by the file pointer offset instruction to transmit the next data packet, so that the waste of time, data flow and bandwidth is caused can be solved.

Referring to fig. 5, fig. 5 is a schematic flowchart illustrating another embodiment of a method for playing a multimedia file online according to the present application.

The method specifically comprises the following steps:

s21: establishing a first link and a second link with an electronic terminal;

when the electronic terminal needs to play the video corresponding to the multimedia file on line, the electronic terminal establishes connection with the server through handshaking, and then initiates a multimedia file data acquisition request to the server through the connection. After receiving a data acquisition request sent by the electronic terminal, the server responds to the request to agree with the establishment of a data acquisition link, and then establishes a link for connecting the multimedia file and the electronic terminal.

In this embodiment, the audio data and the video data of the multimedia file are stored at a relatively long distance, so that after the electronic terminal acquires the audio/video data packet, a file pointer offset instruction needs to be sent to jump to a position far away from the audio/video data packet to acquire a corresponding next audio/video data packet, in order to solve the problem that the file pointer offset instruction is continuously executed in a single link in the acquisition process of audio data and video data, and multimedia data is read at the jump position, a first link and a second link which are respectively connected with the audio data and the video data of a multimedia file are established, the electronic terminal reads the audio data and the video data through the two links respectively, and the audio/video data corresponding to the multimedia file does not need to be obtained through continuous jumping in a single link, so that the reading speed of the audio/video data of the multimedia file can be improved.

S22: respectively and correspondingly sending audio data and video data of the multimedia file to the electronic terminal through the first link and the second link;

in the embodiment, the server transmits audio data and video data of the multimedia file to the electronic terminal through the established first link and second link, respectively. Specifically, in the present embodiment, the server maintains the first link and the second link in a connected state after establishing the first link and the second link. And after receiving the file pointer offset instruction, alternately executing the file pointer offset instruction in the first link and the second link respectively so as to enable the server to send the audio data and the video data of the multimedia file to the electronic terminal in an alternate serial mode through the first link and the second link. After receiving the audio data and the video data, the electronic terminal performs a series of preset processing such as separation and decoding on the audio data and the video data, converts the received audio data and the video data into image data and sound data which can be played normally, and then plays a video corresponding to the multimedia file.

The server alternately switches the first link and the second link to respectively send the audio data and the video data of the non-interlaced multimedia file to the electronic terminal through the first link and the second link, so that the problems of flow and bandwidth waste caused by the fact that the non-interlaced multimedia file reads the audio and video data through a single link to achieve online playing can be solved.

For specific content of this embodiment, reference may be made to the detailed description of an embodiment of the method for online playing a multimedia file in the present application, which is not described herein again.

Referring to fig. 6, fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

In this embodiment, the electronic device 100 includes a display 110, a processor 120, a memory 130, and a communication circuit 140, wherein the memory 130 and the display 110 are coupled to the processor 120. The processor 120 performs data interaction with the server through the communication circuit 140, and the memory 130 may store a related computer program, such as a computer program for receiving the video data and the audio data of the multimedia file through the first link and the second link, or a program instruction for preprocessing the video data and the audio data of the multimedia file. Processor 120 may execute computer program instructions in memory 130 to complete the reception and processing of multimedia file video data and audio data, and to send the raw image data and raw sound data to display 110 for playing the corresponding video.

The processor 120 is used for operation of the electronic device 100, and the processor 120 may also be referred to as a Central Processing Unit (CPU). The processor 120 may be an integrated circuit chip having signal processing capabilities. The processor 120 may also be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor 120 may be any conventional processor or the like.

The communication circuit 140 may be an interface, such as a transceiver, for the electronic device 100 to communicate with a headset or other electronic device. Memory 130 may include Random Access Memory (RAM), read-only memory (ROM), flash memory, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, a hard disk, a removable disk, a CD-ROM, and so forth. Memory 130 may store program data, which may include a single instruction, or many instructions, for example, and may be distributed over several different code segments, among different programs, and across multiple memories 130. Memory 130 may be coupled to processor 120 such that processor 120 can read information from, and write information to, memory 130. Of course, the memory 130 may be integral to the processor 120.

Referring to fig. 7, fig. 7 is a schematic structural diagram of an embodiment of a server according to the present application.

In this embodiment, the server 200 includes a processor 210, a communication circuit 220, and a memory 230, wherein the memory 230 is coupled to the processor 210, and the processor 210 performs data interaction with the electronic terminal through the communication circuit 220, and wherein the memory 230 stores a computer program file, such as a program instruction file for responding to a video playing request or a program instruction file for transmitting multimedia file data. The processor 210 may transmit the video data and the audio data of the multimedia file to the electronic terminal by executing the computer program file in the memory 230, so as to implement the processing of the video data and the audio data of the multimedia file and the playing of the final video on the electronic terminal.

The processor 210 is used for the operation of the server 200, and the processor 210 may also be referred to as a CPU (Central Processing Unit). The processor 210 may be an integrated circuit chip having signal processing capabilities. The processor 210 may also be a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components. A general purpose processor may be a microprocessor or the processor 210 may be any conventional processor or the like.

The communication circuit 220 may be an interface, such as a transceiver, for the electronic terminal to communicate with a headset or other electronic device. Memory 230 may include Random Access Memory (RAM), Read Only Memory (ROM), flash memory, Erasable Programmable Read Only Memory (EPROM), Electrically Erasable Programmable Read Only Memory (EEPROM), registers, a hard disk, a removable disk, a CD-ROM, and so forth. Memory 230 may store program data, which may include a single instruction, or many instructions, for example, and may be distributed over several different code segments, among different programs, and across multiple memories 230. The memory 230 may be coupled to the processor 210 such that the processor 210 can read information from, and write information to, the memory 230. Of course, the memory 230 may be integral to the processor 210.

Referring to fig. 8, fig. 8 is a schematic structural diagram of a computer-readable storage medium according to the present application. Wherein the computer readable storage medium 300 stores a computer program 310 capable of being executed by a processor to perform any of the methods described above. For example, the computer-readable storage medium 300 may store a program file for alternately reading video data and audio data of a multimedia file from a first link and a second link, and may also store program instructions for decapsulating, decoding, rendering, and playing the acquired video data and audio data.

The computer-readable storage medium 300 of this embodiment may be a medium that can store program instructions, such as a usb disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk, or an optical disk, or may be a server that stores the program instructions, and the server may send the stored program instructions to other devices for operation, or may self-execute the stored program instructions.

In the several embodiments provided in the present application, it should be understood that the disclosed method and system may be implemented in other ways. For example, the above-described system embodiments are merely illustrative, and for example, a division of modules or units is merely a logical division, and an actual implementation may have another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

Units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

In addition, functional units in the embodiments of the present application may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present application may be substantially implemented or contributed to by the prior art, or all or part of the technical solution may be embodied in a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, a network device, or the like) or a processor (processor) to execute all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.

The above description is only for the purpose of illustrating embodiments of the present application and is not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes, which are made by the contents of the specification and the drawings of the present application or are directly or indirectly applied to other related technical fields, are also included in the scope of the present application.

In the several embodiments provided in the present application, it should be understood that the disclosed system, apparatus and method may be implemented in other manners.

The above description is only an example of the present application and is not intended to limit the scope of the present application, and all modifications of equivalent structures and equivalent processes performed by the present application and the contents of the appended drawings, which are directly or indirectly applied to other related technical fields, are also included in the scope of the present application.

Claims

1. A multimedia file online playing method is characterized by comprising the following steps:

establishing a first link and a second link with a server;

correspondingly reading the audio data and the video data of the multimedia file in the server through the first link and the second link respectively, wherein the first link is used for reading the audio data, and the second link is used for reading the video data, but the sequence is not strictly limited;

and playing the read audio data and the read video data after preset audio processing.

2. The online playing method according to claim 1, wherein the reading of the audio data and the video data in the server through the first link and the second link respectively comprises:

and correspondingly reading the audio data and the video data in the server through the first link and the second link in an alternate reading mode.

3. The online playing method of claim 2, wherein the establishing the first link and the second link with the server comprises:

establishing one of the first link and the second link matched with an initial data packet format with a server so as to be capable of reading an initial data packet corresponding to the initial data packet format; and if the format of the initial data packet is a video format, creating the first link, and if the format of the initial data packet is an audio format, creating the second link.

4. The online playing method of claim 3, wherein the establishing the first link and the second link with the server comprises:

detecting that a first file pointer offset instruction occurs after the initial data packet is read;

in response to the first file pointer offset instruction, establishing with a server one of the first link and the second link that does not match the initial packet format while maintaining a connection with the server of the one of the first link and the second link that matches the initial packet format.

5. The online playing method according to claim 4, wherein the reading the audio data and the video data in the server through the first link and the second link in an alternate reading manner comprises:

after the first link and the second link are established, detecting a second file pointer offset instruction which occurs after one-time reading is completed through one of the first link and the second link in the alternate reading process;

responding to the second file pointer offset instruction, and judging whether the other one of the first link and the second link is in a disconnected state or a connected state with the server;

if the file is in the disconnected state, reestablishing the connection between the other one of the first link and the second link and the server so as to read the data packet corresponding to the second file pointer offset instruction;

if the file pointer is in a connection state, judging whether the other one of the first link and the second link currently meets the requirement of the second file pointer offset instruction;

if so, reading a data packet corresponding to the second file pointer offset instruction through the other one of the first link and the second link;

if not, disconnecting the other one of the first link and the second link from the server, and reestablishing the connection between the other one of the first link and the second link and the server so as to read the data packet corresponding to the second file pointer offset instruction.

6. The online playback method of claim 5,

the determining whether the other of the first link and the second link currently satisfies the second file pointer offset instruction comprises:

judging whether the current read position of the other one of the first link and the second link in the data storage format is the same as the position of the data packet corresponding to the second file pointer offset instruction or not;

if so, the other of the first link and the second link currently meets the requirement of the second file pointer offset instruction;

if not, the other of the first link and the second link currently does not meet the requirement of the second file pointer offset instruction.

7. The online playing method according to claim 1, wherein the playing after performing the preset audio processing on the read audio data and the read video data comprises:

decapsulating the audio data and the video data to obtain a separated video code stream and an audio code stream;

decoding the video code stream and the audio code stream to obtain original image data and original sound data;

rendering the original image data, and playing the original sound data and the rendered original image data.

8. A multimedia file online playing method is characterized by comprising the following steps:

establishing a first link and a second link in connection with an electronic terminal;

and correspondingly sending the audio data and the video data of the multimedia file to the electronic terminal through the first link and the second link respectively, so that the electronic terminal can play the read audio data and the read video data after preset audio processing.

9. An electronic device comprising a communication circuit, a memory, a processor and a display, wherein the memory and the display are coupled to the processor, the processor performs data interaction with a server through the communication circuit, and the memory stores a computer program, and the processor can read the computer program and execute the computer program to realize the online playing method according to any one of claims 1-7.

10. A server, comprising a communication circuit, a memory and a processor, wherein the memory is coupled to the processor, the processor performs data interaction with an electronic terminal through the communication circuit, the memory stores a computer program, and the processor can read the computer program and execute the computer program to implement the online playing method according to claim 8.

11. A computer-readable storage medium storing a computer program, wherein the computer program is executable by a processor to implement the online playback method according to any one of claims 1 to 8.