CN117640987A - Offline online real-time chorus method, device and medium - Google Patents

Abstract

The application discloses a real-time chorus method, equipment and medium on offline, relates to the technical field of communication, is applied to a cloud server, and comprises the following steps: the real-time audio and video streaming under the pull line of the pull-push service network is pulled and the current marking progress is generated, and the real-time audio and video streaming pushed by the pull-push service network and the current marking progress are issued to the target chorus user terminal by the real-time audio and video service network, so that the target chorus user terminal plays the real-time audio and video streaming and marks the local voice audio streaming by using the current marking progress to obtain a marked voice audio stream; the real-time audio and video service network acquires the marked voice audio stream, the real-time audio and video stream, the current marking progress and the marked voice audio stream are pulled to the cloud confluence service network by the real-time audio and video pulling service network, the cloud confluence service network combines the real-time audio and video stream and the marked voice audio stream, and the combined audio and video stream is pushed to the content distribution network for distribution. Real-time chorus off line on line can be realized.

Description

Offline online real-time chorus method, device and medium

Technical Field

The present invention relates to the field of communications technologies, and in particular, to a method, an apparatus, and a medium for online real-time chorus.

Background

Currently, some chorus application software on the market can realize that users participate in chorus online, i.e. users currently participating in chorus are limited to online modes. For the off-line singing scene, such as a star singing concert, a push stream singing concert and the like, singers and off-desk spectators can participate in chorus off line, and on-line users can only watch the off-line singing concert in real time through the terminal equipment, so that a scheme for realizing real-time chorus of the on-line users and the off-line users is also lacking.

In summary, how to realize real-time chorus of online users and offline users is a problem to be solved at present.

Disclosure of Invention

In view of the above, the present invention aims to provide a method, apparatus and medium for online real-time chorus, which can realize online users and offline users real-time chorus. The specific scheme is as follows:

in a first aspect, the application discloses a real-time chorus method offline, applied to a cloud server, comprising:

pulling a real-time audio and video stream in a singing scene under a line from a content distribution network by utilizing a pulling-pushing service network, performing progress marking on the real-time audio and video stream to obtain a current marking progress, and pushing the real-time audio and video stream and the current marking progress to the real-time audio and video service network;

The real-time audio and video stream and the current marking progress are issued to a target chorus user terminal by utilizing the real-time audio and video service network, so that the target chorus user terminal plays the real-time audio and video stream, and the voice audio stream obtained in real time by the chorus application on the local line is marked by utilizing the current marking progress, so that a marked voice audio stream carrying progress information is obtained;

and acquiring the marked voice audio stream uploaded by the target chorus user terminal through the real-time audio and video service network, pulling the real-time audio and video stream, the current marking progress and the marked voice audio stream to a cloud confluence service network by utilizing the real-time audio and video streaming service network, combining the real-time audio and video stream and the marked voice audio stream by utilizing the cloud confluence service network based on the current marking progress and the progress information, and pushing the combined audio and video stream to the content distribution network for distribution.

Optionally, the pulling, converting and pushing the real-time audio and video stream in the off-line singing scene from the content distribution network by using the service network, and performing progress marking on the real-time audio and video stream to obtain a current marking progress, including:

When the current time interval reaches a preset time interval threshold, pulling a real-time audio and video stream corresponding to the preset time interval threshold from the current target audio and video stream by using a pulling-pushing service network based on the sequence from front to back of time stamps; the current target audio and video stream is an audio and video stream which is not pulled currently and corresponds to an off-line singing scene in a content distribution network;

and determining a current progress serial number corresponding to the current streaming operation, and performing progress marking on the real-time audio and video stream by using the current progress serial number to obtain a current marking progress.

Optionally, when the current time interval reaches a preset time interval threshold, pulling, by using a pull-to-push service network, a real-time audio-video stream corresponding to the preset time interval threshold from a current target audio-video stream based on a sequence from front to back of time stamps, including:

determining a target streaming time stamp corresponding to each streaming operation based on the first streaming time stamp and the preset time interval threshold; the time from the first pulling of the streaming-pushing service network to the real-time audio-video stream in the off-line singing scene in the content distribution network is used as the first streaming time stamp;

And when the current time reaches the target streaming time stamp, pulling the real-time audio and video stream corresponding to the preset time interval threshold value from the current target audio and video stream by using a streaming-push service network based on the sequence from front to back of the time stamp.

Optionally, the determining the current progress serial number corresponding to the current streaming operation, and performing progress marking on the real-time audio and video stream by using the current progress serial number to obtain a current marking progress includes:

determining a current progress serial number corresponding to the current streaming operation based on the historical marking progress corresponding to the last streaming time stamp and the preset time interval threshold; if the current drawing operation is the first drawing operation, the current progress serial number is a preset initial value;

and taking the current progress serial number as the current marking progress of the real-time audio and video stream.

Optionally, the pulling, converting and pushing the real-time audio and video stream in the off-line singing scene from the content distribution network by using the service network, and performing progress marking on the real-time audio and video stream to obtain a current marking progress, including:

and continuously pulling the real-time audio and video stream in the off-line singing scene from the content distribution network by using the pulling-pushing service network, and performing progress marking on the real-time audio and video stream by using a preset progress serial number when the current time interval reaches a preset time interval threshold value to obtain the current marking progress.

Optionally, the offline real-time chorus method further includes:

acquiring online voice audio streams uploaded by other chorus user terminals;

and forwarding the online voice audio streams uploaded by the other chorus user terminals to the target chorus user terminal by utilizing the real-time audio and video service network so that the target chorus user terminal can play the real-time audio and video streams and the online voice audio streams uploaded by the other chorus user terminals simultaneously.

Optionally, the pushing the combined audio and video stream to the content distribution network for distribution includes:

pushing the combined audio and video stream to each content distribution network node of the content distribution network, so that the content distribution network node transmits the combined audio and video stream to the on-line audience user terminal after receiving a data pulling request sent by the on-line audience user terminal; wherein the online audience user terminal screens one content distribution network node from the content distribution network nodes based on the current network communication condition and sends the data pulling request to the content distribution network node.

In a second aspect, the present application discloses a real-time chorus method offline, applied to a target chorus user terminal, including:

Pulling and playing a real-time audio and video stream in an off-line singing scene from a real-time audio and video service network in a cloud server, and acquiring the current marking progress of the real-time audio and video stream; the cloud server pulls the real-time audio and video stream from a content distribution network by utilizing the pull-stream-to-push service network, and performs progress marking on the real-time audio and video stream to obtain the current marking progress;

marking the voice audio stream obtained in real time by the chorus application on the local line by utilizing the current marking progress to obtain a marked voice audio stream carrying progress information;

uploading the marked voice audio stream to the real-time audio and video service network so that the real-time audio and video streaming service network can send the real-time audio and video stream, the current marking progress and the marked voice audio stream which are pulled from the real-time audio and video service network to a cloud confluence service network, combining the real-time audio and video stream and the marked voice audio stream based on the current marking progress and progress information through the cloud confluence service network, and pushing the combined audio and video stream to the content distribution network for distribution.

Optionally, the marking the voice audio stream obtained in real time by the chorus application on the local line by using the current marking progress includes:

starting a local audio acquisition device through a chorus application on a local wire to acquire local audio data through the local audio acquisition device, and denoising the local audio data to obtain a voice audio stream;

and marking the voice audio stream acquired by the chorus application on the local line by utilizing the current marking progress.

In a third aspect, the present application discloses an electronic device comprising:

a memory for storing a computer program;

and the processor is used for executing the computer program to realize the steps of the online real-time chorus method.

In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein the computer program when executed by the processor implements the steps of the off-line online real-time chorus method disclosed previously.

As can be seen, the application discloses a real-time chorus method offline, which is applied to a cloud server and specifically comprises the following steps: pulling a real-time audio and video stream in a singing scene under a line from a content distribution network by utilizing a pulling-pushing service network, performing progress marking on the real-time audio and video stream to obtain a current marking progress, and pushing the real-time audio and video stream and the current marking progress to the real-time audio and video service network; the real-time audio and video stream and the current marking progress are issued to a target chorus user terminal by utilizing the real-time audio and video service network, so that the target chorus user terminal plays the real-time audio and video stream, and the voice audio stream obtained in real time by the chorus application on the local line is marked by utilizing the current marking progress, so that a marked voice audio stream carrying progress information is obtained; and acquiring the marked voice audio stream uploaded by the target chorus user terminal through the real-time audio and video service network, pulling the real-time audio and video stream, the current marking progress and the marked voice audio stream to a cloud confluence service network by utilizing the real-time audio and video streaming service network, combining the real-time audio and video stream and the marked voice audio stream by utilizing the cloud confluence service network based on the current marking progress and the progress information, and pushing the combined audio and video stream to the content distribution network for distribution.

Therefore, after the real-time audio/video stream in the off-line singing scene is pulled from the content distribution network, the real-time audio/video stream is subjected to progress marking to obtain the current marking progress, and the real-time audio/video stream and the current marking progress are pushed to the real-time audio/video service network, namely, the pulling and pushing service in the application provides the functions of pulling, progress marking and pushing of the audio/video stream. Further, the real-time audio and video service network transmits the real-time audio and video stream and the current marking progress to the target chorus user terminal so that the target chorus user terminal plays the acquired real-time audio and video stream, marks the voice audio stream acquired in real time by the chorus application on the local line by utilizing the current marking progress, and acquires the marked voice audio stream carrying progress information; it can be appreciated that by marking the locally acquired voice audio stream with the current marking schedule, the off-line real-time audio and video stream can be correlated with the on-line voice audio stream. And after uploading the marked voice audio stream to the real-time audio and video service network by the target chorus user terminal, pulling the real-time audio and video stream, the current marking progress and the marked voice audio stream to the cloud confluence service network by using the real-time audio and video streaming service network, wherein the real-time audio and video stream and the voice audio stream at the moment are marked with corresponding progress, so that the cloud confluence service network accurately combines the real-time audio and video stream and the marked voice audio stream based on the current marking progress and progress information to obtain a combined audio and video stream, and finally pushing the combined audio and video stream back to the content distribution network for distribution. Through the scheme, the real-time chorus effect of the online user and the offline user is realized.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings that are required to be used in the embodiments or the description of the prior art will be briefly described below, and it is obvious that the drawings in the following description are only embodiments of the present invention, and that other drawings may be obtained according to the provided drawings without inventive effort to a person skilled in the art.

FIG. 1 is a schematic diagram of a system framework to which the offline online real-time chorus scheme disclosed herein is applicable;

fig. 2 is a flowchart of a real-time online chorus method of cloud server end offline disclosed in the present application;

FIG. 3 is a flowchart of a specific offline real-time chorus method disclosed in the present application;

fig. 4 is a flowchart of a real-time online chorus method of off-line chorus user terminal disclosed in the present application;

fig. 5 is a block diagram of an electronic device disclosed in the present application.

Detailed Description

The following description of the technical solutions in the embodiments of the present application will be made clearly and completely with reference to the drawings in the embodiments of the present application, and it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Currently, for offline singing scenes, such as a star singing concert, a push stream singing concert and the like, singers and off-desk spectators can participate in choruses offline, and online users can only watch the offline singing concert in real time through terminal equipment, so that a scheme for realizing real-time choruses of online users and offline users is still lacking. Therefore, the embodiment of the application discloses a method, equipment and medium for online real-time chorus under the line, which can realize the online user and the offline user real-time chorus.

In the offline online real-time chorus scheme of the present application, the system framework adopted may be shown in fig. 1, and may specifically include a cloud server 01, a target chorus user terminal 02, and an online audience user terminal 03.

The cloud server 01 specifically includes a streaming push service network 011, a real-time audio/video service network 012, a real-time audio/video streaming service network 013, and a cloud confluence service network 014.

The main function of the pull-push service network 011 is to provide content pull and push, and in this application, a progress marking function is also provided. It should be noted that, an audio/video acquisition device and a pushing device are installed in the offline singing scene, and are configured to acquire an audio/video stream in the offline singing scene in real time and push the audio/video stream to a content delivery network (Content Delivery Network, i.e., CDN), so that the streaming-to-pushing service network 011 specifically pulls a real-time audio/video stream in the offline singing scene from the content delivery network, and performs progress marking on the real-time audio/video stream to obtain a current marking progress, and then pushes the real-time audio/video stream and the current marking progress to the real-time audio/video service network 012.

The real-time audio and video service network 012 is used to provide low-delay real-time audio and video services, and specifically includes, but is not limited to, a Tencent RTC (wherein RTC is Real Time Communication, real-time audio and video), an acoustic network RTC, and the like. The real-time audio and video service network 012 and the pull-push service network 011, the target chorus user terminal 02 and the real-time audio and video pull-stream service network 013 all have data interaction. The real-time audio and video service network 012 is specifically configured to receive a real-time audio and video stream pushed by the pull-stream-push service network 011 and a current marking progress; the real-time audio and video service network 012 is further configured to send the real-time audio and video stream and the current marking progress to the target chorus user terminal 02, and obtain the marked vocal audio stream uploaded by the target chorus user terminal 02; the real-time audio and video service network 012 is also used for responding to a streaming operation of the real-time audio and video streaming service network 013.

The real-time audio and video streaming service network 013 is configured to pull the real-time audio and video stream, the current marking progress and the marked voice audio stream acquired in the real-time audio and video service network 012 to the cloud confluence service network 014.

The cloud converged service network 014 is configured to combine the real-time audio and video stream and the marked voice audio stream based on the current marking progress and the progress information carried by the marked voice audio stream, and push the combined audio and video stream to the content distribution network for distribution, so as to realize the effect that the online audience user terminal 03 can hear real-time chorus of the online user and the offline user.

The target chorus user terminal 02 is a user terminal which uses online chorus application software to participate in chorus online in a network form, and terminal equipment held by the target chorus user can be smart phones, tablets, computers and other equipment. The target chorus user terminal 02 is configured to obtain and play an offline real-time audio/video stream from the real-time audio/video service network 012, mark a voice audio stream obtained in real time by a chorus application on a local line by using a current marking progress obtained from the real-time audio/video service network 012, obtain a marked voice audio stream carrying progress information, and push the marked voice audio stream to the real-time audio/video service network 012.

The online audience user terminal 03 is a user terminal for listening to chorus through a network form by using online chorus application software, and terminal equipment held by an audience user can be a smart phone, a tablet, a computer and the like. The online audience user terminal 03 is configured to pull the combined audio and video stream from the content distribution network, so as to listen to the real-time chorus effect of the online user and the offline user.

Referring to fig. 2, an embodiment of the application discloses a real-time chorus method offline, which is applied to a cloud server, and includes:

Step S11: and pulling real-time audio and video streams in the singing scene under the line from the content distribution network by utilizing a pulling-pushing service network, performing progress marking on the real-time audio and video streams to obtain a current marking progress, and pushing the real-time audio and video streams and the current marking progress to the real-time audio and video service network.

In this embodiment, after the streaming-pushing service network pulls the real-time audio/video stream in the off-line singing scene from the content distribution network, the real-time audio/video stream is progress marked to obtain the current marking progress, and the real-time audio/video stream and the current marking progress are pushed to the real-time audio/video service network. In one embodiment, the real-time audio and video stream and the current marking progress can be respectively pushed to the real-time audio and video service network as two independent data; in another embodiment, the real-time audio/video stream may carry the current marking progress and then be pushed to the real-time audio/video service network as a whole.

It should be noted that, the main function of the pull-to-push service network is to provide content pull and push, and in this embodiment of the present application, a progress marking function is further provided in the pull-to-push service for performing progress marking on the pulled real-time audio stream.

In addition, it should be noted that an audio/video acquisition device and a push device are installed in the offline singing scene, and are used for acquiring video pictures and sounds of the active site in real time to obtain an audio/video stream, and then push the audio/video stream to the content distribution network. In the related scheme, after the content distribution network acquires the real-time audio and video stream in the off-line singing scene, the content distribution network can perform subsequent content distribution, namely, push the real-time audio and video stream to each content distribution network node, so that the content distribution network node sends the real-time audio and video stream to the on-line audience user terminal after receiving the data pulling request sent by the on-line audience user terminal, and the on-line audience user terminal can watch the off-line singing meeting in real time.

In a specific embodiment, the pulling, converting and pushing the real-time audio and video stream in the off-line singing scene from the content distribution network by using the service network, and performing progress marking on the real-time audio and video stream to obtain a current marking progress, including: and continuously pulling the real-time audio and video stream in the off-line singing scene from the content distribution network by using the pulling-pushing service network, and performing progress marking on the real-time audio and video stream by using a preset progress serial number when the current time interval reaches a preset time interval threshold value to obtain the current marking progress. That is, in this embodiment, the streaming and pushing service continuously pulls real-time audio and video streams in the offline singing scene from the content distribution network, and the streaming and pushing service also needs to perform progress marking on the pulled real-time audio and video streams, specifically, when the current time interval reaches the preset time interval threshold, the current progress marking is performed on the current pulled real-time audio and video streams by using the preset progress serial number to obtain the current marking progress. For example, assuming that 20ms is taken as a frame of data, and the preset time interval threshold is set to 20ms correspondingly, the streaming push service is understood to perform progress marking on the currently pulled real-time audio and video stream every 20 ms. In a specific embodiment, the progress serial number may also be set according to a preset time interval threshold, that is, it is assumed that the first time the real-time audio/video stream is pulled to be the start time 0, and then the progress is continuously accumulated every 20ms, so as to obtain the progress serial numbers of 0, 20, 40, 60 and … …. In addition, the progress serial number may be set according to the number of times of the marking operation, for example, each time the progress marking operation is performed, the progress serial number is accumulated 1 time, so as to obtain the progress serial numbers of 1, 2, and 3 … …. The present embodiment does not limit the setting manner of the progress serial number.

Step S12: and transmitting the real-time audio and video stream and the current marking progress to a target chorus user terminal by utilizing the real-time audio and video service network so that the target chorus user terminal plays the real-time audio and video stream, and marking the voice audio stream obtained in real time by the chorus application on the local line by utilizing the current marking progress to obtain the marked voice audio stream with progress information.

In the embodiment, a real-time audio and video service network transmits a real-time audio and video stream pushed by a pull-to-push service and a current marking progress to a target chorus user terminal so that the target chorus user terminal plays the acquired real-time audio and video stream, and marks a voice audio stream acquired in real time by a chorus application on a local line by using the current marking progress to obtain a marked voice audio stream with progress information; it can be appreciated that by marking the locally acquired voice audio stream with the current marking schedule, the off-line real-time audio video stream can be correlated with the on-line voice audio stream.

The local online chorus application can be application software such as a national chorus or QQ music, and a user can click to join in chorus and withdraw from chorus at will.

Further, the method further comprises the following steps: acquiring online voice audio streams uploaded by other chorus user terminals; and forwarding the online voice audio streams uploaded by the other chorus user terminals to the target chorus user terminal by utilizing the real-time audio and video service network so that the target chorus user terminal can play the real-time audio and video streams and the online voice audio streams uploaded by the other chorus user terminals simultaneously. That is, the target chorus user terminal can play the real-time audio and video streams under the line, and also play the online voice audio streams uploaded by other chorus user terminals at the same time, that is, the chorus user can hear the sounds of the offline chorus user and other online chorus users. In a specific embodiment, a user participating in chorus pulls and plays an online vocal audio stream of other online chorus through a low-delay RTC channel of a real-time audio-video service network. That is, the end users participating in chorus communicate audio streams with each other via channels provided by the real-time audio and video service network.

Step S13: and acquiring the marked voice audio stream uploaded by the target chorus user terminal through the real-time audio and video service network, pulling the real-time audio and video stream, the current marking progress and the marked voice audio stream to a cloud confluence service network by utilizing the real-time audio and video streaming service network, combining the real-time audio and video stream and the marked voice audio stream by utilizing the cloud confluence service network based on the current marking progress and the progress information, and pushing the combined audio and video stream to the content distribution network for distribution.

In this embodiment, after uploading the marked vocal audio stream to the real-time audio and video service network, the target chorus user terminal then utilizes the real-time audio and video streaming service network to pull the real-time audio and video stream, the current marking progress and the marked vocal audio stream to the cloud confluence service network. Because the real-time audio and video stream and the voice audio stream at the moment are marked with corresponding progress, the cloud confluence service network accurately combines the real-time audio and video stream and the marked voice audio stream based on the current marking progress and progress information to obtain combined audio and video stream, thereby realizing accurate alignment of the audio and video streams. And finally, the cloud confluence service pushes the combined audio and video streams back to the content distribution network for distribution, so that the effect that other audiences can hear online chorus together.

As can be seen, the application discloses a real-time chorus method offline, which is applied to a cloud server and specifically comprises the following steps: pulling a real-time audio and video stream in a singing scene under a line from a content distribution network by utilizing a pulling-pushing service network, performing progress marking on the real-time audio and video stream to obtain a current marking progress, and pushing the real-time audio and video stream and the current marking progress to the real-time audio and video service network; the real-time audio and video stream and the current marking progress are issued to a target chorus user terminal by utilizing the real-time audio and video service network, so that the target chorus user terminal plays the real-time audio and video stream, and the voice audio stream obtained in real time by the chorus application on the local line is marked by utilizing the current marking progress, so that a marked voice audio stream carrying progress information is obtained; and acquiring the marked voice audio stream uploaded by the target chorus user terminal through the real-time audio and video service network, pulling the real-time audio and video stream, the current marking progress and the marked voice audio stream to a cloud confluence service network by utilizing the real-time audio and video streaming service network, combining the real-time audio and video stream and the marked voice audio stream by utilizing the cloud confluence service network based on the current marking progress and the progress information, and pushing the combined audio and video stream to the content distribution network for distribution.

Therefore, after the real-time audio/video stream in the off-line singing scene is pulled from the content distribution network, the real-time audio/video stream is subjected to progress marking to obtain the current marking progress, and the real-time audio/video stream and the current marking progress are pushed to the real-time audio/video service network, namely, the pulling and pushing service in the application provides the functions of pulling, progress marking and pushing of the audio/video stream. Further, the real-time audio and video service network transmits the real-time audio and video stream and the current marking progress to the target chorus user terminal so that the target chorus user terminal plays the acquired real-time audio and video stream, marks the voice audio stream acquired in real time by the chorus application on the local line by utilizing the current marking progress, and acquires the marked voice audio stream carrying progress information; it can be appreciated that by marking the locally acquired voice audio stream with the current marking schedule, the off-line real-time audio and video stream can be correlated with the on-line voice audio stream. And after uploading the marked voice audio stream to the real-time audio and video service network by the target chorus user terminal, pulling the real-time audio and video stream, the current marking progress and the marked voice audio stream to the cloud confluence service network by using the real-time audio and video streaming service network, wherein the real-time audio and video stream and the voice audio stream at the moment are marked with corresponding progress, so that the cloud confluence service network accurately combines the real-time audio and video stream and the marked voice audio stream based on the current marking progress and progress information to obtain a combined audio and video stream, and finally pushing the combined audio and video stream back to the content distribution network for distribution. Through the scheme, the real-time chorus effect of the online user and the offline user is realized.

Referring to fig. 3, an embodiment of the present application discloses a specific offline online real-time chorus method, and compared with the previous embodiment, the present embodiment further describes and optimizes a technical solution. The method specifically comprises the following steps:

step S21: when the current time interval reaches a preset time interval threshold, pulling a real-time audio and video stream corresponding to the preset time interval threshold from the current target audio and video stream by using a pulling-pushing service network based on the sequence from front to back of time stamps; and the current target audio and video stream is an audio and video stream which is not pulled currently and corresponds to the off-line singing scene in the content distribution network.

In this embodiment, the pull stream push service may intermittently pull the real-time audio and video stream in the offline singing scene from the content distribution network, but in order to ensure real-time performance, the time delay is reduced, and the time interval threshold is not easy to be set too large, for example, may be set to be pulled from the content distribution network once every 20 ms. Therefore, when the current time interval reaches the preset time interval threshold, the real-time audio and video stream corresponding to the preset time interval threshold is pulled from the current target audio and video stream by using the pulling-pushing service network based on the sequence from front to back of the time stamp.

It can be understood that, in order to avoid pulling repeated audio and video streams and ensure the continuity of the audio and video streams, in this embodiment, real-time audio and video streams corresponding to a preset time interval threshold need to be pulled from audio and video streams that are not currently pulled in the content distribution network according to the sequence from front to back of the time stamps. That is, the pull stream push service pulls 20ms of audio/video stream from the content distribution network every time, and supposing that a section of audio/video stream is recorded as 0-100 ms in units of milliseconds, the first time pulls 0-19 ms of content, the second time pulls 20-39 ms of content, and so on.

In a specific embodiment, when the current time interval reaches a preset time interval threshold, pulling, by using a pull-to-push service network, a real-time audio/video stream corresponding to the preset time interval threshold from a current target audio/video stream based on a sequence from front to back of time stamps, including: determining a target streaming time stamp corresponding to each streaming operation based on the first streaming time stamp and the preset time interval threshold; the time from the first pulling of the streaming-pushing service network to the real-time audio-video stream in the off-line singing scene in the content distribution network is used as the first streaming time stamp; and when the current time reaches the target streaming time stamp, pulling the real-time audio and video stream corresponding to the preset time interval threshold value from the current target audio and video stream by using a streaming-push service network based on the sequence from front to back of the time stamp.

That is, since the streaming push service performs the streaming operation once every preset time interval threshold, after determining that the streaming push service network pulls the time of the real-time audio and video stream in the off-line singing scene from the content distribution network for the first time (i.e., the first streaming time stamp), the target streaming time stamp corresponding to each streaming operation can be determined according to the first streaming time stamp and the preset time interval threshold. In this way, when the current time reaches the target streaming time stamp, the streaming-to-push service network is utilized to pull the real-time audio-video stream corresponding to the preset time interval threshold from the current target audio-video stream based on the sequence from front to back of the time stamp.

Step S22: determining a current progress serial number corresponding to the current streaming operation, performing progress marking on the real-time audio and video stream by using the current progress serial number to obtain a current marking progress, and pushing the real-time audio and video stream and the current marking progress to a real-time audio and video service network.

In this embodiment, after the streaming and pushing service performs the streaming operation once and pulls the real-time audio and video stream, the current progress serial number corresponding to the current streaming operation is used to perform progress marking on the real-time audio and video stream to obtain the current marking progress, and then the real-time audio and video stream and the current marking progress are pushed to the real-time audio and video service network.

In a specific embodiment, determining the current progress serial number corresponding to the current streaming operation, and performing progress marking on the real-time audio and video stream by using the current progress serial number to obtain a current marking progress includes: determining a current progress serial number corresponding to the current streaming operation based on the historical marking progress corresponding to the last streaming time stamp and the preset time interval threshold; if the current drawing operation is the first drawing operation, the current progress serial number is a preset initial value; and taking the current progress serial number as the current marking progress of the real-time audio and video stream. That is, in this embodiment, the marking progress serial number may be set according to the time of the pull stream, specifically, the historical marking progress corresponding to the last pull stream timestamp is determined, and then a preset time interval threshold is added on this basis to obtain the current progress serial number corresponding to the current pull stream operation. For example, assuming that the historical marking progress corresponding to the last pull time stamp is 40, the preset pull time interval threshold is 20, then the current progress serial number corresponding to the current pull operation is 60. Under the condition that the threshold value of the pull-stream time interval is unchanged, the marking progress gradually increases along with the time according to the speed of 20 ms. If the current is the first-time streaming operation, the current progress serial number is a preset initial value, for example, may be set to 0.

Step S23: and transmitting the real-time audio and video stream and the current marking progress to a target chorus user terminal by utilizing the real-time audio and video service network so that the target chorus user terminal plays the real-time audio and video stream, and marking the voice audio stream obtained in real time by the chorus application on the local line by utilizing the current marking progress to obtain the marked voice audio stream with progress information.

Step S24: and acquiring the marked voice audio stream uploaded by the target chorus user terminal through the real-time audio and video service network, and pulling the real-time audio and video stream, the current marking progress and the marked voice audio stream to a cloud confluence service network by utilizing a real-time audio and video streaming service network so as to combine the real-time audio and video stream and the marked voice audio stream by utilizing the cloud confluence service network based on the current marking progress and the progress information.

Step S25: pushing the combined audio and video stream to each content distribution network node of the content distribution network, so that the content distribution network node transmits the combined audio and video stream to the on-line audience user terminal after receiving a data pulling request sent by the on-line audience user terminal; wherein the online audience user terminal screens one content distribution network node from the content distribution network nodes based on the current network communication condition and sends the data pulling request to the content distribution network node.

In this embodiment, after merging the real-time audio and video stream and the marked voice audio stream, the cloud converged service network pushes the merged audio and video stream to each content distribution network node of the content distribution network. And the content distribution network node transmits the combined audio and video stream to the online audience user terminal after receiving the data pulling request sent by the online audience user terminal. It can be understood that, based on the current network communication condition, the online audience user terminal screens one content distribution network node from the content distribution network nodes and sends a data pulling request to the content distribution network node, and in a specific embodiment, the online audience user terminal pulls the combined audio and video stream through the content distribution network node closest to the online audience user terminal, so that the online audience can hear the chorus effect of the offline user and the online user together.

For more specific processing procedures in steps S23 and S24, reference may be made to the corresponding contents disclosed in the foregoing embodiments, and no detailed description is given here.

Therefore, in the embodiment of the application, the pulling, turning and pushing service network can complete the operations of pulling, turning and pushing at intervals of a preset time interval threshold. Specifically, since the pull-to-push service performs the pull operation once at intervals of a preset time interval threshold, after determining the first pull time stamp of the pull-to-push service network, the target pull time stamp corresponding to each pull operation can be determined according to the first pull time stamp and the preset time interval threshold. In this way, once the current time reaches the target pull time stamp, the pull operation is performed. In this process, in order to avoid pulling repeated audio and video streams and ensure the continuity of the audio and video streams, in this embodiment, real-time audio and video streams corresponding to a preset time interval threshold need to be pulled from audio and video streams that are not currently pulled in the content distribution network according to the sequence from front to back of the time stamp. In addition, the user terminal of the online audience can pull the combined audio and video stream through the nearest content distribution network node, so that the effect that the online audience can hear the off-line user and the online user together to chorus is achieved.

Referring to fig. 4, an embodiment of the present application discloses a real-time chorus method offline, applied to a target chorus user terminal, including:

step S31: pulling and playing a real-time audio and video stream in an off-line singing scene from a real-time audio and video service network in a cloud server, and acquiring the current marking progress of the real-time audio and video stream; and the cloud server pulls the real-time audio and video stream from a content distribution network by utilizing the pull-stream-to-push service network, and performs progress marking on the real-time audio and video stream to obtain the current marking progress.

In this embodiment, the target chorus user terminal is a user terminal participating in chorus online, and pulls and plays a real-time audio and video stream in an offline singing scene from a real-time audio and video service network in the cloud server, and obtains a current marking progress of the real-time audio and video stream. It should be noted that, the cloud server specifically pulls the real-time audio and video stream from the content distribution network by using a pull-stream-push service network, performs progress marking on the real-time audio and video stream to obtain a current marking progress, and then pushes the real-time audio and video stream and the current marking progress to the real-time audio and video service network by using the pull-stream-push service.

Step S32: and marking the voice audio stream obtained by the chorus application on the local line in real time by utilizing the current marking progress to obtain a marked voice audio stream carrying progress information.

In this embodiment, the target chorus user terminal marks the voice audio stream acquired by the local audio acquisition device and acquired by the chorus application on the local line by using the acquired current marking progress, so as to obtain a marked voice audio stream carrying progress information. The local audio capturing device may specifically be a microphone.

In a specific embodiment, the marking, by using the current marking progress, the vocal audio stream obtained in real time by the chorus application on the local line includes: starting a local audio acquisition device through a chorus application on a local wire to acquire local audio data through the local audio acquisition device, and denoising the local audio data to obtain a voice audio stream; and marking the voice audio stream acquired by the chorus application on the local line by utilizing the current marking progress. It will be appreciated that when the user confirms to participate in chorus, the local on-line chorus application will turn on the local audio collection device, i.e. the microphone, and then collect local audio data via the microphone, and in this process, the local audio data may be denoised to remove ambient sounds and noise to obtain a vocal audio stream. And then marking the voice audio stream acquired through the local online chorus application by utilizing the current marking progress, and marking locally acquired online voice data by utilizing the first progress information. The real-time audio and video stream off line and the voice audio stream on line can be associated by marking the voice audio stream acquired locally by utilizing the current marking progress.

Step S33: uploading the marked voice audio stream to the real-time audio and video service network so that the real-time audio and video streaming service network can send the real-time audio and video stream, the current marking progress and the marked voice audio stream which are pulled from the real-time audio and video service network to a cloud confluence service network, combining the real-time audio and video stream and the marked voice audio stream based on the current marking progress and progress information through the cloud confluence service network, and pushing the combined audio and video stream to the content distribution network for distribution.

In this embodiment, the target chorus user terminal uploads the marked vocal audio stream to the real-time audio and video service network, further, the real-time audio and video stream in the real-time audio and video service platform, the current marking progress and the marked vocal audio stream are pulled to the cloud confluence service network by the real-time audio and video streaming service network, the real-time audio and video stream and the marked vocal audio stream are accurately combined by the cloud confluence service network based on the current marking progress and progress information, and the combined vocal audio and video stream is obtained, so that the accurate alignment of the off-line real-time audio and video stream and the displayed vocal audio and video stream is realized, and finally, the combined vocal audio and video stream is pushed to the content distribution network by the confluence service network for distribution, so that the effect that other audiences can hear the online users and the off-line users together in a chorus is realized.

The target chorus user terminal is a user terminal participating in chorus online, pulls and plays a real-time audio and video stream in an offline singing scene from a real-time audio and video service network in the cloud server, and acquires the current marking progress of the real-time audio and video stream. Further, the target chorus user terminal marks the voice audio stream acquired by the local audio acquisition equipment and acquired by the chorus application on the local line by utilizing the acquired current marking progress, and the marked voice audio stream carrying the progress information is obtained. And finally, uploading the marked voice audio stream to a real-time audio and video service network by the target chorus user terminal, further, simultaneously pulling the real-time audio and video stream, the current marking progress and the marked voice audio stream in the real-time audio and video service platform to a cloud confluence service network by the real-time audio and video streaming service network, and accurately combining the real-time audio and video stream and the marked voice audio stream by the cloud confluence service network based on the current marking progress and progress information to obtain combined voice and video stream, thereby realizing the accurate alignment of the offline real-time audio and video stream and the displayed voice audio stream, and finally pushing the combined voice and video stream to a content distribution network for distribution by the cloud confluence service network, thereby realizing the effect that other audiences can hear the online user and the offline user together for chorus.

Fig. 5 is a schematic structural diagram of an electronic device according to an embodiment of the present application. Specifically, the method comprises the following steps: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input output interface 25, and a communication bus 26. Wherein the memory 22 is configured to store a computer program that is loaded and executed by the processor 21 to implement relevant steps in the offline live chorus method performed by the electronic device as disclosed in any of the foregoing embodiments.

In this embodiment, the power supply 23 is configured to provide an operating voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and an external device, and the communication protocol to be followed is any communication protocol applicable to the technical solution of the present application, which is not specifically limited herein; the input/output interface 25 is used for acquiring external input data or outputting external output data, and the specific interface type thereof may be selected according to the specific application requirement, which is not limited herein.

Processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 may be implemented in at least one hardware form of DSP (Digital Signal Processing ), FPGA (Field-Programmable Gate Array, field programmable gate array), PLA (Programmable Logic Array ). The processor 21 may also comprise a main processor, which is a processor for processing data in an awake state, also called CPU (Central Processing Unit ); a coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 21 may integrate a GPU (Graphics Processing Unit, image processor) for rendering and drawing of content required to be displayed by the display screen. In some embodiments, the processor 21 may also include an AI (Artificial Intelligence ) processor for processing computing operations related to machine learning.

The memory 22 may be a carrier for storing resources, such as a read-only memory, a random access memory, a magnetic disk, or an optical disk, and the resources stored thereon include an operating system 221, a computer program 222, and data 223, and the storage may be temporary storage or permanent storage.

The operating system 221 is used for managing and controlling various hardware devices on the electronic device 20 and the computer program 222, so as to implement the operation and processing of the processor 21 on the mass data 223 in the memory 22, which may be Windows, unix, linux. The computer program 222 may further comprise a computer program capable of performing other specific tasks in addition to the computer program capable of performing the offline live chorus method performed by the electronic device 20 as disclosed in any of the previous embodiments. The data 223 may include, in addition to data received by the electronic device and transmitted by the external device, data collected by the input/output interface 25 itself, and so on.

Further, the embodiment of the application also discloses a computer readable storage medium, wherein the storage medium stores a computer program, and when the computer program is loaded and executed by a processor, the method steps of online real-time chorus are realized.

In this specification, each embodiment is described in a progressive manner, and each embodiment is mainly described in a different point from other embodiments, so that the same or similar parts between the embodiments are referred to each other.

Those of skill would further appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the various illustrative elements and steps are described above generally in terms of functionality in order to clearly illustrate the interchangeability of hardware and software. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the solution. Those skilled in the art may implement the described functionality using different approaches for each particular application, but such implementation is not to be considered as beyond the scope of this application.

The steps of a method or algorithm described in connection with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software modules may be disposed in random access Memory (Random Access Memory, i.e., RAM), memory, read-Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a compact disc Read-Only Memory (Compact Disc Read-Only Memory, i.e., CD-ROM), or any other form of storage medium known in the art.

Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

The above detailed description of the offline real-time chorus method, device and storage medium provided by the invention applies specific examples to illustrate the principle and implementation of the invention, and the above examples are only used to help understand the method and core idea of the invention; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in accordance with the ideas of the present invention, the present description should not be construed as limiting the present invention in view of the above.

Claims (11)

1. The online real-time chorus method is characterized by being applied to a cloud server and comprising the following steps of:

pulling a real-time audio and video stream in a singing scene under a line from a content distribution network by utilizing a pulling-pushing service network, performing progress marking on the real-time audio and video stream to obtain a current marking progress, and pushing the real-time audio and video stream and the current marking progress to the real-time audio and video service network;

the real-time audio and video stream and the current marking progress are issued to a target chorus user terminal by utilizing the real-time audio and video service network, so that the target chorus user terminal plays the real-time audio and video stream, and the voice audio stream obtained in real time by the chorus application on the local line is marked by utilizing the current marking progress, so that a marked voice audio stream carrying progress information is obtained;

and acquiring the marked voice audio stream uploaded by the target chorus user terminal through the real-time audio and video service network, pulling the real-time audio and video stream, the current marking progress and the marked voice audio stream to a cloud confluence service network by utilizing the real-time audio and video streaming service network, combining the real-time audio and video stream and the marked voice audio stream by utilizing the cloud confluence service network based on the current marking progress and the progress information, and pushing the combined audio and video stream to the content distribution network for distribution.

2. The online real-time chorus method of claim 1, wherein the pulling real-time audio and video streams in the offline singing scene from the content distribution network by using the pull-to-push service network, and performing progress marking on the real-time audio and video streams to obtain a current marking progress, comprises:

when the current time interval reaches a preset time interval threshold, pulling a real-time audio and video stream corresponding to the preset time interval threshold from the current target audio and video stream by using a pulling-pushing service network based on the sequence from front to back of time stamps; the current target audio and video stream is an audio and video stream which is not pulled currently and corresponds to an off-line singing scene in a content distribution network;

and determining a current progress serial number corresponding to the current streaming operation, and performing progress marking on the real-time audio and video stream by using the current progress serial number to obtain a current marking progress.

3. The online real-time chorus method according to claim 2, wherein when the current time interval reaches a preset time interval threshold, pulling, by using a pull-to-push service network, a real-time audio-video stream corresponding to the preset time interval threshold from a current target audio-video stream based on a time stamp from front to back, comprising:

Determining a target streaming time stamp corresponding to each streaming operation based on the first streaming time stamp and the preset time interval threshold; the time from the first pulling of the streaming-pushing service network to the real-time audio-video stream in the off-line singing scene in the content distribution network is used as the first streaming time stamp;

and when the current time reaches the target streaming time stamp, pulling the real-time audio and video stream corresponding to the preset time interval threshold value from the current target audio and video stream by using a streaming-push service network based on the sequence from front to back of the time stamp.

4. The online real-time chorus method of claim 2, wherein determining a current progress serial number corresponding to the current streaming operation, and performing progress marking on the real-time audio and video stream by using the current progress serial number to obtain a current marking progress comprises:

determining a current progress serial number corresponding to the current streaming operation based on the historical marking progress corresponding to the last streaming time stamp and the preset time interval threshold; if the current drawing operation is the first drawing operation, the current progress serial number is a preset initial value;

And taking the current progress serial number as the current marking progress of the real-time audio and video stream.

5. The online real-time chorus method of claim 1, wherein the pulling real-time audio and video streams in the offline singing scene from the content distribution network by using the pull-to-push service network, and performing progress marking on the real-time audio and video streams to obtain a current marking progress, comprises:

and continuously pulling the real-time audio and video stream in the off-line singing scene from the content distribution network by using the pulling-pushing service network, and performing progress marking on the real-time audio and video stream by using a preset progress serial number when the current time interval reaches a preset time interval threshold value to obtain the current marking progress.

6. The offline real-time chorus method according to claim 1, further comprising:

acquiring online voice audio streams uploaded by other chorus user terminals;

and forwarding the online voice audio streams uploaded by the other chorus user terminals to the target chorus user terminal by utilizing the real-time audio and video service network so that the target chorus user terminal can play the real-time audio and video streams and the online voice audio streams uploaded by the other chorus user terminals simultaneously.

7. The online real-time chorus method according to any one of claims 1-6, wherein said pushing the combined audio-video streams to said content distribution network for distribution comprises:

pushing the combined audio and video stream to each content distribution network node of the content distribution network, so that the content distribution network node transmits the combined audio and video stream to the on-line audience user terminal after receiving a data pulling request sent by the on-line audience user terminal; wherein the online audience user terminal screens one content distribution network node from the content distribution network nodes based on the current network communication condition and sends the data pulling request to the content distribution network node.

8. The online real-time chorus method is characterized by being applied to a target chorus user terminal and comprising the following steps:

pulling and playing a real-time audio and video stream in an off-line singing scene from a real-time audio and video service network in a cloud server, and acquiring the current marking progress of the real-time audio and video stream; the cloud server pulls the real-time audio and video stream from a content distribution network by utilizing the pull-stream-to-push service network, and performs progress marking on the real-time audio and video stream to obtain the current marking progress;

Marking the voice audio stream obtained in real time by the chorus application on the local line by utilizing the current marking progress to obtain a marked voice audio stream carrying progress information;

uploading the marked voice audio stream to the real-time audio and video service network so that the real-time audio and video streaming service network can send the real-time audio and video stream, the current marking progress and the marked voice audio stream which are pulled from the real-time audio and video service network to a cloud confluence service network, combining the real-time audio and video stream and the marked voice audio stream based on the current marking progress and progress information through the cloud confluence service network, and pushing the combined audio and video stream to the content distribution network for distribution.

9. The offline real-time chorus method according to claim 8, wherein said marking the voice audio stream obtained in real-time by the local online chorus application using the current marking progress comprises:

starting a local audio acquisition device through a chorus application on a local wire to acquire local audio data through the local audio acquisition device, and denoising the local audio data to obtain a voice audio stream;

And marking the voice audio stream acquired by the chorus application on the local line by utilizing the current marking progress.

10. An electronic device, comprising:

a memory for storing a computer program;

a processor for executing the computer program to implement the steps of the offline real-time chorus method according to any of claims 1 to 9.

11. A computer-readable storage medium storing a computer program; wherein the computer program when executed by a processor implements the steps of the online real-time chorus method of any of claims 1 to 9.