CN116168712A - Audio delay cancellation method, device, equipment and storage medium - Google Patents
Audio delay cancellation method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN116168712A CN116168712A CN202310156606.3A CN202310156606A CN116168712A CN 116168712 A CN116168712 A CN 116168712A CN 202310156606 A CN202310156606 A CN 202310156606A CN 116168712 A CN116168712 A CN 116168712A
- Authority
- CN
- China
- Prior art keywords
- audio stream
- audio
- sound card
- time delay
- stream
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 60
- 238000012545 processing Methods 0.000 claims description 30
- 238000012790 confirmation Methods 0.000 claims description 7
- 238000004590 computer program Methods 0.000 claims description 2
- 230000008569 process Effects 0.000 description 25
- 238000004891 communication Methods 0.000 description 7
- 230000003111 delayed effect Effects 0.000 description 6
- 230000005236 sound signal Effects 0.000 description 5
- 238000010586 diagram Methods 0.000 description 4
- 230000006870 function Effects 0.000 description 4
- 230000001360 synchronised effect Effects 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 230000003340 mental effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000750 progressive effect Effects 0.000 description 1
- 230000001755 vocal effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/003—Changing voice quality, e.g. pitch or formants
- G10L21/007—Changing voice quality, e.g. pitch or formants characterised by the process used
- G10L21/01—Correction of time axis
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Processing of the speech or voice signal to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/04—Time compression or expansion
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
Abstract
The application discloses a method, a device, equipment and a storage medium for audio delay cancellation, wherein the method comprises the following steps: the method comprises the steps of obtaining a background audio stream generated by playing background audio, and obtaining a sound card audio stream corresponding to user audio generated by a user based on sound card equipment, wherein the user audio is matched with the background audio, the playing time delay of the sound card audio stream relative to the background audio stream is determined, and the playing time of the background audio stream or the playing time of the sound card audio stream is calibrated based on the playing time delay. Therefore, the sound card audio stream is obtained based on the user audio generated by the sound card equipment, the sound card audio stream with time delay caused by the sound card equipment can be calibrated under the condition that the sound card audio stream is obtained, and the calibrated sound card audio stream and the background audio stream are fused to obtain the mixed audio stream, so that the audience can feel harmony in hearing, live broadcast is realized without delay, and the quality of live broadcast is improved.
Description
Technical Field
The present application relates to the field of internet live broadcasting technologies, and in particular, to a method, an apparatus, a device, and a storage medium for audio delay cancellation.
Background
With the development of the Internet chat industry, more and more people engage in the live broadcast industry to become a live broadcast platform anchor, and the anchor can chat and interact with the audience to enrich the mental life of the audience on the Internet. The anchor needs to be equipped with good live broadcast facilities to ensure good live broadcast effects, and each anchor uses sound card equipment when necessary during live broadcast so as to better present audio information such as voice, songs and the like to audiences. During live broadcasting, a host needs to input sound and push the sound in real time through an input device, during the period, sound card equipment needs to process sound information sent by a user, then generated audio information is pushed, a spectator end needs to pull the sound, and the pulled sound of the spectator end is a mixed sound stream of a human sound audio stream and a background audio stream, which are processed by the sound card equipment, so that live broadcasting is carried out.
However, there is often a significant lack of synchronization between the background music heard by the audience and the presenter's voice, resulting in poor live quality.
Disclosure of Invention
In view of the above problems, the present application is provided to provide a method, apparatus, device, and storage medium for audio delay cancellation, so as to synchronously implement live broadcast and improve live broadcast quality.
In order to achieve the above object, the following specific solutions are proposed:
a method of audio delay cancellation, comprising:
acquiring a background audio stream generated by playing background audio;
acquiring a sound card audio stream corresponding to user audio generated by a user based on sound card equipment, wherein the user audio is matched with the background audio;
determining the playing time delay of the sound card audio stream relative to the background audio stream;
and calibrating the playing time of the background audio stream or the playing time of the sound card audio stream based on the playing time delay.
Optionally, before said determining a play delay of the sound card audio stream relative to the background audio stream, the method further comprises:
mixing the background audio stream and the sound card audio stream to obtain first mixed stream audio;
pushing the first mixed stream audio to a user terminal.
Optionally, the determining a play delay of the user audio stream relative to the background audio stream includes:
acquiring an alignment time delay parameter fed back by the user terminal;
and determining the alignment time delay parameter as the playing time delay of the sound card audio stream relative to the background audio stream.
Optionally, obtaining the alignment delay parameter fed back by the user terminal includes:
based on the first mixed-stream audio, acquiring one or more preset time delay parameters obtained by performing time delay adjustment operation in a time delay adjustment alignment interface of a user of the user terminal;
and for each preset time delay parameter, if an alignment confirmation instruction obtained by the preset time delay parameter is received in a time delay adjustment alignment interface by a user of the user terminal, determining the preset time delay parameter as an alignment time delay parameter.
Optionally, the determining a play delay of the sound card audio stream relative to the background audio stream includes:
acquiring a target identifier corresponding to the sound card audio stream;
judging whether sound card time delay information corresponding to the target mark exists in a preset time delay information database;
if yes, sound card time delay information corresponding to the target mark is determined to be the playing time delay of the sound card audio stream relative to the background audio stream.
Optionally, the obtaining the sound card audio stream corresponding to the user audio generated by the sound card device includes:
when the background audio is played, acquiring user audio generated by a user based on sound card equipment;
and generating a sound card audio stream based on the user audio.
Optionally, calibrating the playing time of the background audio stream or the playing time of the sound card audio stream based on the playing time delay includes:
the playing time of the sound card audio stream is earlier than the playing time delay, so that mixed stream processing is carried out on the sound card audio stream and the background audio stream to obtain second mixed stream audio;
or alternatively, the first and second heat exchangers may be,
delaying the playing time of the background audio stream by the playing time delay to perform mixed stream processing with the sound card audio stream to obtain a third mixed stream audio;
after said calibrating the play time of the background audio stream or the play time of the sound card audio stream based on the play time delay, the method further comprises:
pushing the second mixed stream audio or the third mixed stream audio to one or more user terminals.
An apparatus of audio delay cancellation, comprising:
the background audio stream acquisition unit is used for acquiring a background audio stream generated by playing background audio;
the sound card audio stream acquisition unit is used for acquiring a sound card audio stream corresponding to user audio generated by a user based on sound card equipment, wherein the user audio is matched with the background audio;
a play time delay determining unit, configured to determine a play time delay of the sound card audio stream relative to the background audio stream;
and the time delay calibration unit is used for calibrating the playing time of the background audio stream or the playing time of the sound card audio stream based on the playing time delay.
Optionally, the apparatus further comprises:
the first mixed stream processing unit is used for carrying out mixed stream processing on the background audio stream and the sound card audio stream to obtain first mixed stream audio before the play time delay of the sound card audio stream relative to the background audio stream is determined;
and the first pushing unit is used for pushing the first mixed stream audio to the user terminal.
Optionally, the play delay determining unit includes:
an alignment time delay parameter obtaining unit, configured to obtain an alignment time delay parameter fed back by the user terminal;
and the parameter time delay determining unit is used for determining the alignment time delay parameter as the playing time delay of the sound card audio stream relative to the background audio stream.
Optionally, the alignment delay parameter obtaining unit includes:
a preset time delay parameter obtaining unit, configured to obtain, based on the first mixed-stream audio, one or more preset time delay parameters obtained by performing a time delay adjustment operation in a time delay adjustment alignment interface by a user of the user terminal;
and the target time delay parameter determining unit is used for determining each preset time delay parameter as an alignment time delay parameter if an alignment confirmation instruction obtained by the preset time delay parameter is received in the time delay adjustment alignment interface by a user of the user terminal.
Optionally, the play delay determining unit includes:
the sound card identification acquisition unit is used for acquiring a target sound card identification corresponding to the sound card equipment;
the sound card delay information existence judging unit is used for judging whether sound card delay information corresponding to the target sound card identifier exists in a preset sound card delay information database, and if yes, the delay information determining unit is executed;
and the time delay information determining unit is used for determining the sound card time delay information corresponding to the target sound card identifier as the playing time delay of the sound card audio stream relative to the background audio stream.
Optionally, the sound card audio stream acquiring unit includes:
the user audio acquisition unit is used for acquiring user audio generated by a user based on sound card equipment when the background audio is played;
and the sound card audio stream generating unit is used for generating a sound card audio stream based on the user audio.
Optionally, the delay calibration unit includes:
the first time delay calibration subunit is used for advancing the playing time of the sound card audio stream by the playing time delay so as to carry out mixed stream processing with the background audio stream to obtain second mixed stream audio;
and the second time delay calibration subunit is used for delaying the playing time of the background audio stream by the playing time delay so as to perform mixed stream processing with the sound card audio stream to obtain third mixed stream audio.
The apparatus further comprises:
and the second pushing unit is used for pushing the second mixed stream audio or the third mixed stream audio to one or more user terminals after the playing time of the background audio stream or the playing time of the sound card audio stream is calibrated based on the playing time delay.
An apparatus for audio delay cancellation includes a memory and a processor;
the memory is used for storing programs;
the processor is configured to execute the program to implement the steps of the method for audio delay cancellation as described above.
A storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of a method of audio delay cancellation as described above.
By means of the technical scheme, the method and the device acquire the background audio stream generated by playing the background audio, acquire the sound card audio stream corresponding to the user audio generated by the sound card equipment, wherein the user audio is matched with the background audio, determine the playing time delay of the sound card audio stream relative to the background audio stream, and calibrate the playing time of the background audio stream or the playing time of the sound card audio stream based on the playing time delay. Therefore, the sound card audio stream is obtained based on the user audio generated by the sound card equipment, and the processing of the sound card equipment has time delay, so that the obtained sound card audio stream has time delay, the sound card audio stream with time delay caused by the sound card equipment can be calibrated under the condition that the sound card audio stream is obtained, and the sound card audio stream and the background audio stream after calibration are fused to obtain the mixed audio stream, so that audiences can feel harmony in hearing, live broadcast is realized without delay, and the quality of live broadcast is improved.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the application. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
fig. 1 is a schematic flow chart of audio delay cancellation according to an embodiment of the present application;
fig. 2 is a schematic structural diagram of an audio delay cancellation device according to an embodiment of the present application;
fig. 3 is a schematic structural diagram of an apparatus for audio delay cancellation according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are only some, but not all, of the embodiments of the present application. All other embodiments, which can be made by one of ordinary skill in the art without undue burden from the present disclosure, are within the scope of the present disclosure.
The scheme of the application can be realized based on the electronic equipment with the data processing capability, the electronic equipment can be a computer, a server, a cloud end and the like, and the exemplary electronic terminal can be a live broadcast terminal used by a host broadcast in live broadcast.
Next, as described in connection with fig. 1, the audio delay cancellation method of the present application may include the steps of:
step S110, obtaining a background audio stream generated by playing the background audio.
The background audio is controlled to be played by a background music system of the local live broadcast end, the background audio stream is a real-time push stream, and the audience end can hear the background audio in real time after pulling the background audio stream.
Step S120, obtaining a sound card audio stream corresponding to user audio generated by a user based on the sound card equipment.
Wherein the user audio may be matched with the background audio, e.g., the background audio may be an accompaniment of a song, and then the user audio may be a vocal singing audio of the song.
Specifically, because the sound card audio stream is obtained through the local sound card device, the local sound card device needs to process the user audio, and a certain delay exists after the current pushing, and the current is not synchronous with the real-time background audio stream. For example, in live broadcast, the host broadcast at the local live broadcast end can feel that the voice singing audio (user audio) and the locally played background audio are synchronously carried out, but in reality, the mixed audio stream formed by mixing the background audio stream pulled by the audience end and the sound card audio stream is asynchronous.
Step S130, determining the playing time delay of the sound card audio stream relative to the background audio stream.
It can be understood that, because the sound card audio stream and the background audio stream are not synchronous, and the sound card audio stream is generated after being processed by the sound card device, on the audience side, the sound card audio stream is played slower than the background audio stream, and in the mixed stream formed by mixing the sound card audio stream and the background audio stream, the sound card audio stream has playing time delay relative to the background audio stream.
Wherein, the playing time delay can depend on the sound card equipment, different sound card equipment causes different playing time delay, and the unit of the playing time delay can be expressed in milliseconds.
Furthermore, the information of the playing time delay can be displayed on the live broadcast end and the audience end, so that the host and the audience can know the audio delay condition of the host in the live broadcast process.
Step S140, based on the playing time delay, the playing time of the background audio stream or the playing time of the sound card audio stream is calibrated.
After calibrating the playing time of the background audio stream or the playing time of the sound card audio stream based on the playing time delay, the calibrated mixed audio can be obtained, and the calibrated mixed audio can be pushed to the audience side.
It can be understood that after the playing time of the real-time background audio stream or the playing time of the sound card audio stream is calibrated, the background audio stream pulled by the viewer has no time difference from the sound card audio stream, and the background audio heard by the viewer and the user audio (such as accompaniment and voice during singing) are aligned/coordinated/synchronized, so that a good live broadcast effect is achieved.
According to the audio delay cancellation method, the background audio stream generated by playing the background audio is obtained, the sound card audio stream corresponding to the user audio generated by the sound card equipment is obtained, wherein the user audio is matched with the background audio, the playing time delay of the sound card audio stream relative to the background audio stream is determined, and the playing time of the background audio stream or the playing time of the sound card audio stream is calibrated based on the playing time delay. Therefore, the sound card audio stream is obtained based on the user audio generated by the sound card equipment, and the processing of the sound card equipment has time delay, so that the obtained sound card audio stream has time delay, the sound card audio stream with time delay caused by the sound card equipment can be calibrated under the condition that the sound card audio stream is obtained, and the sound card audio stream and the background audio stream after calibration are fused to obtain the mixed audio stream, so that audiences can feel harmony in hearing, live broadcast is realized without delay, and the quality of live broadcast is improved.
In some embodiments of the present application, considering that the time delay difference between the sound card audio stream and the background audio stream is experienced from the perspective of the viewer, before determining the playing time delay of the sound card audio stream relative to the background audio stream in step S130, the sound card audio stream and the background audio stream may be pushed to the user terminal, and the process may include:
s1, mixing the background audio stream and the sound card audio stream to obtain first mixed stream audio.
Specifically, the background audio stream and the sound card audio stream can be directly mixed according to the original playing time, and no time adjustment operation is required to be executed.
S2, pushing the first mixed stream audio to the user terminal.
Specifically, the user terminal may be a live broadcast terminal for live broadcast, or may be any viewer terminal. The user terminal may provide a delay adjustment alignment interface to the user for the user to operate in.
It can be understood that, because the first mixed stream audio is obtained by mixing the real-time background audio stream and the sound card audio stream with time delay, after the user terminal pulls the first mixed stream audio, the user on the user terminal can hear the asynchronously played background audio and user audio, and the time delay difference between the sound card audio stream and the background audio stream is fully felt.
According to the audio delay cancellation method provided by the embodiment, before the play time delay of the user audio relative to the background audio is determined, the first mixed stream audio is obtained by obtaining the actual received background audio stream and the sound card audio stream at the audience end and carrying out mixed stream processing, so that the user can fully feel the time delay difference between the sound card audio stream and the background audio stream.
In some embodiments of the present application, the process of determining the play delay of the sound card audio stream relative to the background audio stream in the step S130 is described, and the process may include:
s1, acquiring an alignment time delay parameter fed back by a user terminal.
It can be appreciated that after the first mixed stream audio is obtained, the user on the user terminal can feel the time delay difference between the sound card audio stream and the background audio stream, and a specific amount of the time delay difference. On the basis, a user can execute time delay alignment operation on the time delay adjustment alignment interface, and the user terminal can respond to the time delay alignment operation of the user to generate an alignment time delay parameter and feed the alignment time delay parameter back to the electronic equipment.
Specifically, the process of obtaining the alignment delay parameter fed back by the user terminal may include:
s11, based on the first mixed-stream audio, acquiring one or more preset time delay parameters obtained by the time delay adjustment operation executed by a user of the user terminal in a time delay adjustment alignment interface.
Specifically, a preset time delay interval can be provided in the time delay adjustment alignment interface, a user of the user terminal can perform time delay adjustment operation (such as drag operation) in the preset time delay interval, so as to obtain one or more preset time delay parameters, and the user of the user terminal can determine the preset time delay parameters one by one to adjust whether the background audio stream and the sound card audio stream are aligned after the preset time delay parameters are applied.
S12, for each preset time delay parameter, if an alignment confirmation instruction obtained by the preset time delay parameter is received in a time delay adjustment alignment interface by a user of the user terminal, determining the preset time delay parameter as an alignment time delay parameter.
Specifically, in the process of setting preset delay parameters one by one, a user of the user terminal uses the delay adjustment alignment interface to feed back an alignment confirmation indication of the current preset delay parameter when the background audio stream is aligned with the sound card audio stream, and then the preset delay parameter can be used as the alignment delay parameter.
For example, a host may obtain a mixed stream of a background audio stream of a music accompaniment and a sound card audio stream of a singing music on a terminal, the host hears the singing music sung by the host on the terminal slower than the music accompaniment, the host may adjust the playing time of the singing music by a drag operation on a delay adjustment alignment interface, for example, the playing time may be set value by 0.5s, 1.0s, and 1.5s, when the playing time is set to 1.5s, the mixed stream of the background audio stream of the music accompaniment and the sound card audio stream of the singing music is not delayed, the host may feed back an alignment confirmation prompt for 1.5s on the delay adjustment alignment interface of the terminal, and then an alignment delay parameter for 1.5s may be confirmed and generated.
S2, determining the alignment time delay parameter as the playing time delay of the sound card audio stream relative to the background audio stream.
It can be understood that the alignment delay parameter is information that needs to be adjusted and is fed back by the user of the user terminal based on the first mixed stream audio, so that after the sound card audio stream or the background audio stream in the first mixed stream audio is calibrated according to the alignment delay parameter, the user can hear the synchronous background audio and the user audio, and therefore the alignment delay parameter can be determined as the playing delay of the sound card audio stream relative to the background audio stream.
In some embodiments of the present application, the process of determining the play delay of the sound card audio stream relative to the background audio stream in the step S130 is described, and the process may include:
s1, acquiring a target identifier corresponding to a sound card audio stream.
Specifically, because different sound card devices process the user audio, the time delay generated by processing the user audio by different sound card devices is also different, so the target identifier can be the identifier corresponding to the sound card device generating the sound card audio stream. Meanwhile, the target identifier may also be a user identifier stored in advance for the anchor who generates the sound card audio stream.
S2, judging whether sound card time delay information corresponding to the target mark exists in a preset time delay information database, and if yes, executing S3.
Specifically, the delay information database may be information of delay generated by processing the user audio by a plurality of sound card devices, or information of delay pre-stored by a host who generates the sound card audio stream, and summarizing the information.
S3, determining the sound card time delay information corresponding to the target mark as the playing time delay of the sound card audio stream relative to the background audio stream.
According to the audio delay counteracting method, the target identification corresponding to the sound card audio stream is found from the pre-established delay information database, when the sound card delay information corresponding to the target identification exists in the pre-set delay information database, the sound card delay information corresponding to the target sound card identification is directly inquired to be determined as the playing delay of the sound card audio stream relative to the background audio stream, and therefore synchronization of a plurality of audio streams can be achieved rapidly in a live broadcast process.
In some embodiments of the present application, the process of obtaining the sound card audio stream corresponding to the user audio generated by the sound card device in step S120 is described, where the process may include:
s1, when the background audio is played, user audio generated by a user based on sound card equipment is obtained.
It will be appreciated that since the user audio is matched to the background audio, the user audio is available based on the context in which the background audio is played.
For example, when background music is played, a user can sing a song under the accompaniment of the background music, and the live broadcast end can acquire user audio generated by singing of the user.
S2, generating a sound card audio stream based on the user audio.
Specifically, the user audio may be transmitted to the sound card device, and the sound card device processes the user audio to obtain a sound card audio stream.
In some embodiments of the present application, the process of determining the play delay of the sound card audio stream relative to the background audio stream in the step S130 is described, and the process may include:
s1, acquiring sound card time delay information of local sound card equipment.
Specifically, the sound card delay information of the local sound card device may represent information of total duration used by the process of processing the user sound signal by the local sound card device to obtain the user audio. The process of obtaining sound card delay information of the local sound card apparatus may include:
s11, calculating first time information of a coding result by the local sound card equipment for carrying out DC coding on the user sound signal.
Wherein the user sound signal may be provided by the user in background audio.
Specifically, the local sound card device performs DC encoding on the user sound signal in the first time to obtain an encoding result, where the encoding result needs to be further decoded.
S12, calculating second time information of the decoding result obtained by decoding the encoding result by the local sound card equipment.
Specifically, the local sound card device decodes the encoding result obtained by the DC encoding in the second time to obtain a decoding result, and the decoding result needs further processing.
S13, processing the decoding result by the local sound card computing equipment to obtain third time information of the user audio.
The processing may include 3A processing, sound-changing processing, and reverberation processing, among others.
Specifically, after the local sound card device obtains the decoding result, the local sound card device needs to perform 3A, sound changing and reverberation processing on the decoding result in a third time to obtain the user audio.
S14, accumulating the first time information, the second time information and the third time information to obtain sound card time delay information of the local sound card equipment.
S2, according to the sound card time delay information, determining the playing time delay of the sound card audio stream relative to the background audio stream.
It will be appreciated that the time delay of playing the sound card audio stream relative to the background audio stream is caused by the local sound card device, so that the time delay of playing the sound card audio stream relative to the background audio stream can be determined according to the sound card time delay information.
According to the audio delay cancellation method provided by the embodiment, the time required by the process of processing the user sound signal of the user by the local sound card equipment to obtain the user audio is calculated, and the playing time delay of the user audio relative to the background audio is determined.
In some embodiments of the present application, the procedure of calibrating the playing time of the background audio stream or the playing time of the sound card audio stream based on the playing time delay in the step S140 is described, and the procedure may include the following two cases:
firstly, the playing time of the sound card audio stream is delayed in advance, so that mixed stream processing is carried out on the sound card audio stream and the background audio stream to obtain second mixed stream audio.
It can be understood that before the sound card audio stream is calibrated, the accurate playing time of the sound card audio stream should be before the original playing time, so that the playing time of the user audio can be delayed in advance, and after the sound card audio stream is calibrated, the calibrated sound card audio stream and the background audio stream which is not required to be processed can be mixed to obtain the second mixed stream audio.
The user audio corresponding to the sound card audio stream in the second mixed stream audio and the background audio corresponding to the background audio stream are synchronously performed at the audience side.
And secondly, delaying the playing time of the background audio stream by playing time delay so as to carry out mixed stream processing with the sound card audio stream to obtain third mixed stream audio.
It can be understood that before the background audio stream is calibrated, the accurate playing time of the background audio stream should be after the original playing time, so that the playing time of the background audio stream can be delayed by the playing time delay, and after the background audio stream is calibrated, the calibrated background audio stream and the sound card audio stream which is not required to be processed can be mixed to obtain a third mixed stream audio.
The user audio corresponding to the sound card audio stream in the third mixed stream audio and the background audio corresponding to the background audio stream are synchronously performed at the audience side.
In view of enabling the viewer side to pull the real-time/non-delayed audio stream, in some embodiments of the present application, after calibrating the play time of the background audio stream or the play time of the sound card audio stream based on the play time delay mentioned in the above embodiments, a process of pushing the real-time/non-delayed audio stream to the viewer side may be included, where the process may include:
pushing the second mixed stream audio or the third mixed stream audio to one or more user terminals.
In particular, the one or more user terminals may include a head end, as well as any spectator end.
Further, the anchor may listen to the second mixed-stream audio or the third mixed-stream audio again on the anchor side to check whether the sound card audio stream in the second mixed-stream audio or the third mixed-stream audio is aligned with the background audio stream.
The device for implementing audio delay cancellation provided in the embodiments of the present application is described below, and the device for implementing audio delay cancellation described below and the method for implementing audio delay cancellation described above may be referred to correspondingly.
Referring to fig. 2, fig. 2 is a schematic structural diagram of an apparatus for implementing audio delay cancellation according to an embodiment of the present application.
As shown in fig. 2, the apparatus may include:
a background audio stream obtaining unit 11 for obtaining a background audio stream generated by playing a background audio;
a sound card audio stream obtaining unit 12, configured to obtain a sound card audio stream corresponding to user audio generated by a user based on sound card equipment, where the user audio matches with background audio;
a play delay determining unit 13, configured to determine a play delay of the sound card audio stream relative to the background audio stream;
the time delay calibration unit 14 is configured to calibrate a playing time of the background audio stream or a playing time of the sound card audio stream based on the playing time delay.
Optionally, the apparatus further comprises:
the first mixed stream processing unit is used for carrying out mixed stream processing on the background audio stream and the sound card audio stream to obtain first mixed stream audio before determining the playing time delay of the sound card audio stream relative to the background audio stream;
the first pushing unit is used for pushing the first mixed stream audio to the user terminal.
Optionally, the play delay determining unit includes:
an alignment time delay parameter obtaining unit, configured to obtain an alignment time delay parameter fed back by a user terminal;
and the parameter time delay determining unit is used for determining the alignment time delay parameter as the playing time delay of the sound card audio stream relative to the background audio stream.
Optionally, the alignment delay parameter obtaining unit includes:
the preset time delay parameter acquisition unit is used for acquiring one or more preset time delay parameters obtained by performing time delay adjustment operation in a time delay adjustment alignment interface of a user of the user terminal based on the first mixed stream audio;
the target delay parameter determining unit is used for determining each preset delay parameter as an alignment delay parameter if an alignment confirmation instruction obtained by the preset delay parameter is received from a user of the user terminal in the delay adjustment alignment interface.
Optionally, the play delay determining unit includes:
the sound card identification acquisition unit is used for acquiring a target sound card identification corresponding to the sound card equipment;
the sound card delay information existence judging unit is used for judging whether sound card delay information corresponding to the target sound card identifier exists in a preset sound card delay information database, and if yes, the delay information determining unit is executed;
and the time delay information determining unit is used for determining the sound card time delay information corresponding to the target sound card identifier as the playing time delay of the sound card audio stream relative to the background audio stream.
Optionally, the sound card audio stream acquiring unit includes:
the user audio acquisition unit is used for acquiring user audio generated by a user based on sound card equipment when the background audio is played;
and the sound card audio stream generating unit is used for generating a sound card audio stream based on the user audio.
Optionally, the delay calibration unit includes:
the first time delay calibration subunit is used for delaying the playing time of the sound card audio stream in advance so as to perform mixed stream processing with the background audio stream to obtain second mixed stream audio;
the second time delay calibration subunit is used for delaying the playing time of the background audio stream by playing time delay so as to perform mixed stream processing with the sound card audio stream to obtain third mixed stream audio;
the apparatus further comprises:
and the second pushing unit is used for pushing the second mixed stream audio or the third mixed stream audio to one or more user terminals after calibrating the playing time of the background audio stream or the playing time of the sound card audio stream based on the playing time delay.
The audio delay cancellation device provided by the embodiment of the application can be applied to audio delay cancellation equipment. Optionally, fig. 3 shows a block diagram of a hardware structure of an apparatus for audio delay cancellation, and referring to fig. 3, the hardware structure of the apparatus for audio delay cancellation may include: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;
in the embodiment of the application, the number of the processor 1, the communication interface 2, the memory 3 and the communication bus 4 is at least one, and the processor 1, the communication interface 2 and the memory 3 complete communication with each other through the communication bus 4;
processor 1 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement embodiments of the present invention, etc.;
the memory 3 may comprise a high-speed RAM memory, and may further comprise a non-volatile memory (non-volatile memory) or the like, such as at least one magnetic disk memory;
wherein the memory stores a program, and the processor is operable to invoke the program stored in the memory, the program being operable to:
acquiring a background audio stream generated by playing background audio;
acquiring a sound card audio stream corresponding to user audio generated by a user based on sound card equipment, wherein the user audio is matched with background audio;
determining the playing time delay of the sound card audio stream relative to the background audio stream;
based on the play time delay, the play time of the background audio stream or the play time of the sound card audio stream is calibrated.
Alternatively, the refinement function and the extension function of the program may be described with reference to the above.
The embodiment of the present application also provides a storage medium, where a program adapted to be executed by a processor may be stored, where the program is configured to:
acquiring a background audio stream generated by playing background audio;
acquiring a sound card audio stream corresponding to user audio generated by a user based on sound card equipment, wherein the user audio is matched with background audio;
determining the playing time delay of the sound card audio stream relative to the background audio stream;
based on the play time delay, the play time of the background audio stream or the play time of the sound card audio stream is calibrated.
Alternatively, the refinement function and the extension function of the program may be described with reference to the above.
Finally, it is further noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
In the present specification, each embodiment is described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and may be combined according to needs, and the same similar parts may be referred to each other.
The previous description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the present application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the spirit or scope of the application. Thus, the present application is not intended to be limited to the embodiments shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
Claims (10)
1. A method of audio delay cancellation, comprising:
acquiring a background audio stream generated by playing background audio;
acquiring a sound card audio stream corresponding to user audio generated by a user based on sound card equipment, wherein the user audio is matched with the background audio;
determining the playing time delay of the sound card audio stream relative to the background audio stream;
and calibrating the playing time of the background audio stream or the playing time of the sound card audio stream based on the playing time delay.
2. The method of claim 1, wherein prior to said determining a playback delay of said sound card audio stream relative to said background audio stream, said method further comprises:
mixing the background audio stream and the sound card audio stream to obtain first mixed stream audio;
pushing the first mixed stream audio to a user terminal.
3. The method of claim 2, wherein said determining a playback delay of the user audio stream relative to the background audio stream comprises:
acquiring an alignment time delay parameter fed back by the user terminal;
and determining the alignment time delay parameter as the playing time delay of the sound card audio stream relative to the background audio stream.
4. The method of claim 3, wherein obtaining the alignment delay parameter fed back by the user terminal comprises:
based on the first mixed-stream audio, acquiring one or more preset time delay parameters obtained by performing time delay adjustment operation in a time delay adjustment alignment interface of a user of the user terminal;
and for each preset time delay parameter, if an alignment confirmation instruction obtained by the preset time delay parameter is received in a time delay adjustment alignment interface by a user of the user terminal, determining the preset time delay parameter as an alignment time delay parameter.
5. The method of claim 1, wherein said determining a playback delay of the sound card audio stream relative to the background audio stream comprises:
acquiring a target identifier corresponding to the sound card audio stream;
judging whether sound card time delay information corresponding to the target mark exists in a preset time delay information database;
if yes, sound card time delay information corresponding to the target mark is determined to be the playing time delay of the sound card audio stream relative to the background audio stream.
6. The method of claim 1, wherein the obtaining a sound card audio stream corresponding to user audio generated by the user based on the sound card device comprises:
when the background audio is played, acquiring user audio generated by a user based on sound card equipment;
and generating a sound card audio stream based on the user audio.
7. The method of any of claims 1-6, wherein calibrating the playback time of the background audio stream or the playback time of the sound card audio stream based on the playback delay comprises:
the playing time of the sound card audio stream is earlier than the playing time delay, so that mixed stream processing is carried out on the sound card audio stream and the background audio stream to obtain second mixed stream audio;
or alternatively, the first and second heat exchangers may be,
delaying the playing time of the background audio stream by the playing time delay to perform mixed stream processing with the sound card audio stream to obtain a third mixed stream audio;
after said calibrating the play time of the background audio stream or the play time of the sound card audio stream based on the play time delay, the method further comprises:
pushing the second mixed stream audio or the third mixed stream audio to one or more user terminals.
8. An apparatus for audio delay cancellation, comprising:
the background audio stream acquisition unit is used for acquiring a background audio stream generated by playing background audio;
the sound card audio stream acquisition unit is used for acquiring a sound card audio stream corresponding to user audio generated by a user based on sound card equipment, wherein the user audio is matched with the background audio;
a play time delay determining unit, configured to determine a play time delay of the sound card audio stream relative to the background audio stream;
and the time delay calibration unit is used for calibrating the playing time of the background audio stream or the playing time of the sound card audio stream based on the playing time delay.
9. An apparatus for audio delay cancellation comprising a memory and a processor;
the memory is used for storing programs;
the processor being configured to execute the program to perform the steps of the method of audio delay cancellation as claimed in any one of claims 1 to 7.
10. A storage medium having stored thereon a computer program, which, when executed by a processor, performs the steps of the method of audio delay cancellation as claimed in any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310156606.3A CN116168712A (en) | 2023-02-23 | 2023-02-23 | Audio delay cancellation method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310156606.3A CN116168712A (en) | 2023-02-23 | 2023-02-23 | Audio delay cancellation method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116168712A true CN116168712A (en) | 2023-05-26 |
Family
ID=86411103
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310156606.3A Pending CN116168712A (en) | 2023-02-23 | 2023-02-23 | Audio delay cancellation method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116168712A (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110970045A (en) * | 2019-11-15 | 2020-04-07 | 北京达佳互联信息技术有限公司 | Mixing processing method, mixing processing device, electronic equipment and storage medium |
| CN111583952A (en) * | 2020-05-19 | 2020-08-25 | 北京达佳互联信息技术有限公司 | Audio processing method and device, electronic equipment and storage medium |
| US20210365234A1 (en) * | 2020-05-20 | 2021-11-25 | Shmuel Ur Innovation Ltd. | Streaming of multi-location live events |
| CN113891152A (en) * | 2021-09-28 | 2022-01-04 | 广州华多网络科技有限公司 | Audio playing control method and device, equipment, medium and product thereof |
| CN113938746A (en) * | 2021-09-28 | 2022-01-14 | 广州华多网络科技有限公司 | Network live broadcast audio processing method and device, equipment, medium and product thereof |
| CN114125480A (en) * | 2021-11-17 | 2022-03-01 | 广州方硅信息技术有限公司 | Live broadcasting chorus interaction method, system and device and computer equipment |
| CN115631738A (en) * | 2022-10-26 | 2023-01-20 | 深圳市冠旭电子股份有限公司 | Audio data processing method and device, electronic equipment and storage medium |
-
2023
- 2023-02-23 CN CN202310156606.3A patent/CN116168712A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110970045A (en) * | 2019-11-15 | 2020-04-07 | 北京达佳互联信息技术有限公司 | Mixing processing method, mixing processing device, electronic equipment and storage medium |
| CN111583952A (en) * | 2020-05-19 | 2020-08-25 | 北京达佳互联信息技术有限公司 | Audio processing method and device, electronic equipment and storage medium |
| US20210365234A1 (en) * | 2020-05-20 | 2021-11-25 | Shmuel Ur Innovation Ltd. | Streaming of multi-location live events |
| CN113891152A (en) * | 2021-09-28 | 2022-01-04 | 广州华多网络科技有限公司 | Audio playing control method and device, equipment, medium and product thereof |
| CN113938746A (en) * | 2021-09-28 | 2022-01-14 | 广州华多网络科技有限公司 | Network live broadcast audio processing method and device, equipment, medium and product thereof |
| CN114125480A (en) * | 2021-11-17 | 2022-03-01 | 广州方硅信息技术有限公司 | Live broadcasting chorus interaction method, system and device and computer equipment |
| CN115631738A (en) * | 2022-10-26 | 2023-01-20 | 深圳市冠旭电子股份有限公司 | Audio data processing method and device, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107027050B (en) | Audio and video processing method and device for assisting live broadcast | |
| CN112714330B (en) | Gift presenting method and device based on live broadcast with wheat and electronic equipment | |
| DE112018001871T5 (en) | Audiovisual collaboration process with latency management for large-scale transmission | |
| US20080184870A1 (en) | System, method, device, and computer program product providing for a multiple-lyric karaoke system | |
| US20040044532A1 (en) | System and method for remote audio caption visualizations | |
| CN110910860B (en) | Online KTV implementation method and device, electronic equipment and storage medium | |
| CN110944226B (en) | Network Karaoke system, lyric display method in Karaoke scene and related equipment | |
| CN110741435B (en) | Method, system, and medium for audio signal processing | |
| CN111028818B (en) | Chorus method, apparatus, electronic device and storage medium | |
| CN110856009B (en) | Network karaoke system, audio and video playing method of network karaoke and related equipment | |
| CN113014477A (en) | Gift processing method, device and equipment of voice platform and storage medium | |
| US11902632B2 (en) | Timely addition of human-perceptible audio to mask an audio watermark | |
| CN110536147B (en) | Live broadcast processing method, device and system | |
| CN114466242A (en) | Display device and audio processing method | |
| CN116168712A (en) | Audio delay cancellation method, device, equipment and storage medium | |
| JP5454802B2 (en) | Karaoke equipment | |
| JP2008171194A (en) | Communication system, communication method, server, and terminal | |
| JP2002164862A (en) | Radio program automatic preparation and broadcasting method thereof | |
| CN114528432A (en) | Chorus method, apparatus, device and readable storage medium | |
| WO2022210971A1 (en) | Information processing device and data synchronization method | |
| CN117676184A (en) | Synchronization method for live chorus audio, computer equipment and storage medium | |
| CN117524179A (en) | Song beat data processing method, device, equipment and storage medium | |
| CN113132785A (en) | Multimedia data method, device, electronic equipment and computer storage medium | |
| CN111625677A (en) | Audio playing method, electronic equipment and storage medium | |
| CN117641039A (en) | Singing live broadcasting method, equipment and storage medium of KTV playing live broadcasting room |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |