WO2023284414A1 - 音频合流方法、音频上传方法、设备及程序产品 - Google Patents

音频合流方法、音频上传方法、设备及程序产品 Download PDF

Info

Publication number
WO2023284414A1
WO2023284414A1 PCT/CN2022/094499 CN2022094499W WO2023284414A1 WO 2023284414 A1 WO2023284414 A1 WO 2023284414A1 CN 2022094499 W CN2022094499 W CN 2022094499W WO 2023284414 A1 WO2023284414 A1 WO 2023284414A1
Authority
WO
WIPO (PCT)
Prior art keywords
audio
gain
live broadcast
anchor
anchors
Prior art date
Application number
PCT/CN2022/094499
Other languages
English (en)
French (fr)
Inventor
陈映宜
马行健
Original Assignee
北京字节跳动网络技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京字节跳动网络技术有限公司 filed Critical 北京字节跳动网络技术有限公司
Publication of WO2023284414A1 publication Critical patent/WO2023284414A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/233—Processing of audio elementary streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/21—Server components or server architectures
    • H04N21/218—Source of audio or video content, e.g. local disk arrays
    • H04N21/2187—Live feed
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
    • H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
    • H04N21/236—Assembling of a multiplex stream, e.g. transport stream, by combining a video stream with other content or additional data, e.g. inserting a URL [Uniform Resource Locator] into a video stream, multiplexing software data into a video stream; Remultiplexing of multiplex streams; Insertion of stuffing bits into the multiplex stream, e.g. to obtain a constant bit-rate; Assembling of a packetised elementary stream
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/437—Interfacing the upstream path of the transmission network, e.g. for transmitting client requests to a VOD server
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
    • H04N21/439—Processing of audio elementary streams
    • H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/47—End-user applications
    • H04N21/478—Supplemental services, e.g. displaying phone caller identification, shopping application
    • H04N21/4788—Supplemental services, e.g. displaying phone caller identification, shopping application communicating with other users, e.g. chatting

Definitions

  • Embodiments of the present disclosure relate to audio processing technologies, and in particular, to an audio merging method, an audio uploading method, a device, and a program product.
  • an anchor can enter the room of other anchors, and viewers can watch the live broadcast images of multiple anchors through the terminal, thereby increasing the interest of the live broadcast content.
  • the terminals of each anchor will record audio and video, and send the recorded audio to the server, and the server will merge the audio of each anchor and send The merged audio is sent to the terminal watching the live broadcast.
  • Embodiments of the present disclosure provide an audio merging method, an audio uploading method, a device, and a program product to solve the problem in the prior art that multiple anchors broadcast live in one live broadcast room, and the audio gains of each anchor are different.
  • an embodiment of the present disclosure provides an audio merging method, including:
  • the live broadcast room is a virtual live broadcast room in the network
  • the actual volume gain of the audio of other anchors in each audio is adjusted to the actual volume gain of the audio of the current anchor;
  • the other anchors are anchors other than the current anchor among the anchors in the live broadcast room;
  • the merged audio corresponding to the current anchor is generated according to each audio after the volume gain is adjusted, and the merged audio is used to play the live content of the live room of the current anchor.
  • an audio upload method including:
  • the audio recording information represents the recording information when the audio is obtained
  • the actual volume gain of the anchor audio is used to adjust the gain of the anchor audio, and the adjusted audio is used to generate merged audio .
  • an audio converging device including:
  • the acquisition unit is used to acquire the audio and the actual volume gain of the terminals of the anchors entering the same live broadcast room; wherein, the live broadcast room is a virtual live broadcast room in the network;
  • the adjustment unit is used to adjust the actual volume gain of the audio of other anchors in each audio to the actual volume gain of the audio of the current anchor;
  • the other anchors are anchors other than the current anchor among the anchors in the live broadcast room;
  • the merging unit is configured to generate merging audio corresponding to the current anchor according to each audio after the volume gain is adjusted, and the merging audio is used to play the live content of the live broadcast room of the current anchor.
  • an audio uploading device including:
  • An acquisition unit configured to acquire audio and at least one audio recording information corresponding to the audio during the live broadcast, wherein the audio recording information represents the recording information when the audio is acquired;
  • a gain determination unit configured to determine the actual volume gain of the audio according to the at least one audio recording information
  • the sending unit is configured to send the audio and the actual volume gain of the audio to the server.
  • the actual volume gain of the anchor audio is used to adjust the gain of the anchor audio.
  • the adjusted audio Used to generate merged audio.
  • an embodiment of the present disclosure provides an electronic device, including: at least one processor and a memory;
  • the memory stores computer-executable instructions
  • the at least one processor executes the computer-executed instructions stored in the memory, so that the at least one processor executes the audio combining method described in the above first aspect and various possible designs of the first aspect, or executes the above second aspect And the audio upload method described in various possible designs of the second aspect.
  • the embodiments of the present disclosure provide a computer-readable storage medium, the computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the above first aspect and the first
  • an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the audio combining method described in the above first aspect and various possible designs of the first aspect, Or execute the audio uploading method described in the above second aspect and various possible designs of the second aspect.
  • the embodiments of the present disclosure provide a computer program, which implements the audio combining method described in the above first aspect and various possible designs of the first aspect when the computer program is executed by a processor, or implements the above second aspect
  • the audio merging method, audio uploading method, device, and program product provided in this embodiment can adjust the volume gain of the audio of other anchors in the live broadcast room, so that the audio gain of each other anchor in the live broadcast room is the same as that of the audio of the current anchor.
  • the gain is the same, and then the merged audio corresponding to the current anchor is obtained.
  • the sound volume of each anchor in the merged audio is the same.
  • Fig. 1 is a schematic diagram of a co-shooting process shown in an exemplary embodiment
  • FIG. 2 is a schematic flowchart of an audio combining method shown in an exemplary embodiment of the present disclosure
  • FIG. 3 is an architecture diagram of a live broadcast system shown in an exemplary embodiment of the present disclosure
  • FIG. 4 is a schematic diagram of a live content push flow shown in an exemplary embodiment of the present disclosure
  • Fig. 5 is a schematic flowchart of an audio uploading method shown in an exemplary embodiment of the present disclosure
  • Fig. 6 is a schematic flowchart of an audio uploading method shown in another exemplary embodiment of the present disclosure.
  • Fig. 7 is a schematic structural diagram of an audio combining device shown in an exemplary embodiment of the present disclosure.
  • Fig. 8 is a schematic structural diagram of an audio combining device according to another exemplary embodiment of the present disclosure.
  • Fig. 9 is a schematic structural diagram of an audio uploading device according to an exemplary embodiment of the present disclosure.
  • Fig. 10 is a schematic structural diagram of an audio uploading device according to another exemplary embodiment of the present disclosure.
  • Fig. 11 is a schematic structural diagram of an electronic device according to an exemplary embodiment of the present disclosure.
  • the field of live broadcast includes co-production scenarios.
  • at least one second anchor can enter the live broadcast room of the first anchor, and the live broadcast screen displayed by the user terminal watching the first anchor includes the above-mentioned live broadcast of the first anchor and the second anchor. screen, the user terminal can also play audio during the live broadcast of the first anchor and the second anchor.
  • Fig. 1 is a schematic diagram of a co-shooting process shown in an exemplary embodiment.
  • the first anchor 11 uses the first terminal 12 to perform live broadcast, and the first terminal 12 can collect the picture and audio of the first anchor 11 during the live broadcast.
  • Each second anchor 13 uses each second terminal 14 to perform live broadcast, and each second terminal 14 can collect the picture and audio of each second anchor 13 during live broadcast.
  • the first terminal 12 and each second terminal 14 can send the collected pictures and audio to the cloud 15, and the cloud 15 processes the live pictures of the anchors entering the same live broadcast room to generate a co-shooting picture, and can also process the live pictures of the anchors entering the same live broadcast room.
  • the live audio of each anchor in the room is merged to generate the merged audio.
  • the cloud 15 can push the processed co-production picture and co-production audio to the terminal side of the user 16 watching the live broadcast, so that the user 16 can watch the co-production picture and audio through the terminal.
  • the live broadcast parameters adopted when each terminal is used for live broadcast may also be different, so each audio frequency collected by each terminal for live broadcast The volume gain will also be different.
  • the voices of each anchor will also be different.
  • some anchors have loud voices, and some anchors have low voices, resulting in poor user experience. .
  • the audio gain of each other host in the live broadcast room is adjusted so that the volume gain of the audio of other hosts is the same as the gain of the audio of the current host, so that the multi-channel audio in the live broadcast Gain is the same, and then the audio of each channel is merged to obtain the merged audio.
  • the server can push the merged audio of the current anchor to the user terminal.
  • the audio of each anchor included in the merged audio The volume gain is the same, thereby improving the user experience of watching the live broadcast.
  • Fig. 2 is a schematic flowchart of an audio merging method according to an exemplary embodiment of the present disclosure.
  • the audio confluence method provided by the present disclosure includes:
  • Step 201 acquire the audio and actual volume gain of terminals of each host entering the same live broadcast room; wherein, the live broadcast room is a virtual live broadcast room in the network.
  • the method provided in the present disclosure can be executed by an electronic device with computing capability, for example, it can be executed by a background server of a live broadcast.
  • Fig. 3 is an architecture diagram of a live broadcast system according to an exemplary embodiment of the present disclosure.
  • the live broadcast system may include a live broadcast terminal 31 , a live broadcast server 32 and a user terminal 33 for watching the live broadcast.
  • the live broadcast terminal 31 can be used to record the live broadcast content, and the live broadcast content is uploaded to the live broadcast server 32 through the live broadcast terminal 31, and then the live broadcast server 32 pushes the stream to the user terminal 33.
  • the live broadcast content sent by the live broadcast terminal 31 to the live broadcast server includes recorded audio and its volume gain.
  • the volume gain is used to characterize the volume of the audio.
  • the volume can be set, and the live broadcast terminal can obtain the set volume gain, and determine the actual volume gain of the audio according to the set volume gain.
  • the user can also select information such as special effects of the live broadcast during the live broadcast, and the live broadcast end can also obtain the special effect information during the live broadcast, and determine the actual volume gain of the audio according to the special effect information.
  • the live broadcast end can obtain a plurality of pieces of information used to collect audio during live broadcast, so as to determine the actual volume gain of the audio collected during live broadcast according to the information.
  • the live broadcast end can send the audio and its actual volume gain to the server.
  • the server can obtain the audio uploaded by the terminals of these anchors and the actual volume gain of each audio.
  • the live broadcast room is a virtual live broadcast room in the network.
  • anchors A, B, and C there are anchors A, B, and C, and anchor B and anchor C can enter the live broadcast room of anchor A by operating their respective live broadcast terminals.
  • the server can The live broadcast content uploaded by A, B, and C is merged.
  • the user enters the live broadcast room of anchor A, he can watch the merged live broadcast content.
  • Step 202 adjust the actual volume gain of the audio of other anchors in each audio to the actual volume gain of the audio of the current anchor; the other anchors are anchors other than the current anchor among the anchors in the live broadcast room.
  • each anchor can be the current anchor.
  • anchor A, anchor B, and anchor C are co-produced live together, and when anchor A is the current anchor, anchor B and anchor C are other anchors.
  • anchor B is the current anchor, anchor A and anchor C are other anchors.
  • the actual volume gain of the audio of the current anchor can be used as the target gain, and the actual volume gains of the audio of other anchors in the same live broadcast room can be adjusted to the actual volume gain of the audio of the current anchor, so that the volume of other anchors in the live broadcast room The gain is consistent with the actual volume gain of the current host's audio.
  • each anchor can be the current anchor.
  • the audio volume of each other anchor can be adjusted to the actual volume gain of the audio of the anchor A.
  • anchor B is the current anchor
  • the audio volume of each other anchor can be adjusted to the actual volume gain of the audio of anchor B.
  • Step 203 Generate merged audio corresponding to the current anchor according to each audio after the volume gain is adjusted, and the merged audio is used to play the live content of the live broadcast room of the current anchor.
  • the server may also perform merge processing on the audios in the live broadcast room.
  • the server can merge the audio of the current anchor and the audios of other anchors after adjusting the volume gain to obtain the merged audio of the current anchor. For example, when anchor A is the current anchor, the server may generate the first merged audio corresponding to anchor A, and the gains of the audios of other anchors in the first merged audio are consistent with the actual volume gain of the audio of anchor A. When anchor B is used as the current anchor, the server can also generate a second merged audio corresponding to anchor B, and the gains of the audios of other anchors in the second merged audio are consistent with the actual volume gain of the audio of anchor B.
  • the server side can generate corresponding merged audio for each host performing co-shooting.
  • the server when the server pushes the stream to the user terminal watching the live broadcast room of the current anchor, it can push the combined audio corresponding to the current anchor to the user terminal, and the user terminal can play the combined audio when playing the live content. Since each audio gain in the merged audio is the same, when the user terminal plays the live content, the volume of the voices of each anchor is the same. For example, the first merged audio is sent to the user terminal watching the live broadcast of anchor A, and the second merged audio is sent to the user terminal watching the live broadcast of anchor B.
  • Fig. 4 is a schematic diagram of a process of pushing live content according to an exemplary embodiment of the present disclosure.
  • terminal 41 is the terminal of the current anchor
  • each terminal 43 are examples of terminals of other anchors.
  • the current terminal 41 uploads the audio 42 to the server, and the terminal 43 of at least one other host uploads the audio 44 to the server.
  • each terminal also uploads the actual volume gain of the audio when uploading the audio to the server.
  • the server adjusts the actual volume gain of the audio 44 of each other host to the actual volume gain of the audio 42 of the current host, and obtains each audio 45 after the volume gain has been adjusted.
  • the server may perform merge processing on each audio 45 and audio 42 to obtain merged audio 46, and push it to the user terminal 47 for playing the current anchor.
  • the audio merging method provided by the present disclosure includes: obtaining the audio and the actual volume gain of each host's terminal entering the same live broadcast room; wherein, the live broadcast room is a virtual live broadcast room in the network; Volume gain, adjusted to the actual volume gain of the current anchor’s audio; other anchors are anchors other than the current anchor among the anchors in the live broadcast room; according to each audio after the volume gain is adjusted, the merged audio corresponding to the current anchor is generated, and the merged audio is used It is used to play the live content of the current host's live broadcast room.
  • the audio merging method it is possible to adjust the volume gain of the audio of other hosts in the live broadcast room, so that the audio gain of each other host in the live room is consistent with the audio gain of the current host, and then merge the audio processing, and then obtain the merged audio corresponding to the current anchor, the sound volume of each anchor in the merged audio is the same, and the user terminal can play the merged audio when playing the live content of the current anchor, thereby improving the user experience when watching the live broadcast.
  • the audio merging method provided by the present disclosure may further include:
  • the merged audio of the current host is sent to the user terminal playing the live content of the current host.
  • the server may send the combined audio of the current host to the user terminal.
  • the user can operate the user terminal to watch the live broadcast content, specifically, can watch the live broadcast content of any host in the above live broadcast room.
  • the server can send the merged audio corresponding to the anchor to the user terminal. For example, when the user terminal plays the live broadcast content of anchor A, the server can obtain the merged audio of anchor A and send the merged audio of anchor A to the user terminal, so that the user terminal can play the merged audio when playing the live content.
  • the volume gain of the audio of each anchor in the merged audio is the same, the volume of the voices of each anchor heard by the user when watching the live broadcast is also consistent, thereby improving the experience of the user watching the live broadcast.
  • the method provided by the present disclosure also includes:
  • the server can adjust the volume gain of the audio of other anchors according to the actual volume gain of the audio of the current anchor, and can also merge the audio of each other anchor after the volume has been adjusted, so as to obtain the merged audio of other anchors of the current anchor.
  • the server can send the combined audio of other hosts to the current host, so that the current host can hear the return sound with the same gain, which solves the problem of noisy chat rooms caused by the inconsistent sound gain of each input terminal.
  • anchor A, anchor B, and anchor C perform live broadcast in the same virtual webcast room.
  • the audio gains of anchor B and anchor C can be adjusted to the actual volume gain of the audio of anchor A. And merge the audio of anchor B and the audio of anchor C after the volume is adjusted to obtain the merged audio of other anchors corresponding to anchor A, and send the merged audio of other anchors corresponding to anchor A to the terminal of anchor A.
  • the audio gains of anchor A and anchor C can be adjusted to the actual volume gain of the audio of anchor B. And merge the audio of anchor A and the audio of anchor C after the volume is adjusted to obtain the merged audio of other anchors corresponding to anchor B, and send the merged audio of other anchors corresponding to anchor B to the terminal of anchor B.
  • the audio gain of anchor A and the audio of anchor B can be adjusted to the actual volume gain of the audio of anchor C respectively. And merge the audio of anchor A and the audio of anchor B after the volume is adjusted to obtain the merged audio of other anchors corresponding to anchor C, and send the merged audio of other anchors corresponding to anchor C to the terminal of anchor C.
  • Fig. 5 is a schematic flowchart of an audio uploading method according to an exemplary embodiment of the present disclosure.
  • the audio upload method provided by the present disclosure includes:
  • Step 501 acquire audio and at least one type of audio recording information corresponding to the audio during the live broadcast, wherein the audio recording information represents the recording information when the audio is acquired.
  • the method provided in the present disclosure can be executed by an electronic device with computing capability, for example, it can be executed by a live broadcast terminal for live broadcast.
  • the host can use the live broadcast terminal to perform live broadcast, and the live broadcast terminal can collect images and audio.
  • the host can enable the live broadcast function of the live broadcast terminal, and the live broadcast terminal can open the camera and microphone to collect live video and live audio.
  • the live broadcast end can process the collected sound according to the audio recording information when recording the audio, so as to obtain the recorded audio.
  • the user can adjust the live broadcast volume, so that the live broadcast end can process the collected audio according to the live broadcast volume.
  • users can also set live broadcast special effects, such as adjusting the sound to the special effects of children's voices, then the live broadcast end can perform special effects processing on the collected sounds according to the live broadcast special effects, and then obtain recorded audio.
  • live broadcast special effects such as adjusting the sound to the special effects of children's voices
  • the audio recording information has an influence on the actual volume gain of the audio, therefore, it is necessary to acquire the audio recording information of the audio, so as to determine the actual volume gain of the audio according to the audio recording information.
  • Step 502 Determine an actual audio volume gain according to at least one type of audio recording information.
  • the gain information corresponding to each type of audio recording information can be preset in the live broadcast terminal.
  • the audio recording information is the setting gain
  • the setting gain can be used as the gain information.
  • gain adjustment information corresponding to special effect 1 can be obtained, such as gain increase p, or gain decrease q.
  • the live broadcast end can determine the corresponding gain information according to each audio recording information when recording the audio, and then combine the gain information of various audio recording information to determine the actual volume gain of the audio.
  • the live broadcast end may superimpose the gain information corresponding to other audio recording information on the basis of the gain information corresponding to the set gain, so as to obtain the actual volume gain of the audio.
  • Step 503 Send the audio and the actual volume gain of the audio to the server.
  • the actual volume gain of the audio of the anchor is used to adjust the gain of the audio of the anchor, and the adjusted audio is used to generate a merge audio.
  • the live broadcast end can send both the audio and its actual volume gain to the server, so that the server can include multiple hosts in the live broadcast room, and the server can adjust each audio according to the actual volume gain of each audio, so that the same The volume gain of the audio of each host in the live broadcast room is the same, and then the adjusted audio is merged.
  • the server can process the received audio of each host in the same live broadcast room based on any one of the embodiments shown in FIG. 1 , so that the audio volume of each host in the merged audio tends to be consistent.
  • the live broadcast terminal determines the actual volume gain of the currently recorded audio according to at least one type of audio recording information when recording the audio, so that a more accurate actual volume gain of the audio can be obtained, and then the server can obtain the actual volume gain based on the actual volume When the gain is adjusted for each audio, the volume of each audio can be more consistent.
  • Fig. 6 is a schematic flowchart of an audio uploading method according to another exemplary embodiment of the present disclosure.
  • the audio upload method provided by the present disclosure includes:
  • Step 601 acquire audio and at least one type of audio recording information corresponding to the audio during the live broadcast, wherein the audio recording information represents the recording information when the audio is acquired.
  • step 601 is similar to that of step 501, and will not be repeated here.
  • Step 602 Determine the actual gain of the audio according to each preset volume gain corresponding to each audio recording information and each audio recording information; wherein each preset volume gain corresponding to each audio recording information is preset.
  • each volume preset gain corresponding to each audio recording information can also be preset in the live broadcast terminal, for example, the audio preset gain corresponding to each sound effect can be set, and for another example, The audio preset gain corresponding to each audio collection mode can be preset, and the live broadcast end can determine the actual gain of the audio according to each volume preset gain corresponding to each audio recording information.
  • the live broadcast end can record audio according to each audio recording information, and each audio recording information will have an impact on the final volume gain of the audio. Therefore, the live broadcast end can determine the audio volume gain corresponding to each audio recording information.
  • the live broadcast end may determine each volume current gain corresponding to each audio recording information of the audio according to each volume preset gain corresponding to each audio recording information. For example, in the preset correspondence, there is information 1 corresponding to preset gain 1, information 2 corresponding to preset gain 2, and the audio has audio recording information 1, then the live broadcast end can determine that the current gain of a volume of the audio is gain 1 .
  • the live broadcast end can determine the actual volume gain of the audio according to the current gain of each volume of the audio.
  • the live broadcast end may superimpose the current gains of each volume of an audio, so as to obtain the actual volume gain of the audio.
  • the audio recording information includes any of the following:
  • volume preset gain audio collection method, sound processing method, sound special effect information
  • Volume preset gain refers to the volume gain set by the anchor when live broadcasting.
  • the audio collection method refers to the way the live broadcast end collects audio, such as through an external microphone, or through the built-in microphone of the live broadcast end.
  • the sound processing method refers to the method of performing preliminary processing on the collected sound, such as echo cancellation processing, or denoising processing, etc.
  • the special effect information of the sound refers to the special effect method selected by the anchor during the live broadcast.
  • Step 603 Perform any of the following processing methods on the collected audio: echo cancellation processing, reverberation processing, and sound equalization processing.
  • processing other than echo cancellation processing, reverberation processing, and sound equalization processing can also be performed on the collected audio, which can be set according to requirements.
  • the live broadcast end can perform echo cancellation processing, and/or reverberation processing, and/or sound equalization processing on the collected audio, so that the processed audio meets the live broadcast voice quality requirements.
  • Step 604 sending the processed audio and the actual volume gain of the audio to the server.
  • the live broadcast end can send the processed audio and the actual volume gain of the audio to the server.
  • the actual volume gain of each audio of the anchor is used to adjust each audio of the anchor.
  • the resulting audio is used to generate merged audio.
  • Step 605 Receive the combined audio of other anchors sent by the server.
  • the combined audio of other anchors includes the audio of each other anchor after the volume gain has been adjusted, and the adjusted volume gain of the audio of each other anchor is the actual volume gain.
  • the server can adjust the volume gain of the audio of other anchors to the actual volume gain of the audio collected by the current live broadcast end, and adjust the volume gain of each other anchor. Audio is merged to get other merged audio.
  • the server can send the other merged audio to the current live broadcast end, so that the current anchor can hear the return sound with the same gain, which solves the problem of noisy chat room caused by the inconsistent sound gain of each input end.
  • Fig. 7 is a schematic structural diagram of an audio combining device according to an exemplary embodiment of the present disclosure.
  • the audio converging device 700 includes:
  • the obtaining unit 710 is used to obtain the audio and the actual volume gain of the terminals of the anchors entering the same live broadcast room; wherein, the live broadcast room is a virtual live broadcast room in the network;
  • the adjustment unit 720 is used to adjust the actual volume gain of the audio of other anchors in each audio to the actual volume gain of the audio of the current anchor; the other anchors are anchors other than the current anchor among the anchors in the live broadcast room;
  • the merging unit 730 is configured to generate merging audio corresponding to the current anchor according to each audio after the volume gain is adjusted, and the merging audio is used to play the live content of the live room corresponding to the current anchor.
  • the audio converging device provided in the present disclosure is similar to the embodiment shown in FIG. 2 , and will not be repeated here.
  • Fig. 8 is a schematic structural diagram of an audio combining device according to another exemplary embodiment of the present disclosure.
  • the audio converging device 800 provided by the present disclosure, on the basis of the above-mentioned embodiments, further includes:
  • Push unit 740 used for:
  • the merging unit 730 is also used to perform merging processing on the audio of other anchors whose volume gain has been adjusted to obtain the merged audio of other anchors;
  • the device also includes an echo unit 750, configured to send the combined audio of other anchors to the terminal of the current anchor.
  • Fig. 9 is a schematic structural diagram of an audio uploading device according to an exemplary embodiment of the present disclosure.
  • the audio uploading device 900 includes:
  • An acquisition unit 910 configured to acquire audio and at least one type of audio recording information corresponding to the audio during the live broadcast, wherein the audio recording information represents the recording information when the audio is acquired;
  • a gain determining unit 920 configured to determine the actual volume gain of the audio according to the at least one audio recording information
  • the sending unit 930 is configured to send the audio and the actual volume gain of the audio to the server.
  • the actual volume gain of the anchor audio is used to adjust the gain of the anchor audio.
  • the adjusted Audio is used to generate merged audio.
  • the audio converging device provided in the present disclosure is similar to the embodiment shown in FIG. 5 , and will not be repeated here.
  • Fig. 10 is a schematic structural diagram of an audio uploading device according to another exemplary embodiment of the present disclosure.
  • the audio uploading device 1000 provided by the present disclosure includes:
  • each volume preset gain corresponding to each audio recording information is preset
  • the gain determining unit 920 is specifically configured to:
  • the actual volume gain of the audio is determined according to each preset volume gain corresponding to each audio recording information and each audio recording information.
  • the gain determining unit 920 includes:
  • Each gain determination module 921 is configured to determine each volume current gain corresponding to each audio recording information of the audio according to each volume preset gain corresponding to each audio recording information;
  • the actual gain determining module 922 is configured to determine the actual volume gain of the audio according to the current gain of each volume of the audio.
  • the device also includes a receiving unit 940, configured to receive the combined audio of other anchors sent by the server.
  • the combined audio of other anchors includes the audio of each other anchor after adjusting the volume gain, and the adjusted volume gain of each other anchor's audio. is the actual volume gain.
  • the audio recording information includes any of the following:
  • Volume preset gain audio collection method, sound processing method, sound special effect information.
  • the device further includes a processing unit 940, configured to:
  • the sending unit 930 is further configured to: send the processed audio to the server.
  • An embodiment of the present disclosure further provides a computer program product, including a computer program, and when the computer program is executed by a processor, any one of the above-mentioned audio combining methods or audio uploading methods is implemented.
  • the device provided in this embodiment can be used to implement the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, so this embodiment will not repeat them here.
  • the electronic device 1100 may be a terminal device or a server.
  • the terminal equipment may include but not limited to mobile phones, notebook computers, digital broadcast receivers, personal digital assistants (Personal Digital Assistant, PDA for short), tablet computers (Portable Android Device, PAD for short), portable multimedia players (Portable Media Player, referred to as PMP), mobile terminals such as vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs, desktop computers, etc.
  • PDA Personal Digital Assistant
  • PMP portable multimedia players
  • mobile terminals such as vehicle-mounted terminals (such as vehicle-mounted navigation terminals)
  • fixed terminals such as digital TVs, desktop computers, etc.
  • the electronic device shown in FIG. 11 is only an example, and should not limit the functions and application scope of the embodiments of the present disclosure.
  • an electronic device 1100 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 1101, which may be stored in a program in a read-only memory (Read Only Memory, ROM for short) 1102 or from a storage device. 1108 is loaded into the program in the random access memory (Random Access Memory, referred to as RAM) 1103 to execute various appropriate actions and processes. In the RAM 1103, various programs and data necessary for the operation of the electronic device 1100 are also stored.
  • the processing device 1101, the ROM 1102, and the RAM 1103 are connected to each other through a bus 1104.
  • An input/output (I/O) interface 1105 is also connected to the bus 1104 .
  • an input device 1106 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; ), a speaker, a vibrator, etc.
  • a storage device 1108 including, for example, a magnetic tape, a hard disk, etc.
  • the communication means 1109 may allow the electronic device 1100 to perform wireless or wired communication with other devices to exchange data. While FIG. 11 shows electronic device 1100 having various means, it is to be understood that implementing or having all of the means shown is not a requirement. More or fewer means may alternatively be implemented or provided.
  • embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, where the computer program includes program codes for executing the methods shown in the flowcharts.
  • the computer program may be downloaded and installed from a network via communication means 1109, or from storage means 1108, or from ROM 1102.
  • the processing device 1101 the above-mentioned functions defined in the methods of the embodiments of the present disclosure are executed.
  • the above-mentioned computer-readable medium in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two.
  • a computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.
  • a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
  • a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device .
  • Program code embodied on a computer readable medium may be transmitted by any appropriate medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
  • the above-mentioned computer-readable medium may be included in the above-mentioned electronic device, or may exist independently without being incorporated into the electronic device.
  • the above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device is made to execute the methods shown in the above-mentioned embodiments.
  • Computer program code for carrying out the operations of the present disclosure can be written in one or more programming languages, or combinations thereof, including object-oriented programming languages—such as Java, Smalltalk, C++, and conventional Procedural Programming Language - such as "C" or a similar programming language.
  • the program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
  • the remote computer can be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it can be connected to an external A computer (connected via the Internet, eg, using an Internet service provider).
  • LAN Local Area Network
  • WAN Wide Area Network
  • each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions.
  • the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.
  • the units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. Wherein, the name of the unit does not constitute a limitation of the unit itself under certain circumstances, for example, the first obtaining unit may also be described as "a unit for obtaining at least two Internet Protocol addresses".
  • FPGAs Field Programmable Gate Arrays
  • ASICs Application Specific Integrated Circuits
  • ASSPs Application Specific Standard Products
  • SOCs System on Chips
  • CPLD Complex Programmable Logical device
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device.
  • a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • a machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing.
  • machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read only memory
  • EPROM or flash memory erasable programmable read only memory
  • CD-ROM compact disk read only memory
  • magnetic storage or any suitable combination of the foregoing.
  • an audio merging method including:
  • the live broadcast room is a virtual live broadcast room in the network
  • the actual volume gain of the audio of other anchors in each audio is adjusted to the actual volume gain of the audio of the current anchor;
  • the other anchors are anchors other than the current anchor among the anchors in the live broadcast room;
  • the merged audio corresponding to the current anchor is generated according to each audio after the volume gain is adjusted, and the merged audio is used to play the live content of the live room of the current anchor.
  • an audio upload method including:
  • the audio recording information represents the recording information when the audio is obtained
  • the actual volume gain of the anchor audio is used to adjust the gain of the anchor audio, and the adjusted audio is used to generate merged audio .
  • each volume preset gain corresponding to each audio recording information is preset
  • the determining the actual volume gain of the audio according to the at least one audio recording information includes:
  • the actual volume gain of the audio is determined according to each preset volume gain corresponding to each audio recording information and each audio recording information.
  • the determining the actual volume gain of the audio according to each volume preset gain corresponding to each audio recording information and each audio recording information includes:
  • the actual volume gain of the audio is determined according to the current volume gains of the audio.
  • the combined audio of other anchors sent by the server is received, the merged audio of other anchors includes the audio of each other anchor whose volume gain has been adjusted, and the adjusted volume gain of the audio of each other anchor is the actual volume gain.
  • the audio recording information includes any of the following:
  • Volume preset gain audio collection method, sound processing method, sound special effect information.
  • Sending the audio to the server includes: sending the processed audio to the server.
  • an audio converging device including:
  • the acquisition unit is used to acquire the audio and the actual volume gain of the terminals of the anchors entering the same live broadcast room; wherein, the live broadcast room is a virtual live broadcast room in the network;
  • the adjustment unit is used to adjust the actual volume gain of the audio of other anchors in each audio to the actual volume gain of the audio of the current anchor;
  • the other anchors are anchors other than the current anchor among the anchors in the live broadcast room;
  • the merging unit is configured to generate merging audio corresponding to the current anchor according to each audio after the volume gain is adjusted, and the merging audio is used to play the live content of the live broadcast room of the current anchor.
  • the device further includes a push unit, configured to:
  • the merging unit is further configured to perform merging processing on the audio of other anchors whose volume gain has been adjusted, to obtain the merged audio of other anchors;
  • the device also includes an echo unit, configured to send the combined audio of other anchors to the terminal of the current anchor.
  • an audio uploading device including:
  • An acquisition unit configured to acquire audio and at least one audio recording information corresponding to the audio during the live broadcast, wherein the audio recording information represents the recording information when the audio is acquired;
  • a gain determination unit configured to determine the actual volume gain of the audio according to the at least one audio recording information
  • the sending unit is configured to send the audio and the actual volume gain of the audio to the server.
  • the actual volume gain of the anchor audio is used to adjust the gain of the anchor audio.
  • the adjusted audio Used to generate merged audio.
  • each volume preset gain corresponding to each audio recording information is preset
  • the gain determining unit is specifically used for:
  • the actual volume gain of the audio is determined according to each preset volume gain corresponding to each audio recording information and each audio recording information.
  • the gain determination unit includes:
  • Each gain determination module is configured to determine each volume current gain corresponding to each audio recording information of the audio according to each volume preset gain corresponding to each audio recording information;
  • the actual gain determining module is configured to determine the actual volume gain of the audio according to the current gain of each volume of the audio.
  • the device further includes a receiving unit, configured to receive the combined audio of other anchors sent by the server.
  • the adjusted volume gain of the host's audio is the actual volume gain.
  • the audio recording information includes any of the following:
  • Volume preset gain audio collection method, sound processing method, sound special effect information.
  • the device further includes a processing unit, configured to:
  • the sending unit is further configured to: send the processed audio to the server.
  • an electronic device including: at least one processor and a memory;
  • the memory stores computer-executable instructions
  • the at least one processor executes the computer-executed instructions stored in the memory, so that the at least one processor executes the audio combining method described in the above first aspect and various possible designs of the first aspect, or executes the above second aspect And the audio upload method described in various possible designs of the second aspect.
  • a computer-readable storage medium stores computer-executable instructions, and when a processor executes the computer-executable instructions, Realize the audio merging method described in the above first aspect and various possible designs of the first aspect, or execute the audio uploading method described in the above second aspect and various possible designs of the second aspect.
  • an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the audio combining method described in the above first aspect and various possible designs of the first aspect, Or execute the audio uploading method described in the above second aspect and various possible designs of the second aspect.
  • the embodiments of the present disclosure provide a computer program, which implements the audio combining method described in the above first aspect and various possible designs of the first aspect when the computer program is executed by a processor, or implements the above second aspect

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • General Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Circuit For Audible Band Transducer (AREA)

Abstract

本公开提供一种音频合流方法、音频上传方法、设备及程序产品，涉及音频处理技术，本公开提供的方案能够对直播间内各其他主播的音频的音量增益进行调节，使直播间中各其他主播的音频增益与当前主播的音频的增益一致，进而得到与当前主播对应的合流音频，合流的音频中各主播的声音大小相同，使用合流音频播放当前主播的直播内容时，能够提高观看直播时的用户体验。

Description

音频合流方法、音频上传方法、设备及程序产品
相关申请交叉引用
本申请要求于2021年7月14日提交中国专利局、申请号为202110797609.6、发明名称为“音频合流方法、音频上传方法、设备及程序产品”的中国专利申请的优先权，其全部内容通过引用并入本文。
技术领域
本公开实施例涉及音频处理技术，尤其涉及一种音频合流方法、音频上传方法、设备及程序产品。
背景技术
随着网络技术的发展，网络直播越来越流行，而且直播内容也越来越丰富。比如，一个主播可以进入其他主播的房间，观众可以通过终端观看多个主播的直播画面，从而增加直播内容的趣味性。
其中，多个主播在同一个网络直播间中直播时，各主播的终端会进行音频和视频的录制，并将录制的音频发送到服务器端，由服务器端对各主播的音频进行合流，并将合流后的音频下发到观看直播的终端。
但是，在这种方案中，由于不同主播的终端录制音频时的参数不同，会导致各录制音频的音量增益也不同，进而向观看直播的终端发送的合流后的音频中，不同主播的声音大小差异较大，给用户带来较差的用户体验。
发明内容
本公开实施例提供一种音频合流方法、音频上传方法、设备及程序产品，以解决现有技术中多个主播在一个直播间中直播，各主播的音频增益不同的问题。
第一方面，本公开实施例提供一种音频合流方法，包括：
获取进入同一直播间的各主播的终端的音频及其实际音量增益；其中，所述直播间为网络中的虚拟直播间；
将各音频中其他主播的音频的实际音量增益，调节为当前主播的音频的实际音量增益；所述其他主播为所述直播间的各主播中当前主播以外的主播；
根据调节了音量增益后的各音频生成与所述当前主播对应的合流音频，所述合流音频用于播放所述当前主播的直播间的直播内容。
第二方面，本公开实施例提供一种音频上传方法，包括：
在直播过程中获取音频、以及与所述音频对应的至少一种音频录制信息，其中，所述音频录制信息表征获取所述音频时的录制信息；
根据所述至少一种音频录制信息确定所述音频的实际音量增益；
向服务器发送所述音频以及所述音频的实际音量增益，在直播间中包括多个主播时，主播音频的实际音量增益用于对主播音频的增益进行调节，调节后的音频用于生成合流音频。
第三方面，本公开实施例提供一种音频合流装置，包括：
获取单元，用于获取进入同一直播间的各主播的终端的音频及其实际音量增益；其中，所述直播间为网络中的虚拟直播间；
调节单元，用于将各音频中其他主播的音频的实际音量增益，调节为当前主播的音频的实际音量增益；所述其他主播为所述直播间的各主播中当前主播以外的主播；
合流单元，用于根据调节了音量增益后的各音频生成与所述当前主播对应的的合流音频，所述合流音频用于播放所述当前主播的直播间的直播内容。
第四方面，本公开实施例提供一种音频上传装置，包括：
获取单元，用于在直播过程中获取音频、以及与所述音频对应的至少一种音频录制信息，其中，所述音频录制信息表征获取所述音频时的录制信息；
增益确定单元，用于根据所述至少一种音频录制信息确定所述音频的实际音量增益；
发送单元，用于向服务器发送所述音频以及所述音频的实际音量增益，在直播间中包括多个主播时，主播音频的实际音量增益用于对主播音频的增益进行调节，调节后的音频用于生成合流音频。
第五方面，本公开实施例提供一种电子设备，包括：至少一个处理器和存储器；
所述存储器存储计算机执行指令；
所述至少一个处理器执行所述存储器存储的计算机执行指令，使得所述至少一个处理器执行如上第一方面以及第一方面各种可能的设计所述的音频合流方法，或者执行如上第二方面以及第二方面各种可能的设计所述的音频上传方法。
第六方面，本公开实施例提供一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现如上第一方面以及第一方面各种可能的设计所述的音频合流方法，或者执行如上第二方面以及第二方面各种可能的设计所述的音频上传方法。
第七方面，本公开实施例提供一种计算机程序产品，包括计算机程序，所述计算机程序在被处理器执行时实现如上第一方面以及第一方面各种可能的设计所述的音频合流方法，或者执行如上第二方面以及第二方面各种可能的设计所述的音频上传方法。
第八方面，本公开实施例提供一种计算机程序，所述计算机程序在被处理器执行时实现如上第一方面以及第一方面各种可能的设计所述的音频合流方法，或者执行如上第二方面以及第二方面各种可能的设计所述的音频上传方法。
本实施例提供的音频合流方法、音频上传方法、设备及程序产品，能够对直播间内各其他主播的音频的音量增益进行调节，使直播间中各其他主播的音频增益与当前主播的音频的增益一致，进而得到与当前主播对应的合流音频，合流的音频中各主播的声音大小相同，使用合流音频播放当前主播的直播内容时，能够提高观看直播时的用户体验。
附图说明
为了更清楚地说明本公开实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作一简单地介绍，显而易见地，下面描述中的附图是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为一示例性实施例示出的合拍流程示意图；
图2为本公开一示例性实施例示出的音频合流方法的流程示意图；
图3为本公开一示例性实施例示出的直播系统架构图；
图4为本公开一示例性实施例示出的直播内容推送流程示意图；
图5为本公开一示例性实施例示出的音频上传方法的流程示意图；
图6为本公开另一示例性实施例示出的音频上传方法的流程示意图；
图7为本公开一示例性实施例示出的音频合流装置的结构示意图；
图8为本公开另一示例性实施例示出的音频合流装置的结构示意图；
图9为本公开一示例性实施例示出的音频上传装置的结构示意图；
图10为本公开另一示例性实施例示出的音频上传装置的结构示意图；
图11为本公开一示例性实施例示出的电子设备的结构示意图。
具体实施方式
为使本公开实施例的目的、技术方案和优点更加清楚，下面将结合本公开实施例中的附图，对本公开实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例是本公开一部分实施例，而不是全部的实施例。基于本公开中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本公开保护的范围。
目前，在直播领域中包括合拍场景，比如，至少一个第二主播可以进入第一主播的直播间，观看第一主播的用户终端显示的直播画面中，包括上述第一主播和第二主播的直播画面，用户终端还能够播放第一主播和第二主播的直播时的音频。
图1为一示例性实施例示出的合拍流程示意图。
如图1所示，第一主播11使用第一终端12进行直播，可以由第一终端12采集第一主播11进行直播时的画面和音频。每个第二主播13使用每一第二终端14进行直播，可以由每一第二终端14采集每一第二主播13进行直播时的画面和音频。
第一终端12和每个第二终端14可以将采集的画面和音频发送到云端15，由云端15对进入同一直播间的各主播的直播画面进行处理，生成合拍画面，还可以对进入同一直播间的各主播的直播音频进行合流处理，生成合流后的音频。
云端15可以将处理后的合拍画面以及合拍音频推流到观看直播的用户16的终端侧，从而使用户16能够通过终端观看合拍画面及音频。
但是，由于第一终端12和各第二终端14可能是不同类型或型号的直播设备，因此，使用各终端进行直播时采用的直播参数可能也不同，那么进行直播的各终端采集的各音频的音量增益也会不同，最终合流的音频中，各主播的声音大小也会不同，造成观看直播的用户终端播放直播音频时，有的主播声音大，有的主播声音小，造成较差的用户体验。
为了解决上述技术问题，本公开提供的方案中，对直播间中各其他主播的音频增益进行调整，使得其他主播的音频的音量增益与当前主播的音频的增益相同，使直播中的多路音频增益相同，再对各路音频进行合流处理得到合流音频，在用户终端播放当前主播的直播内容时，服务器可以向该用户终端推送该当前主播的合流音频，该合流音频中包括的各主播的音频的音量增益相同，进而提高用户观看直播的体验。
图2为本公开一示例性实施例示出的音频合流方法的流程示意图。
如图2所示，本公开提供的音频合流方法，包括：
步骤201，获取进入同一直播间的各主播的终端的音频及其实际音量增益；其中，直播间为网络中的虚拟直播间。
其中，本公开提供的方法可以由具备计算能力的电子设备执行，比如可以由直播的后台服务器执行。
图3为本公开一示例性实施例示出的直播系统架构图。
如图3所示，在直播系统中可以包括直播端31，直播服务器32以及观看直播的用户端33。可以使用直播端31录制直播内容，并通过直播端31将直播内容上传到直播服务器32，再由直播服务器32向用户端33推流。
具体的，直播端31向直播服务器发送的直播内容中包括录制的音频及其音量增益。音量增益用于表征该音频的音量的大小。
进一步的，主播使用直播端31进行直播时，可以设置音量大小，直播端可以获取该设置的音量增益，并根据设置的音量增益确定音频的实际音量增益。
实际应用时，用户在直播时，还可以选择直播的特效等信息，直播端还可以获取直播时的特效信息，并根据该特效信息确定音频的实际音量增益。
其中，直播端可以获取直播时用于采集音频的多个信息，从而根据这些信息确定直播时采集的音频的实际音量增益。
具体的，直播端可以将音频及其实际音量增益发送给服务器，当多个主播进入同一直播间时，服务器可以获取这些主播的终端上传的各音频，以及各音频的实际音量增益。
进一步的，直播间为网络中的虚拟直播间，比如，存在主播A、B、C，主播B和主播C可以通过操作各自的直播端，进入主播A的直播间，此时，服务器可以对主播A、B、C各自上传的直播内容进行合流处理，当用户进入主播A的直播间时，能够观看到合流后的直播内容。
步骤202，将各音频中其他主播的音频的实际音量增益，调节为当前主播的音频的实际音量增益；其他主播为直播间的各主播中当前主播以外的主播。
实际应用时，当多个主播进行合拍直播时，每一主播都可以作为当前主播。比如，主播A、主播B、主播C在一起进行合拍直播，主播A为当前主播时，主播B、主播C则为其他主播。主播B为当前主播时，主播A、主播C则为其他主播。
其中，由于同一直播间内各主播的终端录制的音频的实际音量增益可能不同，会导致最后合流的直播内容中有的主播声音大，有的主播声音小，因此，本公开提供的方案中，可以将当前主播的音频的实际音量增益作为目标增益，将同一直播间内其他主播音频的实际音量增益均调整为该当前主播的音频的实际音量增益，从而使该直播间内各其他主播的音量增益与当前主播的音频的实际音量增益保持一致。
具体的，每一主播都可以作为当前主播，比如，当主播A为当前主播时，可以将各其他主播的音频的音量调整为主播A的音频的实际音量增益。当主播B为当前主播时，可以将各其他主播的音频的音量调整为主播B的音频的实际音量增益。
步骤203，根据调节了音量增益后的各音频生成与当前主播对应的合流音频，合流音频用于播放当前主播的直播间的直播内容。
进一步的，服务器调节完该直播间中各其他主播的音频的音量增益之后，服务器还可以对该直播间中的各音频进行合流处理。
实际应用时，服务器可以对该当前主播的音频，以及调节了音量增益后的各其他主播的音频进行合流，得到当前主播的合流音频。比如，主播A为当前主播时，服务器可以生成主播A对应的第一合流音频，该第一合流音频中各其他主播的音频的增益与主播A的音频的实际音量增益一致。将主播B作为当前主播时，服务器还可以生成主播B对应的第二合流音频，该第二合流音频中各其他主播的音频的增益与主播B的音频的实际音量增益一致。
基于本公开提供的方法，在服务器侧针对进行合拍的每一主播，都可以生成对应的合流音频。其中，服务器向观看当前主播的直播间的用户终端推流时，可以将与当前主播对应的合流音频推送给用户终端，用户终端在播放直播内容时可以播放该合流音频。由于合流音频中的各音频增益相同，因此，用户终端播放直播内容时，各主播的声音大小是相同的。比如，向观看主播A的直播的用户终端发送第一合流音频，向观看主播B的直播的用户终端发送第二合流音频。
图4为本公开一示例性实施例示出的直播内容推送流程示意图。
如图4所示，多个主播在同一虚拟的网络直播间内进行直播，各主播的终端可以录制音频，并向服务器发送录制的音频，本实施例以终端41为当前主播的终端，各终端43均为其他主播的终端进行举例说明。
当前终端41向服务器上传音频42，至少一个其他主播的终端43向服务器上传的音频44。
其中，各终端向服务器上传音频时还上传音频的实际音量增益。服务器将各其他主播的音频44的实际音量增益均调节为当前主播的音频42的实际音量增益，得到调节了音量增益后的各音频45。
具体的，服务器可以对各音频45与音频42进行合流处理得到合流音频46，并推送到用于播放当前主播的用户终端47。
本公开提供的音频合流方法，包括：获取进入同一直播间的各主播的终端的音频及其实际音量增益；其中，直播间为网络中的虚拟直播间；将各音频中其他主播的音频的实际音量增益，调节为当前主播的音频的实际音量增益；其他主播为直播间的各主播中当前主播以外的主播；根据调节了音量增益后的各音频生成与当前主播对应的合流音频，合流音频用于播放当前主播的直播间的直播内容。本公开提供的音频合流方法中，能够对直播间内各其他主播的音频的音量增益进行调节，使直播间中各其他主播的音频增益与当前主播的音频的增益一致，再对各音频进行合流处理，进而得到与当前主播对应的合流音频，合流的音频中各主播的声音大小相同，用户终端播放当前主播的直播内容时能够播放该合流音频，从而能够提高观看直播时的用户体验。
在上述实施例基础上，本公开提供的音频合流方法，还可以包括：
向播放当前主播的直播内容的用户终端发送当前主播的合流音频。
若用户终端播放当前主播的直播内容，则服务器可以将该当前主播的合流音频发送给该用户终端。
其中，用户可以操作用户终端观看直播内容，具体可以观看上述直播间中任一主播的直播内容。
用户终端播放哪个主播的直播内容，服务器就可以向该用户终端发送哪个主播对应的合流音频。比如，用户终端播放主播A的直播内容，则服务器可以获取主播A的合流音频，并将主播A的合流音频发送给该用户终端，使得用户终端播放直播内容时可以播放该合流音频。
具体的，由于合流音频中各主播的音频的音量增益大小相同，因此，用户观看直播时听到的各主播的声音大小也一致，进而能够提高用户观看直播的体验。
在一种可选的实施方式中，本公开提供的方法还包括：
对调节了音量增益后的其他主播的音频进行合流处理，得到其他主播合流音频；
将其他主播合流音频发送给所述当前主播的终端。
其中，服务器可以根据当前主播的音频的实际音量增益，调节其他主播的音频的音量增益，还可以对各其他主播调节了音量后的音频进行合流处理，以得到当前主播的其他主播合流音频。
具体的，服务器可以向当前主播发送其他主播合流音频，从而使当前主播可以听见增益大小一致的返声，解决了因为各输入端声音增益大小不一致导致的聊天室嘈杂的问题。
以一个详细的实施例说明本方案。比如主播A、主播B、主播C在同一个虚拟的网络直播间中进行直播。
将主播A作为当前主播时，可以分别将主播B的音频、主播C的音频的增益调节为主播A的音频的实际音量增益。并对调节了音量后的主播B的音频、主播C的音频进行合流，得到与主播A对应的其他主播合流音频，并向主播A的终端发送与主播A对应的其他主播合流音频。
将主播B作为当前主播时，可以分别将主播A的音频、主播C的音频的增益调节为主播B的音频的实际音量增益。并对调节了音量后的主播A的音频、主播C的音频进行合流，得到与主播B对应的其他主播合流音频，并向主播B的终端发送与主播B对应的其他主播合流音频。
将主播C作为当前主播时，可以分别将主播A的音频、主播B的音频的增益调节为主播C的音频的实际音量增益。并对调节了音量后的主播A的音频、主播B的音频进行合流，得到与主播C对应的其他主播合流音频，并向主播C的终端发送与主播C对应的其他主播合流音频。
图5为本公开一示例性实施例示出的音频上传方法的流程示意图。
如图5所示，本公开提供的音频上传方法，包括：
步骤501，在直播过程中获取音频、以及与音频对应的至少一种音频录制信息，其中，音频录制信息表征获取音频时的录制信息。
其中，本公开提供的方法可以由具备计算能力的电子设备执行，比如可以由用于直播的直播端执行。
具体的，主播可以利用直播端进行直播，直播端可以采集画面，还可以采集音频。比如，主播可以开启直播端的直播功能，直播端可以开启摄像头和麦克风，从而采集直播画面和直播音频。
进一步的，直播端可以根据录制音频时的音频录制信息对采集的声音进行处理，以得到录制音频。比如，用户可以调节直播音量，进而使直播端可以根据该直播音量对采集的音频进行处理。
实际应用时，用户还可以设置直播特效，比如将声音调整为小孩子声音的特效，则直播端可以根据直播特效对采集的声音进行特效处理，进而得到录制音频。
其中，音频录制信息对音频实际的音量增益有影响，因此，需要获取音频的音频录制信息，从而根据该音频录制信息确定音频的实际音量增益。
步骤502，根据至少一种音频录制信息确定音频的实际音量增益。
具体的，直播端中可以预先设置与每种音频录制信息对应的增益信息，比如，若音频录制信息为设置增益，则可以将设置增益作为增益信息，再比如，若音频录制信息中包括特效1的信息，则可以获取与特效1对应的增益调整信息，比如增益增加p，或者增益减少q。
进一步的，直播端可以根据录制音频时每一音频录制信息确定对应的增益信息，再结合多种音频录制信息的增益信息，确定该音频的实际音量增益。一种可选的实施方式中，直播端可以在设置增益对应的增益信息的基础上，叠加其他音频录制信息对应的增益信息，进而得到音频的实际音量增益。
步骤503，向服务器发送音频以及音频的实际音量增益，在直播间中包括多个主播时，主播的音频的实际音量增益用于对主播的音频的增益进行调节，调节后的音频用于生成合流音频。
实际应用时，直播端可以将音频及其实际音量增益均发送给服务器，从而使服务器可以在直播间中包括多个主播时，服务器可以根据各音频的实际音量增益对各音频进行调整，使得同一直播间内各主播的音频的音量增益一致，再对调节后的音频进行合流处理。
其中，服务器可以基于图1所示的任一种实施例对接收的处于同一直播间中各主播的音频进行处理，从而使合流后的音频中，各主播的音频音量趋近于一致。
具体的，本公开提供的方法中，直播端根据录制音频时至少一种音频录制信息确定当前录制的音频的实际音量增益，从而能够得到音频更准确的实际音量增益，进而使服务器根据该实际音量增益对各音频进行调整时，能够使各音频的音量更加趋近于一致。
图6为本公开另一示例性实施例示出的音频上传方法的流程示意图。
如图6所示，本公开提供的音频上传方法，包括：
步骤601，在直播过程中获取音频、以及与音频对应的至少一种音频录制信息，其中，音频录制信息表征获取音频时的录制信息。
步骤601与步骤501的实现方式类似，不再赘述。
步骤602，根据每一音频录制信息对应的每一音量预设增益、每一音频录制信息，确定音频的实际增益；其中，预先设置有与每一音频录制信息对应的每一音量预设增益。
其中，本公开提供的方法中，在直播端中还可以预先设置与每一音频录制信息对应的每一音量预设增益，比如可以设置与每一声音特效对应的音频预设增益，再比如，可以预先设 置每一音频采集方式对应的音频预设增益，直播端可以根据每一音频录制信息对应的每一音量预设增益，确定音频的实际增益。
具体的，直播端可以根据各音频录制信息录制音频，每一音频录制信息都会对音频最终的音量增益有影响，因此，直播端可以根据每一音频录制信息对应的音量预设增益，确定该音频最终的实际音量增益
进一步的，直播端可以根据每一音频录制信息对应的每一音量预设增益，确定所述音频的每一音频录制信息对应的每一音量当前增益。比如，在预设的对应关系中，存在信息1对应预设增益1，信息2对应预设增益2，而音频具有音频录制信息1，则直播端可以确定该音频的一个音量当前增益为增益1。
实际应用时，直播端可以根据音频的各音量当前增益，确定音频的实际音量增益。一种可选的实施方式中，直播端可以对一个音频的各音量当前增益进行叠加，从而得到该音频的实际音量增益。
其中，音频录制信息包括以下的任一种：
音量预设增益、音频采集方式、声音处理方式、声音的特效信息
音量预设增益是指主播直播时，设置的音量增益。音频采集方式是指直播端采集音频的方式，比如通过外部的麦克风采集，再比如通过直播端自带的麦克风采集音频。声音处理方式是指对采集的声音进行初步处理的方式，比如回声消除的处理，再比如去噪处理等。声音的特效信息是指直播时，主播选择的特效方式。
步骤603，对采集的音频进行以下任一种处理方式：回声消除处理、混响处理、声音均衡处理。
可选的，还可以对采集的音频进行除回声消除处理、混响处理、声音均衡处理外的处理，具体可以根据需求进行设置。
具体的，直播端可以对采集的音频进行回声消除处理，和/或者混响处理，和/或者声音均衡处理，从而使处理后的音频符合直播语音质量的要求。
步骤604，向服务器发送处理后的音频，以及音频的实际音量增益。
进一步的，直播端可以向服务器发送处理后的音频，及音频的实际音量增益，在直播间中包括多个主播时，主播的各音频的实际音量增益用于对主播的各音频进行调节，调节后的音频用于生成合流音频。
步骤605，接收服务器发送的其他主播合流音频，其他主播合流音频中包括各其他主播的调节了音量增益后的音频，各其他主播的音频调节后的音量增益为实际音量增益。
实际应用时，当使用直播端与其他主播一起合拍时，服务器可以将其他各主播的音频的音量增益调节为当前直播端采集的音频的实际音量增益，并对调节了音量增益的各其他主播的音频进行合流，得到其他合流音频。
其中，服务器可以向当前直播端发送该其他合流音频，从而使当前主播可以听见增益大小一致的返声，解决了因为各输入端声音增益大小不一致导致的聊天室嘈杂问题。
图7为本公开一示例性实施例示出的音频合流装置的结构示意图。
如图7所示，本公开提供的音频合流装置700，包括：
获取单元710，用于获取进入同一直播间的各主播的终端的音频及其实际音量增益；其中，所述直播间为网络中的虚拟直播间；
调节单元720，用于将各音频中其他主播的音频的实际音量增益，调节为当前主播的音频的实际音量增益；所述其他主播为所述直播间的各主播中当前主播以外的主播；
合流单元730，用于根据调节了音量增益后的各音频生成与所述当前主播对应的合流音频，所述合流音频用于播放所述当前主播对应直播间的直播内容。
本公开提供的音频合流装置与图2所示实施例类似，不再赘述。
图8为本公开另一示例性实施例示出的音频合流装置的结构示意图。
如图8所示，本公开提供的音频合流装置800中，在上述实施例基础上，还包括：
推送单元740，用于：
向播放所述当前主播的直播内容的用户终端发送所述当前主播的合流音频。
所述合流单元730还用于对调节了音量增益后的其他主播的音频进行合流处理，得到其他主播合流音频；
所述装置还包括返声单元750，用于将其他主播合流音频发送给所述当前主播的终端。
图9为本公开一示例性实施例示出的音频上传装置的结构示意图。
如图9所示，本公开提供的音频上传装置900，包括：
获取单元910，用于在直播过程中获取音频、以及与所述音频对应的至少一种音频录制信息，其中，所述音频录制信息表征获取所述音频时的录制信息；
增益确定单元920，用于根据所述至少一种音频录制信息确定所述音频的实际音量增益；
发送单元930，用于向服务器发送所述音频以及所述音频的实际音量增益，在直播间中包括多个主播时，主播音频的实际音量增益用于对主播音频的增益进行调节，调节后的音频用于生成合流音频。
本公开提供的音频合流装置与图5所示实施例类似，不再赘述。
图10为本公开另一示例性实施例示出的音频上传装置的结构示意图。
如图所示，本公开提供的音频上传装置1000，包括：
一种可选的实施方式中，预先设置有与每一音频录制信息对应的每一音量预设增益；
所述增益确定单元920具体用于：
根据每一音频录制信息对应的每一音量预设增益、每一音频录制信息，确定所述音频的实际音量增益。
一种可选的实施方式中，所述增益确定单元920包括：
每一增益确定模块921，用于根据每一音频录制信息对应的每一音量预设增益，确定所述音频的每一音频录制信息对应的每一音量当前增益；
实际增益确定模块922，用于根据所述音频的各音量当前增益，确定所述音频的实际音量增益。
所述装置还包括接收单元940，用于接收服务器发送的其他主播合流音频，所述其他主播合流音频中包括各其他主播的调节了音量增益后的音频，各其他主播的音频调节后的音量增益为所述实际音量增益。
一种可选的实施方式中，所述音频录制信息包括以下的任一种：
音量预设增益、音频采集方式、声音处理方式、声音的特效信息。
一种可选的实施方式中，所述装置还包括处理单元940，用于：
对采集的音频进行以下任一种处理方式：回声消除处理、混响处理、声音均衡处理；
所述发送单元930还用于：向服务器发送处理后的音频。
本公开实施例还提供一种计算机程序产品，包括计算机程序，所述计算机程序在被处理器执行时实现如上所述的任一种音频合流方法或者音频上传方法。
本实施例提供的设备，可用于执行上述方法实施例的技术方案，其实现原理和技术效果类似，本实施例此处不再赘述。
参考图11，其示出了适于用来实现本公开实施例的电子设备1100的结构示意图，该电子设备1100可以为终端设备或服务器。其中，终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、个人数字助理(Personal Digital Assistant，简称PDA)、平板电脑(Portable Android Device，简称PAD)、便携式多媒体播放器(Portable Media Player，简称PMP)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图11示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图11所示，电子设备1100可以包括处理装置(例如中央处理器、图形处理器等)1101，其可以根据存储在只读存储器(Read Only Memory，简称ROM)1102中的程序或者从存储装置1108加载到随机访问存储器(Random Access Memory，简称RAM)1103中的程序而执行各种适当的动作和处理。在RAM 1103中，还存储有电子设备1100操作所需的各种程序和数据。处理装置1101、ROM 1102以及RAM 1103通过总线1104彼此相连。输入/输出(I/O)接口1105也连接至总线1104。
通常，以下装置可以连接至I/O接口1105：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置1106；包括例如液晶显示器(Liquid Crystal Display，简称LCD)、扬声器、振动器等的输出装置1107；包括例如磁带、硬盘等的存储装置1108；以及通信装置1109。通信装置1109可以允许电子设备1100与其他设备进行无线或有线通信以交换数据。虽然图11示出了具有各种装置的电子设备1100，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置1109从网络上被下载和安装，或者从存储装置1108被安装，或者从ROM 1102被安装。在该计算机程序被处理装置1101执行时，执行本公开实施例的方法中限定的上述功能。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。
上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备执行上述实施例所示的方法。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言—诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(Local Area Network，简称LAN)或广域网(Wide Area Network，简称WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
描述于本公开实施例中所涉及到的单元可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，单元的名称在某种情况下并不构成对该单元本身的限定，例如，第一获取单元还可以被描述为“获取至少两个网际协议地址的单元”。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
第一方面，根据本公开的一个或多个实施例，提供了一种音频合流方法，包括：
获取进入同一直播间的各主播的终端的音频及其实际音量增益；其中，所述直播间为网络中的虚拟直播间；
将各音频中其他主播的音频的实际音量增益，调节为当前主播的音频的实际音量增益；所述其他主播为所述直播间的各主播中当前主播以外的主播；
根据调节了音量增益后的各音频生成与所述当前主播对应的合流音频，所述合流音频用于播放所述当前主播的直播间的直播内容。
根据本公开的一个或多个实施例，还包括：
向播放所述当前主播的直播内容的用户终端发送所述当前主播的合流音频。
根据本公开的一个或多个实施例，还包括：
对调节了音量增益后的其他主播的音频进行合流处理，得到其他主播合流音频；
将其他主播合流音频发送给所述当前主播的终端。
第二方面，根据本公开的一个或多个实施例，提供了一种音频上传方法，包括：
在直播过程中获取音频、以及与所述音频对应的至少一种音频录制信息，其中，所述音频录制信息表征获取所述音频时的录制信息；
根据所述至少一种音频录制信息确定所述音频的实际音量增益；
向服务器发送所述音频以及所述音频的实际音量增益，在直播间中包括多个主播时，主播音频的实际音量增益用于对主播音频的增益进行调节，调节后的音频用于生成合流音频。
根据本公开的一个或多个实施例，预先设置有与每一音频录制信息对应的每一音量预设增益；
所述根据所述至少一种音频录制信息确定所述音频的实际音量增益，包括：
根据每一音频录制信息对应的每一音量预设增益、每一音频录制信息，确定所述音频的实际音量增益。
根据本公开的一个或多个实施例，所述根据每一音频录制信息对应的每一音量预设增益、每一音频录制信息，确定所述音频的实际音量增益，包括：
根据每一音频录制信息对应的每一音量预设增益，确定所述音频的每一音频录制信息对应的每一音量当前增益；
根据所述音频的各音量当前增益，确定所述音频的实际音量增益。
根据本公开的一个或多个实施例，还包括：
接收服务器发送的其他主播合流音频，所述其他主播合流音频中包括各其他主播的调节了音量增益后的音频，各其他主播的音频调节后的音量增益为所述实际音量增益。
根据本公开的一个或多个实施例，所述音频录制信息包括以下的任一种：
音量预设增益、音频采集方式、声音处理方式、声音的特效信息。
根据本公开的一个或多个实施例，还包括：
对采集的音频进行以下任一种处理方式：回声消除处理、混响处理、声音均衡处理；
向服务器发送所述音频，包括：向服务器发送处理后的音频。
第三方面，根据本公开的一个或多个实施例，提供了一种音频合流装置，包括：
获取单元，用于获取进入同一直播间的各主播的终端的音频及其实际音量增益；其中，所述直播间为网络中的虚拟直播间；
调节单元，用于将各音频中其他主播的音频的实际音量增益，调节为当前主播的音频的实际音量增益；所述其他主播为所述直播间的各主播中当前主播以外的主播；
合流单元，用于根据调节了音量增益后的各音频生成与所述当前主播对应的合流音频，所述合流音频用于播放所述当前主播的直播间的直播内容。
在一种可选的实施方式中，所述装置还包括推送单元，用于：
向播放所述当前主播的直播内容的用户终端发送所述当前主播的合流音频。
在一种可选的实施方式中，所述合流单元还用于对调节了音量增益后的其他主播的音频进行合流处理，得到其他主播合流音频；
所述装置还包括返声单元，用于将其他主播合流音频发送给所述当前主播的终端。
第四方面，根据本公开的一个或多个实施例，提供了一种音频上传装置，包括：
获取单元，用于在直播过程中获取音频、以及与所述音频对应的至少一种音频录制信息，其中，所述音频录制信息表征获取所述音频时的录制信息；
增益确定单元，用于根据所述至少一种音频录制信息确定所述音频的实际音量增益；
发送单元，用于向服务器发送所述音频以及所述音频的实际音量增益，在直播间中包括多个主播时，主播音频的实际音量增益用于对主播音频的增益进行调节，调节后的音频用于生成合流音频。
一种可选的实施方式中，预先设置有与每一音频录制信息对应的每一音量预设增益；
所述增益确定单元具体用于：
根据每一音频录制信息对应的每一音量预设增益、每一音频录制信息，确定所述音频的实际音量增益。
一种可选的实施方式中，所述增益确定单元包括：
每一增益确定模块，用于根据每一音频录制信息对应的每一音量预设增益，确定所述音频的每一音频录制信息对应的每一音量当前增益；
实际增益确定模块，用于根据所述音频的各音量当前增益，确定所述音频的实际音量增益。
一种可选的实施方式中，所述装置还包括接收单元，用于接收服务器发送的其他主播合流音频，所述其他主播合流音频中包括各其他主播的调节了音量增益后的音频，各其他主播的音频调节后的音量增益为所述实际音量增益。
一种可选的实施方式中，所述音频录制信息包括以下的任一种：
音量预设增益、音频采集方式、声音处理方式、声音的特效信息。
一种可选的实施方式中，所述装置还包括处理单元，用于：
对采集的音频进行以下任一种处理方式：回声消除处理、混响处理、声音均衡处理；
所述发送单元还用于：向服务器发送处理后的音频。
第五方面，根据本公开的一个或多个实施例，提供了一种电子设备，包括：至少一个处理器和存储器；
所述存储器存储计算机执行指令；
所述至少一个处理器执行所述存储器存储的计算机执行指令，使得所述至少一个处理器执行如上第一方面以及第一方面各种可能的设计所述的音频合流方法，或者执行如上第二方面以及第二方面各种可能的设计所述的音频上传方法。
第六方面，根据本公开的一个或多个实施例，提供了一种计算机可读存储介质，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现如上第一方面以及第一方面各种可能的设计所述的音频合流方法，或者执行如上第二方面以及第二方面各种可能的设计所述的音频上传方法。
第七方面，本公开实施例提供一种计算机程序产品，包括计算机程序，所述计算机程序在被处理器执行时实现如上第一方面以及第一方面各种可能的设计所述的音频合流方法，或者执行如上第二方面以及第二方面各种可能的设计所述的音频上传方法。
第八方面，本公开实施例提供一种计算机程序，所述计算机程序在被处理器执行时实现如上第一方面以及第一方面各种可能的设计所述的音频合流方法，或者执行如上第二方面以及第二方面各种可能的设计所述的音频上传方法。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开中所涉及的公开范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述公开构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。
此外，虽然采用特定次序描绘了各操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了若干具体实现细节，但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的各种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。
尽管已经采用特定于结构特征和/或方法逻辑动作的语言描述了本主题，但是应当理解所附权利要求书中所限定的主题未必局限于上面描述的特定特征或动作。相反，上面所描述的特定特征和动作仅仅是实现权利要求书的示例形式。

Claims (15)

  1. 一种音频合流方法，其特征在于，包括：
    获取进入同一直播间的各主播的终端的音频及其实际音量增益；其中，所述直播间为网络中的虚拟直播间；
    将各音频中其他主播的音频的实际音量增益，调节为当前主播的音频的实际音量增益；所述其他主播为所述直播间的各主播中所述当前主播以外的主播；
    根据调节了音量增益后的各音频生成与所述当前主播对应的合流音频，所述合流音频用于播放所述当前主播的直播间的直播内容。
  2. 根据权利要求1所述的方法，其特征在于，还包括：
    向播放所述当前主播的直播内容的用户终端发送与所述当前主播对应的合流音频。
  3. 根据权利要求1或2所述的方法，其特征在于，还包括：
    对调节了音量增益后的其他主播的音频进行合流处理，得到其他主播合流音频；
    将所述其他主播合流音频发送给所述当前主播的终端。
  4. 一种音频上传方法，其特征在于，包括：
    在直播过程中获取音频、以及与所述音频对应的至少一种音频录制信息，其中，所述音频录制信息表征获取所述音频时的录制信息；
    根据所述至少一种音频录制信息确定所述音频的实际音量增益；
    向服务器发送所述音频以及所述音频的实际音量增益，在直播间中包括多个主播时，主播的音频的实际音量增益用于对主播的音频的增益进行调节，调节后的音频用于生成合流音频。
  5. 根据权利要求4所述的方法，其特征在于，预先设置有与每一音频录制信息对应的每一音量预设增益；
    所述根据所述至少一种音频录制信息确定所述音频的实际音量增益，包括：
    根据每一音频录制信息对应的每一音量预设增益、每一音频录制信息，确定所述音频的实际音量增益。
  6. 根据权利要求5所述的方法，其特征在于，所述根据每一音频录制信息对应的每一音量预设增益、每一音频录制信息，确定所述音频的实际音量增益，包括：
    根据每一音频录制信息对应的每一音量预设增益，确定所述音频的每一音频录制信息对应的每一音量当前增益；
    根据所述音频的各音量当前增益，确定所述音频的实际音量增益。
  7. 根据权利要求4-6任一种所述的方法，其特征在于，还包括：
    接收服务器发送的其他主播合流音频，所述其他主播合流音频中包括各其他主播的调节了音量增益后的音频，各其他主播的音频调节后的音量增益为所述实际音量增益。
  8. 根据权利要求4-7任一项所述的方法，其特征在于，所述音频录制信息包括以下的任一种：
    音量预设增益、音频采集方式、声音处理方式、声音的特效信息。
  9. 根据权利要求4-8任一项所述的方法，其特征在于，还包括：
    对采集的音频进行以下任一种处理方式：回声消除处理、混响处理、声音均衡处理；
    向服务器发送所述音频，包括：向服务器发送处理后的音频。
  10. 一种音频合流装置，其特征在于，包括：
    获取单元，用于获取进入同一直播间的各主播的终端的音频及其实际音量增益；其中，所述直播间为网络中的虚拟直播间；
    调节单元，用于将各音频中其他主播的音频的实际音量增益，调节为当前主播的音频的实际音量增益；所述其他主播为所述直播间的各主播中所述当前主播以外的主播；
    合流单元，用于根据调节了音量增益后的各音频生成与所述当前主播对应的合流音频，所述合流音频用于播放所述当前主播的直播间的直播内容。
  11. 一种音频上传装置，其特征在于，包括：
    获取单元，用于在直播过程中获取音频、以及与所述音频对应的至少一种音频录制信息，其中，所述音频录制信息表征获取所述音频时的录制信息；
    增益确定单元，用于根据所述至少一种音频录制信息确定所述音频的实际音量增益；
    发送单元，用于向服务器发送所述音频以及所述音频的实际音量增益，在直播间中包括多个主播时，主播音频的实际音量增益用于对主播音频的增益进行调节，调节后的音频用于生成合流音频。
  12. 一种电子设备，其特征在于，包括：至少一个处理器和存储器；
    所述存储器存储计算机执行指令；
    所述至少一个处理器执行所述存储器存储的计算机执行指令，使得所述至少一个处理器执行如权利要求1-3或4-9任一项所述的方法。
  13. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质中存储有计算机执行指令，当处理器执行所述计算机执行指令时，实现如权利要求1-3或4-9任一项所述的方法。
  14. 一种计算机程序产品，其特征在于，包括计算机程序，所述计算机程序在被处理器执行时实现根据权利要求1-3或4-9任一项所述的方法。
  15. 一种计算机程序，其特征在于，所述计算机程序在被处理器执行时实现根据权利要求1-3或4-9任一项所述的方法。
PCT/CN2022/094499 2021-07-14 2022-05-23 音频合流方法、音频上传方法、设备及程序产品 WO2023284414A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110797609.6 2021-07-14
CN202110797609.6A CN113542792B (zh) 2021-07-14 2021-07-14 音频合流方法、音频上传方法、设备及程序产品

Publications (1)

Publication Number Publication Date
WO2023284414A1 true WO2023284414A1 (zh) 2023-01-19

Family

ID=78099178

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/094499 WO2023284414A1 (zh) 2021-07-14 2022-05-23 音频合流方法、音频上传方法、设备及程序产品

Country Status (2)

Country Link
CN (1) CN113542792B (zh)
WO (1) WO2023284414A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113542792B (zh) * 2021-07-14 2023-04-07 北京字节跳动网络技术有限公司 音频合流方法、音频上传方法、设备及程序产品

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040199933A1 (en) * 2003-04-04 2004-10-07 Michael Ficco System and method for volume equalization in channel receivable in a settop box adapted for use with television
US20060217162A1 (en) * 2004-01-09 2006-09-28 Bodley Martin R Wireless multi-user audio system
JP2007053510A (ja) * 2005-08-17 2007-03-01 Sony Corp 記録再生装置、再生装置、音量自動調整方法および音量自動調整プログラム
CN108235052A (zh) * 2018-01-09 2018-06-29 安徽小马创意科技股份有限公司 基于ios可选择多音频通道硬件混音、采集及播放的方法
CN108965904A (zh) * 2018-09-05 2018-12-07 北京优酷科技有限公司 一种直播间的音量调节方法及客户端
CN109036432A (zh) * 2018-07-27 2018-12-18 武汉斗鱼网络科技有限公司 一种连麦方法、装置、设备及存储介质
CN110267064A (zh) * 2019-06-12 2019-09-20 百度在线网络技术（北京）有限公司 音频播放状态处理方法、装置、设备及存储介质
CN110460863A (zh) * 2019-07-15 2019-11-15 北京字节跳动网络技术有限公司 基于显示位置的音视频处理方法、装置、介质和电子设备
CN110915220A (zh) * 2017-07-13 2020-03-24 杜比实验室特许公司 具有流式传输能力的音频输入和输出装置
CN113542792A (zh) * 2021-07-14 2021-10-22 北京字节跳动网络技术有限公司 音频合流方法、音频上传方法、设备及程序产品

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105744293B (zh) * 2016-03-16 2019-04-16 北京小米移动软件有限公司 视频直播的方法及装置
CN105721710A (zh) * 2016-03-28 2016-06-29 联想(北京)有限公司 一种录音方法及装置、电子设备
CN109104616B (zh) * 2018-09-05 2022-01-14 阿里巴巴（中国）有限公司 一种直播间的语音连麦方法及客户端
CN109005419B (zh) * 2018-09-05 2021-03-19 阿里巴巴（中国）有限公司 一种语音信息的处理方法及客户端
CN111739496B (zh) * 2020-06-24 2023-06-23 腾讯音乐娱乐科技（深圳）有限公司 音频处理方法、装置及存储介质
CN111813367A (zh) * 2020-07-22 2020-10-23 广州繁星互娱信息科技有限公司 调整音量的方法、装置、设备以及存储介质
CN112135155B (zh) * 2020-09-11 2022-07-19 上海七牛信息技术有限公司 音视频的连麦合流方法、装置、电子设备及存储介质
CN112272170B (zh) * 2020-10-19 2023-01-10 广州博冠信息科技有限公司 语音通信方法及装置、电子设备、存储介质
CN113112986A (zh) * 2021-05-13 2021-07-13 北京字节跳动网络技术有限公司 音频合成方法、装置、设备、介质及程序产品

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040199933A1 (en) * 2003-04-04 2004-10-07 Michael Ficco System and method for volume equalization in channel receivable in a settop box adapted for use with television
US20060217162A1 (en) * 2004-01-09 2006-09-28 Bodley Martin R Wireless multi-user audio system
JP2007053510A (ja) * 2005-08-17 2007-03-01 Sony Corp 記録再生装置、再生装置、音量自動調整方法および音量自動調整プログラム
CN110915220A (zh) * 2017-07-13 2020-03-24 杜比实验室特许公司 具有流式传输能力的音频输入和输出装置
CN108235052A (zh) * 2018-01-09 2018-06-29 安徽小马创意科技股份有限公司 基于ios可选择多音频通道硬件混音、采集及播放的方法
CN109036432A (zh) * 2018-07-27 2018-12-18 武汉斗鱼网络科技有限公司 一种连麦方法、装置、设备及存储介质
CN108965904A (zh) * 2018-09-05 2018-12-07 北京优酷科技有限公司 一种直播间的音量调节方法及客户端
CN110267064A (zh) * 2019-06-12 2019-09-20 百度在线网络技术（北京）有限公司 音频播放状态处理方法、装置、设备及存储介质
CN110460863A (zh) * 2019-07-15 2019-11-15 北京字节跳动网络技术有限公司 基于显示位置的音视频处理方法、装置、介质和电子设备
CN113542792A (zh) * 2021-07-14 2021-10-22 北京字节跳动网络技术有限公司 音频合流方法、音频上传方法、设备及程序产品

Also Published As

Publication number Publication date
CN113542792B (zh) 2023-04-07
CN113542792A (zh) 2021-10-22

Similar Documents

Publication Publication Date Title
US9071792B2 (en) Adaptive media content scrubbing on a remote device
WO2022152064A1 (zh) 视频生成方法、装置、电子设备和存储介质
WO2019062541A1 (zh) 一种实时数字音频信号混音的方法及装置
US20140328485A1 (en) Systems and methods for stereoisation and enhancement of live event audio
WO2021073368A1 (zh) 视频文件的生成方法、装置、终端及存储介质
US20230421716A1 (en) Video processing method and apparatus, electronic device and storage medium
JP2015206989A (ja) 情報処理装置、情報処理方法及びプログラム
WO2023284437A1 (zh) 媒体文件处理方法、装置、设备、可读存储介质及产品
WO2020259130A1 (zh) 精选片段处理方法、装置、电子设备及可读介质
WO2017101418A1 (zh) 多路流媒体播放方法及设备
US8965181B2 (en) Automatic announcer voice attenuation in a presentation of a broadcast event
WO2022042634A1 (zh) 音频数据的处理方法、装置、设备及存储介质
WO2023284414A1 (zh) 音频合流方法、音频上传方法、设备及程序产品
WO2022134997A1 (zh) 视频跳转播放方法、装置、终端设备及存储介质
US20210390971A1 (en) Method and apparatus and telephonic system for acoustic scene conversion
US11272136B2 (en) Method and device for processing multimedia information, electronic equipment and computer-readable storage medium
WO2024067157A1 (zh) 生成特效视频的方法、装置、电子设备及存储介质
CN113992926A (zh) 界面显示方法、装置、电子设备和存储介质
WO2020253452A1 (zh) 直播间状态消息的推送方法、交互内容的切换方法、装置及设备
WO2023131266A1 (zh) 音频特效的编辑方法、装置、设备及存储介质
US20240040191A1 (en) Livestreaming audio processing method and device
CN113542785B (zh) 应用于直播的音频的输入输出的切换方法、直播设备
CN113948054A (zh) 音轨处理方法、装置、电子设备和存储介质
US11908490B2 (en) Video recording method and device, electronic device and storage medium
US11563857B2 (en) Aggregating hardware loopback

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22841054

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE