WO2017140235A1 - 音频数据处理方法、服务器、客户端以及系统、存储介质 - Google Patents

音频数据处理方法、服务器、客户端以及系统、存储介质 Download PDF

Info

Publication number
WO2017140235A1
WO2017140235A1 PCT/CN2017/073483 CN2017073483W WO2017140235A1 WO 2017140235 A1 WO2017140235 A1 WO 2017140235A1 CN 2017073483 W CN2017073483 W CN 2017073483W WO 2017140235 A1 WO2017140235 A1 WO 2017140235A1
Authority
WO
WIPO (PCT)
Prior art keywords
audio data
client
mixing
data processing
site
Prior art date
Application number
PCT/CN2017/073483
Other languages
English (en)
French (fr)
Inventor
王新亮
李斌
陈静聪
Original Assignee
腾讯科技（深圳）有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 腾讯科技（深圳）有限公司 filed Critical 腾讯科技（深圳）有限公司
Priority to US15/764,737 priority Critical patent/US11196868B2/en
Priority to MYPI2018700977A priority patent/MY189870A/en
Publication of WO2017140235A1 publication Critical patent/WO2017140235A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/765—Media network packet handling intermediate
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M3/00—Automatic or semi-automatic exchanges
    • H04M3/42—Systems providing special services or facilities to subscribers
    • H04M3/56—Arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities
    • H04M3/568—Arrangements for connecting several subscribers to a common circuit, i.e. affording conference facilities audio processing specific to telephonic conferencing, e.g. spatial distribution, mixing of participants
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/762—Media network packet handling at the source 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/60—Network streaming of media packets
    • H04L65/75—Media network packet handling
    • H04L65/764—Media network packet handling at the destination 
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
    • H04L65/80—Responding to QoS
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M2203/00—Aspects of automatic or semi-automatic exchanges
    • H04M2203/50—Aspects of automatic or semi-automatic exchanges related to audio conference
    • H04M2203/5072—Multiple active speakers

Definitions

  • the present invention relates to audio technologies, and in particular, to an audio data processing method, a server, a client, and a system, and a storage medium.
  • Client-side mixing refers to the decoding and mixing operations of multi-channel voice data on the client side. This mixing method requires high bandwidth and processing power for the client.
  • Server mixing means that the client only performs decoding and playback of one voice data, and the communication server decodes, mixes, and encodes voice data of multiple clients.
  • Such a mixing method has a lower requirement on the client side, but since the voice data is subjected to one encoding and decoding process, the distortion and delay of the voice data are easily caused.
  • the embodiments of the present invention provide an audio data processing method, a server, a client, a system, and a storage medium that are compatible with different mixing modes, so as to solve the problem that a user who uses different mixing modes in the related technology is difficult to make a call in the same call system. technical problem.
  • an embodiment of the present invention provides an audio data processing method, including:
  • an embodiment of the present invention further provides an audio data processing server, including:
  • An audio data receiving module configured to receive audio data of the client
  • a mixing mode obtaining module configured to receive an audio data request of the requesting client, and obtain an audio data mixing manner of the requesting client according to the audio data request of the requesting client;
  • a pre-processing module configured to pre-process the audio data of the corresponding client of the audio data request according to the audio data mixing manner of the requesting client
  • a sending module configured to send the pre-processed audio data to the requesting client, where the requesting client obtains the audio data after the mixing process.
  • an embodiment of the present invention further provides an audio data processing system, including any of the foregoing audio data processing servers and a plurality of clients.
  • an embodiment of the present invention further provides an audio data processing method, including:
  • off-site audio data wherein the off-site audio data includes a client identifier
  • an embodiment of the present invention further provides an audio data processing client, including:
  • the remote audio data receiving module is configured to receive the remote audio data, where the remote audio data includes a client identifier of the client from which the remote audio data is originated;
  • a mixing mode determining module configured to acquire the client identifier from the off-site audio data, and determine a mixing manner of the off-site audio data according to the client identifier
  • the audio data playing module is configured to perform playback processing on the off-site mixed data according to the mixing manner of the remote audio data.
  • an embodiment of the present invention further provides an audio data processing system, including a plurality of the foregoing audio data processing clients and at least one audio data processing server.
  • an embodiment of the present invention provides an audio data processing server, including:
  • processor having stored therein executable instructions configured to cause the processor to perform the following operations:
  • an audio data processing client including:
  • processor having stored therein executable instructions configured to cause the processor to perform the following operations:
  • off-site audio data wherein the off-site audio data includes a client identifier
  • an embodiment of the present invention provides an audio data processing server, where the storage is executable A line instruction for performing the audio data processing method of any of the foregoing.
  • an embodiment of the present invention provides a storage medium storing executable instructions, where the executable instructions are used to perform the audio data processing method of any of the foregoing.
  • the audio data processing method, the server, and the system of the embodiment of the present invention perform mixing processing according to the audio data mixing manner of the requesting client, and are compatible. Audio data for different mixing methods.
  • the audio data processing method, the server and the system of the embodiment of the present invention identify the mixing manner of the audio data by using the client identifier, so that the audio data of different mixing modes can be played; and the existing audio data is solved.
  • the technical problems of handling methods, servers, clients, and users of different mixing modes in the system are difficult to make calls in the same call system.
  • FIG. 1 is a flowchart of an audio data processing method according to an embodiment of the present invention.
  • FIG. 2 is a flowchart of an audio data processing method according to an embodiment of the present invention.
  • FIG. 3 is a flowchart of step S203 of the audio data processing method according to an embodiment of the present invention.
  • FIG. 4 is a schematic structural diagram of an audio data processing server according to an embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of an audio data processing server according to an embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of a preprocessing module of an audio data processing server according to an embodiment of the present invention.
  • FIG. 7 is a schematic structural diagram of a second audio data processing unit of a preprocessing module of an audio data processing server according to an embodiment of the present invention.
  • FIG. 8 is a schematic structural diagram of an audio data processing system according to an embodiment of the present invention.
  • FIG. 9 is a flowchart of an audio data processing method according to an embodiment of the present invention.
  • FIG. 10 is a flowchart of an audio data processing method according to an embodiment of the present invention.
  • FIG. 11 is a flowchart of step S1002 of the audio data processing method according to an embodiment of the present invention.
  • FIG. 12 is a flowchart of step S1003 of the audio data processing method according to an embodiment of the present invention.
  • FIG. 13 is a schematic structural diagram of an audio data processing client according to an embodiment of the present invention.
  • FIG. 14 is a schematic structural diagram of an audio data processing client according to an embodiment of the present invention.
  • FIG. 15 is a schematic structural diagram of a mixing mode determining module of an audio data processing client according to an embodiment of the present invention.
  • 16 is a schematic structural diagram of an audio data playing module of an audio data processing client according to an embodiment of the present invention.
  • FIG. 17 is a schematic structural diagram of an audio data processing system according to an embodiment of the present invention.
  • FIG. 18 is a schematic diagram of a working environment structure of an audio data processing server and an electronic device where a client is located according to an embodiment of the present invention.
  • the decoding device of the embodiments of the present invention may be implemented using various electronic devices including, but not limited to, personal computers, server computers, handheld or laptop devices, mobile devices (such as mobile phones, personal digital assistants (PDAs)). , media player, etc.), multiprocessor Systems, consumer electronics, small computers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.
  • PDAs personal digital assistants
  • multiprocessor Systems consumer electronics, small computers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.
  • the electronic device is an audio data processing server or a mobile device running an audio data processing client
  • a compatible process of audio data of different mixing modes (such as server mixing, mobile device mixing) is implemented.
  • FIG. 1 is a flowchart of a method for processing audio data according to an embodiment of the present invention.
  • the audio data processing method of the embodiment of the present invention may be implemented by using an electronic device as an implementation main body, for example, an audio data processing server, where the audio data processing method includes:
  • Step S101 receiving audio data of the client
  • Step S102 receiving an audio data request of the requesting client, and acquiring an audio data mixing manner of the requesting client according to the audio data request of the requesting client;
  • Step S103 preprocessing the audio data of the corresponding client of the audio data request according to the audio data mixing manner of the requesting client;
  • Step S104 Send the pre-processed audio data to the requesting client, so that the requesting client obtains the audio data after the mixing processing based on the pre-processed audio data.
  • step S101 the audio data processing server receives audio data of each client, and the audio data may be voice data or music data sent by the client to other clients. Then it proceeds to step S102.
  • the audio data processing server receives an audio data request requesting the client, and the requesting client sends an audio data request to the electronic device (the electronic device as an implementation body of the audio data processing method) to request to receive voice data or music of other clients.
  • the client of the data is a request to receive voice data or music data of other clients.
  • Electronic device that receives audio data sent by the client and audio data request, and is electronically set For example, the audio data processing server obtains the audio data mixing mode of the requesting client according to the audio data request of the requesting client.
  • the audio data mixing methods here may include client mixing and server mixing. Then it proceeds to step S103.
  • step S101 and step S102 can be described in such a manner:
  • Step S101 receiving audio data of at least two (including two) clients
  • Step S102 receiving an audio data request from the client (the client that sends the audio data request is the requesting client, and may be the client in the at least two clients described in step S101, or may be different from the at least one described in step S101.
  • the client of the two clients acquires the mixing mode (referred to as the audio data type) of the audio data of the requesting client request according to the requesting audio data request of the client.
  • step S103 the audio data processing server requests the corresponding client for the audio data according to the audio data mixing manner of the requesting client acquired in step S102 (the client corresponding to the audio data request, that is, the requesting client requests the audio data through the audio data)
  • the audio data of the source client of the requested audio data is subjected to mixing preprocessing.
  • the client that sends the audio data request can play the audio data pre-processed by the mixing. Then it proceeds to step S104.
  • step S104 the audio data processing server transmits the audio data pre-processed in step S103 to the requesting client to request the client to acquire the audio data after the mixing process. That is, the requesting client obtains the audio data of the corresponding mixed pre-processing, and then obtains the corresponding audio data after the mixing process by using the corresponding audio extraction method, and finally performs the playing operation on the obtained audio data after the mixing process.
  • the audio data processing method of the embodiment of the present invention performs mixing processing according to the audio data mixing manner of the requesting client, and is compatible with audio data of different mixing modes.
  • FIG. 2 is a flowchart of a method for processing audio data according to an embodiment of the present invention.
  • the audio data processing method of the embodiment of the present invention may be implemented by using the foregoing electronic device, in particular, an audio data processing server, where the audio data processing method includes:
  • Step S201 receiving audio data of the client
  • Step S202 receiving an audio data request of the requesting client, and acquiring an audio data mixing manner of the requesting client according to the audio data request of the requesting client;
  • Step S203 preprocessing the audio data of the corresponding client of the audio data request according to the audio data mixing manner of the requesting client;
  • Step S204 the audio data after the mixing pre-processing is sent to the requesting client, so that the requesting client obtains the audio data after the mixing processing based on the audio data after the mixing pre-processing.
  • step S201 the audio data processing server receives audio data of each client, and the audio data may be voice data or music data sent by the client to other clients. Then it proceeds to step S202.
  • the audio data processing server receives the audio data request requesting the client, and requests the client to request to receive the voice data or music data of the other client for sending the audio data request, and the audio data request is for requesting to receive the voice of the other client.
  • Request for data or music data The electronic device that receives the audio data sent by the client and the audio data request takes the audio data processing server as an example, and acquires the audio data mixing manner of the requesting client according to the audio data request of the requesting client.
  • the audio data mixing methods here may include client mixing and server mixing.
  • the audio data mixing method here includes client mixing and server mixing; wherein, the client mixing refers to the client receiving the audio data as the audio data processing client, Performing a mixing operation of a plurality of audio data, and then the audio data processing client plays the audio data after the mixing operation; the server mixing means that the audio data processing server performs a mixing operation of the plurality of audio data, and then The audio data processing server sends the audio data after the mixing operation to the audio data processing client, and the audio data client only serves as a client for playing the audio data, and the audio data processing client performs the playback operation. Then it proceeds to step S203.
  • step S203 the audio data processing server preprocesses the audio data of the corresponding client of the audio data request according to the audio data mixing manner of the requesting client acquired in step S202.
  • the audio data processing server directly performs the collecting operation on the audio data of the corresponding client of the audio data request; after the audio data is collected, the audio data is not processed any.
  • FIG. 3 is a flowchart of step S203 of the audio data processing method according to an embodiment of the present invention.
  • the step S203 includes:
  • Step S301 the audio data processing server performs decoding processing on the audio data of all the clients according to the packet loss condition of the audio data.
  • the audio data processing server determines the packet loss condition of the audio data. If there is no packet loss, the audio data is directly decoded by using the corresponding decoder to obtain the decoded audio. data. If there is a packet loss, the lost error packet is recovered by using a Forward Error Correction (FEC) redundancy packet, and the restored audio data is decoded by using a corresponding decoder to obtain a decoding process. Audio data. If the packet loss cannot be recovered, the packet loss compensation algorithm is used to directly calculate the voice packet before the packet loss, and then the corresponding decoder is used to decode the calculated audio data. Then go to the step S302.
  • FEC Forward Error Correction
  • Step S302 the audio data processing server acquires the audio data of the client corresponding to the audio data request from the audio data after the decoding process acquired in step S301.
  • the audio data processing server can store the received audio data from each client in a data buffer for ready to call. In this way, the audio data processing server can acquire the audio data of the corresponding client according to the audio data request. Then it proceeds to step S303.
  • Step S303 the audio data processing server converts the audio data of the client corresponding to the audio data request into the audio data of the preset format. Since the format of the audio data of each client may be inconsistent with the requirements of the encoder (for example, the format of the audio data encoded by the encoder), all the formats of the audio data are uniformly converted and converted to presets. Formatted audio data for subsequent encoding of the corresponding audio data. Then it proceeds to step S304.
  • Step S304 the audio data processing server performs the mixing process on the audio data of the multiple preset formats acquired in step S303 (the audio data of the corresponding client from the audio data request) to generate the audio data after the mixing process. Then it proceeds to step S305.
  • Step S305 the audio data processing server performs compression coding processing on the audio data after the mixing process obtained in step S304 according to the network situation between the current client and the client. For example, the audio data processing server performs FEC encoding on the audio data after the mixing process according to the network packet loss and the network delay jitter, to generate a corresponding audio data encoding packet, and performs network secure transmission on the audio data encoding packet. After the transfer is completed, it proceeds to step S204.
  • step S204 the audio data processing server transmits the audio data encoding packet acquired in step S203 to the requesting client to request the client to acquire the audio data after the mixing processing.
  • the audio data mixing mode of the requesting client is a client-side mixing
  • the pre-processed audio data obtained by the requesting client is requested to be unmixed multi-channel audio data, so that the requesting client receives the above-mentioned
  • the multi-channel audio data is mixed, and the audio data after the mixing processing is played.
  • the audio data mixing mode of the requesting client is server mixing
  • the pre-processed audio data acquired by the client is requested to be an audio data encoding package, and the requesting client can directly decode and play the audio data encoding package.
  • the audio data processing method of the embodiment of the present invention further includes the following steps: the audio data processing server receives the mixing mode conversion request of the requesting client, and sets the audio data of the requesting client according to the mixing mode conversion request of the requesting client. Sound mode.
  • the audio data processing server can convert the audio data mixing manner of the requesting client according to the mixing mode conversion request of the requesting client, for example, converting the request client of the client mixing mode into the request client of the server mixing mode.
  • the audio data processing method in the embodiment of the present invention encodes audio data to perform secure network transmission on the audio data, and can also convert the audio data of the client according to the request of the client's mixing mode. Timely switching, further improving the transmission security and processing compatibility of audio data of different mixing modes.
  • FIG. 4 is a schematic structural diagram of an audio data processing server according to an embodiment of the present invention.
  • the audio data processing server according to the embodiment of the present invention can be implemented using the description of the audio data processing method described above.
  • the audio data processing server 40 includes an audio data receiving module 41, a mixing mode acquisition module 42, a mixing preprocessing module 43, and a transmitting module 44.
  • the audio data receiving module 41 is configured to receive audio data of the client.
  • the mixing mode obtaining module 42 is configured to receive the audio data request of the requesting client, and obtain the audio data mixing mode of the requesting client according to the audio data request of the requesting client.
  • the pre-processing module 43 is configured to pre-process the audio data of the corresponding client of the audio data request according to the audio data mixing manner of the requesting client.
  • the sending module 44 is configured to send the pre-processed audio data to the requesting client End, in order to request the client to obtain the audio data after the mixing process.
  • the audio data receiving module 41 first receives the audio data of each client, and the audio data may be voice data or music data sent by the client to other clients.
  • the mixing mode acquisition module 42 receives the audio data request of the requesting client, and requests the client to request the client to receive the voice data or music data of other clients, and the audio data request is a request for receiving voice data or music data of other clients.
  • the mixing mode acquisition module 42 acquires the audio data mixing mode of the requesting client according to the audio data request of the requesting client.
  • the audio data mixing methods here may include client mixing and server mixing.
  • the pre-processing module 43 then pre-processes the audio data of the client that is requested by the audio data requesting module according to the audio data mixing mode of the requesting client. So that the corresponding client can play the pre-processed audio data normally.
  • the final sending module 44 sends the audio data preprocessed in the mixing preprocessing module 43 to the requesting client to request the client to obtain the audio data after the mixing processing. That is, the requesting client obtains the corresponding pre-processed audio data, and then adopts the corresponding audio extraction mode, obtains the corresponding audio data after the mixing process based on the pre-processed audio data, and finally obtains the audio after the mixed processing. The data is played back.
  • the audio data processing server of the embodiment of the present invention performs mixing processing according to the audio data mixing manner of the requesting client, and is compatible with audio data of different mixing modes.
  • FIG. 5 is a schematic structural diagram of an audio data processing server according to an embodiment of the present invention.
  • the audio data processing server according to the embodiment of the present invention can be implemented based on the audio data processing method described above.
  • the audio data processing server 50 includes an audio data receiving module 51, a mixing mode acquiring module 52, a mixing preprocessing module 53, a transmitting module 54, and a mixing mode conversion module. 55.
  • the audio data receiving module 51 is configured to receive audio data of the client.
  • the mixing mode obtaining module 52 is configured to receive the audio data request of the requesting client, and obtain the audio data mixing mode of the requesting client according to the audio data request of the requesting client.
  • the mixing pre-processing module 53 is configured to perform pre-mixing pre-processing on the audio data of the corresponding client of the audio data request according to the audio data mixing manner of the requesting client.
  • the sending module 54 is configured to send the pre-processed audio data to the requesting client, so that the requesting client obtains the audio data after the mixing processing based on the audio data after the mixing processing.
  • the mixing mode conversion module 55 is configured to receive a mixing mode conversion request of the requesting client, and set the audio data mixing mode of the requesting client according to the mixing mode conversion request of the requesting client.
  • FIG. 6 is a schematic structural diagram of a preprocessing module of an audio data processing server according to an embodiment of the present invention.
  • the mixing pre-processing module 53 includes a first audio data processing unit 61 and a second audio data processing unit 62.
  • the first audio data processing unit 61 is configured to perform the acquisition processing on the audio data of the corresponding client of the audio data request, if the audio data mixing mode of the requesting client is the client mixing.
  • the second audio data processing unit 62 is configured to, if the audio data mixing mode of the requesting client is the server mixing, sequentially perform decoding, mixing, and encoding processing on the audio data of the corresponding client of the audio data request.
  • FIG. 7 is a schematic structural diagram of a second audio data processing unit of a mixing preprocessing module of an audio data processing server according to an embodiment of the present invention.
  • the second audio data processing unit 62 includes a decoding subunit 71, an audio data acquisition subunit 72, a format conversion subunit 73, a mixing processing subunit 74, and a compression encoding subunit 75.
  • the decoding sub-unit 71 is configured to perform decoding processing on the audio data of all clients according to the packet loss condition of the audio data.
  • the audio data acquisition sub-unit 72 is configured to acquire audio data of the client corresponding to the audio data request from the decoded audio data.
  • Format conversion subunit 73 configuration The audio data of the client corresponding to the audio data request is converted into audio data of a preset format.
  • the mixing processing sub-unit 74 is configured to perform mixing processing on audio data of a preset format.
  • the compression coding sub-unit 75 is configured to perform compression coding processing on the audio data after the mixing processing according to the current network condition.
  • the audio data receiving module 51 first receives the audio data of each client, and the audio data may be voice data or music data sent by the client to other clients.
  • the mixing mode obtaining module 52 receives the audio data request of the requesting client, and requests the client to request the client to receive the voice data or music data of the other client, and the audio data request is a request for receiving the voice data or music data of the other client.
  • the mixing mode acquisition module 52 then acquires the audio data mixing mode of the requesting client according to the audio data request of the requesting client.
  • the audio data mixing methods here include client mix and server mix.
  • the client mix performs a mix operation of multiple audio data for the audio data processing client, and then the audio data processing client plays the audio data after the mix operation.
  • the server mixes a plurality of audio data mixing operations for the audio data processing server, and then the server sends the audio data after the mixing operation to the audio data processing client, and the audio data processing client performs the playback operation.
  • the pre-processing module 53 performs the pre-processing of the audio data of the client that is requested by the audio data requesting module according to the audio data mixing mode of the requesting client acquired by the mixing mode acquisition module 52.
  • the first audio data processing unit 61 of the mixing preprocessing module 53 directly performs the collecting operation on the audio data of the client corresponding to the audio data request; the audio data is collected here. , does not do any processing on the audio data.
  • the second audio data processing unit 62 of the preprocessing module 53 sequentially decodes, mixes, and encodes the audio data of the corresponding client of the audio data request.
  • the process of decoding, mixing, and encoding processing is specifically as follows:
  • the decoding sub-unit 71 of the second audio data processing unit 62 performs decoding processing on the audio data of all the clients in accordance with the packet loss condition of the audio data.
  • the decoding sub-unit 71 determines the packet loss condition of the audio data. If there is no packet loss, the audio data is directly decoded by using the corresponding decoder to obtain the decoded audio. data. If there is a packet loss, the lost voice packet is recovered by using the FEC redundancy packet, and the restored audio data is decoded by using the corresponding decoder to obtain the decoded audio data. If the packet loss cannot be recovered, the packet loss compensation algorithm is used to directly calculate the voice packet before the packet loss, and then the corresponding decoder is used to decode the calculated audio data.
  • the audio data acquisition sub-unit 72 of the second audio data processing unit 62 acquires the audio data of the client corresponding to the audio data request from the decoded audio data acquired by the decoding sub-unit 71.
  • the audio data processing server can store the audio data of each client in the data buffer for ready to call. In this way, the audio data processing server can acquire the audio data of the corresponding client according to the audio data request.
  • the format conversion sub-unit 73 of the second audio data processing unit 62 converts the audio data of the client corresponding to the audio data request into the audio data of the preset format. Since the format of the audio data of each client may be inconsistent with the requirements of the encoder, the format of all the audio data is uniformly converted and converted into audio data of a preset format for subsequent encoding operation of the corresponding audio data.
  • the mixing processing sub-unit 74 of the second audio data processing unit 62 performs a mixing process on the audio data of the multiplex preset format acquired by the format converting sub-unit 73 to generate audio data after one mixing process.
  • the compression coding sub-unit 75 of the second audio data processing unit 62 performs compression coding processing on the audio data after the mixing processing obtained by the mixing processing sub-unit 74 according to the current network condition. That is, the audio data processing server according to network packet loss and network delay jitter, after the mixing process
  • the audio data is FEC encoded to generate a corresponding audio data encoding packet for secure network transmission of the audio data.
  • the final mixing mode conversion module 54 sends the audio data encoding packet acquired by the sending module 53 to the requesting client, so as to request the client to obtain the audio data after the mixing process.
  • the pre-processed audio data obtained by the client is requested to be unprocessed multi-channel audio data, so that the requesting client mixes the multi-channel audio data. And playback processing.
  • the audio data mixing mode of the requesting client is server mixing
  • the pre-processed audio data acquired by the client is requested to be an audio data encoding package, and the requesting client can directly decode and play the audio data encoding package.
  • the mixing mode conversion module 55 of the audio data processing server 50 of the embodiment of the present invention may further receive a mixing mode conversion request of the requesting client, and set a requesting client according to the mixing mode conversion request of the requesting client. Audio data mixing method.
  • the mixing mode conversion module 55 can convert the audio data mixing mode of the requesting client according to the mixing mode conversion request of the requesting client, for example, converting the request client of the client mixing mode into the requesting client of the server mixing mode. Wait.
  • the audio data processing server of the embodiment of the present invention performs encoding processing on the audio data to perform network secure transmission on the audio data, and can also convert the request to the client according to the mixing mode of the requesting client.
  • the audio data mixing mode is switched in time, which further improves the transmission security and processing compatibility of the audio data of different mixing modes.
  • FIG. 8 is a schematic structural diagram of an audio data processing system according to an embodiment of the present invention.
  • the audio data processing system 80 includes The audio data processing server 81 and a plurality of clients, such as a client 82, a client 83, and a client 84.
  • the specific working principle of the audio data processing server 81 is the same as or similar to the description of the audio data processing server in the embodiment of the present invention. For details, refer to the related description of the audio data processing server.
  • the client in the audio data processing system 80 can be a client for performing audio data processing in the related art, such as a call voice client or the like.
  • the audio data processing system of the embodiment of the present invention performs mixing processing according to the audio data mixing manner of the requesting client, and is compatible with audio data of different mixing modes.
  • the audio data processing system of FIG. 8 includes an audio data processing server 81 and a plurality of clients 82, 83, 84, wherein the client includes a client capable of mixing audio data of multiple audio data and can only perform The client of the audio data playback operation.
  • the audio data processing server 81 stores the audio data mixing mode of all clients.
  • the audio data processing server 81 receives the audio data request of a certain client, and acquires the audio data mixing mode of the client, such as client mixing or server mixing.
  • the audio data processing server 81 obtains the audio data of the corresponding client according to the audio data request of the client. If the client 82 requests the audio data of the client 83 and the client 84, the audio data processing server according to the audio data of the client 82. Requesting, obtaining audio data from the client 83 and from the client 84.
  • the audio data processing server 81 directly collects and encodes the acquired audio data of the client 83 and the client 84, and encodes the audio. Data is sent to the client 82;
  • the audio data processing server 81 decodes, mixes, and encodes the acquired audio data of the client 83 and the client 84. Processing and transmitting the encoded audio data to the client 82.
  • the client 82 receives the encoded audio data, and performs decoding, mixing, and playback processing;
  • the client 82 receives the encoded audio data, and performs decoding and playback processing.
  • the client 82 can also send a mix mode conversion request to the audio data processing server 81, and convert the audio data mixing mode of the client 82 from the client mix to the server mix, or convert the server mix to the client. Mixing.
  • FIG. 9 is a flowchart of an audio data processing method according to an embodiment of the present invention.
  • the audio data processing method of the embodiment of the present invention may be implemented by using the foregoing electronic device, in particular, an audio data processing client, where the audio data processing method includes:
  • Step S901 receiving remote audio data, where the remote audio data includes a client identifier
  • Step S902 Obtain a client identifier from the off-site audio data, and determine a mix mode of the off-site audio data according to the client identifier.
  • Step S903 Perform playback processing on the off-site mix data according to the mixing mode of the off-site audio data.
  • the audio data processing client receives the off-site audio data, and the off-site audio data may be audio data transmitted from another audio data processing client or an audio data processing server, and specifically may be processed by the client according to the audio data processing. An audio data request is made and audio data from other audio data processing clients is obtained.
  • the off-site audio data includes a client identifier, that is, each audio data processing client sets a local client identifier on the sent local audio data when the local audio data is sent to other clients or servers, so that Other clients are aware of the source of the audio data. If the audio data processing server processes the audio data and then sends it to other clients, the destination client identifier is set on the sent audio data, so that the destination client knows that the audio data passes through the audio data processing server. deal with. Then it proceeds to step S902.
  • step S902 the audio data processing client obtains the client identifier from the acquired off-site audio data in step S901, and determines the mixing manner of the off-site audio data according to the client identifier.
  • the mixing method of the remote audio data may include client mixing and server mixing. Then it proceeds to step S903.
  • step S903 the audio data processing client performs playback processing on the off-site mix data according to the mixing mode of the off-site audio data acquired in step S902. In this way, audio data can be played in the corresponding playback mode for different audio data of different mixing modes.
  • the audio data processing method of the embodiment of the present invention recognizes the mixing manner of the audio data by the client identifier, so that the audio data of different mixing modes can be played.
  • FIG. 10 is a flowchart of a method for processing audio data according to an embodiment of the present invention.
  • the audio data processing method of the embodiment of the present invention may be implemented by using the foregoing electronic device, in particular, an audio data processing client, where the audio data processing method includes:
  • Step S1001 Receive remote audio data, where the remote audio data includes a client identifier
  • Step S1002 Obtain a client identifier from the off-site audio data, and determine a mix mode of the remote audio data according to the client identifier.
  • Step S1003 playing the off-site mix data according to the mixing mode of the off-channel audio data deal with.
  • the audio data processing client receives the off-site audio data, and the off-site audio data may be audio data transmitted from another audio data processing client or an audio data processing server, and specifically may be sent according to the audio data processing client.
  • the audio data is requested, and the audio data from other audio data processing clients is obtained.
  • the off-site audio data includes a client identifier, that is, each audio data processing client sets a local client identifier on the sent local audio data when the local audio data is sent to other clients or servers, so that Other clients are aware of the source of the audio data. If the audio data processing server processes the audio data and then sends it to other clients, the destination client identifier is set on the sent audio data, so that the destination client knows that the audio data passes through the audio data processing server. deal with. Therefore, if the mobile audio data is a client-side mixing mode, the client identifier is set by other audio data processing clients, such as the mobile audio data is a server mixing mode, and the client identifier is set by the audio data processing server. Then it proceeds to step S1002.
  • step S1002 the audio data processing client acquires the client identifier from the acquired off-site audio data in step S1001, and determines the mixing manner of the off-site audio data according to the client identifier.
  • the mixing method of the remote audio data may include client mixing and server mixing.
  • FIG. 11 is a flowchart of step S1002 of the audio data processing method according to the embodiment of the present invention.
  • the step S1002 includes:
  • Step S1101 the audio data processing client determines whether the client identifier is a local client identifier, if it is a local client identifier, then go to step S1102; if it is not a local client identifier, then go to step S1103;
  • Step S1102 because the client identifier is a local client identifier, indicating off-site audio
  • the data is processed by the audio data processing server, so that the off-site audio data is determined to be a server mixing mode;
  • Step S1103 since the client identifier is not the local client identifier, it indicates that the off-site audio data has not been processed by the audio data processing server, and thus the off-site audio data is determined to be the client-side mixing mode. Then it proceeds to step S1003.
  • step S1003 the audio data processing client performs a playback process on the off-site mix data according to the mix mode of the off-site audio data acquired in step S1002. In this way, audio data can be played in the corresponding playback mode for different audio data of different mixing modes.
  • FIG. 12 is a flowchart of step S1003 of the audio data processing method according to an embodiment of the present invention.
  • the step S1003 includes:
  • step S1201 when the audio data is in the server mixing mode, since the audio data has been mixed by the audio data processing server, the audio data processing client only needs to decode and play the off-site audio data.
  • step S1202 when the audio data is in the client mixing mode, since the audio data is not subjected to mixing processing, the audio data processing client needs to decode, mix, and play the multi-channel audio data.
  • the audio data processing method of the embodiment of the present invention identifies the mixing manner of the audio data by setting the local client identifier, thereby mixing the server and the client.
  • the audio data of the two mixing modes is compatible with the playback operation, which further improves the processing compatibility of the audio data of different mixing modes.
  • FIG. 13 is a schematic structural diagram of an audio data processing client according to an embodiment of the present invention.
  • the audio data processing client of the embodiment of the present invention may be implemented by using the audio data processing method described above, the audio data.
  • the processing client 1300 includes an off-site audio data receiving module 1301, a mixing mode determining module 1302, and an audio data playing module 1303.
  • the off-site audio data receiving module 1301 is configured to receive the off-site audio data, wherein the off-site audio data includes a client identifier.
  • the mixing mode determining module 1302 is configured to acquire the client identifier from the foreign audio data, and determine a mixing mode of the remote audio data according to the client identifier.
  • the audio data playing module 1303 is configured to perform playback processing on the off-site mixed data according to the mixing manner of the remote audio data.
  • the remote audio data receiving module 1301 first receives the remote audio data, and the remote audio data may be the audio data transmitted from another audio data processing client or the audio data processing server. Specifically, it may be audio data obtained from other audio data processing clients according to the audio data request sent by the audio data processing client 1300.
  • the off-site audio data includes a client identifier, that is, each audio data processing client sets a local client identifier on the sent local audio data when the local audio data is sent to other clients or servers, so that the other The client knows the source of the audio data. If the audio data processing server processes the audio data and then sends it to other clients, the destination client identifier is set on the sent audio data, so that the destination client knows that the audio data passes through the audio data processing server. deal with.
  • the mixing mode determining module 1302 then acquires the client identifier from the remote audio data receiving module 1301, and determines the mixing mode of the remote audio data according to the client identifier.
  • the mixing method of the remote audio data may include client mixing and server mixing.
  • the final audio data playing module 1303 performs a playback process on the off-site audio data according to the mixing mode of the off-site audio data acquired by the mixing mode determining module 1302. In this way, audio data can be played in the corresponding playback mode for different audio data of different mixing modes.
  • the audio data processing client of the embodiment of the present invention recognizes the mixing manner of the audio data by using the client identifier, so that the audio data of different mixing modes can be played.
  • FIG. 14 is a schematic structural diagram of an audio data processing client according to an embodiment of the present invention.
  • the audio data processing client of this embodiment can be implemented by using the audio data processing method described above, the audio data processing client 1400 includes an off-site audio data receiving module 1401, a mixing mode determining module 1402, an audio data playing module 1403, and an identifier.
  • the module 1404 and the audio data transmitting module 1405 are set.
  • the off-site audio data receiving module 1401 is configured to receive the off-site audio data, wherein the off-site audio data includes a client identifier.
  • the mixing mode determination module 1402 is configured to acquire a client identifier from the foreign audio data, and determine a mixing mode of the remote audio data according to the client identifier.
  • the audio data playing module 1403 is configured to perform playback processing on the off-site mixed data according to the mixing manner of the remote audio data.
  • the identifier setting module 1404 is configured to set a local client identifier on the local audio data.
  • the audio data transmitting module 1405 is configured to send local audio data to other clients or servers.
  • FIG. 15 is a schematic structural diagram of a mixing mode determining module of an audio data processing client according to an embodiment of the present invention.
  • the mixing mode determining module 1402 includes an identifier determining unit 1501, a first mixing mode determining unit 1502, and a second mixing mode determining unit 1503.
  • the identifier judging unit 1501 is configured to determine whether the client identifier is a local client identifier.
  • the first mixing mode determining unit 1502 is configured to determine that the off-site audio data is the server mixing mode if the client identifier is the local client identifier.
  • the second mixing mode determining unit 1503 is configured to determine that the remote audio data is the client mixing mode if the client identifier is a non-local client identifier.
  • FIG. 16 is an audio data of an audio data processing client according to an embodiment of the present invention.
  • the audio data playing module 1403 includes a first audio data playing unit 1601 and a second audio data playing unit 1602.
  • the first audio data playing unit 1601 is configured to decode and perform playback processing of the off-site audio data when the audio data is the server mixing mode.
  • the second audio data playing unit 1602 is configured to decode, mix, and play the off-site audio data when the audio data is the client-side mixing mode.
  • the remote audio data receiving module 1401 first receives the remote audio data, and the remote audio data may be the audio data transmitted from another audio data processing client or the audio data processing server. Specifically, the audio data obtained from other audio data processing clients is obtained according to the audio data request sent by the audio data processing client.
  • the off-site audio data includes a client identifier, that is, when the audio data sending module 1405 of each audio data processing client sends the local audio data to other clients or servers, it is sent by the identifier setting module 1404.
  • the local client identifier is set on the local audio data so that other clients can know the source of the audio data. If the audio data processing server processes the audio data and then sends it to other clients, the destination client identifier is set on the sent audio data, so that the destination client knows that the audio data passes through the audio data processing server. deal with. Therefore, if the mobile audio data is a client-side mixing mode, the client identifier is set by other audio data processing clients, such as the mobile audio data is a server mixing mode, and the client identifier is set by the audio data processing server.
  • the mixing mode determining module 1402 then acquires the client identifier from the remote audio data receiving module 1401, and determines the mixing mode of the remote audio data according to the client identifier.
  • the mixing method of the remote audio data may include client mixing and server mixing.
  • the specific process for determining the mixing method of the audio data in different places includes:
  • the identifier judging unit 1501 of the mixing mode determining module 1401 determines that the client identifier is No as a local client identifier.
  • the first mixing mode determining unit 1502 of the mixing mode determining module 1401 determines that the remote audio data is the server mixing mode.
  • the second mixing mode determining unit 1503 of the mixing mode determining module 1401 determines that the remote audio data is the client mixing mode.
  • the audio data playing module 1403 determines the mixing mode of the off-site audio data acquired by the module according to the mixing mode, and performs playback processing on the off-site mixed data. In this way, audio data can be played in the corresponding playback mode for different audio data of different mixing modes.
  • the specific process includes:
  • the first audio data playing unit 1601 of the audio data playing module 1403 only needs to decode and play the off-site audio data. .
  • the second audio data playing unit 1602 of the audio data playing module 1403 needs to decode, mix, and play the multi-channel audio data. .
  • the audio data processing client in the embodiment of the present invention identifies the mixing manner of the audio data by setting the local client identifier, thereby mixing the server and the client. Mixing the audio data of the two mixing modes for compatible playback operations further improves the processing compatibility of the audio data of different mixing modes.
  • FIG. 17 is a schematic structural diagram of an audio data processing system according to an embodiment of the present invention.
  • the audio data processing system 1700 An audio data processing server 1701 and a plurality of audio data processing clients 1702, 1703, 1704 are included.
  • the specific working principle of the audio data processing client is the same as or similar to the description of the audio data processing client described above. For details, refer to the related description of the audio data processing client.
  • the audio data processing server 1701 in the audio data processing system 1700 may be a server for performing audio data processing or audio data transfer in the related art, such as a call voice server or the like.
  • the audio data processing system of the embodiment of the present invention recognizes the mixing manner of the audio data by the client identifier, so that the audio data of different mixing modes can be played.
  • the audio data processing system of FIG. 17 includes an audio data processing server 1701 and a plurality of audio data processing clients 1702, an audio data processing client 1703, and an audio data processing client 1704, wherein the audio data processing server includes A server that mixes audio data with multiple channels of audio and a server that can only transfer audio data.
  • the workflow of the audio data processing system 1700 of this embodiment includes:
  • the audio data processing server 1701 receives an audio data request of a certain audio data processing client.
  • the audio data processing server 1701 acquires the audio data of the corresponding audio data processing client according to the audio data request of the audio data processing client. For example, the audio data processing client 1702 simultaneously requests the audio data processing client 1703 and the audio data processing client 1704. The audio data processing server 1701 acquires the audio data of the audio data processing client 1703 and the audio data processing client 1704 in accordance with the audio data request of the audio data processing client 1702.
  • the audio data processing server 1701 can decode, mix, and encode the audio data of the audio data processing client 1703 and the audio data processing client 1704, and set an audio data processing client on the generated encoded audio data.
  • the audio data processing server only performs encoding processing on the audio data of the audio data processing client 1703 and the audio data processing client 1704, so that the audio data processing client 1703 of the encoded audio data processing client 1703 still has the audio data processing client 1703.
  • the client identifier, the audio data of the encoded audio data processing client 1704 still has the client identifier of the audio data processing client 1704, such that the audio data is mixed for the client mix.
  • the audio data processing client 1702 After the audio data processing client 1702 receives the audio data sent by the audio data processing server 1701, if the audio identifier of the audio data processing client 1702 is set on the audio data, the audio data is determined to be a server mix. Thus, the audio data is decoded and played back.
  • the audio data processing client 1703 and the client identifier of the audio data processing client 1704 are set on the audio data, it is determined that the audio data is a client mix, so that the audio data is decoded, mixed, and played. deal with.
  • the audio data processing method, the server, the system, and the storage medium of the embodiment of the present invention perform mixing processing according to the audio data mixing manner of the requesting client, and are compatible with audio data of different mixing modes.
  • the audio data processing method, the server and the system of the embodiment of the present invention identify the mixing manner of the audio data by using the client identifier, so that the audio data of different mixing modes can be played; and the existing audio data is solved.
  • the technical problems of handling methods, servers, clients, and users of different mixing modes in the system are difficult to make calls in the same call system.
  • ком ⁇ онент can be, but is not limited to, a process running on a processor, A processor, an object, an executable application, a thread of execution, a program, and/or a computer.
  • a component can be, but is not limited to, a process running on a processor, A processor, an object, an executable application, a thread of execution, a program, and/or a computer.
  • an application running on a controller and the controller can be a component.
  • One or more components can reside within a process and/or thread of execution, and a component can be located on a computer and/or distributed between two or more computers.
  • the claimed subject matter can be implemented as a method, apparatus, or article of manufacture that uses standard programming and/or engineering techniques to produce software, firmware, hardware, or any combination thereof, to control a computer to implement the disclosed subject matter.
  • article of manufacture as used herein is intended to encompass a computer program accessible from any computer-readable device, carrier, or media.
  • Example electronic device 1812 includes, but is not limited to, a personal computer, a server computer, a handheld or laptop device, a mobile device (such as a mobile phone, a personal digital assistant (PDA), a media player, etc.), a multi-processor system, a consumer Electronic devices, small computers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.
  • a personal computer such as a mobile phone, a personal digital assistant (PDA), a media player, etc.
  • PDA personal digital assistant
  • multi-processor system such as a mobile phone, a personal digital assistant (PDA), a media player, etc.
  • consumer Electronic devices small computers, mainframe computers, distributed computing environments including any of the above systems or devices, and the like.
  • Computer readable instructions may be distributed via computer readable media (discussed below).
  • Computer readable instructions may be implemented as program modules, such as functions, objects, application programming interfaces (APIs), data structures, etc. that perform particular tasks or implement particular abstract data types.
  • program modules such as functions, objects, application programming interfaces (APIs), data structures, etc. that perform particular tasks or implement particular abstract data types.
  • APIs application programming interfaces
  • data structures such as lists, etc. that perform particular tasks or implement particular abstract data types.
  • the functionality of the computer readable instructions can be combined or distributed at will in various environments.
  • FIG. 18 illustrates an example of an electronic device 1812 that includes one or more embodiments of an audio data processing server and client of an embodiment of the present invention.
  • electronic device 1812 includes at least one processing unit 1816 and memory 1818.
  • Memory 1818 can be volatile (such as RAM), non-volatile (such as ROM, flash memory, etc.) or some combination of the two. This configuration is illustrated in Figure 18 by dashed line 1814.
  • electronic device 1812 may include additional features and/or functionality.
  • device 1812 may also include additional storage devices (eg, removable and/or non-removable) including, but not limited to, magnetic storage devices, optical storage devices, and the like.
  • additional storage device is illustrated by storage device 1820 in FIG.
  • computer readable instructions for implementing one or more embodiments provided herein may be in storage device 1820.
  • Storage device 1820 can also store other computer readable instructions for implementing an operating system, applications, and the like.
  • Computer readable instructions may be loaded into memory 1818 for execution by, for example, processing unit 1816.
  • Computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions or other data.
  • Memory 1818 and storage device 1820 are examples of computer storage media.
  • Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage device, magnetic tape cassette, magnetic tape, magnetic disk storage device or other magnetic storage device, Or any other medium that can be used to store desired information and that can be accessed by electronic device 1812. Any such computer storage media may be part of the electronic device 1812.
  • Electronic device 1812 may also include a communication connection 1826 that allows electronic device 1812 to communicate with other devices.
  • Communication connection 1826 may include, but is not limited to, a modem, a network interface card (NIC), an integrated network interface, a radio frequency transmitter/receiver, an infrared port, a USB connection, or other interface for connecting electronic device 1812 to other electronic devices.
  • Communication connection 1826 can include a wired connection or a wireless connection.
  • Communication connection 1826 can transmit and/or receive communication media.
  • Computer readable medium can include a communication medium.
  • Communication media typically embodies computer readable instructions or other data in "modulated data signals" such as carrier waves or other transport mechanisms, and includes any information delivery media.
  • modulated data signal can include Such a signal: one or more of the signal characteristics are set or changed in such a manner as to encode the information into the signal.
  • the electronic device 1812 can include an input device 1824, such as a keyboard, mouse, pen, voice input device, touch input device, infrared camera, video input device, and/or any other input device.
  • Output device 1822 may also be included in device 1812, such as one or more displays, speakers, printers, and/or any other output device.
  • Input device 1824 and output device 1822 can be coupled to electronic device 1812 via a wired connection, a wireless connection, or any combination thereof.
  • an input device or output device from another electronic device can be used as input device 1824 or output device 1822 of electronic device 1812.
  • the components of electronic device 1812 can be connected by various interconnects, such as a bus.
  • interconnects may include Peripheral Component Interconnect (PCI) (such as Fast PCI), Universal Serial Bus (USB), Firewire (IEEE 1394), optical bus architecture, and the like.
  • PCI Peripheral Component Interconnect
  • USB Universal Serial Bus
  • Firewire IEEE 1394
  • optical bus architecture and the like.
  • the components of electronic device 1812 may be interconnected by a network.
  • memory 1818 can be comprised of multiple physical memory units that are interconnected by a network located in different physical locations.
  • storage devices for storing computer readable instructions may be distributed across a network.
  • electronic device 1830 accessible via network 1828 can store computer readable instructions for implementing one or more embodiments of the present disclosure.
  • the electronic device 1812 can access the electronic device 1830 and download a portion or all of the computer readable instructions for execution.
  • electronic device 1812 can download a plurality of computer readable instructions as needed, or some of the instructions can be executed at electronic device 1812 and some of the instructions can be executed at electronic device 1830.
  • the one or more operations may constitute computer readable instructions stored on one or more computer readable media that, when executed by an electronic device, cause the computing device to perform the operations.
  • the order in which some or all of the operations are described should not be construed as implying that the operations must be sequential. Those skilled in the art will An alternative ranking with the benefit of this specification is understood. Moreover, it should be understood that not all operations must be present in every embodiment provided herein.
  • the word "preferred” as used herein is intended to serve as an example, instance, or illustration. Any aspect or design described as “preferred” by the text is not necessarily to be construed as being more advantageous than other aspects or designs. Instead, the use of the word “preferred” is intended to present a concept in a specific manner.
  • the term “or” as used in the present invention is intended to mean an inclusive “or” rather than an exclusive “or”. That is, unless otherwise specified or clear from the context, "X employs A or B” means naturally including any one of the permutations. That is, if X uses A; X uses B; or X uses both A and B, then "X uses A or B" is satisfied in any of the foregoing examples.
  • Each functional unit in the present invention may be integrated into one processing module, or each unit may exist physically separately, or two or more units may be integrated into one module.
  • the above integrated modules can be implemented in the form of hardware or in the form of software functional modules.
  • the integrated module is implemented in the form of a software functional module and is sold as a separate product When sold or used, it can also be stored in a computer readable storage medium.
  • the above mentioned storage medium may be a read only memory, a magnetic disk or an optical disk or the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Telephonic Communication Services (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)
  • Signal Processing For Digital Recording And Reproducing (AREA)

Abstract

本发明实施例提供一种音频数据处理方法，包括：接收客户端的音频数据；接收请求客户端的音频数据请求，并根据请求客户端的音频数据请求，获取请求客户端的音频数据混音方式；根据请求客户端的音频数据混音方式，对音频数据请求对应的客户端的音频数据进行预处理；以及将预处理后的音频数据发送至请求客户端，以便请求客户端获取混音处理后的音频数据。本发明实施例还提供一种音频数据处理服务器、客户端以及系统、存储介质，本发明实施例的音频数据处理服务器、客户端以及系统根据请求客户端的音频数据混音方式进行混音处理，可兼容不同混音方式的音频数据。

Description

音频数据处理方法、服务器、客户端以及系统、存储介质 技术领域
本发明涉及音频技术，特别是涉及一种音频数据处理方法、服务器、客户端以及系统、存储介质。
背景技术
在多人通话系统中，由于有多方参与，如何处理多路语音数据的混音问题是一个关键环节。现有的混音方式包括客户端混音以及服务器混音。
客户端混音是指多路语音数据在客户端进行解码以及混音操作。这种混音方式对客户端的带宽以及处理能力要求较高。
服务器混音是指客户端只进行一路语音数据的解码以及播放，通信服务器对多个客户端的语音数据进行解码、混音以及编码操作。这样混音方式对客户端的要求较低，但是由于语音数据经过一次编码以及解码过程，容易引起语音数据的失真以及延迟。
各种用户会根据自己的需要选择相应的混音方式进行语音数据的混音处理，这样导致不同混音方式的用户难以在同一通话系统中进行通话。
发明内容
本发明实施例提供一种可兼容不同混音方式的音频数据处理方法、服务器、客户端以及系统、存储介质；以解决相关技术中采用不同混音方式的用户难以在同一通话系统中进行通话的技术问题。
第一方面，本发明实施例提供一种音频数据处理方法，包括：
接收客户端的音频数据；
接收请求客户端的音频数据请求，并根据所述请求客户端的音频数据 请求，获取所述请求客户端的音频数据混音方式；
根据所述请求客户端的音频数据混音方式，对所述音频数据请求对应的客户端的音频数据进行预处理；以及，
将预处理后的音频数据发送至所述请求客户端，供所述请求客户端基于所述预处理后的音频数据获取混音处理后的音频数据。
第二方面，本发明实施例还提供一种音频数据处理服务器，包括：
音频数据接收模块，配置为接收客户端的音频数据；
混音方式获取模块，配置为接收请求客户端的音频数据请求，并根据所述请求客户端的音频数据请求，获取所述请求客户端的音频数据混音方式；
预处理模块，配置为根据所述请求客户端的音频数据混音方式，对所述音频数据请求对应的客户端的音频数据进行预处理；以及，
发送模块，配置为将预处理后的音频数据发送至所述请求客户端，供所述请求客户端获取混音处理后的音频数据。
第三方面，本发明实施例还提供一种音频数据处理系统，包括上述任一的音频数据处理服务器以及多个客户端。
第四方面，本发明实施例还提供一种音频数据处理方法，包括：
接收异地音频数据，其中所述异地音频数据包括一客户端标识符；
从所述异地音频数据获取所述客户端标识符，根据所述客户端标识符，确定所述异地音频数据的混音方式；以及，
根据所述异地音频数据的混音方式，对所述异地混音数据进行播放处理。
第五方面，本发明实施例还提供一种音频数据处理客户端，包括：
异地音频数据接收模块，配置为接收异地音频数据，其中所述异地音频数据包括所述异地音频数据所来源客户端的客户端标识符；
混音方式确定模块，配置为从所述异地音频数据获取所述客户端标识符，根据所述客户端标识符，确定所述异地音频数据的混音方式；以及，
音频数据播放模块，配置为根据所述异地音频数据的混音方式，对所述异地混音数据进行播放处理。
第六方面，本发明实施例还提供一种音频数据处理系统，包括多个上述的音频数据处理客户端以及至少一个音频数据处理服务器。
第七方面，本发明实施例提供一种音频数据处理服务器，包括：
处理器和存储器，所述处理器中存储有可执行指令，所述可执行指令配置为引起所述处理器执行以下的操作：
接收客户端的音频数据；
接收请求客户端的音频数据请求，并根据所述请求客户端的音频数据请求，获取所述请求客户端的音频数据混音方式；
根据所述请求客户端的音频数据混音方式，对所述音频数据请求对应的客户端的音频数据进行预处理；
将预处理后的音频数据发送至所述请求客户端，供所述请求客户端获取混音处理后的音频数据。
第八方面，本发明实施例提供一种音频数据处理客户端，包括：
处理器和存储器，所述处理器中存储有可执行指令，所述可执行指令配置为引起所述处理器执行以下的操作：
接收异地音频数据，其中所述异地音频数据包括一客户端标识符；
从所述异地音频数据获取所述客户端标识符，根据所述客户端标识符，确定所述异地音频数据的混音方式；以及，
根据所述异地音频数据的混音方式，对所述异地混音数据进行播放处理。
第九方面，本发明实施例提供一种音频数据处理服务器，存储有可执 行指令，所述可执行指令用于执行前述的任一的音频数据处理方法。
第十方面，本发明实施例提供一种存储介质，存储有可执行指令，所述可执行指令用于执行前述的任一所述的音频数据处理方法。
相较于相关技术的音频数据处理方法、服务器、客户端以及系统、存储介质，本发明实施例的音频数据处理方法、服务器以及系统根据请求客户端的音频数据混音方式进行混音处理，可兼容不同混音方式的音频数据。同时本发明实施例的音频数据处理方法、服务器以及系统通过客户端标识符对音频数据的混音方式进行识别，因此可对不同混音方式的音频数据进行播放操作；解决了现有的音频数据处理方法、服务器、客户端以及系统中不同混音方式的用户难以在同一通话系统中进行通话的技术问题。
附图说明
图1为本发明实施例的音频数据处理方法的的流程图；
图2为本发明实施例的音频数据处理方法的流程图；
图3为本发明实施例的音频数据处理方法的步骤S203的流程图；
图4为本发明实施例的音频数据处理服务器的结构示意图；
图5为本发明实施例的音频数据处理服务器的结构示意图；
图6为本发明实施例的音频数据处理服务器的预处理模块的结构示意图；
图7为本发明实施例的音频数据处理服务器的预处理模块的第二音频数据处理单元的结构示意图；
图8为本发明实施例的音频数据处理系统的结构示意图；
图9为本发明实施例的音频数据处理方法的流程图；
图10为本发明实施例的音频数据处理方法的流程图；
图11为本发明实施例的音频数据处理方法的步骤S1002的流程图；
图12为本发明实施例的音频数据处理方法的步骤S1003的流程图；
图13为本发明实施例的音频数据处理客户端的结构示意图；
图14为本发明实施例的音频数据处理客户端的结构示意图；
图15为本发明实施例的音频数据处理客户端的混音方式确定模块的结构示意图；
图16为本发明实施例的音频数据处理客户端的音频数据播放模块的结构示意图；
图17为本发明实施例的音频数据处理系统的结构示意图；
图18为本发明实施例的音频数据处理服务器及客户端所在的电子设备的工作环境结构示意图。
具体实施方式
请参照图式，其中相同的组件符号代表相同的组件，本发明的原理是以实施在一适当的运算环境中来举例说明。以下的说明是基于所例示的本发明具体实施例，其不应被视为限制本发明未在此详述的其它具体实施例。
在以下的说明中，本发明的具体实施例将参考由一部或多部计算机所执行之作业的步骤及符号来说明，除非另有述明。因此，其将可了解到这些步骤及操作，其中有数次提到为由计算机执行，包括了由代表了以一结构化型式中的数据之电子信号的计算机处理单元所操纵。此操纵转换该数据或将其维持在该计算机之内存系统中的位置处，其可重新配置或另外以本领域技术人员所熟知的方式来改变该计算机之运作。该数据所维持的数据结构为该内存之实体位置，其具有由该数据格式所定义的特定特性。但是，本发明实施例原理以上述文字来说明，其并不代表为一种限制，本领域技术人员将可了解到以下所述的多种步骤及操作亦可实施在硬件当中。
本发明实施例的解码装置可使用各种电子设备进行实施，该电子设备包括但不限于个人计算机、服务器计算机、手持式或膝上型设备、移动设备(比如移动电话、个人数字助理(PDA)、媒体播放器等等)、多处理器 系统、消费型电子设备、小型计算机、大型计算机、包括上述任意系统或设备的分布式计算环境，等等。
特别地，当该电子设备为音频数据处理服务器或运行音频数据处理客户端的移动设备，以实现不同混音方式(如服务器混音、移动设备混音)的音频数据的兼容处理。
请参照图1，图1为本发明实施例提供的音频数据处理方法的流程图。本发明实施例的音频数据处理方法可使用上述的电子设备作为实施主体，例如是音频数据处理服务器进行实施，该音频数据处理方法包括：
步骤S101，接收客户端的音频数据；
步骤S102，接收请求客户端的音频数据请求，并根据请求客户端的音频数据请求，获取请求客户端的音频数据混音方式；
步骤S103，根据请求客户端的音频数据混音方式，对音频数据请求对应的客户端的音频数据进行预处理；
步骤S104，将预处理后的音频数据发送至请求客户端，以便请求客户端基于预处理后的音频数据获取混音处理后的音频数据。
下面详细说明本发明实施例的音频数据处理方法的各步骤的具体流程。
在步骤S101中，音频数据处理服务器接收各个客户端的音频数据，该音频数据可为客户端发送给其他客户端的语音数据或音乐数据等。随后转到步骤S102。
在步骤S102中，音频数据处理服务器接收请求客户端的音频数据请求，请求客户端为向电子设备(电子设备作为音频数据处理方法的实施主体)发送音频数据请求以请求接收其他客户端的语音数据或音乐数据的客户端，音频数据请求为请求接收其他客户端的语音数据或音乐数据的请求。接收到客户端发送的音频数据、以及音频数据请求的电子设备，以电子设 备为音频数据处理服务器为例，根据请求客户端的音频数据请求，获取请求客户端的音频数据混音方式。这里的音频数据混音方式可包括客户端混音以及服务器混音等。随后转到步骤S103。
根据对步骤S101以及步骤S102的说明，步骤S101和步骤S102可以采用这样的方式描述：
步骤S101，接收至少两个(包括两个)客户端的音频数据；
步骤S102，接收客户端的音频数据请求(发送音频数据请求的客户端即为请求客户端，可以为步骤S101记载的至少两个客户端中的客户端，也可以为不同于步骤S101中记载的至少两个客户端的客户端)，根据请求客户端的音频数据请求，获取请求客户端的请求的音频数据的混音方式(简称为音频数据类型)。
在步骤S103中，音频数据处理服务器根据步骤S102获取的请求客户端的音频数据混音方式，对音频数据请求对应的客户端(音频数据请求对应的客户端，也即请求客户端通过音频数据请请求所请求的音频数据的来源客户端)的音频数据进行混音预处理。以便发送音频数据请求的客户端可对混音预处理后的音频数据进行正常播放。随后转到步骤S104。
在步骤S104中，音频数据处理服务器将在步骤S103中混音预处理后的音频数据发送至请求客户端，以便请求客户端获取混音处理后的音频数据。即请求客户端获取相应的混音预处理后的音频数据，随后采用相应的音频提取方式获取相应的混音处理后的音频数据，最后对获取的混音处理后的音频数据进行播放操作。
这样即完成了本发明实施例的音频数据处理方法的音频数据处理过程。
本发明实施例的音频数据处理方法根据请求客户端的音频数据混音方式进行混音处理，可兼容不同混音方式的音频数据。
请参照图2，图2为本发明实施例的音频数据处理方法的流程图。本发明实施例的音频数据处理方法可使用上述的电子设备，特别是音频数据处理服务器进行实施，该音频数据处理方法包括：
步骤S201，接收客户端的音频数据；
步骤S202，接收请求客户端的音频数据请求，并根据请求客户端的音频数据请求，获取请求客户端的音频数据混音方式；
步骤S203，根据请求客户端的音频数据混音方式，对音频数据请求对应的客户端的音频数据进行预处理；
步骤S204，将混音预处理后的音频数据发送至所述请求客户端，以便所述请求客户端基于混音预处理后的音频数据获取混音处理后的音频数据。
下面详细说明本发明实施例的音频数据处理方法的各步骤的具体流程。
在步骤S201中，音频数据处理服务器接收各个客户端的音频数据，该音频数据可为客户端发送给其他客户端的语音数据或音乐数据等。随后转到步骤S202。
在步骤S202中，音频数据处理服务器接收请求客户端的音频数据请求，请求客户端为发送音频数据请求以请求接收其他客户端的语音数据或音乐数据的客户端，音频数据请求为请求接收其他客户端的语音数据或音乐数据的请求。接收到客户端发送的音频数据、以及音频数据请求的电子设备，以音频数据处理服务器为例，根据请求客户端的音频数据请求，获取请求客户端的音频数据混音方式。这里的音频数据混音方式可包括客户端混音以及服务器混音等。
这里的音频数据混音方式包括客户端混音以及服务器混音；其中，客户端混音是指，由接收到音频数据的客户端作为音频数据处理客户端，进 行多个音频数据的混音操作，然后音频数据处理客户端对混音操作后的音频数据进行播放操作；服务器混音是指，由音频数据处理服务器进行多个音频数据的混音操作，然后音频数据处理服务器将混音操作后的音频数据发送至音频数据处理客户端，音频数据客户端仅作为对音频数据进行播放的客户端，由音频数据处理客户端进行播放操作。随后转到步骤S203。
在步骤S203中，音频数据处理服务器根据步骤S202获取的请求客户端的音频数据混音方式，对音频数据请求对应的客户端的音频数据进行预处理。
具体为：
如音频数据混音方式为客户端混音时，音频数据处理服务器直接对音频数据请求对应的客户端的音频数据进行采集操作；这里采集音频数据后，并不对音频数据进行任何处理。
如音频数据混音方式为服务器混音时，音频数据处理服务器对音频数据请求对应的客户端的音频数据依次进行解码、混音以及编码处理。具体请参照图3，图3为本发明实施例的音频数据处理方法的步骤S203的流程图。该步骤S203包括：
步骤S301，音频数据处理服务器根据音频数据的丢包情况，对所有客户端的音频数据进行解码处理。
即音频数据处理服务器接收到音频数据码流后，确定该音频数据的丢包情况，如没有丢包的情况下，则直接使用相应的解码器对该音频数据进行解码，得到解码处理后的音频数据。如具有丢包的情况下，则使用前向纠错(FEC，Forward Error Correction)冗余包恢复丢失的语音包，再使用相应的解码器对该恢复后的音频数据进行解码，得到解码处理后的音频数据。如无法对丢包进行恢复，则使用丢包补偿算法直接计算丢包前的语音包，再使用相应的解码器对计算出的音频数据进行解码。随后转到步骤 S302。
步骤S302，音频数据处理服务器从步骤S301获取的解码处理后的音频数据中，获取音频数据请求对应的客户端的音频数据。
音频数据处理服务器可以将所接收的来自各个客户端的音频数据均存放在一数据缓冲区中，以备随时调用。这样音频数据处理服务器可根据音频数据请求获取相应的客户端的音频数据。随后转到步骤S303。
步骤S303，音频数据处理服务器将音频数据请求对应的客户端的音频数据转换为预设格式的音频数据。由于每个客户端的音频数据的格式可能与编码器的要求(如，与编码器支持进行编码的音频数据的格式)不一致，因此这里对所有的音频数据的格式进行统一转换，均转换为预设格式的音频数据，以便后续对应音频数据进行编码操作。随后转到步骤S304。
步骤S304，音频数据处理服务器对步骤S303获取的多路预设格式的音频数据(来自音频数据请求的对应的客户端的音频数据)进行混音处理，以生成一路混音处理后的音频数据。随后转到步骤S305。
步骤S305，音频数据处理服务器根据当前与客户端之间的网络情况，对步骤S304获取的混音处理后的音频数据进行压缩编码处理。例如，音频数据处理服务器根据网络丢包以及网络延迟抖动情况，对混音处理后的音频数据进行FEC编码，以生成相应的音频数据编码包，对音频数据编码包进行网络安全传输。传输完成后，随后转到步骤S204。
在步骤S204中，音频数据处理服务器将在步骤S203中获取的音频数据编码包发送至请求客户端，以便请求客户端获取混音处理后的音频数据。
如请求客户端的音频数据混音方式为客户端混音，则请求客户端获取的预处理后的音频数据为未经混音处理的多路音频数据，这样，由请求客户端对所接收的上述多路音频数据进行混音，以及，对混音处理后的音频数据进行播放处理。
如请求客户端的音频数据混音方式为服务器混音，则请求客户端获取的预处理后的音频数据为音频数据编码包，则请求客户端可直接对音频数据编码包进行解码以及播放处理。
这样即完成了本发明实施例的音频数据处理方法的音频数据处理过程。
优选的，本发明实施例的音频数据处理方法还包括步骤：音频数据处理服务器接收请求客户端的混音方式转换请求，并根据该请求客户端的混音方式转换请求，设定请求客户端的音频数据混音方式。
即音频数据处理服务器可根据请求客户端的混音方式转换请求对请求客户端的音频数据混音方式进行转换，如将客户端混音方式的请求客户端转换为服务器混音方式的请求客户端等。
此外，本发明实施例实施例的音频数据处理方法会对音频数据进行编码处理，以便对音频数据进行网络安全传输；同时还可根据请求客户端的混音方式转换请求对客户端的音频数据混音方式进行及时切换，进一步提高了不同混音方式的音频数据的传输安全性以及处理兼容性。
本发明实施例还提供一种音频数据处理服务器，请参照图4，图4为本发明实施例的音频数据处理服务器的结构示意图。本发明实施例的音频数据处理服务器可使用上述的音频数据处理方法的记载而进行实施。该音频数据处理服务器40包括音频数据接收模块41、混音方式获取模块42、混音预处理模块43以及发送模块44。
音频数据接收模块41配置为接收客户端的音频数据。混音方式获取模块42配置为接收请求客户端的音频数据请求，并根据请求客户端的音频数据请求，获取请求客户端的音频数据混音方式。预处理模块43配置为根据请求客户端的音频数据混音方式，对音频数据请求对应的客户端的音频数据进行预处理。发送模块44配置为将预处理后的音频数据发送至请求客户 端，以便请求客户端获取混音处理后的音频数据。
本发明实施例的音频数据处理服务器40使用时，首先音频数据接收模块41接收各个客户端的音频数据，该音频数据可为客户端发送给其他客户端的语音数据或音乐数据等。
随后混音方式获取模块42接收请求客户端的音频数据请求，请求客户端为请求接收其他客户端的语音数据或音乐数据的客户端，音频数据请求为请求接收其他客户端的语音数据或音乐数据的请求。随后混音方式获取模块42根据请求客户端的音频数据请求，获取请求客户端的音频数据混音方式。这里的音频数据混音方式可包括客户端混音以及服务器混音等。
然后预处理模块43根据混音方式获取模块42获取的请求客户端的音频数据混音方式，对音频数据请求对应的客户端的音频数据进行预处理。以便相应的客户端可对预处理后的音频数据进行正常播放。
最后发送模块44将在混音预处理模块43中预处理后的音频数据发送至请求客户端，以便请求客户端获取混音处理后的音频数据。即请求客户端获取相应的预处理后的音频数据，随后采用相应的音频提取方式，基于预处理后的音频数据获取相应的混音处理后的音频数据，最后对获取的混音处理后的音频数据进行播放操作。
这样即完成了本发明实施例的音频数据处理服务器40的音频数据处理过程。
本发明实施例的音频数据处理服务器根据请求客户端的音频数据混音方式进行混音处理，可兼容不同混音方式的音频数据。
请参照图5，图5为本发明实施例的音频数据处理服务器的结构示意图。本发明实施例的音频数据处理服务器可基于上述记载的音频数据处理方法而进行实施。该音频数据处理服务器50包括音频数据接收模块51、混音方式获取模块52、混音预处理模块53、发送模块54以及混音方式转换模块 55。
音频数据接收模块51配置为接收客户端的音频数据。混音方式获取模块52配置为接收请求客户端的音频数据请求，并根据请求客户端的音频数据请求，获取请求客户端的音频数据混音方式。混音预处理模块53配置为根据请求客户端的音频数据混音方式，对音频数据请求对应的客户端的音频数据进行混音预处理。发送模块54配置为将预处理后的音频数据发送至请求客户端，以便请求客户端基于混音处理后的音频数据获取混音处理后的音频数据。混音方式转换模块55配置为接收请求客户端的混音方式转换请求，并根据请求客户端的混音方式转换请求设定请求客户端的音频数据混音方式。
请参照图6，图6为本发明实施例的音频数据处理服务器的预处理模块的结构示意图。该混音预处理模块53包括第一音频数据处理单元61以及第二音频数据处理单元62。
第一音频数据处理单元61配置为如请求客户端的音频数据混音方式为客户端混音，则对音频数据请求对应的客户端的音频数据进行采集处理。第二音频数据处理单元62配置为如请求客户端的音频数据混音方式为服务器混音，则对音频数据请求对应的客户端的音频数据依次进行解码、混音以及编码处理。
请参照图7，图7为本发明实施例的音频数据处理服务器的混音预处理模块的第二音频数据处理单元的结构示意图。该第二音频数据处理单元62包括解码子单元71、音频数据获取子单元72、格式转换子单元73、混音处理子单元74以及压缩编码子单元75。
解码子单元71配置为根据音频数据的丢包情况，对所有客户端的音频数据进行解码处理。音频数据获取子单元72配置为从解码处理后的音频数据中获取音频数据请求对应的客户端的音频数据。格式转换子单元73配置 为将音频数据请求对应的客户端的音频数据转换为预设格式的音频数据。混音处理子单元74配置为对预设格式的音频数据进行混音处理。压缩编码子单元75配置为根据当前网络情况，对混音处理后的音频数据进行压缩编码处理。
本发明实施例的音频数据处理服务器50使用时，首先音频数据接收模块51接收各个客户端的音频数据，该音频数据可为客户端发送给其他客户端的语音数据或音乐数据等。
随后混音方式获取模块52接收请求客户端的音频数据请求，请求客户端为请求接收其他客户端的语音数据或音乐数据的客户端，音频数据请求为请求接收其他客户端的语音数据或音乐数据的请求。随后混音方式获取模块52根据请求客户端的音频数据请求，获取请求客户端的音频数据混音方式。
这里的音频数据混音方式包括客户端混音以及服务器混音。客户端混音为音频数据处理客户端进行多个音频数据的混音操作，然后音频数据处理客户端对混音操作后的音频数据进行播放操作。服务器混音为音频数据处理服务器进行多个音频数据的混音操作，然后服务器将混音操作后的音频数据发送至音频数据处理客户端，由音频数据处理客户端进行播放操作。
然后预处理模块53根据混音方式获取模块52获取的请求客户端的音频数据混音方式，对音频数据请求对应的客户端的音频数据进行预处理。
具体为如音频数据混音方式为客户端混音时，混音预处理模块53的第一音频数据处理单元61对则直接对音频数据请求对应的客户端的音频数据进行采集操作；这里采集音频数据，并不对音频数据进行任何处理。
如音频数据混音方式为服务器混音时，预处理模块53的第二音频数据处理单元62则对音频数据请求对应的客户端的音频数据依次进行解码、混音以及编码处理。该解码、混音以及编码处理的过程具体为：
第二音频数据处理单元62的解码子单元71根据音频数据的丢包情况，对所有客户端的音频数据进行解码处理。
即解码子单元71接收到音频数据码流后，确定该音频数据的丢包情况，如没有丢包的情况下，则直接使用相应的解码器对该音频数据进行解码，得到解码处理后的音频数据。如具有丢包的情况下，则使用FEC冗余包恢复丢失的语音包，再使用相应的解码器对该恢复后的音频数据进行解码，得到解码处理后的音频数据。如无法对丢包进行恢复，则使用丢包补偿算法直接计算丢包前的语音包，再使用相应的解码器对计算出的音频数据进行解码。
第二音频数据处理单元62的音频数据获取子单元72从解码子单元71获取的解码处理后的音频数据中获取音频数据请求对应的客户端的音频数据。这里音频数据处理服务器可以将每个客户端的音频数据均存放在数据缓冲区中，以备随时调用。这样音频数据处理服务器可根据音频数据请求获取相应的客户端的音频数据。
第二音频数据处理单元62的格式转换子单元73将音频数据请求对应的客户端的音频数据转换为预设格式的音频数据。由于每个客户端的音频数据的格式可能与编码器的要求不一致，因此这里对所有的音频数据的格式进行统一转换，均转换为预设格式的音频数据，以便后续对应音频数据进行编码操作。
第二音频数据处理单元62的混音处理子单元74对格式转换子单元73获取的多路预设格式的音频数据进行混音处理，以生成一路混音处理后的音频数据。
第二音频数据处理单元62的压缩编码子单元75根据当前网络情况，对混音处理子单元74获取的混音处理后的音频数据进行压缩编码处理。即音频数据处理服务器根据网络丢包以及网络延迟抖动情况，对混音处理后 的音频数据进行FEC编码，以生成相应的音频数据编码包，以便对音频数据进行网络安全传输。
最后混音方式转换模块54将在发送模块53获取的音频数据编码包发送至请求客户端，以便请求客户端获取混音处理后的音频数据。
如请求客户端的音频数据混音方式为客户端混音，则请求客户端获取的预处理后的音频数据为未经处理的多路音频数据，这样请求客户端对上述多路音频数据进行混音以及播放处理。
如请求客户端的音频数据混音方式为服务器混音，则请求客户端获取的预处理后的音频数据为音频数据编码包，则请求客户端可直接对音频数据编码包进行解码以及播放处理。
这样即完成了本实施例的音频数据处理服务器50的音频数据处理过程。
可选地，本发明实施例的音频数据处理服务器50的混音方式转换模块55还可接收请求客户端的混音方式转换请求，并根据该请求客户端的混音方式转换请求，设定请求客户端的音频数据混音方式。
即混音方式转换模块55可根据请求客户端的混音方式转换请求对请求客户端的音频数据混音方式进行转换，如将客户端混音方式的请求客户端转换为服务器混音方式的请求客户端等。
在本发明前述记载的基础上，本发明实施例的音频数据处理服务器会对音频数据进行编码处理，以便对音频数据进行网络安全传输；同时还可根据请求客户端的混音方式转换请求对客户端的音频数据混音方式进行及时切换，进一步提高了不同混音方式的音频数据的传输安全性以及处理兼容性。
本发明实施例还提供一种音频数据处理系统，请参照图8，图8为本发明实施例的音频数据处理系统的结构示意图。该音频数据处理系统80包括 音频数据处理服务器81以及多个客户端，例如客户端82、客户端83和客户端84。
该音频数据处理服务器81的具体工作原理与本发明实施例上述的音频数据处理服务器的记载描述相同或相似，具体请参见上述音频数据处理服务器的相关描述。该音频数据处理系统80中的客户端可为相关技术中的用于进行音频数据处理的客户端，如通话语音客户端等。
本发明实施例的音频数据处理系统根据请求客户端的音频数据混音方式进行混音处理，可兼容不同混音方式的音频数据。
下面通过一具体示例对本发明实施例的音频数据处理系统的具体工作原理。请参照图8，图8的音频数据处理系统包括音频数据处理服务器81以及多个客户端82、83、84，其中客户端包括可进行多路音频数据的混音操作的客户端以及只能进行一路音频数据的播放操作的客户端。
本具体实施例的音频数据处理系统的工作流程包括：
一、音频数据处理服务器81保存所有客户端的音频数据混音方式。
二、音频数据处理服务器81接收某个客户端的音频数据请求，并获取该客户端的音频数据混音方式，如客户端混音或服务器混音。
三、音频数据处理服务器81根据客户端的音频数据请求，获取相应的客户端的音频数据，如客户端82请求客户端83和客户端84的音频数据，则音频数据处理服务器根据客户端82的音频数据请求，获取来自客户端83和来自客户端84的音频数据。
四、如客户端82的音频数据混音方式为客户端混音，则音频数据处理服务器81直接对获取的客户端83和客户端84的音频数据进行采集以及编码处理，并将编码后的音频数据发送至客户端82；
如客户端82的音频数据混音方式为服务器混音，则音频数据处理服务器81对获取的客户端83和客户端84的音频数据进行解码、混音以及编码 处理，并将编码后的音频数据发送至客户端82。
五、如客户端82的音频数据混音方式为客户端混音，则客户端82接收到编码后的音频数据后，进行解码、混音以及播放处理；
如客户端82的音频数据混音方式为服务器混音，则客户端82接收到编码后的音频数据后，进行解码以及播放处理。
六、客户端82还可向音频数据处理服务器81发送混音方式转换请求，将客户端82的音频数据混音方式由客户端混音转换为服务器混音，或由服务器混音转换为客户端混音。
这样即完成了音频数据处理系统80的音频数据处理过程。
本发明实施例还提供一种音频数据处理方法，请参照图9，图9为本发明实施例的音频数据处理方法的的流程图。本发明实施例的音频数据处理方法可使用上述的电子设备，特别是音频数据处理客户端进行实施，该音频数据处理方法包括：
步骤S901，接收异地音频数据，其中异地音频数据包括一客户端标识符；
步骤S902，从异地音频数据获取客户端标识符，并根据客户端标识，确定异地音频数据的混音方式；
步骤S903，根据异地音频数据的混音方式，对异地混音数据进行播放处理。
下面详细说明本发明实施例的音频数据处理方法的各步骤的具体流程。
在步骤S901中，音频数据处理客户端接收异地音频数据，该异地音频数据可为从其他音频数据处理客户端或音频数据处理服务器上传输来的音频数据，具体可为根据音频数据处理客户端通过发出音频数据请求，而获取的来自其他音频数据处理客户端的音频数据。
该异地音频数据包括一客户端标识符，即每个音频数据处理客户端将本地音频数据发送至其他客户端或服务器时，均会在发送出去的本地音频数据上设置本地客户端标识符，以便其他客户端获知该音频数据的来源。如音频数据处理服务器对音频数据进行处理后再发送至其他客户端，则会在发送出去的音频数据上设置目的地客户端标识符，以便目的地客户端获知该音频数据经过音频数据处理服务器的处理。随后转到步骤S902。
在步骤S902中，音频数据处理客户端从步骤S901中获取异地音频数据中获取客户端标识符，并根据该客户端标识符，确定异地音频数据的混音方式。这里异地音频数据的混音方式可包括客户端混音以及服务器混音等。随后转到步骤S903。
在步骤S903中，音频数据处理客户端根据步骤S902获取的异地音频数据的混音方式，对异地混音数据进行播放处理。这样即可针对不同的混音方式的异地音频数据采用相应的播放方式进行音频数据的播放。
这样即完成了本发明实施例的音频数据处理方法的音频数据处理过程。
本发明实施例的音频数据处理方法通过客户端标识符对音频数据的混音方式进行识别，因此可对不同混音方式的音频数据进行播放操作。
请参照图10，图10为本发明实施例的音频数据处理方法的流程图。本发明实施例的音频数据处理方法可使用上述电子设备，特别是音频数据处理客户端进行实施，该音频数据处理方法包括：
步骤S1001，接收异地音频数据，其中异地音频数据包括一客户端标识符；
步骤S1002，从异地音频数据获取客户端标识符，并根据客户端标识，确定异地音频数据的混音方式；
步骤S1003，根据异地音频数据的混音方式，对异地混音数据进行播放 处理。
下面详细说明本发明实施例的音频数据处理方法的各步骤的具体流程。
在步骤S1001中，音频数据处理客户端接收异地音频数据，该异地音频数据可为从其他音频数据处理客户端或音频数据处理服务器传输来的音频数据，具体可为根据音频数据处理客户端发出的音频数据请求，而获取的来自其他音频数据处理客户端的音频数据。
该异地音频数据包括一客户端标识符，即每个音频数据处理客户端将本地音频数据发送至其他客户端或服务器时，均会在发送出去的本地音频数据上设置本地客户端标识符，以便其他客户端获知该音频数据的来源。如音频数据处理服务器对音频数据进行处理后再发送至其他客户端，则会在发送出去的音频数据上设置目的地客户端标识符，以便目的地客户端获知该音频数据经过音频数据处理服务器的处理。因此如移动音频数据为客户端混音方式，则客户端标识符由其他音频数据处理客户端设置，如移动音频数据为服务器混音方式，则客户端标识符由音频数据处理服务器设置。随后转到步骤S1002。
在步骤S1002中，音频数据处理客户端从步骤S1001中获取异地音频数据中获取客户端标识符，并根据该客户端标识符，确定异地音频数据的混音方式。这里异地音频数据的混音方式可包括客户端混音以及服务器混音等。确定异地音频数据的混音方式的具体步骤请参照图11，图11为本发明实施例的音频数据处理方法的步骤S1002的流程图。该步骤S1002包括：
步骤S1101，音频数据处理客户端判断客户端标识符是否为本地客户端标识符，如为本地客户端标识符，则转到步骤S1102；如不是本地客户端标识符，则转到步骤S1103；
步骤S1102，由于客户端标识符为本地客户端标识符，则说明异地音频 数据经过音频数据处理服务器的处理，因此确定该异地音频数据为服务器混音方式；
步骤S1103，由于客户端标识符不是本地客户端标识符，则说明异地音频数据没有经过音频数据处理服务器的处理，因此确定该异地音频数据为客户端混音方式。随后转到步骤S1003。
在步骤S1003中，音频数据处理客户端根据步骤S1002获取的异地音频数据的混音方式，对异地混音数据进行播放处理。这样即可针对不同的混音方式的异地音频数据采用相应的播放方式进行音频数据的播放。具体请参照图12，图12为本发明实施例的音频数据处理方法的步骤S1003的流程图。该步骤S1003包括：
步骤S1201，当音频数据为服务器混音方式时，由于该音频数据已经由音频数据处理服务器进行混音处理，因此音频数据处理客户端只需要对异地音频数据进行解码以及播放处理。
步骤S1202，当音频数据为客户端混音方式时，由于该音频数据没有进行混音处理，因此音频数据处理客户端需要对多路异地音频数据进行解码、混音以及播放处理。
这样即完成了本发明实施例的音频数据处理方法的音频数据处理过程。
在发明实施例前述记载的技术方案的基础上，本发明实施例的音频数据处理方法通过本地客户端标识符的设置，对音频数据的混音方式进行识别，从而对服务器混音和客户端混音两种混音方式的音频数据进行兼容性的播放操作，进一步提高了不同混音方式的音频数据的处理兼容性。
本发明实施例还提供一种音频数据处理客户端，请参照图13，图13为本发明实施例的音频数据处理客户端的结构示意图。本发明实施例的音频数据处理客户端可使用上述的音频数据处理方法的进行实施，该音频数据 处理客户端1300包括异地音频数据接收模块1301、混音方式确定模块1302以及音频数据播放模块1303。
异地音频数据接收模块1301配置为接收异地音频数据，其中异地音频数据包括一客户端标识符。混音方式确定模块1302配置为从异地音频数据获取所述客户端标识符，根据客户端标识符，确定异地音频数据的混音方式。音频数据播放模块1303配置为根据异地音频数据的混音方式，对异地混音数据进行播放处理。
本发明实施例的音频数据处理客户端1300使用时，首先异地音频数据接收模块1301接收异地音频数据，该异地音频数据可为从其他音频数据处理客户端或音频数据处理服务器上传输来的音频数据，具体可为根据音频数据处理客户端1300发出的音频数据请求，而获取的来自其他音频数据处理客户端的音频数据。
该异地音频数据包括一客户端标识符，即每个音频数据处理客户端将本地音频数据发送至其他客户端或服务器时，均会在发送的本地音频数据上设置本地客户端标识符，以便其他客户端获知该音频数据的来源。如音频数据处理服务器对音频数据进行处理后再发送至其他客户端，则会在发送出去的音频数据上设置目的地客户端标识符，以便目的地客户端获知该音频数据经过音频数据处理服务器的处理。
随后混音方式确定模块1302从异地音频数据接收模块1301获取异地音频数据中获取客户端标识符，并根据该客户端标识符，确定异地音频数据的混音方式。这里异地音频数据的混音方式可包括客户端混音以及服务器混音等。
最后音频数据播放模块1303根据混音方式确定模块1302获取的异地音频数据的混音方式，对异地混音数据进行播放处理。这样即可针对不同的混音方式的异地音频数据采用相应的播放方式进行音频数据的播放。
这样即完成了本发明实施例的音频数据处理客户端1300的音频数据处理过程。
本发明实施例的音频数据处理客户端通过客户端标识符对音频数据的混音方式进行识别，因此可对不同混音方式的音频数据进行播放操作。
请参照图14，图14为本发明实施例的音频数据处理客户端的结构示意图。本实施例的音频数据处理客户端可使用上述的音频数据处理方法进行实施，该音频数据处理客户端1400包括异地音频数据接收模块1401、混音方式确定模块1402、音频数据播放模块1403、标识符设定模块1404以及音频数据发送模块1405。
异地音频数据接收模块1401配置为接收异地音频数据，其中异地音频数据包括一客户端标识符。混音方式确定模块1402配置为从异地音频数据获取客户端标识符，根据客户端标识符，确定异地音频数据的混音方式。音频数据播放模块1403配置为根据异地音频数据的混音方式，对异地混音数据进行播放处理。标识符设定模块1404配置为在本地音频数据上设置本地客户端标识符。音频数据发送模块1405配置为将本地音频数据发送至其他客户端或服务器。
请参照图15，图15为本发明实施例的音频数据处理客户端的混音方式确定模块的结构示意图。该混音方式确定模块1402包括标识符判断单元1501、第一混音方式确定单元1502以及第二混音方式确定单元1503。
标识符判断单元1501配置为判断客户端标识符是否为本地客户端标识符。第一混音方式确定单元1502配置为如客户端标识符为本地客户端标识符，则确定异地音频数据为服务器混音方式。第二混音方式确定单元1503配置为如客户端标识符为非本地客户端标识，则确定异地音频数据为客户端混音方式。
请参照图16，图16为本发明实施例的音频数据处理客户端的音频数据 播放模块的结构示意图。该音频数据播放模块1403包括第一音频数据播放单元1601以及第二音频数据播放单元1602。
第一音频数据播放单元1601配置为当音频数据为服务器混音方式时，对异地音频数据进行解码以及播放处理。第二音频数据播放单元1602配置为当音频数据为客户端混音方式时，对异地音频数据进行解码、混音以及播放处理。
本发明实施例的音频数据处理客户端1400使用时，首先异地音频数据接收模块1401接收异地音频数据，该异地音频数据可为从其他音频数据处理客户端或音频数据处理服务器上传输来的音频数据，具体可为根据音频数据处理客户端发出的音频数据请求，而获取的来自其他音频数据处理客户端的音频数据。
该异地音频数据包括一客户端标识符，即每个音频数据处理客户端的音频数据发送模块1405将本地音频数据发送至其他客户端或服务器时，均会通过标识符设定模块1404在发送出去的本地音频数据上设置本地客户端标识符，以便其他客户端获知该音频数据的来源。如音频数据处理服务器对音频数据进行处理后再发送至其他客户端，则会在发送出去的音频数据上设置目的地客户端标识符，以便目的地客户端获知该音频数据经过音频数据处理服务器的处理。因此如移动音频数据为客户端混音方式，则客户端标识符由其他音频数据处理客户端设置，如移动音频数据为服务器混音方式，则客户端标识符由音频数据处理服务器设置。
随后混音方式确定模块1402从异地音频数据接收模块1401获取异地音频数据中获取客户端标识符，并根据该客户端标识符，确定异地音频数据的混音方式。这里异地音频数据的混音方式可包括客户端混音以及服务器混音等。确定异地音频数据的混音方式具体流程包括：
混音方式确定模块1401的标识符判断单元1501判断客户端标识符是 否为本地客户端标识符。
如为本地客户端标识符，则说明异地音频数据经过音频数据处理服务器的处理，因此混音方式确定模块1401的第一混音方式确定单元1502确定该异地音频数据为服务器混音方式。
如不是本地客户端标识符，则说明异地音频数据没有经过音频数据处理服务器的处理，因此混音方式确定模块1401的第二混音方式确定单元1503确定该异地音频数据为客户端混音方式。
然后音频数据播放模块1403根据混音方式确定模块获取的异地音频数据的混音方式，对异地混音数据进行播放处理。这样即可针对不同的混音方式的异地音频数据采用相应的播放方式进行音频数据的播放。具体流程包括：
当音频数据为服务器混音方式时，由于该音频数据已经由音频数据处理服务器进行混音处理，因此音频数据播放模块1403的第一音频数据播放单元1601只需要对异地音频数据进行解码以及播放处理。
当音频数据为客户端混音方式时，由于该音频数据没有进行混音处理，因此音频数据播放模块1403的第二音频数据播放单元1602需要对多路异地音频数据进行解码、混音以及播放处理。
这样即完成了本发明实施例的音频数据处理客户端1400的音频数据处理过程。
在发明实施例前述记载的技术方案的基础上，本发明实施例的音频数据处理客户端通过本地客户端标识符的设置，对音频数据的混音方式进行识别，从而对服务器混音和客户端混音两种混音方式的音频数据进行兼容性的播放操作，进一步提高了不同混音方式的音频数据的处理兼容性。
本发明实施例还提供一种音频数据处理系统，请参照图17，图17为本发明实施例的音频数据处理系统的结构示意图。该音频数据处理系统1700 包括音频数据处理服务器1701以及多个音频数据处理客户端1702、1703、1704。
该音频数据处理客户端的具体工作原理与上述的音频数据处理客户端的描述相同或相似，具体请参见上述音频数据处理客户端的的相关描述。该音频数据处理系统1700中的音频数据处理服务器1701可为相关技术中的用于进行音频数据处理或音频数据转送的服务器，如通话语音服务器等。
本发明实施例的音频数据处理系统通过客户端标识符对音频数据的混音方式进行识别，因此可对不同混音方式的音频数据进行播放操作。
下面通过一具体示例说明本发明实施例的音频数据处理系统的具体工作原理。请参照图17，图17的音频数据处理系统包括音频数据处理服务器1701以及多个音频数据处理客户端1702、音频数据处理客户端1703和音频数据处理客户端1704，其中音频数据处理服务器包括可进行多路音频数据的混音操作的服务器以及只能进行音频数据转送的服务器。
本具体实施例的音频数据处理系统1700的工作流程包括：
一、音频数据处理服务器1701接收某个音频数据处理客户端的音频数据请求。
二、音频数据处理服务器1701根据音频数据处理客户端的音频数据请求，获取相应的音频数据处理客户端的音频数据，如音频数据处理客户端1702同时请求音频数据处理客户端1703和音频数据处理客户端1704的音频数据，则音频数据处理服务器1701根据音频数据处理客户端1702的音频数据请求，获取音频数据处理客户端1703和音频数据处理客户端1704的音频数据。
三、如音频数据处理服务器1701可对音频数据处理客户端1703和音频数据处理客户端1704的音频数据进行解码、混音以及编码处理，同时在生成编码后的音频数据上设置一音频数据处理客户端1702的客户端标识 符，这样该音频数据的混音方式为服务器混音；
如音频数据处理服务器仅对音频数据处理客户端1703和音频数据处理客户端1704的音频数据分别进行编码处理，这样编码后的音频数据处理客户端1703的音频数据上依旧具有音频数据处理客户端1703的客户端标识符，编码后的音频数据处理客户端1704的音频数据上依旧具有音频数据处理客户端1704的客户端标识符，这样该音频数据的混音方式为客户端混音。
四、音频数据处理客户端1702接收到音频数据处理服务器1701发送的音频数据后，如该音频数据上设置的是音频数据处理客户端1702的客户端标识符，则判断该音频数据为服务器混音，这样对该音频数据进行解码以及播放处理。
如该音频数据上设置的是音频数据处理客户端1703和音频数据处理客户端1704的客户端标识符，则判断该音频数据为客户端混音，这样对该音频数据进行解码、混音以及播放处理。
这样即完成了本具体实施例的音频数据处理系统1700的音频数据处理过程。
本发明实施例的音频数据处理方法、服务器以及系统、存储介质，根据请求客户端的音频数据混音方式进行混音处理，可兼容不同混音方式的音频数据。同时本发明实施例的音频数据处理方法、服务器以及系统通过客户端标识符对音频数据的混音方式进行识别，因此可对不同混音方式的音频数据进行播放操作；解决了现有的音频数据处理方法、服务器、客户端以及系统中不同混音方式的用户难以在同一通话系统中进行通话的技术问题。
如本发明实施例所使用的术语“组件”、“模块”、“系统”、“接口”、“进程”等等一般地旨在指计算机相关实体：硬件、硬件和软件的组合、软件或执行中的软件。例如，组件可以是但不限于是运行在处理器上的进程、处 理器、对象、可执行应用、执行的线程、程序和/或计算机。通过图示，运行在控制器上的应用和该控制器二者都可以是组件。一个或多个组件可以有在于执行的进程和/或线程内，并且组件可以位于一个计算机上和/或分布在两个或更多计算机之间。
而且，要求保护的主题可以被实现为使用标准编程和/或工程技术产生软件、固件、硬件或其任意组合以控制计算机实现所公开的主题的方法、装置或制造品。本文所使用的术语“制造品”旨在包含可从任意计算机可读设备、载体或介质访问的计算机程序。当然，本领域技术人员将认识到可以对该配置进行许多修改，而不脱离要求保护的主题的范围或精神。
图18和随后的讨论提供了对实现本发明所述的音频数据处理服务器及客户端所在的电子设备的工作环境的简短、概括的描述。图18的工作环境仅仅是适当的工作环境的一个实例并且不旨在建议关于工作环境的用途或功能的范围的任何限制。实例电子设备1812包括但不限于个人计算机、服务器计算机、手持式或膝上型设备、移动设备(比如移动电话、个人数字助理(PDA)、媒体播放器等等)、多处理器系统、消费型电子设备、小型计算机、大型计算机、包括上述任意系统或设备的分布式计算环境，等等。
尽管没有要求，但是在“计算机可读指令”被一个或多个电子设备执行的通用背景下描述实施例。计算机可读指令可以经由计算机可读介质来分布(下文讨论)。计算机可读指令可以实现为程序模块，比如执行特定任务或实现特定抽象数据类型的功能、对象、应用编程接口(API)、数据结构等等。典型地，该计算机可读指令的功能可以在各种环境中随意组合或分布。
图18图示了包括本发明实施例的音频数据处理服务器及客户端的一个或多个实施例的电子设备1812的实例。在一种配置中，电子设备1812包括至少一个处理单元1816和存储器1818。根据电子设备的确切配置和类型， 存储器1818可以是易失性的(比如RAM)、非易失性的(比如ROM、闪存等)或二者的某种组合。该配置在图18中由虚线1814图示。
在其他实施例中，电子设备1812可以包括附加特征和/或功能。例如，设备1812还可以包括附加的存储装置(例如可移除和/或不可移除的)，其包括但不限于磁存储装置、光存储装置等等。这种附加存储装置在图18中由存储装置1820图示。在一个实施例中，用于实现本文所提供的一个或多个实施例的计算机可读指令可以在存储装置1820中。存储装置1820还可以存储用于实现操作系统、应用程序等的其他计算机可读指令。计算机可读指令可以载入存储器1818中由例如处理单元1816执行。
本文所使用的术语“计算机可读介质”包括计算机存储介质。计算机存储介质包括以用于存储诸如计算机可读指令或其他数据之类的信息的任何方法或技术实现的易失性和非易失性、可移除和不可移除介质。存储器1818和存储装置1820是计算机存储介质的实例。计算机存储介质包括但不限于RAM、ROM、EEPROM、闪存或其他存储器技术、CD-ROM、数字通用盘(DVD)或其他光存储装置、盒式磁带、磁带、磁盘存储装置或其他磁存储设备、或可以用于存储期望信息并可以被电子设备1812访问的任何其他介质。任意这样的计算机存储介质可以是电子设备1812的一部分。
电子设备1812还可以包括允许电子设备1812与其他设备通信的通信连接1826。通信连接1826可以包括但不限于调制解调器、网络接口卡(NIC)、集成网络接口、射频发射器/接收器、红外端口、USB连接或用于将电子设备1812连接到其他电子设备的其他接口。通信连接1826可以包括有线连接或无线连接。通信连接1826可以发射和/或接收通信媒体。
术语“计算机可读介质”可以包括通信介质。通信介质典型地包含计算机可读指令或诸如载波或其他传输机构之类的“己调制数据信号”中的其他数据，并且包括任何信息递送介质。术语“已调制数据信号”可以包括 这样的信号：该信号特性中的一个或多个按照将信息编码到信号中的方式来设置或改变。
电子设备1812可以包括输入设备1824，比如键盘、鼠标、笔、语音输入设备、触摸输入设备、红外相机、视频输入设备和/或任何其他输入设备。设备1812中也可以包括输出设备1822，比如一个或多个显示器、扬声器、打印机和/或任意其他输出设备。输入设备1824和输出设备1822可以经由有线连接、无线连接或其任意组合连接到电子设备1812。在一个实施例中，来自另一个电子设备的输入设备或输出设备可以被用作电子设备1812的输入设备1824或输出设备1822。
电子设备1812的组件可以通过各种互连(比如总线)连接。这样的互连可以包括外围组件互连(PCI)(比如快速PCI)、通用串行总线(USB)、火线(IEEE 1394)、光学总线结构等等。在另一个实施例中，电子设备1812的组件可以通过网络互连。例如，存储器1818可以由位于不同物理位置中的、通过网络互连的多个物理存储器单元构成。
本领域技术人员将认识到，用于存储计算机可读指令的存储设备可以跨越网络分布。例如，可经由网络1828访问的电子设备1830可以存储用于实现本发明所提供的一个或多个实施例的计算机可读指令。电子设备1812可以访问电子设备1830并且下载计算机可读指令的一部分或所有以供执行。可替代地，电子设备1812可以按需要下载多条计算机可读指令，或者一些指令可以在电子设备1812处执行并且一些指令可以在电子设备1830处执行。
本文提供了实施例的各种操作。在一个实施例中，所述的一个或多个操作可以构成一个或多个计算机可读介质上存储的计算机可读指令，其在被电子设备执行时将使得计算设备执行所述操作。描述一些或所有操作的顺序不应当被解释为暗示这些操作必需是顺序相关的。本领域技术人员将 理解具有本说明书的益处的可替代的排序。而且，应当理解，不是所有操作必需在本文所提供的每个实施例中存在。
而且，本文所使用的词语“优选的”意指用作实例、示例或例证。奉文描述为“优选的”任意方面或设计不必被解释为比其他方面或设计更有利。相反，词语“优选的”的使用旨在以具体方式提出概念。如本发明中所使用的术语“或”旨在意指包含的“或”而非排除的“或”。即，除非另外指定或从上下文中清楚，“X使用A或B”意指自然包括排列的任意一个。即，如果X使用A；X使用B；或X使用A和B二者，则“X使用A或B”在前述任一示例中得到满足。
而且，尽管已经相对于一个或多个实现方式示出并描述了本公开，但是本领域技术人员基于对本说明书和附图的阅读和理解将会想到等价变型和修改。本公开包括所有这样的修改和变型，并且仅由所附权利要求的范围限制。特别地关于由上述组件(例如元件、资源等)执行的各种功能，用于描述这样的组件的术语旨在对应于执行所述组件的指定功能(例如其在功能上是等价的)的任意组件(除非另外指示)，即使在结构上与执行本文所示的本公开的示范性实现方式中的功能的公开结构不等同。此外，尽管本公开的特定特征已经相对于若干实现方式中的仅一个被公开，但是这种特征可以与如可以对给定或特定应用而言是期望和有利的其他实现方式的一个或多个其他特征组合。而且，就术语“包括”、“具有”、“含有”或其变形被用在具体实施方式或权利要求中而言，这样的术语旨在以与术语“包含”相似的方式包括。
本发明中的各功能单元可以集成在一个处理模块中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个模块中。上述集成的模块既可以采用硬件的形式实现，也可以采用软件功能模块的形式实现。所述集成的模块如果以软件功能模块的形式实现并作为独立的产品销 售或使用时，也可以存储在一个计算机可读取存储介质中。上述提到的存储介质可以是只读存储器，磁盘或光盘等。上述的各装置或系统，可以执行相应方法实施例中的方法。
以上所述，仅为本发明的具体实施方式，但本发明的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本发明揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本发明的保护范围之内。因此，本发明的保护范围应以所述权利要求的保护范围为准。

Claims (24)

  1. 一种音频数据处理方法，包括：
    接收客户端的音频数据；
    接收请求客户端的音频数据请求，并根据所述请求客户端的音频数据请求，获取所述请求客户端所请求的音频数据的混音方式；
    根据所述请求客户端所请求的音频数据的混音方式，对所述音频数据请求对应的客户端的音频数据进行预处理；以及，
    将预处理后的音频数据发送至所述请求客户端，供所述请求客户端基于所述预处理后的音频数据获取混音处理后的音频数据。
  2. 根据权利要求1所述的音频数据处理方法，其中，所述请求客户端的音频数据混音方式包括客户端混音以及服务器混音；
    所述根据所述请求客户端的音频数据混音方式，对所述音频数据请求对应的客户端的音频数据进行预处理，包括：
    如所述请求客户端的音频数据混音方式为客户端混音，则对所述音频数据请求对应的客户端的音频数据进行采集处理；
    如所述请求客户端的音频数据混音方式为服务器混音，则对所述音频数据请求对应的客户端的音频数据依次进行解码、混音以及编码处理。
  3. 根据权利要求2所述的音频数据处理方法，其中，所述对所述音频数据请求对应的客户端的音频数据依次进行解码、混音以及编码处理，包括：
    根据所述音频数据的丢包情况，对所有客户端的音频数据进行解码处理；
    从解码处理后的音频数据中获取所述音频数据请求对应的客户端的音频数据；
    将所述音频数据请求对应的客户端的音频数据转换为预设格式的音频 数据；
    对所述预设格式的音频数据进行混音处理；以及，
    根据当前网络情况，对混音处理后的音频数据进行压缩编码处理。
  4. 根据权利要求1所述的音频数据处理方法，其中，还包括：
    接收所述请求客户端的混音方式转换请求，并根据所述请求客户端的混音方式，转换请求设定所述请求客户端的音频数据混音方式。
  5. 一种音频数据处理服务器，包括：
    音频数据接收模块，配置为接收客户端的音频数据；
    混音方式获取模块，配置为接收请求客户端的音频数据请求，并根据所述请求客户端的音频数据请求，获取所述请求客户端的音频数据混音方式；
    预处理模块，配置为根据所述请求客户端的音频数据混音方式，对所述音频数据请求对应的客户端的音频数据进行预处理；
    发送模块，配置为将预处理后的音频数据发送至所述请求客户端，以便所述请求客户端获取混音处理后的音频数据。
  6. 根据权利要求5所述的音频数据处理服务器，其中，所述请求客户端的音频数据混音方式包括客户端混音以及服务器混音；
    其中所述预处理模块，包括：
    第一音频数据处理单元，配置为如所述请求客户端的音频数据混音方式为客户端混音，则对所述音频数据请求对应的客户端的音频数据进行采集处理；
    第二音频数据处理单元，配置为如所述请求客户端的音频数据混音方式为服务器混音，则对所述音频数据请求对应的客户端的音频数据依次进行解码、混音以及编码处理。
  7. 根据权利要求6所述的音频数据处理服务器，其中，所述第二音频 数据处理单元包括：
    解码子单元，配置为根据所述音频数据的丢包情况，对所有客户端的音频数据进行解码处理；
    音频数据获取子单元，配置为从解码处理后的音频数据中获取所述音频数据请求对应的客户端的音频数据；
    格式转换子单元，配置为将所述音频数据请求对应的客户端的音频数据转换为预设格式的音频数据
    混音处理子单元，配置为对所述预设格式的音频数据进行混音处理；
    压缩编码子单元，配置为根据当前网络情况，对混音处理后的音频数据进行压缩编码处理。
  8. 根据权利要求5所述的音频数据处理服务器，其中，所述音频数据处理服务器还包括：
    混音方式转换模块，配置为接收所述请求客户端的混音方式转换请求，并根据所述请求客户端的混音方式转换请求，设定所述请求客户端的音频数据混音方式。
  9. 一种音频数据处理系统，包括权利要求5-8中任一的音频数据处理服务器以及多个客户端。
  10. 一种音频数据处理方法，包括：
    接收异地音频数据，其中所述异地音频数据包括所述异地音频数据所来源客户端的客户端标识符；
    从所述异地音频数据获取所述客户端标识符，并根据所述客户端标识符，确定所述异地音频数据的混音方式；以及，
    根据所述异地音频数据的混音方式，对所述异地混音数据进行播放处理。
  11. 根据权利要求10所述的音频数据处理方法，其中，所述根据所述 客户端标识符，确定所述异地音频数据的混音方式，包括：
    判断所述客户端标识符是否为本地客户端标识符；
    如所述客户端标识符为本地客户端标识符，则确定所述异地音频数据为服务器混音方式；
    如所述客户端标识符为非本地客户端标识，则确定所述异地音频数据为客户端混音方式。
  12. 根据权利要求10所述的音频数据处理方法，其中，所述根据所述异地音频数据的混音方式，对所述异地混音数据进行播放处理，包括：
    当所述音频数据为服务器混音方式时，对所述异地音频数据进行解码以及播放处理；
    当所述音频数据为客户端混音方式时，对所述异地音频数据进行解码、混音以及播放处理。
  13. 根据权利要求10所述的音频数据处理方法，其中，还包括：
    在本地音频数据上设置本地客户端标识符；以及，
    将所述本地音频数据发送至其他客户端或服务器。
  14. 根据权利要求10所述的音频数据处理方法，其中，如所述移动音频数据为服务器混音方式，则所述客户端标识符由音频数据处理服务器设置到所述异地音频数据中。
  15. 一种音频数据处理客户端，包括：
    异地音频数据接收模块，配置为接收异地音频数据，其中所述异地音频数据包括一客户端标识符；
    混音方式确定模块，配置为从所述异地音频数据获取所述客户端标识符，根据所述客户端标识符，确定所述异地音频数据的混音方式；以及
    音频数据播放模块，配置为根据所述异地音频数据的混音方式，对所述异地混音数据进行播放处理。
  16. 根据权利要求15所述的音频数据处理客户端，其中，所述混音方式确定模块包括：
    标识符判断单元，配置为判断所述客户端标识符是否为本地客户端标识符；
    第一混音方式确定单元，配置为如所述客户端标识符为本地客户端标识符，则确定所述异地音频数据为服务器混音方式；
    第二混音方式确定单元，配置为如所述客户端标识符为非本地客户端标识，则确定所述异地音频数据为客户端混音方式。
  17. 根据权利要求15所述的音频数据处理客户端，其中，所述音频数据播放模块包括：
    第一音频数据播放单元，配置为当所述音频数据为服务器混音方式时，对所述异地音频数据进行解码以及播放处理；
    第二音频数据播放单元，配置为当所述音频数据为客户端混音方式时，对所述异地音频数据进行解码、混音以及播放处理。
  18. 根据权利要求15所述的音频数据处理客户端，其中，所述音频数据处理客户端包括：
    标识符设定模块，配置为在本地音频数据上设置本地客户端标识符；以及
    音频数据发送模块，配置为将所述本地音频数据发送至其他客户端或服务器。
  19. 根据权利要求15所述的音频数据处理客户端，其中，如所述移动音频数据为服务器混音方式，则所述客户端标识符由音频数据处理服务器设置到所述异地音频数据中。
  20. 一种音频数据处理系统，包括多个权利要求15-19中任一的音频数据处理客户端以及至少一个音频数据处理服务器。
  21. 一种音频数据处理服务器，包括：
    处理器和存储器，所述处理器中存储有可执行指令，所述可执行指令配置为引起所述处理器执行以下的操作：
    接收客户端的音频数据；
    接收请求客户端的音频数据请求，并根据所述请求客户端的音频数据请求，获取所述请求客户端的音频数据混音方式；
    根据所述请求客户端的音频数据混音方式，对所述音频数据请求对应的客户端的音频数据进行预处理；
    将预处理后的音频数据发送至所述请求客户端，以便所述请求客户端获取混音处理后的音频数据。
  22. 一种音频数据处理客户端，包括：
    处理器和存储器，所述处理器中存储有可执行指令，所述可执行指令配置为引起所述处理器执行以下的操作：
    接收异地音频数据，其中所述异地音频数据包括一客户端标识符；
    从所述异地音频数据获取所述客户端标识符，根据所述客户端标识符，确定所述异地音频数据的混音方式；以及
    根据所述异地音频数据的混音方式，对所述异地混音数据进行播放处理。
  23. 一种存储介质，存储有可执行指令，所述可执行指令用于执行权利要求1至5任一项所述的音频数据处理方法。
  24. 一种存储介质，存储有可执行指令，所述可执行指令用于执行权利要求5至9任一项所述的音频数据处理方法。
PCT/CN2017/073483 2016-02-18 2017-02-14 音频数据处理方法、服务器、客户端以及系统、存储介质 WO2017140235A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US15/764,737 US11196868B2 (en) 2016-02-18 2017-02-14 Audio data processing method, server, client and server, and storage medium
MYPI2018700977A MY189870A (en) 2016-02-18 2017-02-14 Audio data processing method, server, client and server

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610092373.5 2016-02-18
CN201610092373.5A CN105721469B (zh) 2016-02-18 2016-02-18 音频数据处理方法、服务器、客户端以及系统

Publications (1)

Publication Number Publication Date
WO2017140235A1 true WO2017140235A1 (zh) 2017-08-24

Family

ID=56156111

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/073483 WO2017140235A1 (zh) 2016-02-18 2017-02-14 音频数据处理方法、服务器、客户端以及系统、存储介质

Country Status (4)

Country Link
US (1) US11196868B2 (zh)
CN (1) CN105721469B (zh)
MY (1) MY189870A (zh)
WO (1) WO2017140235A1 (zh)

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN105721469B (zh) * 2016-02-18 2019-09-20 腾讯科技（深圳）有限公司 音频数据处理方法、服务器、客户端以及系统
US20220208210A1 (en) * 2019-02-19 2022-06-30 Sony Interactive Entertainment Inc. Sound output control apparatus, sound output control system, sound output control method, and program
CN109889549B (zh) * 2019-04-03 2021-09-28 海能达通信股份有限公司 媒体数据下发方法及装置、关键任务一键通系统
CN110536171B (zh) * 2019-08-26 2022-03-11 腾讯科技（深圳）有限公司 互动场景中的多媒体处理方法、装置及电子设备
CN111768782A (zh) * 2020-06-30 2020-10-13 广州酷狗计算机科技有限公司 音频识别方法、装置、终端及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1984353A (zh) * 2006-04-19 2007-06-20 华为技术有限公司 一种实现混音的方法和装置
CN103327014A (zh) * 2013-06-06 2013-09-25 腾讯科技（深圳）有限公司 一种语音处理方法、装置及系统
WO2014085093A1 (en) * 2012-11-29 2014-06-05 Dolby Laboratories Licensing Corporation Systems for providing services in a voice conferencing environment
CN104917671A (zh) * 2015-06-10 2015-09-16 腾讯科技（深圳）有限公司 基于移动终端的音频处理方法和装置
CN105721469A (zh) * 2016-02-18 2016-06-29 腾讯科技（深圳）有限公司 音频数据处理方法、服务器、客户端以及系统

Family Cites Families (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7945006B2 (en) * 2004-06-24 2011-05-17 Alcatel-Lucent Usa Inc. Data-driven method and apparatus for real-time mixing of multichannel signals in a media server
US7769806B2 (en) * 2007-10-24 2010-08-03 Social Communications Company Automated real-time data stream switching in a shared virtual area communication environment
US8407287B2 (en) * 2009-07-14 2013-03-26 Radvision Ltd. Systems, methods, and media for identifying and associating user devices with media cues
US8958567B2 (en) * 2011-07-07 2015-02-17 Dolby Laboratories Licensing Corporation Method and system for split client-server reverberation processing
US20140369528A1 (en) * 2012-01-11 2014-12-18 Google Inc. Mixing decision controlling decode decision
CN103248882A (zh) * 2012-02-02 2013-08-14 腾讯科技（深圳）有限公司 多媒体数据传输的方法、多媒体数据传输装置及系统
WO2014100374A2 (en) * 2012-12-19 2014-06-26 Rabbit, Inc. Method and system for content sharing and discovery
CN104137523B (zh) * 2012-12-31 2017-02-08 华为技术有限公司 一种实现会议接入的方法、设备和系统
US20160196131A1 (en) * 2014-07-07 2016-07-07 Symphony Teleca Corporation Remote Embedded Device Update Platform Apparatuses, Methods and Systems
US10511718B2 (en) * 2015-06-16 2019-12-17 Dolby Laboratories Licensing Corporation Post-teleconference playback using non-destructive audio transport

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1984353A (zh) * 2006-04-19 2007-06-20 华为技术有限公司 一种实现混音的方法和装置
WO2014085093A1 (en) * 2012-11-29 2014-06-05 Dolby Laboratories Licensing Corporation Systems for providing services in a voice conferencing environment
CN103327014A (zh) * 2013-06-06 2013-09-25 腾讯科技（深圳）有限公司 一种语音处理方法、装置及系统
CN104917671A (zh) * 2015-06-10 2015-09-16 腾讯科技（深圳）有限公司 基于移动终端的音频处理方法和装置
CN105721469A (zh) * 2016-02-18 2016-06-29 腾讯科技（深圳）有限公司 音频数据处理方法、服务器、客户端以及系统

Also Published As

Publication number Publication date
CN105721469B (zh) 2019-09-20
CN105721469A (zh) 2016-06-29
MY189870A (en) 2022-03-15
US11196868B2 (en) 2021-12-07
US20180270360A1 (en) 2018-09-20

Similar Documents

Publication Publication Date Title
WO2017140235A1 (zh) 音频数据处理方法、服务器、客户端以及系统、存储介质
WO2018121014A1 (zh) 视频播放控制方法、装置及终端设备
RU2631137C2 (ru) Связывание устройств
TW201517572A (zh) 一種數據處理方法、裝置及系統
WO2015188575A1 (zh) 视频数据提供方法、视频播放方法、装置及系统
JP2017525247A (ja) 効率的なフレームレンダリング
US20180063481A1 (en) Human interface device (hid) based control of video data conversion at docking station
US9948732B2 (en) Method for providing status messaging service in electronic device and the electronic device thereof
CN108809924B (zh) 执行网络实时通信的方法和装置
US10834164B2 (en) Virtualizing audio and video devices using synchronous A/V streaming
KR102014774B1 (ko) 단말의 음성인식을 제어하는 서버 및 방법, 그리고 단말
CN111478916B (zh) 基于视频流的数据传输方法、设备和存储介质
KR102327912B1 (ko) 디지털 디바이스를 이용하여 원격지 pc와 연결하는 방법
WO2013078614A1 (zh) 一种虚拟桌面场景下音频压缩算法选择方法、装置及通信系统
TW201327370A (zh) 資源分享系統與方法及其播放裝置
WO2017177802A1 (zh) 音视频播放方法、装置及计算机存储介质
TW200947307A (en) Resource management methods and systems, and related operating systems and machine readable medium thereof
WO2019075896A1 (zh) 一种设备间互连的方法、装置、系统、设备和存储介质
US20120030383A1 (en) Digital media providing device with expandable transcoding capability and method thereof
AU2018403361B2 (en) Data transmission
US10104422B2 (en) Multimedia playing control method, apparatus for the same and system
WO2016107174A1 (zh) 多媒体文件数据的处理方法及系统、播放器和客户端
CN104754400B (zh) 一种基于移动终端的封套信息共享方法及装置
WO2015135409A1 (zh) 一种会话建立方法、终端及系统
WO2015154621A1 (zh) 一种通讯录信息的传输方法、装置及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17752667

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 15764737

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17752667

Country of ref document: EP

Kind code of ref document: A1