CN110166185B

CN110166185B - Audio data processing method and device, storage medium and processor

Info

Publication number: CN110166185B
Application number: CN201910408187.1A
Authority: CN
Inventors: 田建超
Original assignee: Guangzhou Shiyuan Electronics Thecnology Co Ltd; Guangzhou Shizhen Information Technology Co Ltd
Current assignee: Guangzhou Shiyuan Electronics Thecnology Co Ltd; Guangzhou Shizhen Information Technology Co Ltd
Priority date: 2019-05-15
Filing date: 2019-05-15
Publication date: 2021-07-09
Anticipated expiration: 2039-05-15
Also published as: CN110166185A

Abstract

The invention discloses a method and a device for processing audio data, a storage medium and a processor. The method comprises the following steps: respectively receiving first audio data of one or more first clients in a plurality of clients to obtain target audio data; under the condition that first target audio data of a target client is missing in the target audio data, acquiring a target state of the target client, wherein the plurality of clients comprise the target client, and the target state is used for indicating that the forwarding of the first target audio data is forbidden or the first target audio data is allowed to be forwarded to a client associated with the target client; and under the condition that the target state indicates that the first target audio data is forbidden to be forwarded, forbidding sending a target request to the target client, wherein the target request is used for requesting the target client to retransmit the first target audio data. By the method and the device, the effect of improving the efficiency of processing the audio data is achieved.

Description

Audio data processing method and device, storage medium and processor

Technical Field

The present invention relates to the field of data processing, and in particular, to a method and an apparatus for processing audio data, a storage medium, and a processor.

Background

At present, when audio data is processed, a server receives audio data sent by all clients, and determines the audio data that needs to be forwarded to other clients. The audio data of some clients do not need to be forwarded to other clients, for example, if the volume emitted by a certain client is small, the audio data does not need to be forwarded to other clients.

If the audio data sent by a certain client side is missing, the server directly sends a packet loss retransmission (NACK) request to the client side. However, if the audio data of the client does not need to be forwarded to other clients, when the audio data is lost, the server still sends a NACK request to the client, and the actually retransmitted audio data is not forwarded to other clients, so that the NACK request and retransmission are useless, and the efficiency of processing the audio data is low.

Aiming at the problem of low efficiency of processing audio data in the prior art, no effective solution is provided at present.

Disclosure of Invention

The invention mainly aims to provide a method, a device, a storage medium and a processor for processing audio data, so as to at least solve the technical problem of low efficiency of processing the audio data.

In order to achieve the above object, according to an aspect of an embodiment of the present invention, there is provided a method of processing audio data. The method comprises the following steps: respectively receiving first audio data of one or more first clients in a plurality of clients to obtain target audio data; under the condition that first target audio data of a target client is missing in the target audio data, acquiring a target state of the target client, wherein the plurality of clients comprise the target client, and the target state is used for indicating that the forwarding of the first target audio data is forbidden or the first target audio data is allowed to be forwarded to a client associated with the target client; and under the condition that the target state indicates that the first target audio data is forbidden to be forwarded, forbidding sending a target request to the target client, wherein the target request is used for requesting the target client to retransmit the first target audio data.

Optionally, after obtaining the target status of the target client, the method further includes: sending a target request to the target client if the target status indicates permission to forward the first target audio data to a client associated with the target client; receiving first target audio data sent by a target client in response to a target request; the first target audio data is forwarded to a client associated with the target client.

Optionally, before receiving the first audio data of one or more first clients of the plurality of clients, respectively, the method further comprises: receiving second audio data of a plurality of clients; and determining the target state of each client according to the second audio data of each client.

Optionally, determining the target state of each client according to the second audio data of each client includes: extracting first volume data from the second audio data of each client, wherein the first volume data is used for indicating the volume of the audio emitted by the corresponding client; determining a first target client with a first target number associated with each client from the plurality of clients according to the first volume data of each client, wherein each client is used for receiving second audio data of each first target client associated with each client, the volume of the audio indicated by the first volume data of each first target client is greater than the volume of the audio indicated by the first volume data of the second client, and the second client is any one of the plurality of clients except each client and the first target clients with the first target number; determining states of a first target number of first target clients as first target states respectively, wherein the first target states are used for indicating that the second audio data of the first target clients are allowed to be forwarded to the clients associated with the first target clients; and respectively determining the states of second target clients with a second target number in the plurality of clients as second target states, wherein the second target clients with the second target number are clients except the first target clients with the first target number in the plurality of clients, and the second target states are used for indicating that the forwarding of second audio data of the second target clients is forbidden.

Optionally, after prohibiting sending the target request to the target client, the method further includes: receiving third audio data of a plurality of clients; and updating the target state of each client according to the third audio data of each client.

Optionally, the updating the target state of each client according to the third audio data of each client includes: extracting second volume data from the third audio data of each client, wherein the second volume data is used for indicating the volume of the audio emitted by the corresponding client; determining a third target client with a third target number associated with each client from the plurality of clients according to the second volume data of each client, wherein each client is used for receiving the third audio data of each third target client associated with each client, the volume of the audio indicated by the second volume data of each third target client is greater than the volume of the audio indicated by the second volume data of each third client, and the third client is any one of the plurality of clients except each client and the third target clients with the third target number; updating the states of a third target number of third target clients to a first target state respectively, wherein the first target state is used for indicating that the third audio data of the third target clients are allowed to be forwarded to the client associated with the third target client; and respectively updating the states of fourth target clients with a fourth target number in the multiple clients to second target states, wherein the fourth target clients with the fourth target number are clients except the third target clients with the third target number in the multiple clients, and the second target states are used for indicating that the forwarding of third audio data of the fourth target clients is forbidden.

Optionally, receiving the third audio data of the plurality of clients comprises: third audio data of the plurality of clients is received every target time period.

Optionally, the updating the target state of each client according to the third audio data of each client includes: and updating the target state of each client according to the third audio data of each client every other target time period.

Optionally, before obtaining the target status of the target client, the method further includes: acquiring a target list, wherein the target list comprises identification information of the first audio data of each first client; in a case where the identification information of the first target audio data is not included in the target list, it is determined that the first target audio data is missing in the target audio data.

Optionally, respectively receiving first audio data of one or more first clients of the multiple clients, and obtaining the target audio data includes: the method comprises the steps of respectively obtaining first audio data of one or more first clients in a conference scene to obtain target audio data.

In order to achieve the above object, according to another aspect of the embodiments of the present invention, there is also provided a method of transmitting target audio data to a server, wherein the target audio data includes first audio data of one or more first clients among a plurality of clients; and under the condition that the target audio data lacks the first target audio data of the target client and the target state of the target client indicates that the server is prohibited from forwarding the first target audio data of the target client, the target client is prohibited from receiving a target request sent by the server, wherein the plurality of clients include the target client, and the target request is used for requesting the target client to retransmit the first target audio data.

In order to achieve the above object, according to another aspect of the embodiments of the present invention, there is also provided an apparatus for processing audio data. The device includes: the receiving unit is used for respectively receiving first audio data of one or more first clients in the plurality of clients to obtain target audio data; the device comprises an acquisition unit, a processing unit and a processing unit, wherein the acquisition unit is used for acquiring a target state of a target client under the condition that first target audio data of the target client is missing in the target audio data, the plurality of clients comprise the target client, and the target state is used for indicating that the forwarding of the first target audio data is forbidden or the forwarding of the first target audio data to a client associated with the target client is allowed; and the forbidding unit is used for forbidding sending a target request to the target client under the condition that the target state indicates that the first target audio data are forbidden to be forwarded, wherein the target request is used for requesting the target client to retransmit the first target audio data.

In order to achieve the above object, according to another aspect of the embodiments of the present invention, there is also provided an apparatus for processing audio data. The device comprises a device sending unit, a server and a processing unit, wherein the device sending unit is used for sending target audio data to the server, and the target audio data comprises first audio data of one or more first clients in a plurality of clients; the target client is configured to receive a target request sent by the server when the target state of the target client indicates that the server is prohibited from forwarding the first target audio data of the target client, where the plurality of clients include the target client, and the target request is used to request the target client to retransmit the first target audio data.

In order to achieve the above object, according to another aspect of the present invention, there is also provided a storage medium. The storage medium comprises a stored program, wherein the apparatus on which the storage medium is located is controlled to perform the method of the embodiments of the present invention when the program is run.

To achieve the above object, according to another aspect of the present invention, there is also provided a processor. The processor is configured to run a program, wherein the program performs the method of the embodiment of the present invention when running.

In the embodiment of the invention, first audio data of one or more first clients in a plurality of clients are respectively received to obtain target audio data; under the condition that first target audio data of a target client is missing in the target audio data, acquiring a target state of the target client, wherein the plurality of clients comprise the target client, and the target state is used for indicating that the forwarding of the first target audio data is forbidden or the first target audio data is allowed to be forwarded to a client associated with the target client; and under the condition that the target state indicates that the first target audio data is forbidden to be forwarded, forbidding sending a target request to the target client, wherein the target request is used for requesting the target client to retransmit the first target audio data. That is to say, when the first target audio data of the target client is missing in the obtained target audio data, it is determined whether the first target audio data is audio data (active audio) that needs to be forwarded, and if not, the request for obtaining the missing first target audio data from the target client is prohibited, so that request retransmission of the audio data of the client can be reduced, the technical problem of low efficiency in processing the audio data is solved, and the technical effect of improving the efficiency in processing the audio data is achieved.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the invention and, together with the description, serve to explain the invention and not to limit the invention. In the drawings:

fig. 1A is a flowchart of a method of processing audio data according to an embodiment of the present invention;

fig. 1B is a flowchart of another audio data processing method according to an embodiment of the present invention;

FIG. 2 is a schematic diagram of audio data transmission between a server and a client according to an embodiment of the present invention;

fig. 3 is a flowchart of another audio data processing method according to an embodiment of the present invention;

fig. 4 is a flowchart of another audio data processing method according to an embodiment of the present invention;

fig. 5 is a schematic diagram of an apparatus for processing audio data according to an embodiment of the present invention; and

fig. 6 is a schematic diagram of another audio data processing apparatus according to an embodiment of the present invention.

Detailed Description

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present invention will be described in detail below with reference to the embodiments with reference to the attached drawings.

In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.

It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be used. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Example 1

The embodiment of the invention provides a method for processing audio data. The audio data processing method according to the embodiment of the present invention will be described below from the server side.

Fig. 1A is a flowchart of a method of processing audio data according to an embodiment of the present invention. As shown in fig. 1A, the method comprises the steps of:

step S102, receiving first audio data of one or more first clients in the plurality of clients respectively to obtain target audio data.

In the technical solution provided by step S102 of the present invention, the server receives the first audio data of one or more first clients of the multiple clients, respectively, to obtain the target audio data. The Client (Client) is a source end for generating audio data, and may be an audio Client, for example, an audio Client used in a multi-end voice conference scenario. Optionally, the client of this embodiment may include an audio receiving module (AudioRecvStream) and an audio transmitting module (AudioSendStream), where the audio receiving module of the client may be configured to receive audio data generated by other clients and forwarded by the server, and the audio transmitting module of the client may be configured to transmit the audio data generated by the client to the server. The audio data is used to indicate audio generated by the client during the operation process, and may be an audio data packet including a plurality of audio data, for example, volume data (audio level) including audio, a duration of the audio, a time when the audio starts, a time when the audio ends, and the like. The volume data is used to indicate the sound level of the audio generated by the client, and may be carried by the header of the audio data packet.

Optionally, the server of this embodiment receives audio data sent by a plurality of clients, where the server also includes an audio receiving module and an audio sending module, the audio receiving module of the server may receive the audio data sent by the clients, and the audio sending module of the server may forward the received audio data of the clients to other clients. The server of this embodiment may select some of the received audio data of the clients and forward the selected audio data to other clients, that is, the server selectively forwards some of the received audio data of all the clients instead of forwarding all the received audio data of the clients to the clients. Optionally, the server selects N paths of audio data with a larger volume from the received audio data of all the clients, where N may be a natural number greater than or equal to 3, determines the N paths of audio data as active audio data, and determines audio data generated by the clients except the N paths of audio data from among the audio data sent by the multiple clients as inactive audio data. The active audio data is the audio data which can be forwarded to other clients by the server, and the active audio data is the audio data which is not forwarded to other clients by the server, so that each client can receive N paths of audio data with larger volume except the own audio data, and when a plurality of users speak simultaneously through the clients, each user can be ensured to receive N paths of audio data with larger volume than the own audio data through the clients, so that the purpose of reducing bandwidth is achieved, and the effect of filtering out ambient interference sounds can be achieved.

And step S104, acquiring the target state of the target client under the condition that the first target audio data of the target client is missing in the target audio data.

In the technical solution provided in step S104 of the present invention, after receiving the first audio data of one or more first clients in the multiple clients respectively and obtaining the target audio data, the server obtains a target state of the target client under the condition that the first target audio data of the target client is missing in the target audio data, where the multiple clients include the target client, and the target state is used to indicate that forwarding of the first target audio data is prohibited or the first target audio data is permitted to be forwarded to a client associated with the target client.

In this embodiment, the server may receive audio data sent by all the clients under normal conditions, but due to factors such as network and line fault, the audio data generated by the clients may not be sent to the server normally, so that a packet loss situation occurs, and the server receives incomplete target audio data. The multiple clients of the embodiment include a target client, which may be any one of the multiple clients and may generate first target Audio data, and if the first target Audio data is missing in the target Audio data, the first target Audio data is a missing packet of the target Audio data, at this time, the server obtains a target state (inactive) of the target client, which may be used to indicate that forwarding of the Audio data generated by the target client is prohibited, that is, the first target Audio data generated by the target client does not belong to Active Audio data (Active Audio), and does not need to be forwarded to other clients, except the target client, of the multiple clients; the target status may also be used to indicate that the audio data generated by the target client is allowed to be forwarded to the client associated with the target client, i.e. the first target audio data generated by the target client belongs to active audio data, which needs to be forwarded to other clients of the plurality of clients than the target client.

Optionally, the server of this embodiment includes an endpoint (endpoint) corresponding to each client, and the endpoint may be configured to record a target state of the corresponding client, and the target state may be updated at target time intervals.

It should be noted that, this embodiment is not limited to the audio data missing in the target audio data, and the audio data of two or more clients may be missing.

And step S106, in the case that the target state indicates that the forwarding of the first target audio data is forbidden, the target request is forbidden to be sent to the target client.

In the technical solution provided by step S106 of the present invention, after the target state of the target client is obtained, and when the target state indicates that forwarding of the first target audio data is prohibited, a target request is prohibited from being sent to the target client, where the target request is used to request the target client to retransmit the first target audio data.

In this embodiment, when the target status of the target client indicates that forwarding of the first target audio data is prohibited, the first target audio data is output as inactive audio data at the current time, for example, audio data with a small volume, and even if the server acquires the first target audio data, the server does not need to forward the first target audio data to other clients.

In the embodiment, when the target state indicates that the forwarding of the first target audio data is forbidden, the target request is forbidden to be sent to the target client, and excessive NACK requests can be avoided under the condition of packet loss, so that the sending and receiving end analyses of the server end are reduced, the retransmission of the audio data of the client can be reduced, and the purposes of reducing the uplink bandwidth of the client and the downlink bandwidth of the server are achieved.

In the embodiment, first audio data of one or more first clients in a plurality of clients are respectively received, and target audio data are obtained; under the condition that first target audio data of a target client is missing in the target audio data, acquiring a target state of the target client, wherein the plurality of clients comprise the target client, and the target state is used for indicating that the forwarding of the first target audio data is forbidden or the first target audio data is allowed to be forwarded to a client associated with the target client; and under the condition that the target state indicates that the first target audio data is forbidden to be forwarded, forbidding sending a target request to the target client, wherein the target request is used for requesting the target client to retransmit the first target audio data. That is to say, when the first target audio data of the target client is missing in the obtained target audio data, it is determined whether the first target audio data is audio data (active audio) that needs to be forwarded, and if not, the request for obtaining the missing first target audio data from the target client is prohibited, so that request retransmission of the audio data of the client can be reduced, the technical problem of low efficiency in processing the audio data is solved, and the technical effect of improving the efficiency in processing the audio data is achieved.

As an optional implementation manner, after acquiring the target status of the target client in step S104, the method further includes: sending a target request to the target client if the target status indicates permission to forward the first target audio data to a client associated with the target client; receiving first target audio data sent by a target client in response to a target request; the first target audio data is forwarded to a client associated with the target client.

In this embodiment, the target state may indicate that the audio data generated by the target client is allowed to be forwarded to the client associated with the target client, that is, the first target audio data generated by the target client is active audio data, for example, the first target audio data is audio data generated by a client with the largest volume among the plurality of clients, so that in the case that the target state indicates that the first target audio data is allowed to be forwarded to the client associated with the target client, the server sends a target request to the target client, requests the target client to retransmit the first target audio data, and the target client forwards the first target audio data to the client associated with the target client in response to the target request after receiving the target request.

For example, the plurality of clients are client 1, client 2, client 3 and client 4, where the volume indicated by the audio data generated by client 1 is the largest, the audio data is determined to be active audio data, and the target state of the audio data allows forwarding the audio data to client 2, client 3 and client 4 associated with client 1, so that client 2, client 3 and client 4 can only receive the audio data of client 1 with the largest sound except for itself. It should be noted that, the association here refers to the association between the target client and the client that can receive the audio data.

As an optional implementation manner, before receiving the first audio data of one or more first clients of the plurality of clients respectively in step S102, the method further includes: receiving second audio data of a plurality of clients; and determining the target state of each client according to the second audio data of each client.

In this embodiment, before receiving the first audio data of one or more first clients of the plurality of clients, respectively, the target status of each client needs to be determined, the second audio data of the plurality of clients may be received, and the target status of each client is determined by the second audio data of the plurality of clients, for example, the status of prohibiting forwarding the audio data generated by each client or the status of allowing forwarding the audio data generated by each client to the client associated with the target client is determined.

The method of determining the target state of each client in accordance with the second audio data of each client of the embodiment is described below.

As an optional implementation, determining the target state of each client according to the second audio data of each client includes: extracting first volume data from the second audio data of each client, wherein the first volume data is used for indicating the volume of the audio emitted by the corresponding client; determining a first target client with a first target number associated with each client from the plurality of clients according to the first volume data of each client, wherein each client is used for receiving second audio data of each first target client associated with each client, the volume of the audio indicated by the first volume data of each first target client is greater than the volume of the audio indicated by the first volume data of the second client, and the second client is any one of the plurality of clients except each client and the first target clients with the first target number; determining states of a first target number of first target clients as first target states respectively, wherein the first target states are used for indicating that the second audio data of the first target clients are allowed to be forwarded to the clients associated with the first target clients; and respectively determining the states of second target clients with a second target number in the plurality of clients as second target states, wherein the second target clients with the second target number are clients except the first target clients with the first target number in the plurality of clients, and the second target states are used for indicating that the forwarding of second audio data of the second target clients is forbidden.

In this embodiment, the second audio data of each client includes first volume data, which may be carried by a header of the second audio data, for indicating the volume of audio emitted by the client that generated the second audio data. After the first volume data is extracted from the second audio data of each client, a first target client of a first target number associated with each client may be determined from the plurality of clients according to the first volume data of each client, each client is configured to receive the second audio data of each first target client associated with each client, the volume of the audio indicated by the first volume data of each first target client is greater than the volume of the audio indicated by the first volume data of the second client, and the second client is any one of the plurality of clients except for each client and the first target clients of the first target number, that is, each client may receive the second audio data generated by the clients of the first target number except for itself, which are of the first target number. The volumes indicated by the first volume data of the clients other than each client among the plurality of clients may be ranked, from which N (N < ═ 3) clients with higher volumes are determined, which are determined as clients associated with each client that may receive the second audio data generated by the client associated with each client.

For example, the plurality of clients are client 1, client 2, client 3, and client 4, and the volumes indicated by the second audio data generated by client 1, client 2, client 3, and client 4 are sorted, and the volume indicated by the second audio data generated by client 1 > the volume indicated by the second audio data generated by client 2 > the volume of client 3 > the volume of client 4. Setting the target number to be 1, and then for the client 1, the volume indicated by the second audio data generated by the client 2 is the largest among the volumes indicated by the second audio data generated by the client 2, the client 3 and the client 4, and then the client 2 is a client associated with the client 1, and the client 1 can receive the second audio data of the client 2 forwarded by the server; for the client 2, the volume indicated by the second audio data generated by the client 1 is the largest among the volumes indicated by the second audio data generated by the client 1, the client 3 and the client 4, then the client 1 is a client associated with the client 2, and the client 2 can receive the second audio data of the client 1 forwarded by the server; for the client 3, the volume indicated by the second audio data generated by the client 1 is the largest among the volumes indicated by the second audio data generated by the client 1, the client 2 and the client 4, then the client 1 is a client associated with the client 3, and the client 3 can receive the second audio data of the client 1 forwarded by the server; for the client 4, the volume indicated by the second audio data generated by the client 1 is the largest among the volumes indicated by the second audio data generated by the client 1, the client 2 and the client 3, then the client 1 is a client associated with the client 4, and the client 4 can receive the second audio data of the client 1 forwarded by the server.

After determining a first target number of first target clients associated with each client from the plurality of clients, the state of the first target number of first target clients for indicating permission to forward the second audio data of the first target clients to the client associated with the first target client, that is, the second audio data of the first target clients is active audio data, for example, the plurality of clients are client 1, client 2, client 3 and client 4, the volume indicated by the second audio data generated by client 1 > the volume indicated by the second audio data generated by client 2 > the volume of client 3 > the volume of client 4, the target state of client 1 is determined as the state for indicating permission to forward the second audio data of client 1 to client 2 associated with client 1, A first target state of the client 3, the client 4, and the second audio data generated by the client 1 is active audio data. Whereas for client 1 the target state of client 2 is the first target state indicating that the second audio data of client 2 is allowed to be forwarded to client 1 associated with client 2.

In this embodiment, after a first target number of first target clients associated with each client is determined from the plurality of clients, states of a second target number of second target clients in the plurality of clients are respectively determined as second target states, the second target states are used to indicate that forwarding of second audio data of the second target number of second target clients is prohibited, and the second target number of second target clients is a client other than the first target number of first target clients in the plurality of clients, that is, second audio data generated by the second target number of second target clients is inactive audio data. For example, the plurality of clients are client 1, client 2, client 3 and client 4, the volume indicated by the second audio data generated by client 1 > the volume indicated by the second audio data generated by client 2 > the volume of client 3 > the volume of client 4, the target state of client 1 is determined as a first target state indicating permission of forwarding the second audio data of client 1 to client 2, client 3 and client 4 associated with client 1, the target state of client 2, client 3 and client 4 is determined as a second target state prohibiting forwarding of the second audio data thereof, and the second audio data generated by client 2, client 3 and client 4 is inactive audio data. For the client 1, the target state of the client 2 is a first target state for indicating that the second audio data of the client 2 is allowed to be forwarded to the client 1 associated with the client 2, the target states of the client 1, the client 3 and the client 4 are second target states for prohibiting the forwarding of the second audio data thereof, and the second audio data generated by the client 1, the client 3 and the client 4 are inactive audio data, so that the target states of a plurality of clients are determined, the retransmission of the audio data of the clients is reduced, the purpose of reducing the uplink bandwidth of the client and the downlink bandwidth of the server is achieved, and the efficiency of processing the audio data is improved

As an optional implementation manner, after prohibiting sending the target request to the target client in step S106, the method further includes: receiving third audio data of a plurality of clients; and updating the target state of each client according to the third audio data of each client.

In this embodiment, after inhibiting the sending of the target request to the target client, the target state of the client may change with the generated audio data over time, and the embodiment may update the target state of each client at a later time. Second audio data for a plurality of clients may be received, and a target status for each client may be determined from the second audio data for the plurality of clients, such as a status that inhibits forwarding of audio data generated by each client or a status that permits forwarding of audio data generated by each client to a client associated with the target client. And updating the target state of each client, wherein the updated target state can be the same as or different from the target state before updating. The target state of the client may be updated to an endpoint (endpoint) in the server corresponding to the client.

As an optional implementation, the updating the target status of each client according to the third audio data of each client includes: extracting second volume data from the third audio data of each client, wherein the second volume data is used for indicating the volume of the audio emitted by the corresponding client; determining a third target client with a third target number associated with each client from the plurality of clients according to the second volume data of each client, wherein each client is used for receiving the third audio data of each third target client associated with each client, the volume of the audio indicated by the second volume data of each third target client is greater than the volume of the audio indicated by the second volume data of each third client, and the third client is any one of the plurality of clients except each client and the third target clients with the third target number; updating the states of a third target client of a third target number to a first target state respectively, wherein the first target state is used for indicating that the audio data of the third target client are allowed to be forwarded to the client associated with the third target client; and respectively updating the states of fourth target clients with a fourth target number in the multiple clients to second target states, wherein the fourth target clients with the fourth target number are clients except the third target clients with the third target number in the multiple clients, and the second target states are used for indicating that the forwarding of the audio data of the fourth target clients is forbidden.

In this embodiment, the third audio data of each client includes second volume data, which may be carried by a header of the third audio data, for indicating a volume of audio emitted by the client that generated the third audio data. After the second volume data is extracted from the third audio data of each client, a third target client with a third target number associated with each client may be determined from the multiple clients according to the second volume data of each client, each client is configured to receive the third audio data of each third target client associated with each client, the volume of the audio indicated by the second volume data of each third target client is greater than the volume of the audio indicated by the second volume data of the third client, and the third client is any one of the multiple clients except for each client and the third target clients with the third target number, that is, each client may receive the third audio data generated by the clients with the third target number of higher volumes except for itself. The volumes indicated by the second volume data of the clients other than each client among the plurality of clients may be ranked, from which N (N < ═ 3) clients with higher volumes are determined, which are determined as clients associated with each client that may receive the third audio data generated by the client associated with each client.

After a third target number of third target clients associated with each client is determined from the plurality of clients, the statuses of the third target clients of the third target number may be updated to the first target status, respectively, where the first target status is used to indicate that the third audio data of the third target clients are allowed to be forwarded to the client associated with the third target client, that is, the second audio data of the third target client is active audio data.

In this embodiment, after a third target number of third target clients associated with each client is determined from the multiple clients, the statuses of a fourth target number of fourth target clients in the multiple clients are respectively updated to a second target status, where the second target status is used to indicate that forwarding of third audio data of the fourth target number of fourth target clients is prohibited, and the fourth target number of fourth target clients is a client other than the first target number of third target clients in the multiple clients, that is, second audio data generated by the fourth target number of fourth target clients is inactive audio data, so that updating of the target statuses of the multiple clients is achieved, retransmission of the audio data of the clients is reduced, and the purpose of reducing the uplink bandwidth of the clients and the downlink bandwidth of the server is achieved, the efficiency of processing audio data is improved.

As an optional implementation, receiving the third audio data of the plurality of clients includes: receiving third audio data of a plurality of clients at intervals of a target time period; updating the target state of each client according to the third audio data of each client comprises: and updating the target state of each client according to the third audio data of each client every other target time period.

In this embodiment, when the target statuses of the plurality of clients are updated, the third audio data of the plurality of clients may be received every target time period, for example, the third audio data generated by the plurality of clients may be received every 15 minutes, which is not limited herein. After receiving the third audio data of the multiple clients, the target state of each client may be updated according to the received third audio data of each client in the above method, so that the target state of each client is updated every other target time period according to the third audio data of each client, thereby reducing retransmission of the audio data of the clients, achieving the purpose of reducing uplink bandwidth of the clients and downlink bandwidth of the server, and improving efficiency of processing the audio data.

As an optional implementation manner, before acquiring the target status of the target client in step S104, the method further includes: acquiring a target list, wherein the target list comprises identification information of the first audio data of each first client; in a case where the identification information of the first target audio data is not included in the target list, it is determined that the first target audio data is missing in the target audio data.

In this embodiment, a plurality of clients send audio data packets to a server, and in a case that a target list is not empty, the target list includes identification information of first audio data sent by each first client received by the server, and since the audio data are sent to the server in time sequence and carry sequence numbers, the identification information in this embodiment may be a sequence number, for example, if the plurality of clients are 10 clients, the plurality of clients send audio data with sequence numbers from 1 to 10 to the server. After the target list is obtained, in a case that the identification information of the first target audio data is not included in the target list, it is determined that the first target audio data is missing in the target audio data, where the first target audio data is a missing packet, this embodiment may determine any audio data missing in the target list as the first target audio data, for example, when the server does not receive audio data with sequence number 5, and audio data packets after and before sequence number 5 are received, the server may determine that the audio data with sequence number 5 is a missing packet.

After determining that the first target audio data is missing in the target audio data, in the case that the target state indicates that forwarding of the first target audio data is prohibited, sending a target request to the target client is prohibited, in the case that the target state indicates that forwarding of the first target audio data to a client associated with the target client is permitted, sending the target request to the target client, and receiving the first target audio data sent by the target client in response to the target request; the first target audio data are forwarded to the client side associated with the target client side, so that retransmission of the audio data of the client side is reduced, the purposes of reducing uplink bandwidth of the client side and downlink bandwidth of the server are achieved, and the efficiency of processing the audio data is improved.

As an optional implementation manner, in step S102, respectively receiving first audio data of one or more first clients in the multiple clients, and obtaining target audio data includes: the method comprises the steps of respectively obtaining first audio data of one or more first clients in a conference scene to obtain target audio data.

The audio data processing method of the embodiment is suitable for a conference scene, for example, the conference scene is a multi-terminal voice conference scene. The method comprises the steps that a server respectively obtains first audio data of one or more first clients in a conference scene to obtain target audio data, the server obtains a target state of the target client under the condition that the first target audio data of the target client is lost in the target audio data, and the server prohibits sending a target request for requesting the target client to retransmit the first target audio data to the target client in the conference scene under the condition that the target state indicates that the first target audio data are prohibited to be forwarded.

For example, in a conference scenario, four users join the conference by using the client 1, the client 2, the client 3, and the client 4, respectively. The server acquires the audio data sent by the client 1, the client 3 and the client 4, the audio data of the client 2 has a packet loss condition, and the target state of the client 2 indicates that the server is prohibited from forwarding the audio data of the client 2, so that the server prohibits sending a target request for requesting the target client to retransmit the first target audio data to the client 2 in the conference scene, which can reduce the request retransmission of the audio data of the client in the conference scene, and further achieve the effect of improving the efficiency of processing the audio data in the conference scene.

The following describes a method for processing audio data according to an embodiment of the present invention from a terminal side.

Fig. 1B is a flowchart of another audio data processing method according to an embodiment of the present invention. As shown in fig. 1B, the method includes the steps of:

step S202, sending the target audio data to the server.

In the technical solution provided by step S202 of the present invention, the terminal sends target audio data to the server, where the target audio data includes first audio data of one or more first clients in the multiple clients.

In this embodiment, the terminal transmits target audio data to the servers, and the servers receive the target audio data, respectively. The client is a source end for generating audio data, and may be an audio client, for example, an audio client used in a multi-end voice conference scene. Optionally, the client of this embodiment may include an audio receiving module and an audio sending module, where the audio receiving module of the client may be configured to receive audio data generated by other clients forwarded by the server, and the audio sending module of the client may be configured to send the audio data generated by the client to the server.

Step S204, when the first target audio data of the target client is missing in the target audio data, and the target state of the target client indicates that the server is prohibited from forwarding the first target audio data of the target client, the target client is prohibited from receiving the target request sent by the server.

In the technical solution provided by step S204 of the present invention, the plurality of clients include a target client, and the target request is used to request the target client to retransmit the first target audio data.

In this embodiment, the terminal may send all the audio data generated by the client to the server, but due to network and line faults, the audio data generated by the client may not be sent to the server normally, so that a packet loss situation occurs. The multiple clients of the embodiment include a target client, which may be any one of the multiple clients, and may generate first target Audio data, if the first target Audio data is missing in the target Audio data, the first target Audio data is a missing packet of the target Audio data, and in a case that a target state of the target client indicates that the server is prohibited from forwarding the first target Audio data of the target client, the terminal prohibits the target client from receiving a target request sent by the server, that is, the first target Audio data generated by the target client does not belong to Active Audio data (Active Audio), and does not need to be forwarded to other clients except the target client in the multiple clients.

As an alternative example, in the case of the audio data packet loss of the client, the embodiment requests retransmission of only the audio data of the client with a fixed number of loud sounds, and the audio data of the client with a small volume does not request retransmission even if the packet loss occurs.

The embodiment sends the target audio data to the server, and determines whether the first target audio data is audio data (active audio) that needs to be forwarded or not through the server when the first target audio data of the target client is missing from the target audio data, and if not, prohibits the target client from receiving a request for requesting to acquire the missing first target audio data, so that request retransmission of the audio data of the client can be reduced, data processing amount of analysis of sending and receiving ends of the server is reduced, retransmission of audio data packets of the client can be reduced, uplink bandwidth of the client and downlink bandwidth of the server are reduced, the technical problem of low efficiency of processing the audio data is solved, and the technical effect of improving efficiency of processing the audio data is achieved.

Example 2

The embodiments of the present invention are illustrated below with reference to preferred embodiments. Specifically, an audio data transmission method in a multi-terminal semantic conference scene is exemplified.

In this embodiment, in a multi-port voice conference scenario, there are multiple clients, each client is configured to generate audio data and send the audio data to a server, and the server receives, by default, the audio data sent by all the clients, where the audio data is at least used to indicate a volume of audio sent by the client. In the audio data sent by the client, the server can select N paths of audio data with larger volume, determine the N paths of audio data as active audio data, and send the active audio data to other clients, so that each client can only receive the N paths of audio data with the maximum volume except the own audio data, and therefore when a plurality of users speak simultaneously through the clients, each user can be ensured to receive the N paths of audio data with the larger volume than the own audio data through the clients, the bandwidth is reduced, and meanwhile, the ambient interference sound is filtered.

In this embodiment, packet loss retransmission (NACK) is that the server sends a retransmission request, that is, a NACK request, to the sender when it does not receive data. By notifying the sender of the retransmission request to retransmit the data, a message, such as "i did not receive message," may be sent to the notifier. Alternatively, the audio data packets of this embodiment are all transmitted in time sequence, and may all carry sequence numbers, where the sequence numbers may be sequential and ascending in sequence according to the transmission time. The NACK request of this embodiment carries the sequence number of the lost audio data packet, so that the sender retransmits the lost audio data packet.

Fig. 2 is a schematic diagram of audio data transmission between a server and a client according to an embodiment of the present invention. As shown in fig. 2, the server 20, the Client (Client)11, the Client 12, the Client 13, and the Client 14, wherein the Client 11 includes an audio transmission module (AudioSendStream)111 and an audio reception module (AudioRecvStream)112, the Client 12 includes an audio transmission module 121 and an audio reception module 122, the Client 13 includes an audio transmission module 131 and an audio reception module 132, and the Client 14 includes an audio transmission module 141 and an audio reception module 142; the server includes an endpoint (endpoint)21, an endpoint 22, an endpoint 23, and an endpoint 24, where the endpoint 21 includes an audio receiving module 211, an audio sending module 212, and an endpoint input pin 213, the endpoint 22 includes an audio receiving module 221, an audio sending module 222, and an endpoint input pin 223, the endpoint 23 includes an audio receiving module 231, an audio sending module 232, and an endpoint input pin 233, the endpoint 24 includes an audio receiving module 241, an audio sending module 242, and an endpoint input pin 243, and the server further includes a filter (excepts self filter) 25.

In this embodiment, the client 11 corresponds to an endpoint 21 of the server 20, the endpoint 21 being used for transmission of audio data between the server 20 and the client 11; the client 12 corresponds to an endpoint 22 of the server 20, the endpoint 22 being used for the transmission of audio data between the server 20 and the client 12; the client 13 corresponds to an endpoint 13 of the server, the endpoint 13 being used for transmission of audio data between the server 20 and the client 13; the client 14 corresponds to an endpoint 14 of the server, the endpoint 14 being used for transmission of audio data between the server 20 and the client 14.

The client 11, the client 12, the client 13, the client 14 and the corresponding endpoint 21, the endpoint 22, the endpoint 23 and the endpoint 24 in this embodiment all have an audio sending module (the audio sending module 121, the audio sending module 131, the audio sending module 141, the audio sending module 212, the audio sending module 222, the audio sending module 232 and the audio sending module 242) and an audio receiving module (the audio receiving module 112, the audio receiving module 122, the audio receiving module 132, the audio receiving module 142, the audio receiving module 211, the audio receiving module 231 and the audio receiving module 241), where the audio sending module, that is, the audio stream sending module, is used for sending audio data, and the audio receiving module, that is, the audio stream receiving module, is used for receiving audio data.

In the conference scenario of this embodiment, the client 11, the client 12, the client 13, and the client 14 join the conference, and the server receives audio data generated by the client 11, the client 12, the client 13, and the client 14 respectively during the conference. The volume of the audio data currently generated by the client 11 is greater than the volume of the audio data generated by the client 12, the client 13, and the client 14. When a plurality of users speak simultaneously through the corresponding clients, the attention of the listener is limited, and the server may select only the audio data of the client 11 with the largest volume for forwarding (N ═ 1). That is, although the server receives the audio data generated by the client 12, the client 13, and the client 14 during the conference, the server does not forward the audio data to other clients, but only forwards the audio data generated by the client 11, and the volumes of the audios indicated by the audio data received by the client 12, the client 13, and the client 14 are all greater than the volume of the audio indicated by the audio data generated by itself, that is, other clients can only receive one path of audio data with the largest volume of the audio, so that the bandwidth is reduced, and ambient interference sounds can be filtered.

Alternatively, in this embodiment, although the volume of audio indicated by the audio data of the client 11 is the largest among the volumes indicated by the audio data of the clients 12, 13, 14, the client 11 cannot receive its own audio data. The server can select the audio data of the client with the largest volume from the client 12, the client 13 and the client 14, store the audio data into the filter 25 through the input pin 213, and forward the audio data to the client 11 through the filter 25, so that the client 11 is only subjected to one path of audio data with the largest volume except for the client itself, thereby reducing the bandwidth and filtering out ambient environmental interference sounds.

Fig. 3 is a flowchart of another audio data processing method according to an embodiment of the present invention. As shown in fig. 3, the method comprises the steps of:

step S301, the server obtains N paths of audio data with the maximum volume in the current conference.

In this embodiment, the server obtains multiple channels of audio data sent by multiple clients, where each channel of audio data corresponds to each client one to one. The method includes selecting N (which may be a natural number of N < 3) audio data with a large volume from multiple paths of audio data sent by multiple clients, and determining the N paths of audio data as active audio data (active audio).

Optionally, in this embodiment, the audio data sent by the client may be an audio data packet, where a header of the audio data packet may carry volume information (audioLevel) of an audio currently generated by the client, and the volume information indicates a volume level of the audio. The server extracts the volume information in each audio data packet according to the audio data packet sent by each client in the conference scene every other target time period, sequences the volume information in each audio data packet, and takes the client corresponding to the N paths of audio data packets with the maximum volume indicated by the volume information as the client capable of generating active audio. The target time period is also an update period of the active audio data.

In step S302, the server updates the status of whether each audio data of each client is active audio data to each endpoint of the server.

After acquiring the N paths of audio data with the largest volume in the current conference and determining the N paths of audio data as active audio data, updating a state of whether each audio data of each client is active audio data to each endpoint of the server, for example, the audio data sent by the client 11 is active audio data, updating the state to an endpoint 21 corresponding to the client 11, and the client 12, the client 13, and the client 14 are inactive audio data, updating the state to a corresponding endpoint 22, an endpoint 23, and an endpoint 24, respectively.

In this embodiment, after determining active audio data in the audio data of the multiple clients, the connection state between the filter (explicit self filter) of the server and the endpoint input pins in different endpoints is updated, and the endpoint input pins in the endpoints corresponding to inactive audio data may be removed, for example, the audio data of the client 12, the client 13, and the client 14 are inactive audio data, and the endpoint input pin 223 corresponding to the client 12, the endpoint input pin 233 corresponding to the client 13, and the endpoint input pin 243 corresponding to the client 14 are removed, so that the audio data of the client 12, the client 13, and the client 14 are not forwarded to other clients; in this embodiment, the endpoint input pin corresponding to the new active audio is connected to the filter, for example, the audio data of the client 11 is active audio data, and the endpoint input pin 213 corresponding to the client 11 is connected to the filter 25, so that the server can forward the active audio data to other clients.

Step S303, the audio receiving module of each endpoint of the server determines whether to request retransmission of the audio data packet according to whether the audio data of the corresponding client is active audio data and the packet loss condition of the audio data packet.

Optionally, in this embodiment, after the server updates the status of whether each audio data of each client is active audio data to each endpoint of the server, the audio receiving module of each endpoint of the server may subsequently receive the audio data sent by the corresponding client. Optionally, the audio receiving module of each endpoint of the server determines whether audio data sent by a corresponding client is received, if audio data sent by a corresponding client is not received, it is determined whether the state of the audio data of the client recorded by the endpoint is active audio data, if the audio data of the corresponding client is active audio data, a retransmission of an audio data packet is requested to the corresponding client, and if the audio data of the corresponding client is inactive audio data, no retransmission of an audio data packet is requested to the corresponding client, so that when packet loss of the inactive audio data occurs, the server still sends a NACK request to the client generating the inactive audio data, which causes useless NACK requests and retransmissions, thereby avoiding waste of bandwidth resources.

Optionally, in this embodiment, the sending of the NACK request is performed in the audio receiving module of each endpoint, that is, after the server receives the audio data of the multiple clients, it is determined whether there is an audio data packet loss condition, and optionally, the audio data packets of this embodiment are sent in time sequence, and may all have sequence numbers, where the sequence numbers may be sequence numbers that are arranged in ascending order according to the sending time and are consecutive, for example, the multiple clients send audio data packets with sequence numbers from 1 to 10, when the server does not receive the audio data packet with sequence number 5, and the audio data packets before and after sequence number 5 are received, the server may determine that the audio data packet with sequence number 5 is lost, if the client that generates the audio data with sequence number 5 has been determined as the client that generates active audio data in the latest update period of active audio data, the server needs to send a NACK request to the client, where the NACK request carries a sequence number of the lost audio data packet, for example, 5, so that the client resends the lost audio data packet. After receiving the NACK request, the client retransmits the lost audio data packet to the server through an audio transmission module of the client, so that the server receives an active audio data packet.

Fig. 4 is a flowchart of another audio data processing method according to an embodiment of the present invention. As shown in fig. 4, the method comprises the steps of:

step S401, acquiring audio data packets sent by multiple clients.

Step S402, the missing packet sequence number of the audio data packet is obtained from the module for sending the NACK request.

The missing packet sequence number (nacksist) of this embodiment is a missing packet sequence number calculated according to the sequence number of the received audio data packet, for example, if a plurality of clients send audio data packets with sequence numbers from 1 to 10, and if the server does not receive an audio data packet with sequence number 5 and audio data packets with sequence numbers after 5 are received, the server may determine that the audio data packet with sequence number 5 is lost.

Step S403, determine whether the missing packet sequence number is empty.

Step S404, determining whether the client state corresponding to the missing packet sequence number is a state of generating active audio data.

And under the condition that the sequence number of the missing packet is not empty, judging whether the state of the client corresponding to the sequence number of the missing packet is the state of generating active audio data.

Step S405, requesting to retransmit the audio data of the client corresponding to the missing packet sequence number.

And if the state of the client corresponding to the missing packet sequence number is judged to be the state of generating active audio data, requesting to retransmit the audio data of the client corresponding to the missing packet sequence number.

And when the missing packet serial number is null or the state of the client corresponding to the missing packet serial number is not the state of generating active audio data, ending the flow of the method for processing the audio data.

The embodiment can be used for a method that a server side only requests to retransmit active audio in a conference scene, and under the condition that the audio data acquired by the server contains lost audio data, whether the lost audio data is the active audio data which needs to be forwarded to other clients is determined to determine whether the lost audio data needs to be retransmitted, if the lost audio data does not need to be retransmitted, the request for acquiring the lost audio data from a target client side is forbidden, so that excessive NACK (negative acknowledgement) requests can be prevented from being sent to the client side, the data processing amount of analysis of a server side sending and a receiving end is reduced, the retransmission of audio data packets of the client side is reduced, and the uplink bandwidth of the client side and the downlink bandwidth of the server are reduced.

It should be noted that the steps illustrated in the flowcharts of the figures may be performed in a computer system such as a set of computer-executable instructions and that, although a logical order is illustrated in the flowcharts, in some cases, the steps illustrated or described may be performed in an order different than presented herein.

Example 3

The embodiment of the invention also provides a device for processing the audio data. It should be noted that the audio data processing apparatus of this embodiment may be used to execute the audio data processing method shown in fig. 1A according to the embodiment of the present invention.

Fig. 5 is a schematic diagram of an apparatus for processing audio data according to an embodiment of the present invention. As shown in fig. 5, the audio data processing apparatus 500 includes: a receiving unit 10, an acquiring unit 20 and a prohibiting unit 30.

The receiving unit 10 is configured to receive first audio data of one or more first clients of the multiple clients, respectively, to obtain target audio data.

The obtaining unit 20 is configured to obtain a target status of a target client in a case that first target audio data of the target client is missing in the target audio data, where the plurality of clients includes the target client, and the target status is used to indicate that forwarding of the first target audio data is prohibited or is allowed to be performed on the first target audio data to a client associated with the target client.

A prohibiting unit 30, configured to prohibit sending a target request to the target client if the target status indicates that forwarding of the first target audio data is prohibited, where the target request is used to request the target client to retransmit the first target audio data.

It should be noted that the receiving unit 10 in this embodiment may be configured to execute step S102 in this embodiment, the obtaining unit 20 in this embodiment may be configured to execute step S104 in this embodiment, and the prohibiting unit 30 in this embodiment may be configured to execute step S106 in this embodiment.

In the embodiment, when the first target audio data of the target client is missing in the acquired target audio data, whether the first target audio data is audio data (active audio) which needs to be forwarded is determined, and if not, the request for acquiring the missing first target audio data from the target client is prohibited, so that the request for retransmission of the audio data of the client can be reduced, the technical problem of low efficiency in processing the audio data is solved, and the technical effect of improving the efficiency in processing the audio data is further achieved.

The embodiment of the invention also provides a device for processing the audio data. It should be noted that the audio data processing apparatus of this embodiment may be used to execute the audio data processing method shown in fig. 1B according to the embodiment of the present invention.

Fig. 6 is a schematic diagram of another audio data processing apparatus according to an embodiment of the present invention. As shown in fig. 6, the audio data processing apparatus 600 includes: a transmitting unit 40 and a disabling unit 50.

A sending unit 40, configured to send target audio data to a server, where the target audio data includes first audio data of one or more first clients of the multiple clients.

The prohibiting unit 50 is configured to, in a case that the first target audio data of the target client is missing in the target audio data, and in a case that the target status of the target client indicates that the server is prohibited from forwarding the first target audio data of the target client, prohibit the target client from receiving a target request sent by the server, where the plurality of clients include the target client, and the target request is used to request the target client to retransmit the first target audio data.

It should be noted that the sending unit 40 in this embodiment may be configured to execute step S202 in this embodiment, and the prohibiting unit 50 in this embodiment may be configured to execute step S204 in this embodiment.

Example 4

The embodiment of the invention also provides a storage medium. The storage medium includes a stored program, wherein the apparatus in which the storage medium is located is controlled to execute the audio data processing method in the embodiment of the present invention when the program is executed.

Example 5

The embodiment of the invention also provides a processor. The processor is used for running a program, wherein the program executes the audio data processing method in the embodiment of the invention.

It will be apparent to those skilled in the art that the modules or steps of the present invention described above may be implemented by a general purpose computing device, they may be centralized on a single computing device or distributed across a network of multiple computing devices, and they may alternatively be implemented by program code executable by a computing device, such that they may be stored in a storage device and executed by a computing device, or fabricated separately as individual integrated circuit modules, or fabricated as a single integrated circuit module from multiple modules or steps. Thus, the present invention is not limited to any specific combination of hardware and software.

The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method of processing audio data, comprising:

respectively receiving first audio data of one or more first clients in a plurality of clients to obtain target audio data;

acquiring a target state of a target client under the condition that first target audio data of the target client is missing in the target audio data, wherein the plurality of clients include the target client, and the target state is used for indicating that forwarding of the first target audio data is prohibited or the first target audio data is allowed to be forwarded to the client associated with the target client;

and under the condition that the target state indicates that the first target audio data is forbidden to be forwarded, forbidding sending a target request to the target client, wherein the target request is used for requesting the target client to retransmit the first target audio data.

2. The method of claim 1, wherein after obtaining the target status of the target client, the method further comprises:

sending the target request to the target client if the target status indicates permission to forward the first target audio data to the client associated with the target client;

receiving the first target audio data sent by the target client in response to the target request;

forwarding the first target audio data to the client associated with the target client.

3. The method of claim 1, wherein prior to receiving the first audio data of the one or more first clients of the plurality of clients, respectively, the method further comprises:

receiving second audio data of the plurality of clients;

determining the target state of each of the clients according to the second audio data of each of the clients.

4. The method of claim 3, wherein determining the target state for each of the clients according to the second audio data for each of the clients comprises:

extracting first volume data from the second audio data of each client, wherein the first volume data is used for indicating the volume of audio emitted by the corresponding client;

determining a first target number of first target clients associated with each client from the plurality of clients according to the first volume data of each client, wherein each client is used for receiving the second audio data of each first target client associated with each client, the volume of audio indicated by the first volume data of each first target client is greater than the volume of audio indicated by the first volume data of a second client, and the second client is any one of the plurality of clients except each client and the first target number of first target clients;

determining the states of the first target clients of the first target number as first target states respectively, wherein the first target states are used for indicating that the second audio data of the first target clients are allowed to be forwarded to the clients associated with the first target clients;

determining states of a second target client with a second target number in the plurality of clients as second target states respectively, wherein the second target clients with the second target number are clients except the first target clients with the first target number in the plurality of clients, and the second target states are used for indicating that the second audio data of the second target clients are prohibited to be forwarded.

5. The method of claim 1, wherein after inhibiting sending of target requests to the target client, the method further comprises:

receiving third audio data of the plurality of clients;

and updating the target state of each client according to the third audio data of each client.

6. The method of claim 5, wherein updating the target state for each of the clients according to the third audio data for each of the clients comprises:

extracting second volume data from the third audio data of each client, wherein the second volume data is used for indicating the volume of audio emitted by the corresponding client;

determining a third target number of third target clients associated with each client from the plurality of clients according to the second volume data of each client, wherein each client is used for receiving the third audio data of each third target client associated with each client, the volume of audio indicated by the second volume data of each third target client is greater than the volume of audio indicated by the second volume data of each third client, and the third client is any one of the plurality of clients except each client and the third target number of third clients;

updating the statuses of the third target clients of the third target number to first target statuses, respectively, wherein the first target statuses are used to indicate permission to forward the third audio data of the third target clients to the clients associated with the third target clients;

respectively updating states of a fourth target client with a fourth target number in the plurality of clients to a second target state, wherein the fourth target client with the fourth target number is a client of the plurality of clients except the third target client with the third target number, and the second target state is used for indicating that the third audio data of the fourth target client is prohibited to be forwarded.

7. The method of claim 5, wherein obtaining third audio data for the plurality of clients comprises:

and acquiring the third audio data of the plurality of clients every other target time period.

8. The method of claim 7, wherein updating the target status for each of the clients according to the third audio data for each of the clients comprises:

and updating the target state of each client according to the third audio data of each client every other target time period.

9. The method according to any of claims 1 to 8, wherein before obtaining the target status of the target client, the method further comprises:

acquiring a target list, wherein the target list comprises identification information of the first audio data of each first client;

determining that the first target audio data is missing in the target audio data in a case where the identification information of the first target audio data is not included in the target list.

10. The method of any one of claims 1 to 8, wherein receiving first audio data of one or more first clients of the plurality of clients respectively, and obtaining the target audio data comprises:

and respectively acquiring the first audio data of the one or more first clients in a conference scene to obtain the target audio data.

11. A method of processing audio data, comprising:

sending target audio data to a server, wherein the target audio data comprises first audio data of one or more first clients of a plurality of clients;

and when the target state of the target client indicates that the server is prohibited from forwarding the first target audio data of the target client, prohibiting the target client from receiving a target request sent by the server, wherein the plurality of clients include the target client, and the target request is used for requesting the target client to retransmit the first target audio data.

12. An apparatus for processing audio data, comprising:

the receiving unit is used for respectively receiving first audio data of one or more first clients in the plurality of clients to obtain target audio data;

an obtaining unit, configured to obtain a target status of a target client when first target audio data of the target client is missing in the target audio data, where the plurality of clients include the target client, and the target status is used to indicate that forwarding of the first target audio data is prohibited or is allowed to be performed on the first target audio data to the client associated with the target client;

a prohibiting unit, configured to prohibit sending a target request to the target client when the target status indicates that forwarding of the first target audio data is prohibited, where the target request is used to request the target client to retransmit the first target audio data.

13. An apparatus for processing audio data, comprising:

the device comprises a sending unit, a receiving unit and a processing unit, wherein the sending unit is used for sending target audio data to a server, and the target audio data comprises first audio data of one or more first clients in a plurality of clients;

a prohibiting unit, configured to prohibit, when first target audio data of a target client is missing in the target audio data, and when a target status of the target client indicates that the server is prohibited from forwarding the first target audio data of the target client, the target client from receiving a target request sent by the server, where the plurality of clients include the target client, and the target request is used to request the target client to retransmit the first target audio data.

14. A storage medium, characterized in that the storage medium comprises a stored program, wherein the program, when executed, controls an apparatus in which the storage medium is located to perform the method of any one of claims 1 to 11.

15. A processor, characterized in that the processor is configured to run a program, wherein the program when running performs the method of any of claims 1 to 11.