CN102354498A

CN102354498A - Method, device and system for processing mixed sound

Info

Publication number: CN102354498A
Application number: CN2011102364032A
Authority: CN
Inventors: 孙常春; 汪丽娟; 高瑞娟
Original assignee: Shenzhen Ipanel TV Inc
Current assignee: Shenzhen Ipanel TV Inc
Priority date: 2011-08-17
Filing date: 2011-08-17
Publication date: 2012-02-15

Abstract

The invention discloses a method, device and system for processing a mixed sound. The method comprises the following steps of: establishing a mixed sound channel between an audio frequency server and a terminal when the terminal confirms that the mixed sound processing of a sound is needed; on the established mixed sound channel, indicating each audio frequency sound Ur1 needing to be processed during the mixed sound processing to the audio frequency server; and receiving a mixed sound processing result sent by the audio frequency server, wherein the mixed sound processing result is acquired by performing the mixed sound processing on each audio frequency by the audio frequency server. The method, device and system can be used for promoting the processing speed and efficiency of the terminal.

Description

A kind of mixed audio processing method, Apparatus and system

Technical field

The present invention relates to the audio treatment technology, relate in particular to a kind of mixed audio processing method, Apparatus and system.

Background technology

In the prior art, all need the terminal that audio is carried out audio mixing under a lot of situation and handle.For example; A simple game on line; In the process that the user plays; The client at user place possibly sent many sound simultaneously; The sound of walking about such as background sound, keypad tone, personage, score sound or the like; These sound send simultaneously, just need resolve voice data synthetic, the process of audio mixing just.

In the prior art, all be to be handled by the audio mixing that sound is carried out at the terminal, concrete: the audio mixing that the terminal is read the source file of each sound and carried out sound is handled, and audio mixing is handled the sound that obtains be sent to audio decoder and play.Above-mentioned audio mixing processing procedure need be accomplished in the internal memory of terminal, and well-known, the internal memory at terminal is generally smaller; Be used to deposit those and need the summary responses file; Handle if carry out audio mixing more simultaneously, the terminal internal memory that takies is very big, has reduced the processing speed and the efficient at terminal.

Summary of the invention

In view of this, the technical matters that the present invention will solve is, a kind of mixed audio processing method, Apparatus and system are provided, and can improve the processing speed and the efficient at terminal.

For this reason, the embodiment of the invention adopts following technical scheme:

The embodiment of the invention provides a kind of mixed audio processing method, comprising:

When the terminal is confirmed to carry out the audio mixing processing of sound, and set up the audio mixing passage between the audio server;

On the audio mixing passage of being set up, the sound Url of each road audio frequency of required processing in audio server indication audio mixing is handled;

Receive the audio mixing result that audio server is sent, said audio mixing result is carried out the audio mixing processing by audio server to each road audio frequency and is obtained.

Setting up the audio mixing passage between said terminal and the audio server comprises:

The SETUP request message is sent to audio server in the terminal, and said SETUP message comprises: the Url address and the data transfer mode of audio server;

The response message of the said SETUP request message that the reception audio server is sent comprises in the said response message: first code status and data transfer mode; Said first code status is used to indicate the audio mixing passage whether to set up success.

Said in audio server indication audio mixing is handled the sound Url of each road audio frequency of required processing comprise:

The ADD request message is sent to audio server in the terminal, and said ADD request message comprises: the sound Url and the audio attribute of audio server Url, audio frequency; Said audio frequency is for carrying out the audio frequency that audio mixing is handled;

Receive the ADD response message that said audio server is sent, said ADD response message comprises: the sign of second code status and this road audio frequency; Said second code status is used to indicate this road audio frequency whether to add success.

Said in audio server indication audio mixing is handled the sound Url of each road audio frequency of required processing also comprise:

Confirm to send corresponding control request message according to control types to audio server to carrying out road audio frequency that audio mixing handles when controlling; Said control request message comprises: the sign of the audio frequency of audio server Url and required control

Receive the corresponding control response message of said control request message that audio server is sent, comprise in the said control response message: third generation sign indicating number sign, third generation sign indicating number sign are used to indicate for the whether success of the control and treatment of the audio frequency of said required control.

Said control types comprises: suspend the audio mixing of the audio mixing of audio mixing, deletion one road audio frequency of the audio mixing of one road audio frequency, road audio frequency that cancellation suspends, the audio mixing that suspends all road audio frequency, the audio mixing of all road audio frequency of recovering to be suspended, the audio mixing of deleting all road audio frequency, replacement one road audio frequency, in audio mixing to one road audio frequency be fade-in fade-out processing, adjust, the broadcasting time of one road audio frequency adjusted for the volume of one road audio frequency.

Also comprise: when the terminal confirms that the audio mixing processing finishes, the audio mixing passage of setting up between destruction and the audio server.

The audio mixing passage of setting up between terminal destruction and the audio server comprises:

The TEARDOWN request message is sent to audio server in the terminal, comprises in the said TEARDOWN request message: the Url of audio server.

Also comprise:

After the audio mixing passage was set up, the terminal is every sent heartbeat message at a distance from Preset Time, comprises in the said heartbeat message: audio server Url and heartbeat message sign;

Receive the heartbeat response message that audio server sends, comprise in the said heartbeat response message: the 5th code status and heartbeat message sign; Whether said the 5th code status is used to indicate said audio mixing passage unimpeded.

The embodiment of the invention also provides a kind of mixed audio processing method, comprising:

When the terminal is confirmed to carry out the audio mixing processing of sound, set up the audio mixing passage between audio server and the terminal;

On the audio mixing passage of being set up, the sound Url of each road audio frequency of required processing during the audio server receiving terminal is indicated, audio mixing is handled;

Said each road audio frequency is carried out audio mixing handle, the audio mixing result is sent to the terminal.

Setting up the audio mixing passage between audio server and the terminal comprises:

The SETUP request message that the audio server receiving terminal is sent, said SETUP message comprises: the Url address and the data transfer mode of audio server;

Audio server sends the response message of said SETUP request message to the terminal, comprises in the said response message: first code status and data transfer mode; Said first code status is used to indicate the audio mixing passage whether to set up success.

The sound Url of each road audio frequency of required processing comprised during said audio server receiving terminal is indicated, audio mixing was handled:

The ADD request message that the audio server receiving terminal is sent, said ADD request message comprises: the sound Url and the audio attribute of audio server Url, audio frequency; Said audio frequency is for carrying out the audio frequency that audio mixing is handled;

Audio server obtains audio frequency according to the sound Url of said audio frequency; One of audio frequency that the audio frequency that gets access to is handled as audio mixing carries out audio mixing to be handled;

Send the ADD response message to the terminal, said ADD response message comprises: the sign of second code status and this road audio frequency; Said second code status is used to indicate this road audio frequency whether to add success.

The sound Url of each road audio frequency of required processing also comprised during said audio server receiving terminal is indicated, audio mixing was handled:

The control request message that the audio server receiving terminal is sent according to control types; Said control request message comprises: the sign of the audio frequency of audio server Url and required control;

Audio server carries out the corresponding control and treatment of said control types to the audio frequency of said required control;

Send the corresponding control response message of said control request message to the terminal, comprise in the said control response message: third generation sign indicating number sign, third generation sign indicating number sign are used to indicate for the whether success of the control and treatment of the audio frequency of said required control.

Said control types comprises: suspend the audio mixing of audio mixing, replacement one road audio frequency of audio mixing, deletion one road audio frequency of the audio mixing of one road audio frequency, road audio frequency that cancellation suspends, in audio mixing to one road audio frequency be fade-in fade-out processing, adjust, the broadcasting time of one road audio frequency adjusted for the volume of one road audio frequency.

Also comprise:

When the terminal confirms that the audio mixing processing finishes, the audio mixing passage of setting up between destruction and the audio server.

The TEARDOWN request message that the audio server receiving terminal is sent comprises in the said TEARDOWN request message: the Url of audio server, so that carry out the destruction of said audio mixing passage according to the TEARDOWN request message.

Also comprise:

After the audio mixing passage was set up, the every heartbeat message that sends at a distance from Preset Time of receiving terminal comprised in the said heartbeat message: audio server Url and heartbeat message sign;

Audio server sends the heartbeat response message to the terminal, comprises in the said heartbeat response message: the 5th code status and heartbeat message sign; Whether said the 5th code status is used to indicate said audio mixing passage unimpeded.

The embodiment of the invention provides a kind of audio mixing treating apparatus, comprising:

First sets up the unit, when being used to confirm to carry out the audio mixing processing of sound, and sets up the audio mixing passage between the audio server;

First indicating member is used at the audio mixing passage of being set up, the sound Url of each road audio frequency of required processing in audio server indication audio mixing is handled;

First receiving element is used to receive the audio mixing result that audio server is sent, and said audio mixing result is carried out the audio mixing processing by audio server to each road audio frequency and obtained.

First sets up the unit comprises:

First sends subelement, is used for sending the SETUP request message to audio server, and said SETUP message comprises: the Url address and the data transfer mode of audio server;

First receives subelement, is used to receive the response message of the said SETUP request message that audio server sends, and comprises in the said response message: first code status and data transfer mode; Said first code status is used to indicate the audio mixing passage whether to set up success.

First indicating member comprises:

Second sends subelement, is used for sending the ADD request message to audio server, and said ADD request message comprises: the sound Url and the audio attribute of audio server Url, audio frequency; Said audio frequency is for carrying out the audio frequency that audio mixing is handled;

Second receives subelement, is used to receive the ADD response message that said audio server is sent, and said ADD response message comprises: the sign of second code status and this road audio frequency; Said second code status is used to indicate this road audio frequency whether to add success.

First indicating member also comprises:

The 3rd sends subelement, is used for confirming sending corresponding control request message according to control types to audio server to carrying out road audio frequency that audio mixing handles when controlling; Said control request message comprises: the sign of the audio frequency of audio server Url and required control;

The 3rd receives subelement; Be used to receive the corresponding control response message of said control request message that audio server is sent; Comprise in the said control response message: third generation sign indicating number sign, third generation sign indicating number sign are used to indicate for the whether success of the control and treatment of the audio frequency of said required control.

The embodiment of the invention also provides a kind of audio mixing treating apparatus, comprising:

Second sets up the unit, when being used for the terminal and confirming to carry out the audio mixing of sound and handle, and sets up the audio mixing passage between the terminal;

Second indicating member is used at the audio mixing passage of being set up, the sound Url of each road audio frequency of required processing during receiving terminal is indicated, audio mixing is handled;

First transmitting element is used for that said each road audio frequency is carried out audio mixing and handles, and the audio mixing result is sent to the terminal.

Second sets up the unit comprises:

The 4th receives subelement, is used for the SETUP request message that receiving terminal is sent, and said SETUP message comprises: the Url address and the data transfer mode of audio server;

The 4th sends subelement, is used for sending to the terminal response message of said SETUP request message, comprises in the said response message: first code status and data transfer mode; Said first code status is used to indicate the audio mixing passage whether to set up success.

Second indicating member comprises:

The 5th receives subelement, is used for the ADD request message that receiving terminal is sent, and said ADD request message comprises: the sound Url and the audio attribute of audio server Url, audio frequency; Said audio frequency is for carrying out the audio frequency that audio mixing is handled;

First handles subelement, is used for obtaining audio frequency according to the sound Url of said audio frequency; One of audio frequency that the audio frequency that gets access to is handled as audio mixing carries out audio mixing to be handled;

The 5th sends subelement, is used for sending the ADD response message to the terminal, and said ADD response message comprises: the sign of second code status and this road audio frequency; Said second code status is used to indicate this road audio frequency whether to add success.

Second indicating member also comprises:

The 6th receives subelement, is used for the control request message that receiving terminal is sent according to control types; Said control request message comprises: the sign of the audio frequency of audio server Url and required control

Second handles subelement, is used for the audio frequency of said required control is carried out the corresponding control and treatment of said control types;

The 6th sends subelement; Be used for sending the corresponding control response message of said control request message to the terminal; Comprise in the said control response message: third generation sign indicating number sign, third generation sign indicating number sign are used to indicate for the whether success of the control and treatment of the audio frequency of said required control.

A kind of audio mixing disposal system of the embodiment of the invention comprises:

The audio mixing passage when being used to confirm to carry out the audio mixing processing of sound, and is set up in the terminal between the audio server; On the audio mixing passage of being set up, the sound Url of each road audio frequency of required processing in audio server indication audio mixing is handled; Receive the audio mixing result that audio server is sent, said audio mixing result is carried out the audio mixing processing by audio server to each road audio frequency and is obtained;

Audio server, be used for and the terminal between set up the audio mixing passage; On the audio mixing passage of being set up, the sound Url of each road audio frequency of required processing during receiving terminal is indicated, audio mixing is handled; Said each road audio frequency is carried out audio mixing handle, the audio mixing result is sent to the terminal.

Technique effect analysis for technique scheme is following:

By the terminal the required audio frequency that carries out the audio mixing processing is indicated to audio server; Carrying out audio mixing by audio server handles; The audio mixing result is sent to the terminal; Carry out representing of audio mixing result; Thereby the terminal need not to carry out the audio mixing processing of audio frequency again; Thereby reduced the data processing amount at terminal, improved the data processing speed and the efficient at terminal.

Description of drawings

Fig. 1 is a kind of mixed audio processing method schematic flow sheet of the embodiment of the invention;

Fig. 2 is the another kind of mixed audio processing method schematic flow sheet of the embodiment of the invention;

Fig. 3 is the third mixed audio processing method schematic flow sheet of the embodiment of the invention;

Fig. 4 is a kind of audio mixing treating apparatus of embodiment of the invention structural representation;

Fig. 5 is the another kind of audio mixing treating apparatus of an embodiment of the invention structural representation;

Fig. 6 is a kind of audio mixing disposal system of embodiment of the invention structural representation;

Fig. 7 is the application example of the embodiment of the invention.

Embodiment

Below, be described with reference to the accompanying drawings the realization of embodiment of the invention mixed audio processing method, Apparatus and system.

Fig. 1 is a kind of mixed audio processing method schematic flow sheet of the application embodiment, and this method is applicable to end side; As shown in Figure 1, this method comprises:

Step 101: when the terminal is confirmed to carry out the audio mixing processing of sound, and set up the audio mixing passage between the audio server;

Step 102: on the audio mixing passage of being set up, the sound URL(uniform resource locator) (Url) of terminal each road audio frequency of required processing in audio server indication audio mixing is handled;

Step 103: the terminal receives the audio mixing result that audio server is sent, and said audio mixing result is carried out the audio mixing processing by audio server to each road audio frequency and obtained.

Fig. 2 is the another kind of mixed audio processing method schematic flow sheet of the embodiment of the invention, is applicable to audio server one side, and as shown in Figure 2, this method comprises:

Step 201: when the terminal is confirmed to carry out the audio mixing processing of sound, set up the audio mixing passage between audio server and the terminal;

Step 202: on the audio mixing passage of being set up, the sound Url of each road audio frequency of required processing during the audio server receiving terminal is indicated, audio mixing is handled;

Step 203: said each road audio frequency is carried out audio mixing handle, the audio mixing result is sent to the terminal.

In the mixed audio processing method illustrated in figures 1 and 2; By the terminal the required audio frequency that carries out the audio mixing processing is indicated to audio server; Carrying out audio mixing by audio server handles; The audio mixing result is sent to the terminal; Carry out representing of audio mixing result; Thereby the terminal need not to carry out the audio mixing processing of audio frequency again, thereby has reduced the data processing amount at terminal, has improved the data processing speed and the efficient at terminal.

On the basis of Fig. 1 and Fig. 2, the sound mixed audio processing method of the application embodiment is carried out more detailed explanation through Fig. 3.As shown in Figure 3, this method comprises:

Step 301: when the terminal is confirmed to carry out the audio mixing processing of sound, send DESCRIBE message, comprise in the DESCRIBE message: audio server URL, Cseq, User-Agent to audio server;

Wherein, the order of Cseq sign session is so that terminal and audio server can be confirmed the corresponding relation between request message (request) and the response message (response) according to Cseq; User-Agent is used for the sign and the version number of indicating terminal.

For example, said DESCRIBE message can for:

iamp：//192.168.45.242:5056/mixaudio?IAMP/1.0\r\n；

CSeq：1\r\n；

User-Agent：iPanel?V1.0\r\n。

Step 302: audio server judges according to the DESCRIBE message that receives whether own resource satisfies the audio mixing treatment requirement, if send DESCRIBE normal response message to the terminal; Otherwise, send redirect response message to the terminal.

Wherein, can comprise in the DESCRIBE normal response message: return code state, CSeq, server; Redirect response message can comprise: conversation order, server, the Location of return code state, correspondence.

Wherein, the return code state in the normal response message can be set to normal condition, as r, and the return code state in the redirect response message can be set to: error condition, like 302 mistakes etc.; Server representes audio service end sign and version number; Location representes the Redirect Address that audio server returns.After set-top box is received this Redirect Address; Again carry out the describe request again; It can have a field redirectionTimes; Show redirected number of times, whenever be redirected once that this value adds 1; Can maximal value be set for this field redirectionTimes; For example 10, when its value reaches maximal value and still returns redirect response message, just illustrate that audio server does not have enough resources.

For example, said normal response message can for:

IAMP/1.0?200?OK\r\n

CSeq：1\r\n

Server：iPanelAudioServer?V1.0\r\n。

Redirect response message can for:

IAMP/1.0?302?Moved?Temporarily\r\n

CSeq：1\r\n

Server：iPanelAudioServer?V1.0\r\n

Location：iamp://192.168.45.243:5056/mixaudio?redirectionTimes＝1\r\n

Wherein, step 301 and step 302 are optional step.

Step 303: when the return code state in the terminal judges response message is normal condition, send the SETUP request message to audio server; Said SETUP request message is used for setting up the audio mixing passage to the audio server request.

Can comprise in the said SETUP message: the Url of audio server, CSeq, Transport;

Wherein, Transport is used to represent data transfer mode.

For example, said SETUP request message can for:

SETUP?iamp：//192.168.45.242:5056/mixaudio?IAMP/1.0\r\n

CSeq：1\r\n

Transport：MP2T/AVP/UDP；unicast；destination＝10.11.22.33；port＝10000\r\n

(or Transport:MP2T/AVP/TCP; R)

Wherein, Transport representes data transfer mode, and this parameter has a plurality of information, with "; " separate, as follows:

-MP2T/AVP/UDP: the expression data channel, it is coded format that udp protocol MP2T/MP2E/PCM is adopted in transmission;

-unicast: clean culture is adopted in expression, has only MP2T/AVP/UDP then to come into force;

-destination: the IP of expression clean culture generally is the IP address of set-top box;

-port: the port of expression clean culture, this value is specified a not port of usefulness by set-top box;

-MP2T/AVP/TCP: the expression data channel is walked IP, and Transmission Control Protocol is adopted in transmission;

-http: expression transmission The data http mode, have only MP2T/AVP/TCP to come into force;

-address: using when returning, only is to adopt " MP2T/AVP/TCP; Http " time comes into force, and is the download address of http.

Step 304: audio server sends the SETUP response message according to the SETUP request message that receives to the terminal.

When the code status in the SETUP response message was normal condition, the audio mixing passage between expression terminal and the audio server was set up successfully.

Wherein, can comprise in the SETUP response message: first code status, CSeq, Session, Transport, StreamAudioPid, UdpNatPort.

Wherein, the code status in the embodiment of the invention is used for all representing whether corresponding request message processing of request completes successfully; Here, first code status is used to indicate said audio mixing passage whether to set up success.In practical application, for example can carry out the sign of said code status through following mode, concrete:

First bit digital of state code has defined the classification of responding, the not concrete classification of back two digits.First digit has 5 kinds of values possibilities:

-1xx: keep, use in the future;

-2xx: success-operation be received, understand, accept (received, understood, accepted);

-3xx: be redirected (Redirection)-accomplish request must further operate;

-4xx: the terminal grammar mistake of makeing mistakes-ask maybe can't realize;

-5xx: server end makes mistakes-and server can't realize legal request.

Session is a session identification, and this value is that session between audio server and each terminal distributes a unique value by audio server so that in the session at follow-up and this terminal, carry, carry out and this terminal between the sign of session; StreamAudioPid is the pid value of audio stream, and is general, and this value has only when transport stream in the transmission channel is ts, just can come into force.UdpNatPort:5057: be used for illustrating port from udp nat information to server that send out.

For example, said SETUP response message can for:

IAMP/1.0?200?OK\r\n

CSeq：1\r\n

Session：91972680469864186\r\n

(or

Transport：MP2T/AVP/TCP；http；address＝http://192.168.10.252:8001/23.mp2\r\n)

StreamAudioPid：123\r\n

UdpNatPort：5057

Step 305: when the code status that carries in the terminal judges SETUP response message is normal condition; The terminal begins and the interval Preset Time sends heartbeat message (GET_PARAMETER), can comprise in the heartbeat message: audio server URL, Cseq, heartbeat content type (Content-Type), Session.

For example, said heartbeat message can for:

GET_PARAMETER?iamp：//192.168.45.242:5056/mixaudio?IAMP/1.0\r\n

CSeq：6\r\n

Content-Type：text/parameters\r\n

Session：91972680469864186\r\n

Step 306: audio server receives said heartbeat message; State according to audio mixing passage between terminal and the audio server; Send the heartbeat response message to the terminal, can comprise in the heartbeat response message: the 5th code status, CSeq, Content-Type, Session.

Whether the 5th code status is used between indicating terminal and the audio server audio mixing passage unimpeded.

For example, said heartbeat response message can for:

IAMP/1.0?200?OK\r\n

CSeq：6\r\n

Content-Type：text/parameters

Session：91972680469864186\r\n。

Step 307: when code status is normal condition in the terminal judges SETUP response message, send the ADD request message to audio server;

Can comprise in the ADD request message: audio server URL, need carry out sound Url, audio attribute, FadeInTime, StartVolumeSize (initial volume of the effect of fading in), Cseq, the Session of the audio frequency that audio mixing handles, audio attribute can comprise: VolumeSize (volume), Loop (number of times that this road audio mixing circulation is broadcast) etc.

Wherein, can be provided with and send an ADD request message then is audio mixing processing interpolation one road audio frequency, and repeatedly ADD can be sent in the terminal, carries out audio mixing and handles to realize the interpolation multichannel voice frequency.

For example, said ADD request message can for:

ADD?iamp：//192.168.45.242:5056/mixaudio?IAMP/1.0\r\n

Url：file://1.wav\r\n

(or Url:igmp: //r)

FadeInTime：5000\r\n

StartVolumeSize：0\r\n

VolumeSize：100\r\n

Loop：-1\r\n

CSeq：4\r\n

Session：91972680469864186\r\n；

Wherein, Loop: the number of times that this road audio frequency circulation is broadcast is an integer; If then broadcast for circulation forever its value＜=0, recommended value is-1; If its value＞0, the then number of times of broadcasting for circulation.

MixId: in a session, this road audio frequency of unique identification, 32, generate by service end, suggestion identified with the time: this value is in a session, to each ADD, Replace request;

-in same session, same audio frequency url address, if when twice ADD instruction, MixId is different;

-this value terminal will use;

-npt=beginning-: broadcast from the starting position, finish to playing.

-FadeInTime: fade in the time, the ms of unit, value is represented not implementation effect smaller or equal to 0, directly begins to play by the VolumeSize volume.

-StartVolumeSize: the effect of fading in initial volume.Value is handled by 0 smaller or equal to 0.

-VolumeSize: volume, when carrying out, the effect of fading in representes that effect finishes volume.Its value can be 1～400, expression 1%～400%; At this moment, 100 is normal quantity.Adjustment back volume equals the initial volume size and multiply by number percent.

Step 308: audio server obtains audio frequency according to the sound Url of said audio frequency; One of audio frequency that the audio frequency that gets access to is handled as audio mixing carries out audio mixing to be handled, and sends the ADD response message to the terminal;

Wherein, can comprise in the ADD response message: second code status, MixId, CSeq, Session.

Wherein, said second code status is used for indicating the audio frequency of ADD request message request interpolation whether to add success.MixID is the audio setting that described request is added by audio server, is used for this road audio frequency of unique identification.

Wherein, said ADD response message can for:

IAMP/1.0?200?OK\r\n

MixId：12\r\n

CSeq：4\r\n

Session：91972680469864186\r\n

Above step 307 and step 308 repeat, and then can need carry out the audio frequency of audio mixing to the audio server interpolation.

And in practical application, except the audio frequency that adds audio mixing, carry out in the process of audio mixing at the request audio server, can also the audio frequency that add be controlled, for example, can include but not limited to following control types:

Suspend the audio mixing of the audio mixing of audio mixing, deletion one road audio frequency of the audio mixing of one road audio frequency, road audio frequency that cancellation suspends, the audio mixing that suspends all road audio frequency, the audio mixing of all road audio frequency of recovering to be suspended, the audio mixing of deleting all road audio frequency, replacement one road audio frequency, in audio mixing to one road audio frequency be fade-in fade-out processing, adjust, the broadcasting time of one road audio frequency adjusted for the volume of one road audio frequency.

Also promptly, this method can also comprise:

Step 309: the terminal is confirmed to send corresponding control request message according to control types to audio server to carrying out road audio frequency that audio mixing handles when controlling; Said control request message comprises: the sign of the audio frequency of audio server Url and required control;

Step 310: audio server receives said control request message; Audio frequency to said required control carries out the corresponding control and treatment of said control types; Send control response message to the terminal; Comprise in the said control response message: third generation sign indicating number sign, third generation sign indicating number sign are used to indicate for the whether success of the control and treatment of the audio frequency of said required control.

Concrete:

(1) control types is: when suspending the audio mixing of one road audio frequency; Said control request message can be the PAUSE request message, and this message can comprise: the sign of the audio frequency of audio server URL, MixId, FadeOutTime (fading out the time), EndVolumeSize the end volume of effect (audio frequency fade out), CSeq, Session and required time-out; Said control response message can be the PAUSE response message, and this message comprises: third generation sign indicating number state, CSeq, Session.Whether this third generation sign indicating number state is used to indicate audio server successful for the time-out of this road audio frequency.

For example, this PAUSE request message can for:

PAUSE?iamp：//192.168.18.157:554/mixaudio?IAMP/1.0\r\n

MixId：12\r\n

FadeOutTime：5000\r\n

EndVolumeSize：0\r\n

CSeq：3\r\n

Session：91972680469864186\r\n

The PAUSE response message can for:

IAMP/1.0?200?OK\r\n

CSeq：3\r\n

Session：91972680469864186\r\n。

(2) during the audio mixing of the road audio frequency that suspends for cancellation when control types; Said control request message can for: the RESUME request message can comprise in this message: the sign of the audio frequency that audio server URL, MixId, FadeInTime, StartVolumeSize (initial volume of the effect of fading in), Cseq, Session, Range (scope of beginning is broadcast in expression) and required cancellation suspend; Said control response message can for: the RESUME response message can comprise in this message: third generation sign indicating number state, CSeq, Session, Range; This third generation sign indicating number state is used to indicate the time-out for said audio frequency whether to cancel success.

For example, said RESUME request message can for:

RESUME?iamp：//192.168.18.157:554/mixaudio?IAMP/1.0\r\n

MixId：12\r\n

FadeInTime：5000\r\n

StartVolumeSize：0\r\n

CSeq：2\r\n

Session：91972680469864186\r\n

The RESUME response message can for:

IAMP/1.0?200?OK\r\n

CSeq：2\r\n

Session：91972680469864186\r\n

This instruction meeting is after the user sends the PAUSU instruction, and the RESUME that redispatches instruction for current location begins to continue to play, finishes up to playing

Wherein, Range: the scope of beginning is broadcast in expression, can use the npt form to specify:

-npt: until what time expression finishes from when beginning; "-" is preceding to be the start time point, "-" back expression concluding time point, and unit is a millisecond;

-npt=0.0-: expression finishes up to playing since section broadcast 0.0 normal time;

-npt=now-: begin from current location, finish to playing;

-npt=beginning-: broadcast from the starting position, finish to playing.

(3) when control types is the audio mixing of replacement one road audio frequency; Said control request message can be the REPLACE request message; Can comprise in this message: sound Url (requiring the sound address of the new audio frequency of audio mixing), FadeOutTime (fading out the time), FadeInTime (fading in the time), VolumeSize, CSeq, the Session of the sign of the audio frequency of audio server URL, required replacement (for example, can be MixID), new audio frequency; Said control response message can for: the REPLACE response message can comprise in this message: third generation sign indicating number state (be used to indicate said replacement whether success), MixId, CSeq, Session.

For example, the REPLACE request message can for:

REPLACE?iamp：//192.168.45.242:5056/mixaudio?IAMP/1.0\r\n

MixId：12\r\n

Url：file://1.wav

(or Url:igmp: //r)

FadeOutTime：5000\r\n

FadeInTime：3000\r\n

VolumeSize：100\r\n

Loop：10\t\n

CSeq：5\r\n

Session：91972680469864186\r\n

The REPLACE response message can for:

IAMP/1.0?200?OK\r\n

MixId：12\r\n

CSeq：5\r\n

Session：91972680469864186\r\n

Wherein, FadeOutTime: fade out the time, the ms of unit, value can be represented not implementation effect smaller or equal to 0, directly finishes preceding this road voice playing of replacement.

FadeInTime: fade in the time, the ms of unit, value can be represented not implementation effect smaller or equal to 0, this road audio frequency after the replacement directly begins to play by the VolumeSize volume.

(4) when control types in audio mixing, one road audio frequency being fade-in fade-out when handling, it handles with the adjustment of volume similar, does not give unnecessary details here.

This control types is used to control the effect of being fade-in fade-out of a certain road audio frequency.

In addition; When audio frequency is fade-in fade-out processing; Can combine other control types together audio frequency to be controlled; For example can be in interpolation, deletion, time-out, a certain road of recovery audio frequency; Indication is carried out controls such as corresponding time-out, recovery, deletion, interpolation for this road audio frequency through the mode of fading in or fading out, for example, and when adding a certain audio frequency; Carry out the interpolation of this road audio frequency through the mode of fading in, the corresponding time and the effect of fading in of fading in set begins volume or the like; Perhaps, when deleting a certain road audio frequency, carry out the deletion of this road audio frequency through the mode of fading out, corresponding time of fading out and the effect of the fading out end volume set suspended perhaps to recover to wait and operated similarly, and in the literary composition of back description arranged, and do not give unnecessary details here.

(5) when control types when adjusting for the volume of one road audio frequency; Control request message can for: the VOLUME request message can comprise in this message: audio server URL, MixId (need carry out the sign of the audio frequency of volume adjustment), VolumeSize, CSeq, Session; Control response message can for: the VOLUME response message can comprise in this message: third generation sign indicating number state, CSeq, MixId, VolumeSize, Session.Whether the volume adjustment that the third generation sign indicating number state here is used for indicative audio is successful.

For example, the VOLUME request message can for:

VOLUME?iamp：//192.168.45.242:5056/mixaudio?IAMP/1.0\r\n

MixId：12\r\n

VolumeSize：156\r\n

CSeq：14\r\n

Session：91972680469864186\r\n

The VOLUME response message can for:

IAMP/1.0?200?OK\r\n

CSeq：14\r\n

MixId：12\r\n

VolumeSize：156\r\n

Session：91972680469864186\r\n

Wherein ,-VolumeSize: volume, its value can be 1～400, expression 1%～400%; 100 is normal quantity.Adjustment back volume equals the initial volume size and multiply by number percent.

(6) when control types when the broadcasting time of one road audio frequency adjusted, it is similar that its disposal route and volume are adjusted, and do not give unnecessary details here.

(7) when control types when one road audio frequency deleted; Control request message can be the REMOVE request message, and this message can comprise: the sign (like MixID) of the audio frequency of audio server URL, needs deletion, FadeOutTime (fading out the time), EndVolumeSize the end volume of effect (audio frequency fade out), CSeq, Session; Said control response message can for: the REMOVE response message can comprise in this message: third generation sign indicating number state (whether the audio frequency of expression request deletion deletes success), CSeq, Session.

For example, the REMOVE request message can for:

REMOVE?iamp：//192.168.45.242:5056/mixaudio?IAMP/1.0\r\n

MixId：12\r\n

FadeOutTime：5000\r\n

EndVolumeSize：0\r\n

CSeq：13\r\n

Session：91972680469864186\r\n

The REMOVE response message can for:

IAMP/1.0?200?OK\r\n

CSeq：13\r\n

Session：91972680469864186\r\n。

(8) when control types is the audio mixing of all road audio frequency of time-out; Control request message can be the PAUSEALL request message, and this message can comprise: audio server URL, FadeOutTime (fading out the time), EndVolumeSize the end volume of effect (audio frequency fade out), CSeq, Session; Said control response message can be the PAUSEALL response message, and this message can comprise: third generation sign indicating number state (the expression request suspends all audio frequency and whether suspends success), CSeq, Session.

For example, said PAUSEALL request message can for:

PAUSEALL?iamp：//192.168.18.157:554/mixaudio?IAMP/1.0\r\n

FadeOutTime：5000\r\n

EndVolumeSize：0\r\n

CSeq：3\r\n

Session：91972680469864186\r\n

The PAUSEALL response message can for:

IAMP/1.0?200?OK\r\n

CSeq：3\r\n

Session：91972680469864186\r\n

Wherein, FadeOutTime representes fading out the time of audio frequency, the ms of unit, and value is represented not implementation effect smaller or equal to 0, directly suspends and plays;

EndVolumeSize representes the fade out end volume of effect of audio frequency.Value can be handled by 0 smaller or equal to 0.

(9) when control types during for the audio mixing of all road audio frequency of recovering to be suspended; Control request message can be the RESUMEALL request message, and this message can comprise: audio server URL, FadeInTime (fading in the time), EndVolumeSize the end volume of effect (audio frequency fade in), CSeq, Session; Said control response message can be the RESUMEALL response message, and this message can comprise: third generation sign indicating number state (the expression request recovers all audio frequency and whether recovers success), CSeq, Session.

For example, the RESUMEALL request message can for:

RESUMEALL?iamp：//192.168.18.157:554/mixaudio?IAMP/1.0\r\n

FadeInTime：5000\r\n

EndVolumeSize：0\r\n

CSeq：3\r\n

Session：91972680469864186\r\n

The RESUMEALL response message can for:

IAMP/1.0?200?OK\r\n

CSeq：3\r\n

Session：91972680469864186\r\n

Wherein, FadeInTime representes fading in the time of audio frequency, the ms of unit, and value is represented not implementation effect smaller or equal to 0, directly recovers to play.

EndVolumeSize represent the to fade in end volume of effect.Value can be handled by 0 smaller or equal to 0.

(10) when control types is the audio mixing of deletion (also can be regarded as and stop) all road audio frequency; Control request message can be the REMOVEALL request message, and this message can comprise: audio server URL, FadeOutTime (fading out the time), EndVolumeSize the end volume of effect (audio frequency fade out), CSeq, Session; Said control response message can be the REMOVEALL response message, and this message can comprise: third generation sign indicating number state (whether all audio frequency of expression request deletion delete success), CSeq, Session.

For example, the REMOVEALL request message can for:

REMOVEALL?iamp：//192.168.18.157:554/mixaudio?IAMP/1.0\r\n

FadeOutTime：5000\r\n

EndVolumeSize：0\r\n

CSeq：3\r\n

Session：91972680469864186\r\n

The REMOVEALL response message can for:

IAMP/1.0?200?OK\r\n

CSeq：3\r\n

Session：91972680469864186\r\n

Wherein, FadeOutTime representes fading out the time of audio frequency, the ms of unit, and value is represented not implementation effect smaller or equal to 0, directly stops to play.

EndVolumeSize representes the fade out end volume of effect of audio frequency.Value is handled by 0 smaller or equal to 0.

Above control procedure and the interpolation process of audio frequency can be intersected execution, and execution sequence can not limit according to the practical situations setting in practical application here.

Step 311: audio server carries out the audio mixing processing to the audio frequency of terminal indication, and the audio mixing result is sent to the terminal.

Wherein, do not have execution sequence between step 311 and the above-mentioned steps 307～step 310 yet, carry out said interpolation or control and treatment in execution in step 311 processes again.

Step 312: when the audio mixing of audio server is finished dealing with, send audio mixing state notifying (ANNOUNCE), can comprise in the ANNOUNCE message: audio server URL, Cseq, Notice (content of needs notice), Session to the terminal.

For example, ANNOUNCE message can for:

ANNOUNCE?iamp：//192.168.45.242:5056/mixaudio?IAMP/1.0\r\n

CSeq：1\r\n

Notice：type＝1；mixId＝3；state＝0\r\n

Session：91972680469864186\r\n

Annotate:

When-this instruction is used for service end notice terminal related information, initiatively sent by service end, the terminal need not returned;

Value among the-Notice is the content of needs notice:

-type: notification type: can value be 1, represent the current audio mixing state that is in; In practical application, type can also have other possibilities, can come with this type of audio mixing state phase region other through get different values for Type.

-mixId: the id of a certain road audio mixing;

-State: audio mixing state: 0, stop audio mixing; 1, the beginning audio mixing;

-only need broadcast naturally finish stop in transmission audio mixing state notifying;

-in the time of add, replace, remove instruction, need not send this notice (terminal is known).

Step 313: when the terminal confirms that the audio mixing processing finishes, the audio mixing passage of setting up between destruction and the audio server.

The concrete realization of this step can for:

The TEARDOWN request message is sent to audio server in the terminal, comprises in the said TEARDOWN request message: the Url of audio server, Session, Cseq;

Here, the TEARDOWN request message that audio server can directly be sent according to the terminal carries out the destruction of said audio mixing passage, can no longer send corresponding response message to the terminal.

Perhaps; Also can send the TEARDOWN response message to the terminal by audio server; At this moment; The terminal receives the TEARDOWN response message that audio server sends; Comprise in the said TEARDOWN response message: whether the 4th code status, CSeq, Session, said the 4th code status are used to indicate the destruction for the audio mixing passage successful.

For example the TEARDOWN request message can for:

TEARDOWN?iamp：//192.168.45.242:5056/mixaudio?IAMP/1.0\r\n

Session：91972680469864186\r\n

CSeq：5\r\n

The TEARDOWN response message can for:

IAMP/1.0?200?OK\r\n

CSeq：5\r\n

Session：91972680469864186\r\n。

Cseq in the present embodiment and session also can be optional parameter.

Corresponding with above mixed audio processing method, the embodiment of the invention also provides a kind of audio mixing treating apparatus and system.

Fig. 4 is a kind of audio mixing treating apparatus of embodiment of the invention structural representation, and this device can be arranged at the terminal, for example in the set-top box; As shown in Figure 4, this device can comprise:

First sets up unit 410, when being used to confirm to carry out the audio mixing processing of sound, and sets up the audio mixing passage between the audio server;

First indicating member 420 is used at the audio mixing passage of being set up, the sound Url of each road audio frequency of required processing in audio server indication audio mixing is handled;

First receiving element 430 is used to receive the audio mixing result that audio server is sent, and said audio mixing result is carried out the audio mixing processing by audio server to each road audio frequency and obtained.

Preferably, first set up unit 410 and can comprise:

Preferably, first indicating member 420 can comprise:

Preferably, first indicating member can also comprise:

Wherein, said control types can comprise: suspend the audio mixing of audio mixing, replacement one road audio frequency of audio mixing, deletion one road audio frequency of the audio mixing of one road audio frequency, road audio frequency that cancellation suspends, the audio mixing that suspends all road audio frequency, all road audio frequency of recovering to be suspended audio mixing, delete the audio mixing of all road audio frequency, in audio mixing to one road audio frequency be fade-in fade-out processing, adjust, the broadcasting time of one road audio frequency adjusted for the volume of one road audio frequency.

Preferably, returning apparatus can also comprise:

The first destruction unit, when being used for confirming that the audio mixing processing finishes, the audio mixing passage of setting up between destruction and the audio server.

The first destruction unit can comprise:

First sends submodule, is used for sending the TEARDOWN request message to audio server, comprises in the said TEARDOWN request message: the Url of audio server, so that audio server carries out the destruction of audio mixing passage.

Further, the first destruction unit further comprises:

First receives submodule, is used to receive the TEARDOWN response message that audio server sends, and comprises in the said TEARDOWN response message: whether the 4th code status, said the 4th code status are used to indicate the destruction for the audio mixing passage successful.

Preferably, this device can also comprise:

The heartbeat message transmitting element after being used for the audio mixing passage and setting up, whenever sends heartbeat message at a distance from Preset Time, comprises in the said heartbeat message: audio server Url and heartbeat message sign;

Heartbeat response receiving element is used to receive the heartbeat response message that audio server sends, and comprises in the said heartbeat response message: the 5th code status and heartbeat message sign; Whether said the 5th code status is used to indicate said audio mixing passage unimpeded.

Fig. 5 is the another kind of audio mixing treating apparatus of an embodiment of the invention structural representation, and this device goes in the audio server; As shown in Figure 5, this device can comprise:

Second sets up unit 510, when being used for the terminal and confirming to carry out the audio mixing of sound and handle, and sets up the audio mixing passage between the terminal;

Second indicating member 520 is used at the audio mixing passage of being set up, the sound Url of each road audio frequency of required processing during receiving terminal is indicated, audio mixing is handled;

First transmitting element 530 is used for that said each road audio frequency is carried out audio mixing and handles, and the audio mixing result is sent to the terminal.

Preferably, second set up unit 510 and can comprise:

Preferably, second indicating member 520 can comprise:

Preferably, second indicating member 520 can also comprise:

Said control types comprises: suspend the audio mixing of audio mixing, replacement one road audio frequency of audio mixing, deletion one road audio frequency of the audio mixing of one road audio frequency, road audio frequency that cancellation suspends, the audio mixing that suspends all road audio frequency, all road audio frequency of recovering to be suspended audio mixing, delete the audio mixing of all road audio frequency, in audio mixing to one road audio frequency be fade-in fade-out processing, adjust, the broadcasting time of one road audio frequency adjusted for the volume of one road audio frequency.

Preferably, this device can also comprise:

The second destruction unit, when being used for the terminal and confirming that the audio mixing processing finishes, destroy and the terminal between the audio mixing passage set up.

The second destruction unit can comprise:

Second receives submodule, is used for the TEARDOWN request message that receiving terminal is sent, and comprises in the said TEARDOWN request message: the Url of audio server, so that carry out the destruction of audio mixing passage according to said TEARDOWN request message.

Preferably, the second destruction unit further comprises:

Second sends submodule, is used for sending the TEARDOWN response message to the terminal, comprises in the said TEARDOWN response message: whether the 4th code status, said the 4th code status are used to indicate the destruction for the audio mixing passage successful.

Preferably, can also comprise:

The heartbeat message receiving element, after being used for the audio mixing passage and setting up, the every heartbeat message that sends at a distance from Preset Time of receiving terminal comprises in the said heartbeat message: audio server Url and heartbeat message sign;

Heartbeat response transmitting element is used for sending the heartbeat response message to the terminal, comprises in the said heartbeat response message: the 5th code status and heartbeat message sign; Whether said the 5th code status is used to indicate said audio mixing passage unimpeded.

Fig. 6 is a kind of audio mixing disposal system of the embodiment of the invention, and as shown in Figure 6, this system can comprise:

The audio mixing passage when being used to confirm to carry out the audio mixing processing of sound, and is set up in terminal 610 between the audio server; On the audio mixing passage of being set up, the sound Url of each road audio frequency of required processing in audio server indication audio mixing is handled; Receive the audio mixing result that audio server is sent, said audio mixing result is carried out the audio mixing processing by audio server to each road audio frequency and is obtained;

Audio server 620, be used for and the terminal between set up the audio mixing passage; On the audio mixing passage of being set up, the sound Url of each road audio frequency of required processing during receiving terminal is indicated, audio mixing is handled; Said each road audio frequency is carried out audio mixing handle, the audio mixing result is sent to the terminal.

In audio mixing treating apparatus shown in Fig. 4～6 and the system; By the terminal the required audio frequency that carries out the audio mixing processing is indicated to audio server; Carrying out audio mixing by audio server handles; The audio mixing result is sent to the terminal; Carry out representing of audio mixing result; Thereby the terminal need not to carry out the audio mixing processing of audio frequency again, thereby has reduced the data processing amount at terminal, has improved the data processing speed and the efficient at terminal.

The application that gives an actual example to above-mentioned mixed audio processing method, Apparatus and system describes, as shown in Figure 7:

For audio A: first ADD, at t3 constantly, send REMOVE

For audio frequency B: send the ADD instruction, broadcast once

For audio frequency C: send the ADD instruction, circulation is broadcast repeatedly

For audio frequency D: send the REPLACE instruction, from the beginning it is interrupted broadcast.

One of ordinary skill in the art will appreciate that; The process of the method for realization the foregoing description can be accomplished through the relevant hardware of programmed instruction; Described program can be stored in the read/write memory medium, and this program when carrying out the corresponding step in the said method.Described storage medium can be like ROM/RAM, magnetic disc, CD etc.

The above only is a preferred implementation of the present invention; Should be pointed out that for those skilled in the art, under the prerequisite that does not break away from the principle of the invention; Can also make some improvement and retouching, these improvement and retouching also should be considered as protection scope of the present invention.

Claims

1. a mixed audio processing method is characterized in that, comprising:

2. method according to claim 1 is characterized in that, sets up the audio mixing passage between said terminal and the audio server and comprises:

3. method according to claim 1 is characterized in that, said in audio server indication audio mixing is handled the sound Url of each road audio frequency of required processing comprise:

4. method according to claim 3 is characterized in that, said in audio server indication audio mixing is handled the sound Url of each road audio frequency of required processing also comprise:

5. method according to claim 4; It is characterized in that said control types comprises: suspend the audio mixing of the audio mixing of audio mixing, deletion one road audio frequency of the audio mixing of one road audio frequency, road audio frequency that cancellation suspends, the audio mixing that suspends all road audio frequency, the audio mixing of all road audio frequency of recovering to be suspended, the audio mixing of deleting all road audio frequency, replacement one road audio frequency, in audio mixing to one road audio frequency be fade-in fade-out processing, adjust, the broadcasting time of one road audio frequency adjusted for the volume of one road audio frequency.

6. according to each described method of claim 1 to 4, it is characterized in that, also comprise:

7. method according to claim 6 is characterized in that, the audio mixing passage of setting up between terminal destruction and the audio server comprises:

8. according to each described method of claim 1 to 4, it is characterized in that, also comprise:

9. a mixed audio processing method is characterized in that, comprising:

10. method according to claim 9 is characterized in that, sets up the audio mixing passage between audio server and the terminal and comprises:

11. method according to claim 9 is characterized in that, the sound Url of each road audio frequency of required processing comprised during said audio server receiving terminal is indicated, audio mixing was handled:

12. method according to claim 11 is characterized in that, the sound Url of each road audio frequency of required processing also comprised during said audio server receiving terminal is indicated, audio mixing was handled:

13. method according to claim 12; It is characterized in that said control types comprises: suspend the audio mixing of audio mixing, replacement one road audio frequency of audio mixing, deletion one road audio frequency of the audio mixing of one road audio frequency, road audio frequency that cancellation suspends, in audio mixing to one road audio frequency be fade-in fade-out processing, adjust, the broadcasting time of one road audio frequency adjusted for the volume of one road audio frequency.

14. according to each described method of claim 9 to 12, it is characterized in that, also comprise:

15. method according to claim 14 is characterized in that, the audio mixing passage of setting up between terminal destruction and the audio server comprises:

16. according to each described method of claim 9 to 12, it is characterized in that, also comprise:

17. an audio mixing treating apparatus is characterized in that, comprising:

18. device according to claim 17 is characterized in that, first sets up the unit comprises:

19. device according to claim 17 is characterized in that, first indicating member comprises:

20. device according to claim 19 is characterized in that, first indicating member also comprises:

21. device according to claim 20; It is characterized in that said control types comprises: suspend the audio mixing of the audio mixing of audio mixing, deletion one road audio frequency of the audio mixing of one road audio frequency, road audio frequency that cancellation suspends, the audio mixing that suspends all road audio frequency, the audio mixing of all road audio frequency of recovering to be suspended, the audio mixing of deleting all road audio frequency, replacement one road audio frequency, in audio mixing to one road audio frequency be fade-in fade-out processing, adjust, the broadcasting time of one road audio frequency adjusted for the volume of one road audio frequency.

22. an audio mixing treating apparatus is characterized in that, comprising:

23. device according to claim 22 is characterized in that, second sets up the unit comprises:

24. device according to claim 22 is characterized in that, second indicating member comprises:

25. device according to claim 24 is characterized in that, second indicating member also comprises:

26. device according to claim 25; It is characterized in that said control types comprises: suspend the audio mixing of the audio mixing of audio mixing, deletion one road audio frequency of the audio mixing of one road audio frequency, road audio frequency that cancellation suspends, the audio mixing that suspends all road audio frequency, the audio mixing of all road audio frequency of recovering to be suspended, the audio mixing of deleting all road audio frequency, replacement one road audio frequency, in audio mixing to one road audio frequency be fade-in fade-out processing, adjust, the broadcasting time of one road audio frequency adjusted for the volume of one road audio frequency.

27. an audio mixing disposal system is characterized in that, comprising: