WO2010130193A1

WO2010130193A1 - Device, method for controlling audio media packet transmission and audio media server

Info

Publication number: WO2010130193A1
Application number: PCT/CN2010/072616
Authority: WO
Inventors: 丁向军
Original assignee: 中兴通讯股份有限公司
Priority date: 2009-05-13
Filing date: 2010-05-11
Publication date: 2010-11-18
Also published as: CN101567853A; CN101567853B

Abstract

A device for controlling audio media packet sending includes a predictive packet transmission (PPT) control module and a time management packet transmission control module. The PPT control module is set to determine a predicted transmission time for a media message to be sent, and to link said media message with a link list indicated by a pointer array element corresponding to said predicted transmission time. The time management packet transmission control module is set to browse the link list indicated by the pointer array element according to an index of said element and determine whether a media message exists in the link list; if it does, the time management packet transmission control module transmits said media message at its predicted transmission time, and browses the link list indicated by the pointer array element according to the index of the next pointer array element. Correspondingly, the present invention also provides a method for controlling media packet transmission and an audio media server. Based on Host Media Processing (HMP), the present invention enables accurate control of packing intervals, and, in terms of capacity, is as good as a dedicated Digital Signal Processor (DSP).

Description

Audio media delivery control device, method and audio media server

TECHNICAL FIELD The present invention relates to a network communication technology in the field of computer applications, and in particular to an audio/media delivery control device based on Linux/vxworks, a media delivery control method, and an audio media server.

Background technique

The media server is an important device in the Next Generation Network (NGN), and its location in the NGN network is shown in Figure 1. The media processing system in the existing media server involves key technologies such as parsing, streaming, and precise control of the packing interval. For these key technologies, media processing systems in media servers in the industry are mainly implemented in two ways: hardware mode - Digital Singnal Processing (DSP) and software mode - Host Media Processing (HMP) technology. However, their drawbacks are: The hardware method is costly, and the software method is cost-effective, but the capacity is too small. Once the capacity is increased, the fluidization and packing interval precision control performance is degraded. The dedicated DSP technology is to process the media data through a specific algorithm in the dedicated chip. Generally, the dedicated DSP chip has strong media processing capability, and the channel density is also large. The price of each channel is fixed, and the channel density is higher. HMP technology is the development direction of the future media server, but compared with the dedicated DSP, the host media processing capacity is general, so in the case of large capacity, the packaging precision control will be significantly reduced, and the large capacity and packaging accuracy become its development. The bottleneck.

SUMMARY OF THE INVENTION The technical problem to be solved by the present invention is to provide an audio media delivery control device, a media delivery control method, and an audio media server, which improve the accuracy of media packet packing interval.

In order to solve the above technical problem, the present invention provides an audio media delivery control device, where the device includes an expected delivery control module and a time management delivery control module, wherein The expected delivery control module is configured to: determine an expected transmission time of the media message to be sent, and attach the media message to a linked list indicated by the pointer array element corresponding to the expected transmission time;

The time management delivery control module is configured to: browse whether there is a media message on the linked list indicated by the pointer array element according to the pointer array element index, and if there is a media message, the expected delivery time of the media message is The media message is sent out, and the linked list pointed by the pointer array element is browsed according to the next pointer array element index. The time management delivery control module is further configured to: when there is a media message on the linked list indicated by the pointer array element according to the pointer array element index, if there is no media message, delay until the expected transmission of the pointer array element Time, and browse the linked list of pointer array elements according to the next pointer array element index. The expected delivery control module is configured to determine an expected transmission time of the media message by: reading an expected transmission time of a current pointer array element being processed by the time management delivery control module; On the basis of time, a certain amount of time is added as the estimated sending time of the media message, so as to ensure that the time management sending module browses the media message attached to the corresponding pointer array element. The expected allocation time of the pointer array element index and the media message on the linked list indicated by the pointer array element has a corresponding relationship, and the expected transmission time interval of the adjacent pointer array elements is fixed. The predictive delivery control module and the time management delivery control module employ multi-thread processing technology.

In order to solve the above technical problem, the present invention further provides a media packet sending control method, including: a hooking step, comprising: determining an expected sending time of a media message to be sent and attaching the media packet to the device The step of the pointer array element corresponding to the expected sending time; and the sending step, comprising: browsing, according to the pointer array element index, whether there is a media message on the linked list indicated by the current pointer array element, and if there is a media message, The expected transmission time of the media message sends the media message, and browses the pointer array element according to the next pointer array element index. The linked list; if there is no media message, the delay is until the pointer array element is expected to send the time, according to the next pointer array element to read the linked list of pointer array elements. The attaching step is implemented by a plurality of expected delivery threads, and the sending step is implemented by a plurality of time management delivery control threads. Before determining an expected transmission time of the media message to be sent, the attaching step further includes: It is expected that the sending thread browses the channel it is responsible for; when it is found that a channel requires to send a media message, the estimated sending time of the media message to be sent is determined as follows:

A: The read time manages the index index of the current pointer array element currently being processed in the dispatch control thread and the expected send time foresndtime;

B: Determine the hook index of the first media message to be sent nextpkt_tindex and the expected transmission time nextpkt_foresndtime, where nextpkt_tindex=(index+n)% the number of elements of the pointer array; nextpkt_ foresndtime= Foresndtime+n* The time interval represented by each pointer array 1; where "%" means modulo, "*" means multiplication; n is a preset index margin to ensure that the time management packet control thread browses To the media message attached to the corresponding pointer array element;

C: Determine the hook index of the subsequent media message nextpkt—tindex and the expected sending time nextpkt— foresndtime , until all media messages are processed, where: nextpkt_index= nextpkt index + (packaging interval / time interval represented by each index) %

(the number of elements of the pointer array), wherein the packing interval is an integer multiple of the time interval represented by each index; "/" means division; nextpkt_foresndtime= nextpkt_foresndtime+ (packaging interval). The sending step is implemented by multiple time management delivery control threads;

Each time management delivery control thread performs the sending step as follows: A: whether there is a media message to be sent on the linked list indicated by the pointer array element of the current pointer array element index, and if so, the media message is read, and the address and load information of the media message is read. And the expected sending time parameter of the batch of media messages is sent to the send message (sendmsg) function; the sendmsg function sends the batch of media messages in batches at the expected sending time, and then returns at the expected sending time, executing B; if in the current pointer The array element index pointer array element refers to the linked list does not have to send a media message, the time management packet control thread calls the delay function (DelayTime) delay until the current index list of the expected transmission time, execute B;

B: Update the current index index and the expected sending time of the media message on the linked list of the current index, and return A; where, the current index: Index= ( Index+1 ) % (the maximum number of elements in the pointer array); the prediction corresponding to the current index Send time: foresndtime=foresndtime+ The time interval represented by each index. The attaching step further includes: adding, by each media message, a media message counter corresponding to the linked list by one; in the sending step, the time management sending packet control thread determines the current index of the linked list according to the media message counter Is there a media message to send?

In order to solve the above technical problem, the present invention provides an audio media server, including an audio media delivery control device, where the device includes an expected delivery control module and a time management delivery control module, wherein the predicted delivery control module is configured to: The expected sending time of the sent media message, and attaching the media message to the linked list indicated by the pointer array element corresponding to the expected sending time;

The time management delivery control module is configured to: browse whether there is a media message on the linked list indicated by the pointer array element according to the pointer array element index, and if there is a media message, the expected delivery time of the media message is The media message is sent out and browsed according to the next pointer array element index. The linked list pointed to by the pointer array element; if there is no media message, the delay is until the expected sending time of the element pointer array and the linked list indicated by the pointer array element is browsed according to the next pointer array element index. The expected delivery control module is configured to determine an expected transmission time of the media message by: reading an expected transmission time of a pointer array element being processed by the time management delivery control module; and an expected transmission time at the read Assuming a certain amount of time is added as the estimated sending time of the media message, to ensure that the time management sending module browses to the media message attached to the corresponding pointer array element. The audio media server further includes a signaling forwarding unit, a main control unit, and a media storage and forwarding unit, where the signaling forwarding unit is configured to: implement unified forwarding of internal and external signaling of the server; Receiving and parsing the control signaling forwarded by the signaling forwarding unit, and generating a corresponding control message to control the media storage and forwarding unit; and the media storage and forwarding unit has an external network port, and the media storage and forwarding unit is configured to: The recording and the voice playing are implemented according to the control message sent by the main control unit, wherein in the recording and the voice playing, the audio media sending control device implements the packet sending control of the media message. The media storage and forwarding unit is configured to implement recording and voice playback by using a multi-thread processing technology. The audio media server further includes a media processing unit, where the media processing unit includes a call proxy module, a transceiver number module, and a conference processing module, where the call proxy module is configured to: implement a control message between the master control unit and the media processing unit. Interacting, if the control message is a transceiver number message, the notification transceiver module is processed, and if the control message is a transcoding or conference message, the conference processing module is notified; the transceiver number module is set as: performing the receiver resource setting, and is in the media stream. The valid number is used for validity identification; and the control message generation number of the main control unit is set; the conference processing module is configured to: process the media stream from each participant, and combine the input speech frames of different participants into a mixed speech frame. And the voice frame output to each participant is The mixed voice frame filters out the input speech frame of the participant; the main control unit is further configured to: generate a control message for the media processing unit; and the media storage and forwarding unit is further configured to: include a multi-tone dual frequency The destination address of the media message of the DTMF number is converted into the address corresponding to the transceiver number channel.

Compared with the prior art, the present invention provides a packet control apparatus and method for managing an accurate and effective control of a media message pointer array mount index and an estimated transmission time by using a packet thread to manage a pointer array with time information, and can efficiently and efficiently All media packets at the index location are sent in batches at a predetermined transmission time, which achieves precise control of the packetization interval, provides high-precision Qos (Quality of Service) performance, and uses multi-thread processing technology with large capacity. Features. The audio media server of the present invention is based on HMP, has low cost, and not only realizes precise control of the packaging interval, but also has the capacity comparable to a dedicated DSP, and can provide media processing functions in basic and enhanced services, including service sound supply, conference, and interactive. Features such as answering, notification, and advanced voice services.

1 is a schematic diagram of a location of a media server in an NGN network; FIG. 2 is a schematic diagram of various functional modules of the media server of the present invention; FIG. 3 is a flowchart of audio source file parsing; Figure 5 is a block diagram showing the structure of the audio media packet transmission control apparatus of the present invention; Figure 6 is a schematic diagram of the audio packet delivery control method of the present invention; Figure 7 is a flowchart of the processing of the estimated packet delivery control module of the present invention; Time management delivery control module processing flow chart; Figure 9 is a DTMF number transmission and reception number flow chart; Figure 10 is a flow chart of the conference call.

A preferred embodiment of the present invention is shown in FIG. 2. The audio media server of the present invention includes a signaling forwarding unit, a main control unit connected to the signaling forwarding unit, a control/media switching unit connected to the main control unit, and a control/ a media storage and forwarding unit and a media processing unit connected to the media switching unit, wherein the main control unit, the media storage and forwarding unit, and the media processing unit mutually control the flow or media stream through the control/media exchange unit, the media storage and forwarding unit, and the media processing unit It is a carrier for implementing the media processing technology of the present invention. The signaling forwarding unit is configured to: mainly implement the isolation between the internal network of the system and the external network, and the control signaling is uniformly forwarded by the signaling forwarding unit to implement a single external Internet Protocol (IP) address presentation; wherein, the control signaling The method includes: a Session Initiation Protocol (SIP), a UI 248, and a Media Gateway Control Protocol (MGCP). The main control unit is configured to: receive and parse control signaling, and generate corresponding control messages to control media storage. The forwarding unit and the media processing unit perform corresponding channel processing on playback, recording, transceiving number, conference, and transcoding; control signaling is a message in an international standard format, and the control message interprets each field in the control signaling after translation , composed of messages suitable for the delivery format inside the system. The control/media exchange unit is configured to: complete server internal control message exchange and media data packet exchange, and implement the function of the switch, which is a bridge for communication between other modules, and details are not described herein again. The media storage and forwarding unit is configured to: perform a large amount of multimedia data storage, including a voice library that implements a voice playback function and a storage of user recordings when the recording function is implemented. The media storage and forwarding unit is provided with an external network port. When the media does not need to perform additional conversion processing, the media is directly sent and received through the external network port on the media storage and forwarding unit. The media packet sent through the external network port is a streamed media packet. The packet is a packet conforming to the international standard format. When the recording service is implemented, the media stream is received through the external network port, if the terminal (such as a fixed battery) The terminal wants to record, and during the recording process, if no DTMF number (0-9, *, #) is pressed, then the DTMF number will not be included in the media stream; if the recording is completed (for example: # The number indicates the end of the recording.) Press # at this time, then the received media stream will contain the DTMF number (##). After the ## is detected, it will be reported to the main control unit, and the main control unit will issue the control. The message is used to close the channel to stop recording. The media storage and forwarding unit comprises an internal call proxy module, a recording media processing module, a sound media processing module, a delivery control module, a collection control module and a storage module, wherein the internal call (proxy) proxy module is set as: implementation and main control module The interaction of control messages is implemented by the control plane of the control/media switching unit. If the received control message is analyzed as a recording, the recording processing module is notified; if it is playing, the playback processing module is notified. The recording media processing module is configured to: after sorting the media messages received by the receiving control module, extract the payload into a specified file format (WAV/AMR). The playback media processing module is configured to: find a sound source file from the storage module according to parameters in the playback control message, parse the audio source file, read valid data of the sound source, and package and stream the media message suitable for network transmission; If the media packet encoding format of the source data stream is inconsistent with the encoding format required by the terminal, it needs to be sent to the media processing unit for processing; the packet sending control module is configured to: send the media packet through the internal and external network ports, specifically, the sending packet The control module, that is, the delivery control device, is the core of the present invention and will be described in detail below. The receiving control module is configured to: receive media packets through the internal and external network ports, and if the media message received from the external network port is inconsistent with the required stored source file format (WAV/AMR), it needs to be forwarded to the media processing unit. After being transcoded by the media processing unit, it is sent to the internal network port of the receiving control module, and then processed by the recording media processing module, and the destination address of the media message containing the DTMF number is converted into an address corresponding to the transceiver number channel. The storage module is set to: Store the source file. The media processing unit is set to: complete voice codec mode conversion, implement dual tone multi-frequency (Dual

Tone Multi-Frequency, DTMF) Transceiver number and conference mixing function. The media processing unit has Ethernet port and time division multiplexing (Time Division Multiplex and Multiplexer, TDM) interface to meet the individual needs of IP users and Public Switched Telephone Network (PSTN) users. The media processing unit includes a call proxy module, a transceiver number module, and a conference processing module. The call proxy module is configured to: implement interaction of control messages between the master control unit and the media processing unit, and notify the sending and receiving number if the control message is a transceiver number message. The module processes, if the control message is a transcoding or a conference message, notifies the conference processing module that the transceiver number module is set to: perform a receiver resource setting, and perform validity identification on the valid number in the media stream; and according to the main control unit The control message generation number is set; the conference processing module is configured to: process the media stream from each participant, combine the input speech frames of different participants into a mixed speech frame; and output the speech frame to each participant as a mixed speech frame. The input speech frame of the participant is filtered out, thus preventing the user from hearing his own echo. The audio media server of the present invention is a media processing technology implementation scheme suitable for various network users based on the Linux/vxworks operating system, and provides low cost and large capacity to overcome the shortage of the above DSP and HMP based on the improvement of the HMP mode. The media server of the present invention provides media processing functions required for various services under the control of the application server, including: media processing technologies such as playback, recording, DTMF receiving, conference, and transcoding. The main points of the media processing technology of the present invention are as follows: 1. Analysis and construction of the audio source file format: Since different types of files have different organization forms of audio sample data, it is necessary to analyze according to the corresponding format. In general, a file contains a description part and a data part of the data. At present, the processing of the audio source file supports waveform types (Wave Form, WAV) and Adaptive Multi-Rate (AMR). Table 1 shows the WAV file format, and Table 2 shows the AMR file format. Of course, other source types can be supported as required, completely controlled by software.

The WAV file header is one of the sound wave file formats used in multimedia, and is based on the Resource Interchange File Format (RIFF) format. AMR full name Adaptive Multi-Rate, adaptive multi-rate encoding, mainly used for audio in mobile devices, the compression ratio is relatively large. Table 1: WAV file format

RIFF WAVE Chunk

ID = 'RIFF'

RiffType = 'WAVE'

Format Chunk

ID = 'fmt '

Fact Chunk(optional)

ID = 'fact'

Data Chunk

ID = 'data' Table 2: AMR file format

+ +

I Header | t t | P| FT |Q|p|p|

One two two two +-+-+-+-+-+-+-+-+

I speech frame π |

+ +

0 1 2 3

0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + | P | FT=2 |Q| P| P| I +

I I

+ speech bits fmr f rame - b, cick n, channel k +

I I

+ +-+-+

I |P| P| + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one + one +

The block diagram of the analysis of the WAV file and the AMR file is shown in Figure 3. The construction flow is similar. The process of parsing the WAV file includes: reading a file header of the WAV file; extracting file header information, including data encoding mode, channel Number, sample frequency, and number of bits required for each sample; skip file description information; read data from audio data portion. The parsing process of the AMR file includes: reading the magic number of the file; reading the first frame data; calculating the size of the frame data according to the frame header; reading the frame header; determining whether the frame is bad, and reading the next frame header if the frame is bad, Otherwise, the frame voice data is read. 2. Streaming of voice data and voice structure of streamed data Streaming processing means that the data in the read source file is packaged by Real-time Transport Protocol (RTP) and constructed to be transmitted on the IP network. The RFC3550-compliant media ^艮文. This includes constructing IP/User Data (User Datagram Protocol, UDP)/RTP headers and payload data. If the encoding format of the voice data is the same as the encoding and decoding format required by the terminal user, the conversion processing is not required, and is directly streamed by the playback processing module of the media storage and forwarding unit and then sent through the external network port; if the voice data is encoded The format is inconsistent with the codec format required by the end user (such as G.711a/u, G.729, G.723), and voice data conversion is required. The process of streaming processing is shared by the media storage and forwarding unit and the media processing unit. Upon completion, the media stream interaction between the two is achieved through the media plane of the control/media switching unit.

The streaming process can be implemented in the following two ways: Method 1: The playback processing module of the media storage and forwarding unit restores the voice data to the PCM8 or PCM16 format, and the media processing unit converts it to the corresponding G.711/ by an algorithm. The encoded data of G.729/G.723, and finally the playback processing module of the media storage and forwarding unit performs the composition processing based on RTP. Manner 2: The playback processing module of the media storage and forwarding unit directly streams the voice data in the audio file, such as a/u-law or AMR frame, into an RTP message in the G.71 la/u or RFC3267 payload format. The conference processing module of the media processing unit then transcodes the RTP processed data into the destination codec G.729/G.723 mode. The second method is recommended for use in the present invention. Of course, for the streaming of WAV files, you need to know the source data encoding in the WAV file. (the source data encoding format is in the file header). The source and destination address information of IP/UDP is determined by the parameters brought by the main control unit, and the length of the payload data is determined by the packetization time information brought by the main control unit. For the streaming of AMR files, it is necessary to refer to the rfc3267 standard to change the storage format to the streaming data of the transport format. This involves the issue of different payload formats: byte-aligned format and bandwidth-efficient format. It is necessary to select different formats according to the Session Description Protocol (SDP) description in the signaling, and then perform format conversion of the audio voice frame in the payload. The voice structure of the streaming data is reversed: the payload in the streamed data is changed to the source data for storage. In this process, the received RTP data needs to be sorted and dithered, and format converted so that the voice data is stored in the correct file type in the correct format. Third, large-capacity multi-threaded processing technology

Each operation implemented by each module of the media storage and forwarding unit includes file header information construction during recording processing, RTP packet payload extraction, audio source parsing during effective voice storage and playback processing, data reading, streaming, packaging, and delivery. All use multi-thread processing technology, that is, each operation has multiple threads processed in parallel, thus achieving the characteristics of multi-channel and large capacity.

One of the most common models for manipulating files is the synchronous blocking I/O model. In this model, the user space application executes a system call, which causes the application to block. As shown in FIG. 4, taking a read operation as an example, the present invention utilizes a thread family (that is, multiple threads) to simulate an asynchronous read operation; a read request thread determines whether to initiate a read operation, and a read file thread reads data; Such a read request thread and a read file thread constitute a read request thread family and a read file thread family; thus, the asynchronous read file operation can be simulated by concurrently reading the file thread, and the structure of the RTP package thread family for packing processing is also shown in the figure. And the family of dispatching threads that perform the processing of the delivery package. The simulation of writing files is similar. When a file operation needs to be performed, a communication queue is used between the read/write request thread and the read-write file thread to communicate between the two thread families. Once some channels need to send media streams, they are constantly packed and packetized for the voice data being read. If it is handled by a process or thread, it will definitely cause "congestion", which will limit the large capacity; and the application of multiple threads concurrent processing solves these problems. The foundation laid the foundation. 4. High-precision packet transmission control technology A large number of media streams are sent from the inner and outer network ports of the packet transmission control unit of the media storage and forwarding unit. In order to accurately control the packetization interval of the streaming data (RTP data), the delivery control device of the present invention implements a method for managing an array of pointers with time information by sending a packet thread. As shown in FIG. 5, the delivery control device of the present invention includes an expected delivery control module. And a time management delivery control module, wherein: the expected delivery control module is configured to: cache the media message after the streaming media processing module is streamed, determine the estimated transmission time and attach to the linked list of the elements corresponding to the sending time of the pointer array. . Each network port corresponds to a certain number of caches for storing media packets. When no media packets are to be processed, all caches are on the idlelist. If media packets need to be processed, the packet control module is expected to be The application cache is used to load the media packet on the idle list idlelist, and then the packet is attached to the pointer array element managed by the time management delivery control module corresponding to the network port. After the time management packet control module sends the message on the array element, the cache is attached to the idle list idlelist of the network port. The estimated transmission time of the media message is determined as follows: reading the expected transmission time of the current pointer array element being processed by the time management delivery control module; adding a certain time remainder based on the expected transmission time of the reading The quantity is used as the expected sending time of the media message to ensure that the time management sending module browses the media message attached to the corresponding pointer array element. The index of the pointer array element and the expected transmission time of the media message on the linked list pointed by the pointer array element have a corresponding correspondence relationship, and the expected transmission time interval of the adjacent pointer array element is fixed. The time management packet control module is configured to: sequentially read the media message on the linked list of the pointer array element according to the pointer array element index, and send it out at the expected sending time; if the pointer array element chain header does not carry any media message, then A delay is required until the element is expected to be sent.

The expected delivery control module and the time management delivery control module adopt multi-thread processing technology Now. Correspondingly, as shown in FIG. 6, the media packet sending control method of the present invention includes the following steps: Step 601: The attaching step includes: determining an expected sending time of a media packet to be sent, and attaching the media packet to the packet On the linked list indicated by the pointer array element corresponding to the expected transmission time;

Step 602: The sending step includes: sequentially reading, according to the pointer array element index, the media message on the linked list indicated by the pointer array element, and sending the media message at the expected sending time; if the pointer array element If there is no media message on the list, it is delayed until the expected delivery time of the element. Further, the sending step includes: browsing, according to the pointer array element index, whether there is a media message on the linked list indicated by the current pointer array element, and if there is a media message, sending the media message at the expected sending time of the media message, and According to the next pointer array element index browsing the linked list of the pointer array element; if there is no media message, the delay until the pointer array element is expected to send time, according to the next pointer array element index read the pointer array element refers to the linked list

The function and steps of the module are described in detail below with reference to the accompanying drawings: The flow chart of the expected packet control module is shown in Figure 7. When it is expected that the expected packet-issuing thread family of the packet-issuing control module is created by the scheduling process, it enters the permanent while loop. In this while loop, the thread will be responsible for all the channels. Once a channel is found to require sending a media message, it will enter the packing function. In the packing function, it will check whether the media message to be sent is the first one. , then proceed as follows: A. The read time manages the index index of the array of pointers currently being processed in the thread control thread and the expected send time foresndtime; in order to be able to send the first media message, then the first medium The index and expected transmission time of the message must be the index and time after index and foresndtime, so it is necessary to reserve a certain margin (index margin and time margin) on index and foresndtime, so that the time management package control thread will browse to (Time management delivery control thread is browsed by index); currently The index of the first media message nextpkt_tindex and the expected transmission time nextpkt_foresndtime are calculated as: nextpkt- tindex=(index+n)% the number of elements in the pointer array; nextpkt—foresndtime=foresndtime+n* The time interval represented by the pointer array 1; Where "%" means modulo;

*" indicates multiplication; n is a preset index margin (such as η = 5) to ensure that the time management packet control thread browses to the media message attached to the corresponding pointer array element; corresponding time The remainder is the product of the index margin n and the time interval represented by each pointer array. The current time interval represented by each pointer array index is the fixed transmission interval of the adjacent pointer array elements (eg 10ms). The index and the expected sending time are in fact corresponding to each other. The first media packet is indexed by the current index, and the index is moved backward by n indexes. The corresponding estimated sending time is the current estimated sending time. The time interval is found. After the index number is found, the media message is attached to the linked list of the corresponding index of the pointer array, and the media message counter in the linked list structure is incremented by one.

B. Calculate the index and expected transmission time of the next media message to be sent in the channel. The algorithm is as follows: nextpkt_index= nextpkt index + (packaging interval/time interval represented by each cable) % (number of elements in the pointer array) Nextpkt_foresndtime= nextpkt— foresndtime+ (packaging interval) where “/” means divide; the packing interval here is an integer multiple of the time interval represented by each index. The simplest case is that the packing interval and the time interval represented by each index are identical. The second media message sent by the channel is used to find the linked list of the corresponding index of the pointer array by using the above calculated index number and expected transmission time, and the media message counter in the linked list structure is incremented by one.

C. The index number and expected transmission time of the subsequent media message are repeated in step B until the media message is processed. According to the above manner, the media packet that can be guaranteed by the pointer array element index of the media message is sent, and no packet loss occurs. There is no limit on the number of media messages stored on each pointer element. Even if media packets with multiple channels need to be sent at the same time, as long as the pointer array element index is determined according to the above principles, the media packets of the above multiple channels can be stored in the same Pointer array element The corresponding linked list is introduced, thereby providing a prerequisite for batch sending of media messages. The flow chart of the time management delivery control module is shown in Figure 8. When the time management of the time management delivery control module is created by the scheduling process, the current system time is obtained, and the time is rounded to the second as a pointer. The expected transmission time of the media message carried on the first index 0 list of the array is TO, the current index index=0, and then the always while loop, in this while loop will:

A. Browse the pointer array to see if there is any media message to be sent (through the media message counter) on the linked list of the current index. Once the media list of the current index is found, the media message will be removed, and then the media message will be removed. The address and payload information of the media messages and the expected transmission time parameters of the batch of media messages are transmitted to the sendmsg function, and the batch messages are sent in batches at the expected transmission time, and then returned at the expected transmission time. Execution B; If there is no media message to be sent on the current index list, the delay function DelayTime is called to delay the expected transmission time of the index list, and B is executed.

B. Update the current index index and the expected transmission time of the media message on the linked list of the current index, and execute C. Update method: Current index:

Index= ( Index+1 ) % (the maximum number of elements in the pointer array) The estimated transmission time corresponding to the current index: foresndtime+=the time interval represented by each index (10ms) where "+=" is the force assignment operator. Indicates the foresndtime=foresndtime+ time interval represented by each index;

C, repeat step eight, B, forever loop

The code is implemented as follows:

Void* TimeQueueThread(void *arg) gettimeofday(&tpSndTime, NULL); tpSndTime.tv — sec += 3;

tpSndTime.tv—usee = 0;

tpLastSndTime = tpSndTime;

wQStartlndex = 0;

Pointer Array [wQStartIndex].QForeCastSndTime

=tpSndTime; while(l) wQIndex = wQStartlndex;

wEleNum= Pointer Array [wQIndex] . wElememtNum

If(!wEleNum)

DelayTime();

Else

Sendmsg();

NextTv(&tpLastSndTime,MinPktTime*1000,&tpSndTime); tpLastSndTime = tpSndTime; wQStartlndex += 1;

wQStartlndex = wQStartlndex %QUEUE_MAX;

Pointer Array [wQStartIndex].QForeCastSndTime

=tpSndTime; Because for the end user, if the packetization interval is too large, it will seriously affect the voice quality. This packing interval accuracy is an important indicator of Qos, which directly affects the operator's evaluation of product quality. It can be said that in the case that the file read data and the package can be supplied in time, Qos completely depends on the accuracy of the packaging interval. The delivery control device of the invention can efficiently and efficiently send all the media messages of the index position at a predetermined transmission time by accurately and effectively controlling the media message in the pointer array and predicting the transmission time. . Five, DTMF transceiver number function, support in-band mode and RFC2833 mode

The DTMF transceiver number function is implemented on the transceiver module of the media processing unit. As shown in FIG. 9, when the transceiver unit of the media processing unit receives the request from the main control unit to open the transceiver number, the first series is performed. The setting of the receiver resource (including the creation of the transceiver number channel; setting the voice channel parameters, the detection mode of the DTMF number, and the activation of the receiver resource), and then the media storage and forwarding unit passes the network address translation (NAT) to the external network. The incoming media stream containing the DTMF number is forwarded to the receiving resource of the corresponding media processing unit, and the valid number in the media stream is detected and validated by the receiving resource (in accordance with the national standard GB9038-88) The sending number is to generate a valid DTMF number according to GB9038-88 and forward it to the destination terminal through the NAT of the media storage and forwarding unit. The above describes the process of transmitting and receiving numbers on the IP side. The transceiver number of the TDM side needs to introduce the Pulse Code Modulation (PCM) stream to the receiving resource of the media processing unit through E1/T1 for the same process. The configuration parameters are different only when the transceiver number is configured. The main control module configures the parameters of the transceiver number through the control message, so that the transceiver number is in the working state. When the number is desired, the media stream is forwarded through the nat of the media storage and forwarding unit, because the external network device can only When the external network port of the media storage and forwarding unit is displayed, the destination address of the media packet is also the address of the external network port (ip+port). Therefore, the media storage and forwarding unit needs to replace the destination address of the media packet with the external network port. Address corresponding to the channel, and then sent to the transceiver channel The mapping relationship between the address of the external network port and the address corresponding to the transceiver number channel is saved in the nat table; when the number is to be sent, the number needs to be generated according to the instruction of the master control number message. Sixth, support for 3-party and multi-party conferencing functions, maximum support for 64-party mixing conference/transcoding is implemented on the conference processing module of the media processing unit; the conference allows multiple users to participate in a conference, each user You can hear conversations from other users. Participants in the conference can be either TDM users or IP users. There are key processing methods in conference/transcoding, which can combine the input speech frames of different participants into a mixed speech frame; the speech frame output to each participant is to filter the mixed speech frames out of the participants. Enter a speech frame to prevent the user from hearing their own echo. Each participant can be a different codec; transcoding is just a special form of meeting. Participants in the conference can be set to a "mute" state, at which point the mixer will silently process the participant, but the participant can hear the voices of other participants; participants of the conference can be set to "keep" State, at this point the mixer treats the participant as silent, and the participant does not hear the voices of other participants; the participants of the conference can be set to the "announce hold" state, at which point the mixer takes the participation As a mute process, the participant can hear the voice of the source participant; the participant of the conference can be set to the "source participant" state, at which point the mixer inputs the participant's input to the other "announcement hold" Participants, the participants of other "announcement" can hear the voice of "source participants";"sourceparticipants" are typically used to play music, news, announcements, and so on. The implementation process of creating a conference is as shown in FIG. 10. When the conference processing module receives the create conference message sent by the main control unit, if the conference ID already exists, the conference control unit is reported, otherwise the conference ID and the participant ID are created, and the participant parameters are set. The session corresponding to the participant is enabled, the process of performing the mixed voice frame and filtering the participant's own voice frame is performed during the conference; when the participant joins the conference message, the participant ID is also executed and the participant parameter is set. The process of enabling the channel corresponding to the participant; if the message of the revocation conference participant sent by the main control unit is received during the conference, the conference ID is cancelled.

The present invention provides a packet control device and method for managing an array of pointers with time information by using an issue thread The method realizes accurate and effective control of the index of the media message pointer array and the expected transmission time, and can efficiently send all the media messages of the index position in batches at a predetermined sending time, thereby realizing the accuracy of the packing interval. Control, can provide high-quality quality of service (Qos) performance, in addition to multi-threaded processing technology, with large capacity.

Industrial Applicability The audio media server of the present invention is based on HMP implementation, has low cost, and not only realizes precise control of the packaging interval, but also has the capacity comparable to a dedicated DSP, and can provide media processing functions in basic and enhanced services, including service sound supply, conference, and Features such as interactive answering, notifications, and advanced voice services.

Claims

Claim

An audio media delivery control device, comprising an estimated delivery control module and a time management delivery control module, wherein the predicted delivery control module is configured to: determine an expected transmission time of a media message to be sent, and set the media The message is hooked onto the linked list indicated by the pointer array element corresponding to the expected sending time;

The time management delivery control module is configured to: browse whether there is a media message on the linked list indicated by the pointer array element according to the pointer array element index, and if there is a media message, the expected delivery time of the media message is The media message is sent out, and the linked list pointed by the pointer array element is browsed according to the next pointer array element index.

2. The apparatus according to claim 1, wherein: the time management packet control module is further configured to: if there is a media message on the linked list indicated by the pointer array element according to the pointer array element index, if there is no media message , then delay until the expected sending time of the pointer array element, and browse the linked list pointed by the pointer array element according to the next pointer array element index.

3. The apparatus according to claim 1, wherein: the expected delivery control module is configured to determine an expected transmission time of the media message by: reading a current pointer that the time management delivery control module is processing The estimated transmission time of the array element; adding a certain time margin to the expected transmission time of the media message based on the expected transmission time of the read, to ensure that the time management delivery module browses to the corresponding pointer The media message of the array element.

4. Apparatus according to claim 1 or 2 or 3, wherein: said pointer array element index and relationship, and the expected transmission time interval of adjacent pointer array elements is fixed.

The apparatus according to claim 1 or 2 or 3, wherein: said predictive delivery control module and said time management delivery control module employ multi-thread processing technology.

A media packet control method, comprising: a hooking step, comprising: determining an expected sending time of a media message to be sent and attaching the media message to a pointer array element corresponding to the expected sending time And the sending step, comprising: displaying, according to the pointer array element index, whether there is a media message on the linked list indicated by the current pointer array element, and if there is a media message, the estimated sending time of the media message Sending the media message, and browsing the linked list of the pointer array element according to the next pointer array element index; if there is no media message, delaying until the pointer array element is expected to send time, according to the next pointer array The element) reads the linked list of pointer array elements.

The method according to claim 6, wherein: the attaching step is implemented by a plurality of expected packet sending threads, and the sending step is implemented by a plurality of time management packet sending control threads; determining an expected sending of the media message to be sent Before the time, the attaching step further includes: each expected packet sending thread browses the channel it is responsible for; when it is found that a channel requires to send a media message, the estimated sending time of the media message to be sent is determined as follows:

A: read time management The index index of the current pointer array element currently being processed in the dispatch control thread and the preamble send time foresndtime;

B: Determine the hook index of the first media message to be sent nextpkt_tindex and the expected transmission time nextpkt_foresndtime, where, nextpkt_tindex=(index+n)% the number of elements of the pointer array; nextpkt_ foresndtime= Foresndtime+n* The time interval represented by each pointer array 1; where "%" means modulo, "*" means multiplication; n is a preset index margin to ensure that the time management packet control thread browses To the media message attached to the corresponding pointer array element; C: Determine the hook index of the subsequent media message nextpkt_tindex and the expected sending time nextpkt foresndtime until all media packets are processed, where: nextpkt_index= nextpkt index + (packaging interval/time interval represented by each cable 1) (the number of elements of the pointer array), wherein the packing interval is an integer multiple of the time interval represented by each index; "/" means division; nextpkt_foresndtime= nextpkt_foresndtime+ (packaging interval).

8. The method according to claim 6, wherein: said transmitting step is implemented by a plurality of time management delivery control threads; each time management delivery control thread performs said transmitting step as follows: A: browsing current pointer array element index Whether there is a media message to be sent on the linked list indicated by the pointer array element, and if so, reading the media message, and the address and payload information of the media message and the prediction of the batch of media messages The send time parameter is passed to the send message (sendmsg) function; the sendmsg function sends the batch of media messages in batches at the expected send time, and then returns at the expected send time, executing B; if the pointer array element in the current pointer array element index There is no media message to be sent on the linked list, and the time management packet control thread calls the delay function (DelayTime) delay until the expected transmission time of the current index list, and executes B;

B: Update the current index index and the expected sending time of the media message on the linked list of the current index, and return A; where, the current index: Index= ( Index+1 ) % (the maximum number of elements in the pointer array); the prediction corresponding to the current index Send time: foresndtime=foresndtime+ The time interval represented by each index.

The method according to claim 7 or 8, wherein: the attaching step further comprises: adding one media message to the media message counter corresponding to the linked list; In the sending step, the time management packet control thread determines, according to the media message counter, whether there is a media message to be sent on the linked list of the current index.

An audio media server, comprising an audio media delivery control device, the device comprising an expected delivery control module and a time management delivery control module, wherein the predicted delivery control module is configured to: determine an estimate of a media message to be sent Transmitting time, and attaching the media message to a linked list indicated by a pointer array element corresponding to the expected sending time;

The time management delivery control module is configured to: browse whether there is a media message on the linked list indicated by the pointer array element according to the pointer array element index, and if there is a media message, the expected delivery time of the media message is The media message is sent out, and the linked list of the pointer array element is browsed according to the next pointer array element index; if there is no media message, the delay is until the expected sending time of the element pointer array and according to the next pointer array element index Browse the linked list of pointer array elements.

11. The audio media server of claim 10, wherein: the expected delivery control module is configured to determine an expected transmission time of the media message by: reading the time management delivery control module is processing The estimated transmission time of the pointer array element; adding a certain time margin to the expected transmission time of the media message based on the expected transmission time of the read, to ensure that the time management delivery module browses to the corresponding connection The media message of the pointer array element.

The audio media server according to claim 10 or 11, wherein the audio media server further comprises a signaling forwarding unit, a main control unit, and a media storage and forwarding unit, wherein the signaling forwarding unit is configured to: Unified forwarding of the internal and external signaling of the server; the main control unit is configured to: receive and parse the control signaling forwarded by the signaling forwarding unit, and generate a corresponding control message to control the media storage and forwarding unit; and the media The storage and forwarding unit has an external network port, and the media storage and forwarding unit is configured to: perform recording and voice playback according to a control message sent by the main control unit, where In the sound playing, the audio media packet sending control device implements the packet control of the media message.

13. The audio media server of claim 12, wherein: the media storage and forwarding unit is configured to implement recording and voice playback using a multi-thread processing technology.

The audio media server of claim 12, wherein: the audio media server further comprises a media processing unit, the media processing unit comprises a call proxy module, a transceiver number module, and a conference processing module, wherein the call proxy module is configured The method is: implementing the interaction of the control message between the main control unit and the media processing unit, and notifying the sending and receiving number module if the control message is a sending and receiving number message, and notifying the conference processing module if the control message is a transcoding or a conference message; Set to: set the receiver resource, and identify the validity of the valid number in the media stream; and generate the control message of the main control unit;

The conference processing module is configured to: process media streams from various participants, combine input speech frames of different participants into a mixed speech frame, and output the speech frames to each participant to filter out the mixed speech frames. Inputting a voice frame; the main control unit is further configured to: generate a control message to the media processing unit; and the media storage and forwarding unit is further configured to: media the message containing the multi-tone dual-frequency DTMF number The destination address is translated into the address corresponding to the transceiver channel.