JP3492561B2 - Communication voice processing device and storage medium storing voice processing program - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a communication voice processing apparatus suitable for decoding and encoding voice when performing voice communication using a packet communication network, and a storage medium storing a voice processing program.

[0002]

2. Description of the Related Art ITU-T Recommendation H.264 is to be observed when performing voice communication through a packet communication network. 323. ITU-T Recommendation H.264. 323
Is related to a standard for performing multimedia communication on a packet communication network. When communicating multimedia information such as voice and images in real time, ITU
T Recommendation H. It is required to use a communication terminal compliant with H.323.

ITU-T Recommendation H.264 FIG. 6 shows an example of a communication device used for a voice communication terminal conforming to H.323.
The communication device shown in FIG. 6 includes a terminal control unit 11 and a connection control unit 1.
2, a voice input unit 13, a voice output unit 14, a packet transmitting / receiving unit 15, an asynchronous communication network 16, and a voice processing unit 40. The terminal control unit 11 manages and controls the operations of the connection control unit 12 and the voice processing unit 40. The connection control unit 12
According to an instruction from the terminal control unit 11, negotiation is performed with the partner terminal at the time of connection, connection conditions with the partner terminal are determined,
Perform connection processing. The voice input unit 13 inputs voice data by a voice input means (not shown) such as a microphone and transfers the voice data to the voice processing unit 40. The voice output unit 14 is the voice processing unit 4.
Audio data from 0 is reproduced as audio through an audio output device (not shown) such as a speaker. Packet transmitter / receiver 1
5 performs packet transmission / reception processing through the asynchronous communication network 16. The asynchronous communication network 16 is a communication network adopted in a packet communication network or the like, through which packets are transferred between terminals.

The voice processing unit 40 receives the ITU-T recommendation G.264 according to an instruction from the terminal control unit 11. The audio data is encoded / decoded according to the 723.1 audio encoding method. Here, G. 723.1 is an audio data encoding / decoding method defined by ITU-T. When audio is encoded, the audio waveform is predicted and encoded, so that a high compression rate is achieved. Although it is possible to encode with, the encoding process takes time. In addition, G. In 723.1, encoding of audio data is performed based on a unit called a frame.
Performs decryption processing. One frame takes 30 msec (24
PCM data of 0 sample).

As shown in FIG. 7, the conventional speech processing section 40 is composed of a speech control section 41, a speech coding section 42, and a speech decoding section 43. The voice control unit 41 manages the voice encoding unit 42 and the voice decoding unit 43, and alternately issues an operation instruction to each unit. According to the instruction from the voice control unit 41, the voice encoding unit 42 converts the voice data input from the voice input unit 13 into the G.34. It is encoded according to the 723.1 encoding method. Next, the encoded bit stream is packetized and written in the packet transmitting / receiving unit 15. The audio decoding unit 43 reads the packet input from the packet transmitting / receiving unit 15 and extracts the encoded bitstream from the packet according to the instruction from the audio control unit 41. Next, a decoding process is performed, and the decoded voice data is output from the voice output unit 14 as voice.

[0006]

Conventionally, when a voice is reproduced from a packet, the voice decoding section 43 has a voice control section 41.
Each time it received an operation instruction from, one packet was read from the packet transmitting / receiving unit 15 and decoded. However, in this case, as shown in FIG. 8, when the interval from the voice decoding unit 43 receiving an operation instruction to the next operation instruction becomes longer than the packet arrival interval from the partner terminal, the reception packet Reading will not be in time. Therefore, the received packet is stored in the receiving buffer of the packet transmitting / receiving unit 15. When the receive buffer becomes full, the number of received packets that cannot be stored will be lost. For this reason, a sound break occurs during reproduction. This phenomenon is more remarkable as the capacity of the packet transmitting / receiving unit 15 is smaller.

Further, as described in the section of the prior art,
G. In a voice encoding / decoding method that performs predictive encoding such as the 723.1 encoding method, when encoding is compared with decoding, the encoding takes longer time. Therefore,
If the audio encoding unit 42 and the audio decoding unit 43 are operated at a ratio of 1: 1, the decoding process may not be in time. FIG. 9 is a scheduling diagram of processing when the encoding processing is performed every three frames and the decoding processing is performed every one frame. As shown in FIG. 9, the audio signal is encoded while the audio packet from the partner terminal is being decoded, and the output of the audio data is delayed. Therefore, the conventional audio encoding / decoding method for performing predictive encoding has a drawback in that audio interruption occurs during reproduction.

The present invention has been made in view of the above circumstances, and provides a communication voice processing apparatus and a storage medium storing a voice processing program capable of preventing dropping of received packets and causing no interruption of voice during reproduction. Is intended.

[0009]

SUMMARY OF THE INVENTION A first aspect of the present invention is a communication voice processing apparatus for encoding / decoding a voice signal for voice communication using a packet communication network. A voice encoding unit that encodes a signal, a packet storage unit that temporarily stores the received voice signal packet, a voice decoding unit that reads and decodes the voice signal packet from the packet storage unit, and gives an operation instruction to each unit. And a voice control unit for sending.

A second aspect of the present invention is the communication voice processing apparatus according to the first aspect of the invention, wherein when there are a number of received packets that cannot be stored in the packet storage unit at one time, the voice control unit is used. Is characterized in that the packet stored in the packet storage unit is deleted in order from the earliest stored time and a new received packet is stored.

A third aspect of the present invention is the communication voice processing apparatus according to the first aspect of the present invention, wherein when one or more packets are received, the voice control unit receives all received packets at once. The communication voice processing device according to claim 1, wherein the voice processing device is stored in the packet storage unit.

A fourth invention is the communication voice processing apparatus according to the first invention, wherein the packet storage section includes a ring buffer for storing the received voice signal packet, and a packet which may be received. It is characterized by having the same number of stages as the maximum number.

A fifth aspect of the present invention is the voice processing apparatus for communication according to the fourth aspect, wherein the interval for reading a received voice signal packet into the ring buffer is Recv _Interval , the maximum jitter of packet arrival is J _Max , and the received packet is included in the received packet. The amount of audio information being _recorded is P _Frame , and the number of stages of the ring buffer is Re
If cv _Maxl , then Recv _Maxl = {(J _Max + Recv _Interval ) /
It is characterized in that P _Frame } +1 holds.

A sixth invention is a communication voice processing apparatus according to the fifth invention, wherein the jitter J _Max is determined by using statistical information of RTCP.

A seventh aspect of the present invention is the communication voice processing apparatus according to the first aspect, wherein the voice control unit determines the ratio of the number of times the encoding process and the decoding process are performed, to the code of the voice signal. It is characterized in that the time during which the decoding process is performed is equal to the time during which the decoding process is performed.

An eighth aspect of the present invention is the communication voice processing apparatus according to the seventh aspect, wherein the voice control unit determines the ratio of the time required for the encoding process and the decoding process from the amount of voice data. Is characterized by.

According to a ninth aspect of the present invention, a voice encoding unit that encodes a voice signal when transmitting, a packet storage unit that temporarily stores a received voice signal packet, and a voice signal packet that is read from the packet storage unit and decoded. In a voice processing apparatus for communication, which comprises a voice decoding unit to be converted and a voice control unit for sending an operation instruction to each unit, when there are a number of received packets that cannot be stored in the packet storage unit at one time,
It is a storage medium recording a program for causing the voice control unit to delete a packet having the earliest storage time in the packet storage unit in order and store a new received packet.

In a tenth aspect of the invention, when one or more packets are received, a program for causing the voice control section to store all the received packets in the packet storage section at a time is recorded. It is a storage medium according to the ninth invention.

An eleventh aspect of the present invention is a voice coding unit for coding a voice signal when transmitting, a packet storage unit for temporarily storing a received voice signal packet, and a voice signal packet read from the packet storage unit for decoding. In a voice processing apparatus for communication, comprising: a voice decoding unit for converting into audio; and a voice control unit for sending an operation instruction to each unit, the packet storage unit includes a ring buffer for storing a received voice signal packet, and is capable of receiving. It is a storage medium in which a program for causing the voice control unit to have the same number of stages as the maximum number of effective packets is recorded.

A twelfth aspect of the invention is to set an interval for reading a received voice signal packet into the ring buffer as Recv.
_Interval , the maximum jitter of packet arrival is J _Max , the voice information amount included in the received packet is P _Frame , and the number of stages of the ring buffer is Recv _Maxl , then Recv _Maxl = {(J _Max + Recv _Interval ) /
The storage medium according to the eleventh aspect of the present invention has recorded therein a program for causing the audio control unit to execute the operation of making P _Frame } +1 hold.

The thirteenth aspect of the present invention is the storage medium according to the eleventh aspect of the present invention, in which a program for causing the voice control section to execute the determination of the jitter J _Max using the statistical information of RTCP is recorded.

A fourteenth aspect of the present invention is a voice encoding section for encoding a voice signal when transmitting, a packet storage section for temporarily storing a received voice signal packet, and a voice signal packet read from the packet storage section for decoding. In a speech processing apparatus for communication comprising a speech decoding unit for encoding and a speech control unit for sending an operation instruction to each unit, the ratio of the number of times the encoding process and the decoding process are executed is performed by encoding the audio signal. The storage medium stores a program for causing the audio control unit to make the time for performing the decoding process equal to the time for performing the decoding process.

In a fifteenth aspect of the present invention, the voice control section records a program for causing the voice control section to execute determination of a ratio of time required for encoding processing and decoding processing from the amount of voice data. 14 is a storage medium according to the invention.

In order to achieve the above object, in the present invention, a voice control unit, a voice encoding unit, and a voice decoding unit are additionally provided in the voice processing unit, and a packet storage unit having a new buffer is provided in the voice transmitting / receiving unit. The packet is transferred to and stored in the buffer of the packet storage unit before the decoding process, and the packet is read from the buffer and decoded when the packet is decoded. With this configuration, when one or more packets are received by the packet transmitting / receiving unit, all the packets are transferred to and stored in a buffer in the packet storage unit, for example, a ring buffer having one or more stages at a time, at once. It is possible to suppress dropping of packets in the packet transmitting / receiving unit.

Further, by using a communication device having a voice processing unit having a voice control unit, a voice encoding unit, and a voice decoding unit, the encoding process and the encoding process are performed based on the ratio of time required for the encoding process and the decoding process. When the number of times the decoding process is executed is determined, the received packet can be efficiently processed even when the voice coding / decoding method that performs predictive coding is adopted. That is, the number of times the voice control unit issues an operation instruction to the voice encoding unit and the voice decoding unit is determined depending on the time required for the voice encoding process in the voice encoding unit and the voice decoding process in the voice decoding unit. , Increase the number of operations of the speech decoding unit. By not biasing the processing to one of the voice encoding process and the voice decoding process, the voice data can be efficiently processed. The communication device of the present invention is
Since the audio processing unit as described above is provided, even when the amount of audio data received from the partner terminal device is large, audio interruption does not occur during reproduction.

[0026]

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

FIG. 1 is a block diagram showing the configuration of a communication device having a voice processing device according to the present invention. Since this communication device has the same configuration as the conventional voice communication device (FIG. 7) except for the voice processing unit, common parts are denoted by the same reference numerals and detailed description thereof will be omitted. This communication device is similar to the voice communication device of the prior art, and is ITU-T Recommendation H.264. It is used for a voice communication terminal compliant with H.323. As shown in FIG. 1, the voice processing unit 20 is configured to include a voice control unit 21, a voice encoding unit 22, a packet storage unit 23, and a voice decoding unit 24.

The operation (function) of the voice processing section 20 will be described below. When performing the packet encoding process, the voice control unit 21 issues an operation instruction to the voice encoding unit 22. The voice encoding unit 22 is a voice control unit 21.
In accordance with an operation instruction from G., the input voice data is read from the voice input unit 13, and the input voice data is input to G.I. 72
3.1 Encoding according to the encoding method. Next, the encoded bit stream is packetized to form the packet transmission / reception unit 1
Write to 5.

When performing the packet decoding process, the voice control unit 21 first issues an operation instruction to the packet storage unit 23. The packet storage unit 23 is a voice control unit 21.
According to the instruction from, the packet of the voice signal from the partner terminal is read from the reception buffer of the packet transmission / reception unit 15 into the buffer of the packet storage unit 23, for example, the ring buffer. Next, the voice control unit 21 issues an operation instruction to the voice decoding unit 24. The audio decoding unit 24 reads the packet from the buffer of the packet storage unit 23 and extracts the encoded bitstream from the packet according to the instruction from the audio control unit 21. Next, the audio decoding unit 24 performs a decoding process, and the decoded audio data is output from the audio output unit 14.

Here, the processes of the packet storage unit 23 and the voice control unit 21 will be described in detail. First, the processing of the packet storage unit 23 will be described with reference to FIG. FIG. 2 is a block diagram showing an example of the packet storage unit 23. The packet storage unit 23 includes a packet monitoring unit 31 and a ring buffer 32. And
When there is an instruction from the voice control unit 21, the packet monitoring unit 3
1 confirms whether or not a packet exists in the packet transmitting / receiving unit 15. At this time, when one or more packets are received by the packet transmitting / receiving unit 15, all the packets are transferred to the ring buffer 32 and stored at once. When storing packets in the ring buffer 32, if more packets than the free area of the ring buffer 32 are received by the packet transmitting / receiving unit 15, the packets are deleted in order from the earliest stored time in the ring buffer 32. A new packet is stored.

FIG. 3 is an explanatory diagram showing a state in which a packet is stored in the ring buffer 32. As shown in FIG. 3, the reception buffer 17 can store seven packets at a time, and the ring buffer 32 can store four packets at a time. In FIG. 3A, the ring buffer 32
The number of packets that is less than the number of
It is shown stored in the. The packet storage unit 23 stores the packets 1a, 2a, 3a from the reception buffer 17 in the packet transmission / reception unit 15 to the ring buffer 32 in the packet storage unit according to an instruction from the voice control unit 21.

On the other hand, when the number of packets is larger than the number of stages of the ring buffer 32, for example, as shown in FIG.
It is assumed that a, 2a, 3a, 4a, 5a, 6a are stored in the reception buffer 17 of the packet transmitting / receiving unit 15. When the next packet is received, the ring buffer 32 can store only four packets.
A, 2a, 3a, 4a are stored in the ring buffer 32. Next, the packets 1a and 2a stored in the ring buffer 32 whose storage time is old are deleted, and the packets 5a and 6a are stored in the storage areas thereof.

Next, the number of stages of the ring buffer 32 is determined as follows. Let Recv _{Interval be} the interval at which packets are read from the packet transmitter / receiver 15, J _{max be} the maximum packet arrival jitter, and P _Frame be the audio information amount (the number of encoded frames) contained in the received packet. The maximum number of packets Recv _Maxl that may be received is: Recv _Maxl = {(J _max + Recv _Interval ) /
P _Frame } +1. Therefore, since the maximum number of packets that may be transferred to the ring buffer 32 at one time is Recv _Maxl , this value is set as the number of stages of the ring buffer 32.
The value of J _max is ITU-T Recommendation H.264. 245
Maximum Audio Delay Jitter or ITU-T Recommendation H.264. 225 RTCP interarrival jitter parameters are used.

By setting the number of stages of the ring buffer 32 in this way, the received packets stored in the receive buffer 17 of the packet transmitter 15 can always be transferred to the ring buffer 32 at one time.

FIG. 4 shows means for solving the conventional scheduling problem shown in FIG. That is, the interval from when the voice processing unit 20 receives an operation instruction from the terminal control unit 11 until when the next operation instruction is received may be longer than the arrival interval of packets from the partner terminal. In that case,
In the present invention, as shown in FIG.
Packets received by the packet storage unit 23 and not yet transferred to the packet storage unit 23 can be transferred to the ring buffer 32 in the packet storage unit 23 at one time. Therefore, the storage area of the reception buffer 17 of the packet transmission / reception unit 15 is less likely to be filled with packets, and the packets can be prevented from overflowing from the reception buffer 17.

Next, a detailed description will be given regarding the processing of the voice control unit 21. The number of times that the voice control unit 21 issues an operation instruction to the voice encoding unit 22 and the voice decoding unit 24 can be determined as follows. In the audio encoding unit 22, the time required for performing the encoding process of one frame is TE, the audio data amount (the number of frames) encoded in one encoding process is F _E , G. The time required for performing the decoding process of the bit stream for one frame in the audio decoding unit 24 in accordance with G.723.1 is T _D , and the amount of audio data to be decoded in one decoding process (the number of frames) To F
_{Assuming D} , the time E _Time required for the operation of the speech encoding unit 22 and the time D _Time required for the operation of the speech decoding unit 24 are respectively expressed by the following equations. E _Time = T _E × F _E D _Time = T _D × F _D

Here, when the number of operations of the speech encoding unit 22 is E _Count and the number of operations of the speech decoding unit 24 is D _Count , the operation times of the speech encoding unit 22 and the speech decoding unit 24 are made equal. E _Time × E _Count = D _Time × D _Count Here, _assuming that the operation count E _Count of the speech encoding unit 22 is 1, the operation count D _Count of the speech decoding unit 24 is D _Count = E _Time / D It will be _Time . The voice control unit determines the ratio of the number of times the encoding process and the decoding process are executed so that the time required for the voice encoding process and the time required for the decoding process are equal.
For example, in the case of E _Time : D _Time = 7: 1, the speech decoding unit 2 operates between the speech processing unit 21 issuing an operation command to the speech encoding unit 22 and the next operation command.
The operation command may be issued 7 times to 4.

FIG. 5 shows means for solving the conventional scheduling problem shown in FIG. FIG. 5 is an explanatory diagram showing the time relationship between the decoding process and the encoding process. As shown in FIG. 5, the processing per frame becomes longer in the order of decoding processing, encoding processing, and audio reproduction processing. It is shown that three voice signal packets from the partner terminal are decoded and then the voice signal is encoded for three frames. Then, the audio reproduction processing of the decoded three packets is performed while the encoding processing of the three frames is performed. In this way, as is clear from FIG. 5, even if the encoding processing time is longer than the decoding processing time, the voice input from the partner terminal can be reproduced without interruption.

Further, a program for executing the above method, a program for determining the ratio of time required for the encoding process and the decoding process from the audio data amount, and 1 when one or more packets are received by the packet transmitting / receiving unit. A program that allows all packets to be transferred from the packet transmitter / receiver to the receive buffer in the packet storage unit every time, and the buffer in the packet storage unit is a ring buffer, and the number of packets that cannot be stored in this buffer is stored. In this case, the communication device may include a recording medium in which a program that deletes the buffer with the oldest storage time in the ring buffer in order and stores a new packet is recorded. Further, such a necessary program may be recorded in the voice control unit of the voice processing unit. As described above, the present invention is not limited to the above-described embodiment, and various modifications can be made within the scope of the claims.

[0040]

According to the present invention, the packet transmission / reception unit has one
When two or more packets are received, all packets can be transferred from the packet transmitter / receiver to the receive buffer in the packet storage unit at one time and stored, so that the packets overflow and drop from the receive buffer of the packet transmitter / receiver. It is possible to suppress that. In addition, since it is possible to perform decoding processing for a plurality of packets at one time, it is possible to suppress interruption of audio during reproduction due to delay in decoding processing of packets, and to realize comfortable audio communication.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a voice communication device having a voice processing device according to the present invention.

FIG. 2 is a block diagram showing a configuration of a packet storage unit.

FIG. 3 is an explanatory diagram showing a state in which a packet is stored in a ring buffer of a packet storage unit.

FIG. 4 is an explanatory diagram showing a timing at which a packet storage unit reads a packet.

FIG. 5 is an explanatory diagram showing a timing at which a voice decoding unit performs voice decoding.

FIG. 6 is a block diagram showing a basic configuration of a conventional voice communication device.

FIG. 7 is a block diagram of a voice communication device including a configuration of a conventional voice processing unit.

FIG. 8 is an explanatory diagram showing a timing at which a packet storage unit of a conventional communication device reads a packet.

FIG. 9 is an explanatory diagram showing a timing at which a speech decoding unit of a conventional communication device decodes speech.

[Explanation of symbols]

11 Terminal control unit 12 Connection control unit 13 Voice input section 14 Audio output section 15 Packet transmitter / receiver 16 Asynchronous communication network 20 Voice processing unit 21 Voice control unit 22 Speech coding unit 23 Packet storage 24 Speech decoding unit 21 Packet monitoring unit 22 ring buffer

Claims (6)

(57) [Claims] 1. A speech processing apparatus for communication which encodes / decodes a speech signal in order to perform speech communication using a packet communication network, and a speech coding unit for coding a speech signal when transmitting. includes a packet storage unit for temporarily storing the audio signal packets received, the audio decoding unit to decode the read audio signal packets from the packet storage unit, and a sound control unit which sends an instruction for operation to each unit, the packet The storage unit stores the received voice signal packet.
It is equipped with a ring buffer to store and may receive packets.
Has the same number of stages as the maximum number of packets and reads the received voice signal packet into the ring buffer.
The interval to be inserted is Recv _Interval , the maximum packet arrival
Is J _Max , and the amount of voice information included in the received packet is
P _Frame , where Recv _Maxl is the number of stages of the ring buffer , Recv _Maxl = {(J _Max + Recv _Interval ) / P
_A speech processing apparatus for communication, characterized in that _Frame } +1 holds . 2. The zip using the statistical information of RTCP.
The communication voice processing apparatus according to claim 1, wherein the data J _Max is defined . 3. Sound is produced using a packet communication network.
A voice signal is encoded and decoded for voice communication.
In the credit voice processing device, a voice encoding unit that encodes a voice signal when transmitting,
A packet storage unit for temporarily storing received voice signal packets
And read the audio signal packet from the packet storage unit.
Audio decoding unit to decode and sound to send operation instructions to each unit
With a voice control unit, the time required to perform one frame of encoding processing
T _E , the audio data amount to be encoded in one encoding process
F _E , the decoding process of the bit stream for one frame
The time required to perform is T _D , and decoding is performed in one decoding process
Necessary for encoding process, if the amount of audio data to be encoded is F _D
Time E _Time and time required for composite processing
D _Time is E _Time = TE × F _E , D _Time = T _D × F _D , and E _{Count is} the number of operations of the voice encoding process , and voice recovery
When the number of operations of Nos processing and D _Count, the voice control unit, a communication speech processing apparatus and controls so that _{E Time × E Count = D Time} × D Count. 4. Audio for encoding an audio signal when transmitting
Temporarily stores the received voice signal packet with the encoding unit
A packet storage unit and an audio signal packet from the packet storage unit.
Voice decoding unit that reads and decodes the packets and operates in each unit
Communication voice processing apparatus including a voice control unit for sending the instruction
In the packet storage unit,
A ring buffer for storing packets is available for reception
To have the same number of rounds as the maximum number of active packets
Read the received voice signal packet into the ring buffer.
The interval to be inserted is Recv _Interval , the maximum packet arrival
Is J _Max , and the amount of voice information included in the received packet is
P _Frame, a number of Recv _MaxL of the ring buffer
Then, Recv _Maxl = {(J _Max + Recv _Interval ) / P
_Frame } +1 is made to cause the voice control unit to execute.
A storage medium in which a program for recording is recorded. 5. The jig using the statistical information of RTCP.
The voice control unit is made to execute the definition of the data J _Max .
The storage medium according to claim 4, wherein a program for recording is recorded. 6. Audio for encoding an audio signal when transmitting
Temporarily stores the received voice signal packet with the encoding unit
A packet storage unit and an audio signal packet from the packet storage unit.
Voice decoding unit that reads and decodes the packets and operates in each unit
Communication voice processing apparatus including a voice control unit for sending the instruction
In, the time required to perform the encoding process of 1 frame is
T _E , the audio data amount to be encoded in one encoding process
F _E , the decoding process of the bit stream for one frame
The time required to perform is T _D , and decoding is performed in one decoding process
Necessary for encoding process, if the amount of audio data to be encoded is F _D
Time E _Time and time required for composite processing
D _Time is E _Time = TE × F _E , D _Time = T _D × F _D , and E _{Count is} the number of operations of the voice encoding process , and voice recovery
_{Letting the} voice control unit execute control so that E _Time × E _Count = D _Time × D _Count , where D _Count is the number of operations of the encoding process.
A storage medium in which a program for recording is recorded.