KR20170072783A - Channel adaptive audio mixing method for multi-point conference service - Google Patents

Abstract

The embodiments of the present invention relate to a method of mixing audio in a multi-party conference service, wherein a channel state adaptive audio mixing method in a multi-party conferencing service includes receiving audio data from a plurality of conference terminals step; Confirming whether each of the plurality of conference terminals has a channel state detection function and a channel state information exchange capability; Receiving channel state information from a conference terminal having a channel state detection function and a channel state information exchange capability; Obtaining channel state information from voice data received from a conference terminal having no channel state detection function and channel state information exchange capability; And a step of checking a channel where the howling or echo is generated based on the channel state information and mixing the voice data of the remaining channels except the voice data of the identified channel to generate a conference voice. According to embodiments of the present invention, audio data of a channel in which a howling, an echo, or excessive noise occurs is originally excluded from the audio mixing process, thereby preventing the interruption of the multi-party conference service and improving the voice quality.

Description

[0001] The present invention relates to a channel adaptive audio mixing method for a multi-party conference service,

Embodiments of the present invention relate to a method of audio mixing in a multipoint conference service.

In a multi-point control unit (MCU) that provides a multi-party conference service (multi-party video conferencing service or multi-party voice conferencing service) through a communication network, the following three methods are generally used for mixing voice data do. First, there is a simple mixing method in which all the voice data received from conference terminals participating in a multi-party conference are mixed (simply mixed) at a uniform rate. Second, it detects all the conference terminals corresponding to the speaker, amplifies the volume of the voice data of the conference terminal corresponding to the speaker, and mixes (mixes the speaker) with the voice data of the remaining conference terminals to emphasize the voice size of the speaker There is a mixing method. Third, the voice data of the conference terminal corresponding to the main speaker is searched for the main speaker among the speakers, and the voice data of the conference terminal is mixed with the voice data of the remaining conference terminals after amplifying the volume, .

In the case of the multi-party conference service, the multi-point access control apparatus mixes the voice data so that the conference can be effectively performed, and transmits the mixed voice data to all the conference terminals participating in the conference. At this time, the biggest problem is that the influence of howling, echo, or excessive noise generated in any one unspecified conferencing terminal is transmitted to all other conferencing terminals, thereby degrading the service quality.

A typical solution for solving this problem is to prevent degradation of voice data quality by providing echo cancellation, noise suppression function, and howling detection and cancellation function for each conference terminal. However, when the position of the microphone or the speaker is changed in the environment where the conference terminal is installed, the echo path changes and echo or howling occurs. Also, the noise or noise generated at any one of the conference terminals Is included in the mixed voice signal, and is spread to all the conference terminals participating in the multi-party conference.

Japanese Unexamined Patent Application Publication No. 2003-23499 (conference server apparatus and conference system)

Embodiments of the present invention provide a method for performing audio mixing except audio data of howling, echo, or excessive noise of a conference terminal participating in a multi-party conference service.

In embodiments of the present invention, a multi-point access control apparatus adaptively detects a channel state information according to presence or absence of a channel state information detection function of a conference terminal, and provides feedback on howling, echo or excessive noise To prevent the interruption of multi-party conferencing services.

A channel state adaptive audio mixing method in a multi-party conference service according to an embodiment of the present invention includes: receiving voice data from a plurality of conference terminals; Confirming whether each of the plurality of conference terminals has a channel state detection function and a channel state information exchange capability; Receiving channel state information from a conference terminal having a channel state detection function and a channel state information exchange capability; Obtaining channel state information from voice data received from a conference terminal having no channel state detection function and channel state information exchange capability; And a step of checking a channel where the howling or echo is generated based on the channel state information and mixing the voice data of the remaining channels except the voice data of the identified channel to generate a conference voice.

According to embodiments of the present invention, voice data of a channel in which a howling, an echo, or excessive noise occurs is originally excluded from the audio mixing process, thereby preventing the interruption of the multi-party conference service and improving the voice quality of the multi-party conference service.

According to embodiments of the present invention, a multi-point access control apparatus adaptively detects channel state information according to whether a conference terminal has a channel state information detection function, It is possible to provide a multi-party conference service having data quality.

1 is an exemplary view for explaining a multi-party conference system to which embodiments of the present invention are applied;
FIG. 2 is an exemplary diagram for explaining a multi-point connection control apparatus according to an embodiment of the present invention;
3 is a flowchart illustrating a channel state adaptive audio mixing method in a multi-point connection control apparatus according to an embodiment of the present invention.

In the following description of the embodiments of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

FIG. 1 is an exemplary diagram for explaining a multi-party conference system to which embodiments of the present invention are applied.

The conference terminal 200 may include a video photographing module such as a camera, a video output module such as a monitor, a voice input module such as a microphone, and a voice output module such as a speaker. The conference terminal 200 can transmit the user's image data and the user's voice data acquired through the image capturing module and the voice input module to the multi-point access control apparatus 100 through a network such as the Internet.

The multi-point connection control apparatus 100 can receive video data and audio data from a plurality of conference terminals 200. [ The multi-point access control apparatus 100 may mix the received video data and audio data, and may transmit the conference video and the conference voice generated as a result of the mixing to the conference terminal 200.

2 is an exemplary diagram for explaining a multi-point connection control apparatus according to an embodiment of the present invention.

The multi-point connection control apparatus according to an embodiment of the present invention includes an audio mixing unit 110, a howling detection unit 120, an echo detection unit 130, and a control unit 140.

The audio mixing unit 110 can receive channel-specific audio data from a plurality of conference terminals. The audio mixing unit 110 may generate the conference voice by mixing the remaining voice data except the howling or echo generated voice data among the voice data received from the plurality of conference terminals. At this time, the audio mixing unit 110 can generate the conference voice by performing the mixing by excluding the voice data received from the conference terminal to receive the conference voice. The audio mixing unit 110 generates mixing state information including a channel number of a conference terminal in which howling, echo, or excessive noise is found and information indicating that speech data of the conference terminal is excluded from the conference speech, Status information can be provided to a plurality of conference terminals. To this end, the audio mixing unit 110 may have a noise detection function that can confirm a channel in which excessive noise is generated.

The howling detection unit 120 can receive channel-specific voice data from a plurality of conference terminals. The howling detection unit 120 determines whether howling has occurred in the audio data of the channel in which the control unit 140 instructs the control unit 140 to detect the channel state information among the audio data received from the plurality of conference terminals, To the control unit 140. The control unit 140 may be configured to transmit the channel state information to the control unit 140. [ For example, the howling detection unit 120 buffers voice data packetized in units of about 20 msec time, converts the buffered voice data into a frequency domain signal, and then, based on the energy size of the frequency domain signal, The frequency can be selected. If the ratio of the power spectrum of the Howling candidate frequency to the short average power spectrum and the ratio of the long mean power spectrum are larger than a preset threshold value, It can be determined that howling occurs at the howling candidate frequency.

The echo detecting unit 130 can receive channel-specific audio data from a plurality of conference terminals. The echo detecting unit 130 determines whether echo has occurred in the audio data of the channel in which the control unit 140 instructs the control unit 140 to detect the channel state information among the audio data received from the plurality of conference terminals, To the control unit 140. The control unit 140 may be configured to transmit the channel state information to the control unit 140. [

The control unit 140 can perform communication with the conference terminal and confirm whether or not each conference terminal has the channel state detection function and the channel state information exchange capability.

If the conference terminal has the channel state detection function and the channel state information exchange capability, the control unit 140 notifies the howling detection unit 120 and the echo detection unit 130 that channel state information is not detected in the voice data received from the corresponding conference terminal, . Then, the control unit 140 can receive channel state information from the conference terminal.

If the conference terminal does not have the channel state detection function and the channel state information exchange capability, the control unit 140 controls the wayl detection unit 120 and the echo detection unit 130 to detect the channel state information in the voice data received from the conference terminal ). Accordingly, the howling detection unit 120 and the echo detection unit 130 can detect channel information in the voice data received from the corresponding conference terminal, and provide the detected channel information to the control unit 140.

The control unit 140 can check whether there is a channel where the howling and echo are generated based on the channel state information, and notify the audio mixing unit 110 of the channel where the howling or echo occurs.

3 is a flowchart illustrating a channel state adaptive audio mixing method in a multi-point access control apparatus according to an embodiment of the present invention.

In operation 301, the audio mixing unit 110 may receive the number of conference terminals participating in the video conference and the channel number from the control unit 140. [ The audio mixing unit 110 may receive voice data corresponding to the number of conference terminals participating in the video conference. That is, the audio mixing unit 110 can receive channel-specific audio data from a plurality of conference terminals participating in the video conference.

In step 303, the control unit 140 communicates with a plurality of conference terminals participating in the video conference to determine whether or not the conference terminals have a channel state detection function (howling detection function and echo detection function) Functions can be used to verify whether or not it is capable of exchanging acquired channel state information with a multi-point access control device.

In step 305, the controller 140 controls the howling detection unit 120 and the echo detection unit 120 to not perform the channel state information detection on the voice data received from the conference terminal having the channel state detection function and the channel state information exchange capability 130). Then, the control unit 140 can receive channel state information of the conference terminal from the conference terminal.

In step 307, the control unit 140 controls the howling detection unit 120 and the echo detection unit 130 to perform channel state information detection on the voice data received from the conference terminal having the channel state detection function and the channel state information exchange capability ). Accordingly, the howling detecting unit 120 and the echo detecting unit 130 may detect the channel state information from the voice data received from the conference terminal, and provide the detected channel state information to the controller 140.

In step 309, the control unit 140 can check whether there is a channel in which at least one of echo and howling occurs. If there is a channel where at least one of echo and howling occurs, the control unit 140 can notify the audio mixing unit 110 of what channel the corresponding channel is.

In step 311, the audio mixing unit 110 may generate the conference voice by mixing the voice data of the remaining channels except for the voice data of the channel where at least one of echo and howling occurs. At this time, the audio mixing unit 110 may generate the conference voice by mixing the voice data of the remaining channels except the voice data of the channel (conference terminal) to receive the conference voice. If there is no channel in which the echo or howling occurs, the audio mixing unit 110 may generate a conference voice by mixing only voice data of a channel to receive the conference voice and voice data of the remaining channels.

In step 313, the audio mixing unit 110 receives mixing status information including information indicating that the channel number of the conference terminal in which howling, echo, or excessive noise is found and the voice data of the conference terminal are excluded from the conference voice And provide the generated mixing state information to a plurality of conference terminals.

The embodiments of the invention described above may be implemented in any of a variety of ways. For example, embodiments of the present invention may be implemented using hardware, software, or a combination thereof. When implemented in software, it may be implemented as software running on one or more processors using various operating systems or platforms. Additionally, such software may be written using any of a number of suitable programming languages, and may also be compiled into machine code or intermediate code executable in a framework or virtual machine.

Also, when embodiments of the present invention are implemented on one or more processors, one or more programs for carrying out the methods of implementing the various embodiments of the invention discussed above may be stored on a processor readable medium (e.g., memory, A floppy disk, a hard disk, a compact disk, an optical disk, a magnetic tape, or the like).

Claims (1)

Receiving voice data from a plurality of conference terminals;
Confirming whether each of the plurality of conference terminals has a channel state detection function and a channel state information exchange capability;
Receiving channel state information from a conference terminal having a channel state detection function and a channel state information exchange capability;
Obtaining channel state information from voice data received from a conference terminal having no channel state detection function and channel state information exchange capability; And
Checking a channel in which howling or echo occurs based on the channel state information, and generating a conference voice by mixing voice data of the remaining channels except for the voice data of the confirmed channel
And a channel state adaptive audio mixing method in a multi-party conference service.

Images