CN113450821B - Multi-party conference call system, method and computing device based on distributed computing - Google Patents
Multi-party conference call system, method and computing device based on distributed computing Download PDFInfo
- Publication number
- CN113450821B CN113450821B CN202110656110.3A CN202110656110A CN113450821B CN 113450821 B CN113450821 B CN 113450821B CN 202110656110 A CN202110656110 A CN 202110656110A CN 113450821 B CN113450821 B CN 113450821B
- Authority
- CN
- China
- Prior art keywords
- conference call
- calculation
- kth
- devices
- calculation force
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 18
- 238000004364 calculation method Methods 0.000 claims abstract description 80
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 21
- 238000004891 communication Methods 0.000 claims abstract description 7
- 238000004590 computer program Methods 0.000 claims description 4
- 238000003491 array Methods 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/02—Details
- H04L12/16—Arrangements for providing special services to substations
- H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
- H04L12/1813—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast for computer conferences, e.g. chat rooms
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
- G10L2021/02161—Number of inputs available containing the signal or the noise to be suppressed
- G10L2021/02166—Microphone arrays; Beamforming
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Software Systems (AREA)
- Signal Processing (AREA)
- General Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Acoustics & Sound (AREA)
- Human Computer Interaction (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Health & Medical Sciences (AREA)
- Computer Networks & Wireless Communication (AREA)
- Mathematical Physics (AREA)
- Quality & Reliability (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Telephonic Communication Services (AREA)
Abstract
The embodiment of the invention discloses a multi-party conference call system, a multi-party conference call method and a multi-party conference call computing device based on distributed computing. The system comprises: a microphone array for picking up voice data to be processed; the conference call devices are used for carrying out distributed computation on the voice data to be processed to obtain target voice data; and a plurality of conference call devices communicate with each other. In the embodiment of the invention, the voice is picked up by the microphone array, the calculation handshake mechanism is established at the communication end of the conference call equipment, and the calculation force of the local partial algorithm is distributed to the point-to-point equipment for distributed calculation, so that the cost of the chip module is reduced.
Description
Technical Field
The invention relates to the technical field of sound processing, in particular to a multi-party conference call system and method based on distributed computing and computing equipment.
Background
With the development of global economy, teleconferencing is used more and more frequently. Conventional single microphone teleconferencing systems are difficult to meet in terms of sound pickup quality, echo and ambient noise suppression. The teleconferencing system with the multi-microphone array greatly improves the voice call quality by adopting the multi-channel dereverberation and echo cancellation technology.
However, the multi-microphone array introduces new technical problems while improving the voice call quality. Because the computation complexity of the multi-microphone array is high, the chip of the conference call device is required to have high computation power, thereby increasing the cost of the chip module.
Disclosure of Invention
In view of the above technical drawbacks, an objective of an embodiment of the present invention is to provide a multi-party conference call system, a method and a computing device for distributed computing.
To achieve the above object, in a first aspect, an embodiment of the present invention provides a multi-party conference call system for distributed computing, including:
a microphone array for picking up voice data to be processed;
the conference call devices are used for carrying out distributed computation on the voice data to be processed to obtain target voice data; and a plurality of conference call devices communicate with each other.
The actual computing power of the kth conference call device is as follows:
Gsk+G*(Tk-Gsk)/(T1+T2+……+Tk+……+Tn-Gs1-……Gsk-……-Gsn)
Gsk is the pre-calculation force of the kth device, gsn is the pre-calculation force of the nth device, G is the calculation force requirement of the microphone array algorithm, tk is the original calculation force of the kth conference call device, and Tn is the original calculation force of the nth conference call device.
In a second aspect, an embodiment of the present invention provides a multiparty conference call method based on distributed computing, including:
Receiving voice data to be processed picked up by a microphone array;
performing distributed computation on voice data to be processed through a plurality of conference call devices to obtain target voice data; and a plurality of conference call devices communicate with each other.
As a specific embodiment of the present application, before receiving the voice data to be processed picked up by the microphone array, the method further includes:
a power handshake mechanism is established between a plurality of conference call devices.
As a specific embodiment of the present application, the method further includes:
calculating the actual calculation force of each conference call device;
and carrying out distributed calculation on the voice data to be processed according to the actual calculation power of each conference call device.
In a third aspect, an embodiment of the present invention provides a computing device, including a processor, an input device, an output device, and a memory, where the processor, the input device, the output device, and the memory are connected to each other, and where the memory is configured to store a computer program, the computer program including program instructions, the processor being configured to invoke the program instructions to perform the steps of:
Receiving voice data to be processed picked up by a microphone array;
Calculating the voice data to be processed by adopting actual calculation force to obtain target voice data;
the actual calculation force is calculated according to calculation force requirements of a plurality of conference call devices and microphone arrays; the actual computing power of the kth conference call device is as follows:
Gsk+G*(Tk-Gsk)/(T1+T2+……+Tk+……+Tn-Gs1-……Gsk-……-Gsn)
Wherein Gsk is the front calculation force of the kth device, gsn is the front calculation force of the nth device, G is the calculation force requirement of the microphone array algorithm, tk is the original calculation force of the kth conference call device, and Tn is the original calculation force of the nth conference call device.
By implementing the method and the system thereof, the microphone array picks up the voice, the communication end of the conference call equipment establishes the calculation handshake mechanism, and the calculation force of the local part of algorithm is distributed to the point-to-point equipment for distributed calculation, thereby reducing the cost of the chip module.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below.
FIG. 1 is a schematic diagram of a computational power calculation according to an embodiment of the present invention;
fig. 2 is a block diagram of a multiparty conference call system based on distributed computing provided in an embodiment of the present invention;
fig. 3 is a flowchart of a multi-party conference call method based on distributed computing according to an embodiment of the present invention;
FIG. 4 is a block diagram of a computing device provided by an embodiment of the invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and fully with reference to the accompanying drawings, in which it is evident that the embodiments described are some, but not all embodiments of the invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Referring to fig. 1, the inventive concept of the present invention is:
And establishing a computational power handshake mechanism at a communication end of the conference call equipment, and distributing the computational power of the local part algorithm to point-to-point equipment for distributed computation. As shown in the following diagram, when N devices communicate, the calculation power of the kth device is Tk, the voice of the kth device needs to be broadcast to N-1 devices to complete a conference call, the calculation power part of the microphone array which needs to be completed on the kth device is put on the N-1 devices to perform distributed calculation, the calculation power requirement of the microphone array algorithm is G, the front calculation power (the calculation power required by the algorithm part which needs to be put on the local terminal) is Gsk, and then the kth device only needs to have calculation power of gsk+g (Tk-Gsk)/(t1+t2+ … … +tk+ … … +tn-Gs1- … … Gsk- … … -Gsn), and the average calculation power required by the devices participating in the distributed calculation is not lower than the front calculation power Gsk of the algorithm.
Based on the above inventive concept, an embodiment of the present invention provides a multiparty conference call system based on distributed computing, as shown in fig. 2, including:
a microphone array for picking up voice data to be processed;
the conference call devices are used for carrying out distributed computation on the voice data to be processed to obtain target voice data; and a plurality of conference call devices communicate with each other.
The actual computing power of the kth conference call device is as follows:
Gsk+G*(Tk-Gsk)/(T1+T2+……+Tk+……+Tn-Gs1-……Gsk-……-Gsn)
Gsk is the pre-calculation force of the kth device, gsn is the pre-calculation force of the nth device, G is the calculation force requirement of the microphone array algorithm, tk is the original calculation force of the kth conference call device, and Tn is the original calculation force of the nth conference call device.
By implementing the system, the voice is picked up through the microphone array, the calculation handshake mechanism is established at the communication end of the conference call equipment, and the calculation force of the local partial algorithm is distributed to the point-to-point equipment for distributed calculation, so that the cost of the chip module is reduced.
Based on the same inventive concept, the embodiment of the invention also provides a multiparty conference call method based on distributed computing, which can include:
s1, establishing a computing power handshake mechanism among a plurality of conference call devices.
S2, calculating the actual calculation power of each conference call device.
For example, the actual computing power of the kth conference call device is:
Gsk+G*(Tk-Gsk)/(T1+T2+……+Tk+……+Tn-Gs1-……Gsk-……-Gsn)
Wherein Gsk is the front calculation force of the kth device, gsn is the front calculation force of the nth device, G is the calculation force requirement of the microphone array algorithm, tk is the original calculation force of the kth conference call device, and Tn is the original calculation force of the nth conference call device.
S3, receiving the voice data to be processed picked up by the microphone array.
S4, carrying out distributed computation on the voice data to be processed through a plurality of conference call devices to obtain target voice data.
Specifically, the voice data to be processed is calculated in a distributed mode according to the actual calculation power of each conference call device calculated in the previous step, so that the calculation power of the local part algorithm is distributed to other point-to-point device merchants, the voice quality is ensured, and meanwhile, the cost of the chip module is reduced.
Referring again to FIG. 4, an embodiment of the present invention provides a computing device comprising: one or more processors 101, one or more input devices 102, one or more output devices 103, and a memory 104, the processors 101, input devices 102, output devices 103, and memory 104 being interconnected by a bus 105. The memory 104 is used for storing a computer program comprising program instructions, which the processor 101 is configured to invoke for performing the method of the above-described method embodiment part.
It should be appreciated that in embodiments of the present invention, the Processor 101 may be a central processing unit (Central Processing Unit, CPU), a deep learning graphics card (e.g., NPU, injedag GPU, google TPU), other general purpose Processor, digital signal Processor (DIGITAL SIGNAL Processor, DSP), application SPECIFIC INTEGRATED Circuit (ASIC), off-the-shelf Programmable gate array (Field-Programmable GATE ARRAY, FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware components, or the like. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The input device 102 may include a keyboard or the like, and the output device 103 may include a display (LCD or the like), a speaker or the like.
The memory 104 may include read only memory and random access memory and provides instructions and data to the processor 101. A portion of the memory 104 may also include non-volatile random access memory. For example, the memory 104 may also store information of device type.
While the invention has been described with reference to certain preferred embodiments, it will be understood by those skilled in the art that various changes and substitutions of equivalents may be made and equivalents will be apparent to those skilled in the art without departing from the scope of the invention. Therefore, the protection scope of the invention is subject to the protection scope of the claims.
Claims (5)
1. A distributed computing-based multi-party conferencing system, comprising:
a microphone array for picking up voice data to be processed;
The n conference call devices are used for carrying out distributed computation on voice data to be processed according to the actual computing power of each conference call device, the distributed computation is to establish a computing power handshake mechanism at a communication end of the n conference call devices, the computing power of a local part algorithm is distributed to the point-to-point conference call devices to carry out distributed computation, when the n conference call devices communicate, the original computing power of the kth conference call device is Tk, the voice of the kth conference call device is broadcast to the n-1 conference call devices to complete conference call, and the microphone array computing power part to be completed on the kth conference call device is put on the n-1 devices to carry out distributed computation, so that target voice data are obtained; and a plurality of conference call devices are communicated with each other;
the actual computing power of the kth conference call device is as follows:
Gsk+G*(Tk-Gsk)/(T1+T2+……+Tk+……+Tn-Gs1-……Gsk-……-Gsn)
Wherein Gsk is the pre-calculation force of the kth conference call device, gsn is the pre-calculation force of the nth conference call device, G is the calculation force requirement of the microphone array algorithm, tk is the original calculation force of the kth conference call device, tn is the original calculation force of the nth conference call device, and the pre-calculation force is the calculation force required by the algorithm part which has to be executed at the local end.
2. A multi-party conference call method based on distributed computing, comprising:
Receiving voice data to be processed picked up by a microphone array;
The method comprises the steps that distributed computation is carried out on voice data to be processed according to actual computing power of each conference call device through n conference call devices, the distributed computation is that a computing power handshake mechanism is established at a communication end of the n conference call devices, computing power of a local part algorithm is distributed to point-to-point conference call devices to carry out distributed computation, when the n conference call devices communicate, original computing power of the kth conference call device is Tk, voice of the kth conference call device is broadcast to the n-1 conference call devices to complete conference call, and microphone array computing power parts needed to be completed on the kth conference call devices are placed on the n-1 devices to carry out distributed computation, so that target voice data are obtained; and a plurality of conference call devices are communicated with each other;
the actual computing power of the kth conference call device is as follows:
Gsk+G*(Tk-Gsk)/(T1+T2+……+Tk+……+Tn-Gs1-……Gsk-……-Gsn)
Wherein Gsk is the pre-calculation force of the kth conference call device, gsn is the pre-calculation force of the nth conference call device, G is the calculation force requirement of the microphone array algorithm, tk is the original calculation force of the kth conference call device, tn is the original calculation force of the nth conference call device, and the pre-calculation force is the calculation force required by the algorithm part which has to be executed at the local end.
3. The distributed computing-based multiparty conference call method according to claim 2, wherein before receiving the pending voice data picked up by the microphone array, the method further comprises:
And establishing a computing power handshake mechanism among the n conference call devices.
4. The distributed computing-based multiparty conference call method according to claim 3, further comprising:
The actual computing power of each conference call device is calculated.
5. A computing device comprising a processor, an input device, an output device, and a memory, the processor, the input device, the output device, and the memory being interconnected, wherein the memory is configured to store a computer program comprising program instructions, the processor being configured to invoke the program instructions to perform the steps of:
Receiving voice data to be processed picked up by a microphone array;
Calculating to-be-processed voice data according to the actual calculation power of each conference call device through n conference call devices, establishing a calculation power handshake mechanism for the communication end of the n conference call devices by distributed calculation, distributing the calculation power of a local part algorithm to the point-to-point conference call devices to perform distributed calculation, broadcasting the original calculation power of the kth conference call device to the n-1 conference call devices to complete conference call when the n conference call devices communicate, and placing the calculation power part of a microphone array to be completed on the kth conference call device to the n-1 devices to perform distributed calculation to obtain target voice data;
the actual calculation force is calculated according to calculation force requirements of a plurality of conference call devices and microphone arrays; the actual computing power of the kth conference call device is as follows:
Gsk+G*(Tk-Gsk)/(T1+T2+……+Tk+……+Tn-Gs1-……Gsk-……-Gsn)
Wherein Gsk is the pre-calculation force of the kth conference call device, gsn is the pre-calculation force of the nth conference call device, G is the calculation force requirement of the microphone array algorithm, tk is the original calculation force of the kth conference call device, tn is the original calculation force of the nth conference call device, and the pre-calculation force is the calculation force required by the algorithm part which has to be executed at the local end.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110656110.3A CN113450821B (en) | 2021-06-11 | 2021-06-11 | Multi-party conference call system, method and computing device based on distributed computing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110656110.3A CN113450821B (en) | 2021-06-11 | 2021-06-11 | Multi-party conference call system, method and computing device based on distributed computing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113450821A CN113450821A (en) | 2021-09-28 |
| CN113450821B true CN113450821B (en) | 2024-05-07 |
Family
ID=77811319
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110656110.3A Active CN113450821B (en) | 2021-06-11 | 2021-06-11 | Multi-party conference call system, method and computing device based on distributed computing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113450821B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1543181A (en) * | 2003-04-30 | 2004-11-03 | 华为技术有限公司 | A distributed mix processing method |
| CN101252452A (en) * | 2007-03-31 | 2008-08-27 | 红杉树（杭州）信息技术有限公司 | Distributed type tone mixing system in multimedia conference |
| CN101573955A (en) * | 2006-12-27 | 2009-11-04 | 诺基亚公司 | Distributed teleconference multichannel architecture, system, method, and computer program product |
| CN105068048A (en) * | 2015-08-14 | 2015-11-18 | 南京信息工程大学 | Distributed microphone array sound source positioning method based on space sparsity |
| CN106027946A (en) * | 2015-03-27 | 2016-10-12 | 阿尔卡特朗讯企业通信国际公司 | Method for allocating video conferencing task to processing device |
| CN108712584A (en) * | 2018-05-16 | 2018-10-26 | 中国电子科技集团公司第二十八研究所 | A kind of distributed sound mixing method for videoconference |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7496188B2 (en) * | 2004-09-20 | 2009-02-24 | International Business Machines Corporation | N-ways conference system using only participants' telephony devices without external conference server |
| US11539844B2 (en) * | 2018-09-21 | 2022-12-27 | Dolby Laboratories Licensing Corporation | Audio conferencing using a distributed array of smartphones |
-
2021
- 2021-06-11 CN CN202110656110.3A patent/CN113450821B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1543181A (en) * | 2003-04-30 | 2004-11-03 | 华为技术有限公司 | A distributed mix processing method |
| CN101573955A (en) * | 2006-12-27 | 2009-11-04 | 诺基亚公司 | Distributed teleconference multichannel architecture, system, method, and computer program product |
| CN101252452A (en) * | 2007-03-31 | 2008-08-27 | 红杉树（杭州）信息技术有限公司 | Distributed type tone mixing system in multimedia conference |
| CN106027946A (en) * | 2015-03-27 | 2016-10-12 | 阿尔卡特朗讯企业通信国际公司 | Method for allocating video conferencing task to processing device |
| CN105068048A (en) * | 2015-08-14 | 2015-11-18 | 南京信息工程大学 | Distributed microphone array sound source positioning method based on space sparsity |
| CN108712584A (en) * | 2018-05-16 | 2018-10-26 | 中国电子科技集团公司第二十八研究所 | A kind of distributed sound mixing method for videoconference |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113450821A (en) | 2021-09-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111951819A (en) | Echo cancellation method, device and storage medium | |
| US10546593B2 (en) | Deep learning driven multi-channel filtering for speech enhancement | |
| CN108076226B (en) | Method for adjusting call quality, mobile terminal and storage medium | |
| US20220188619A1 (en) | Microcontroller Interface for Audio Signal Processing | |
| US11488019B2 (en) | Lossless model compression by batch normalization layer pruning in deep neural networks | |
| US9886966B2 (en) | System and method for improving noise suppression using logistic function and a suppression target value for automatic speech recognition | |
| CN110769352B (en) | Signal processing method and device and computer storage medium | |
| US9997170B2 (en) | Electronic device and reverberation removal method therefor | |
| US20200365168A1 (en) | Method for acquiring noise-refined voice signal, and electronic device for performing same | |
| CN110503973B (en) | Audio signal transient noise suppression method, system and storage medium | |
| CN113450821B (en) | Multi-party conference call system, method and computing device based on distributed computing | |
| CN112289336B (en) | Audio signal processing method and device | |
| CN111615045B (en) | Audio processing method, device, equipment and storage medium | |
| US11875783B2 (en) | Method and system of audio input bit-size conversion for audio processing | |
| CN110996208B (en) | Wireless earphone and noise reduction method thereof | |
| CN112243182A (en) | Pickup circuit, method and device | |
| US10366701B1 (en) | Adaptive multi-microphone beamforming | |
| CN112307161B (en) | Method and apparatus for playing audio | |
| Jiang et al. | An improved practical state-space FDAF with fast recovery of abrupt echo-path changes | |
| CN111650560B (en) | Sound source positioning method and device | |
| CN111724808A (en) | Audio signal processing method, device, terminal and storage medium | |
| CN112785997B (en) | Noise estimation method and device, electronic equipment and readable storage medium | |
| CN113077809B (en) | Echo cancellation method, device, equipment and storage medium | |
| CN110969217A (en) | Method and device for processing image based on convolutional neural network | |
| CN113409802B (en) | Method, device, equipment and storage medium for enhancing voice signal |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |