CN113450821A - Multi-party conference call system, method and computing device based on distributed computing - Google Patents
Multi-party conference call system, method and computing device based on distributed computing Download PDFInfo
- Publication number
- CN113450821A CN113450821A CN202110656110.3A CN202110656110A CN113450821A CN 113450821 A CN113450821 A CN 113450821A CN 202110656110 A CN202110656110 A CN 202110656110A CN 113450821 A CN113450821 A CN 113450821A
- Authority
- CN
- China
- Prior art keywords
- conference call
- gsk
- voice data
- calculation
- calculation power
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 19
- 238000004364 calculation method Methods 0.000 claims abstract description 61
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 15
- 238000004590 computer program Methods 0.000 claims description 4
- 238000004891 communication Methods 0.000 abstract description 8
- 238000010586 diagram Methods 0.000 description 2
- 238000012545 processing Methods 0.000 description 2
- 238000013135 deep learning Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001629 suppression Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F9/00—Arrangements for program control, e.g. control units
- G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
- G06F9/46—Multiprogramming arrangements
- G06F9/50—Allocation of resources, e.g. of the central processing unit [CPU]
- G06F9/5061—Partitioning or combining of resources
- G06F9/5072—Grid computing
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L12/00—Data switching networks
- H04L12/02—Details
- H04L12/16—Arrangements for providing special services to substations
- H04L12/18—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast
- H04L12/1813—Arrangements for providing special services to substations for broadcast or conference, e.g. multicast for computer conferences, e.g. chat rooms
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
- G10L21/0216—Noise filtering characterised by the method used for estimating noise
- G10L2021/02161—Number of inputs available containing the signal or the noise to be suppressed
- G10L2021/02166—Microphone arrays; Beamforming
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Signal Processing (AREA)
- Software Systems (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Acoustics & Sound (AREA)
- Human Computer Interaction (AREA)
- Mathematical Physics (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Quality & Reliability (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Computer Networks & Wireless Communication (AREA)
- Telephonic Communication Services (AREA)
Abstract
The embodiment of the invention discloses a multi-party conference call system, a multi-party conference call method and computing equipment based on distributed computing. The system comprises: the microphone array is used for picking up voice data to be processed; the conference call equipment is used for carrying out distributed calculation on the voice data to be processed to obtain target voice data; and a plurality of conference call devices communicate with each other. In the embodiment of the invention, the microphone array is used for picking up the voice, a calculation force handshake mechanism is established at the communication end of the conference communication equipment, and the calculation force of partial algorithm of the microphone is distributed to the point-to-point equipment for distributed calculation, so that the cost of the chip module is reduced.
Description
Technical Field
The invention relates to the technical field of sound processing, in particular to a multi-party conference call system based on distributed computing, a method and computing equipment.
Background
With the development of global economy, teleconferences are used more and more frequently. Conventional single-microphone teleconferencing systems have difficulty meeting requirements for pickup quality, echo, and ambient noise suppression. The multi-microphone array teleconference system greatly improves the voice call quality by adopting multi-channel dereverberation and echo cancellation technologies.
However, the multi-microphone array introduces a new technical problem while improving the voice call quality. Due to the high computational complexity of the multi-microphone array, the conference call equipment chip is required to have high computational power, thereby increasing the cost of the chip module.
Disclosure of Invention
In view of the foregoing technical defects, an embodiment of the present invention provides a distributed computing multi-party conference call system, method and computing device.
In order to achieve the above object, in a first aspect, an embodiment of the present invention provides a distributed computing multi-party conference call system, including:
the microphone array is used for picking up voice data to be processed;
the conference call equipment is used for carrying out distributed calculation on the voice data to be processed to obtain target voice data; and a plurality of conference call devices communicate with each other.
Wherein, the actual computational power that the kth conference call equipment possesses is:
Gsk+G*(Tk-Gsk)/(T1+T2+……+Tk+……+Tn-Gs1-……Gsk-……-Gsn)
gsk is the pre-calculation power of the kth device, Gsn is the pre-calculation power of the nth device, G is the calculation power requirement of the microphone array algorithm, Tk is the original calculation power of the kth conference call device, and Tn is the original calculation power of the nth conference call device.
In a second aspect, an embodiment of the present invention provides a multiparty conference call method based on distributed computing, including:
receiving voice data to be processed picked up by a microphone array;
carrying out distributed calculation on the voice data to be processed through a plurality of conference call devices to obtain target voice data; and a plurality of conference call devices communicate with each other.
As a specific embodiment of the present application, before receiving the voice data to be processed picked up by the microphone array, the method further includes:
a computational handshake mechanism between a plurality of conference call devices is established.
As a specific embodiment of the present application, the method further includes:
calculating the actual calculation force of each conference call device;
and performing distributed calculation on the voice data to be processed according to the actual calculation power of each conference call device.
In a third aspect, an embodiment of the present invention provides a computing device, including a processor, an input device, an output device, and a memory, where the processor, the input device, the output device, and the memory are connected to each other, where the memory is used to store a computer program, the computer program includes program instructions, and the processor is configured to call the program instructions to perform the following steps:
receiving voice data to be processed picked up by a microphone array;
calculating the voice data to be processed by adopting actual calculation power to obtain target voice data;
wherein the actual computational power is obtained by computing according to computational power requirements of a plurality of conference call devices and a microphone array; the actual computational power that the kth conference call device possesses is:
Gsk+G*(Tk-Gsk)/(T1+T2+……+Tk+……+Tn-Gs1-……Gsk-……-Gsn)
where Gsk is the pre-calculation power of the kth device, Gsn is the pre-calculation power of the nth device, G is the calculation power requirement of the microphone array algorithm, Tk is the original calculation power of the kth conference call device, and Tn is the original calculation power of the nth conference call device.
The method and the system thereof pick up the voice through the microphone array, establish the computing power handshake mechanism at the communication end of the conference communication equipment, distribute the computing power of partial algorithms of the local machine to the point-to-point equipment for distributed computation, thereby reducing the cost of the chip module.
Drawings
In order to more clearly illustrate the detailed description of the invention or the technical solutions in the prior art, the drawings that are needed in the detailed description of the invention or the prior art will be briefly described below.
FIG. 1 is a computational force calculation schematic according to an embodiment of the present invention;
FIG. 2 is a block diagram of a multi-party conference call system based on distributed computing according to an embodiment of the present invention;
FIG. 3 is a flowchart of a multi-party conference call method based on distributed computing according to an embodiment of the present invention;
FIG. 4 is a block diagram of a computing device provided by an embodiment of the invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, the inventive concept of the present invention is:
and establishing a calculation handshake mechanism at a communication end of the conference call equipment, and distributing the calculation power of the local part of algorithm to point-to-point equipment for distributed calculation. As shown in the following figure, when N devices perform communication, the computation force of the kth device is Tk, the voice of the kth device needs to be broadcast to N-1 devices to complete a conference call, the microphone array computation force part that needs to be completed on the kth device is placed on the N-1 devices to perform distributed computation, the computation force requirement of the microphone array algorithm is G, and the pre-computation force (the computation force required by the algorithm part that must be placed on the local end) is Gsk, then the kth device only needs to have the computation force of Gsk + G (Tk-Gsk)/(T1+ T2+ … … + Tk + … … + TN-Gs1- … … k- … … -Gsn), and the average computation force required by the devices participating in the distributed computation is not lower than the algorithm pre-computation force Gsk.
Based on the above inventive concept, an embodiment of the present invention provides a multiparty conference call system based on distributed computing, as shown in fig. 2, including:
the microphone array is used for picking up voice data to be processed;
the conference call equipment is used for carrying out distributed calculation on the voice data to be processed to obtain target voice data; and a plurality of conference call devices communicate with each other.
Wherein, the actual computational power that the kth conference call equipment possesses is:
Gsk+G*(Tk-Gsk)/(T1+T2+……+Tk+……+Tn-Gs1-……Gsk-……-Gsn)
gsk is the pre-calculation power of the kth device, Gsn is the pre-calculation power of the nth device, G is the calculation power requirement of the microphone array algorithm, Tk is the original calculation power of the kth conference call device, and Tn is the original calculation power of the nth conference call device.
The system of the invention is implemented, the voice is picked up by the microphone array, a calculation force handshake mechanism is established at the communication end of the conference communication equipment, and the calculation force of partial algorithm of the system is distributed on the point-to-point equipment for distributed calculation, thereby reducing the cost of the chip module.
Based on the same inventive concept, an embodiment of the present invention further provides a multiparty conference call method based on distributed computing, as shown in fig. 3, which may include:
and S1, establishing a computational handshake mechanism among the conference call devices.
S2, calculating the actual calculation force of each conference call device.
For example, the actual computational effort of the kth conference call device is:
Gsk+G*(Tk-Gsk)/(T1+T2+……+Tk+……+Tn-Gs1-……Gsk-……-Gsn)
where Gsk is the pre-calculation power of the kth device, Gsn is the pre-calculation power of the nth device, G is the calculation power requirement of the microphone array algorithm, Tk is the original calculation power of the kth conference call device, and Tn is the original calculation power of the nth conference call device.
And S3, receiving the voice data to be processed picked up by the microphone array.
And S4, performing distributed calculation on the voice data to be processed through a plurality of conference call devices to obtain target voice data.
Specifically, distributed computation is performed on the voice data to be processed according to the computed actual computation power of each conference call device, so that the computation power of the local partial algorithm is distributed to other point-to-point equipment merchants, the voice quality is guaranteed, and the cost of the chip module is reduced.
Referring to fig. 4 again, an embodiment of the present invention provides a computing device, including: one or more processors 101, one or more input devices 102, one or more output devices 103, and memory 104, the processors 101, input devices 102, output devices 103, and memory 104 being interconnected via a bus 105. The memory 104 is used for storing a computer program comprising program instructions, the processor 101 being configured for invoking the program instructions for performing the methods of the above-described method embodiment parts.
It should be understood that, in the embodiment of the present invention, the Processor 101 may be a Central Processing Unit (CPU), a deep learning graphics card (e.g., NPU, england GPU, google TPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an FPGA (Field-Programmable Gate Array) or other Programmable logic device, a discrete Gate or transistor logic device, a discrete hardware component, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
The input device 102 may include a keyboard or the like, and the output device 103 may include a display (LCD or the like), a speaker, or the like.
The memory 104 may include read-only memory and random access memory, and provides instructions and data to the processor 101. A portion of the memory 104 may also include non-volatile random access memory. For example, the memory 104 may also store device type information.
While the invention has been described with reference to specific embodiments, the invention is not limited thereto, and various equivalent modifications and substitutions can be easily made by those skilled in the art within the technical scope of the invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (7)
1. A multi-party conference call system based on distributed computing, comprising:
the microphone array is used for picking up voice data to be processed;
the conference call equipment is used for carrying out distributed calculation on the voice data to be processed to obtain target voice data; and a plurality of conference call devices communicate with each other.
2. The multi-party conference call system based on distributed computing of claim 1, wherein the k-th conference call device has an actual computational power of:
Gsk+G*(Tk-Gsk)/(T1+T2+……+Tk+……+Tn-Gs1-……Gsk-……-Gsn)
where Gsk is the pre-calculation power of the kth device, Gsn is the pre-calculation power of the nth device, G is the calculation power requirement of the microphone array algorithm, Tk is the original calculation power of the kth conference call device, and Tn is the original calculation power of the nth conference call device.
3. A multi-party conference call method based on distributed computing is characterized by comprising the following steps:
receiving voice data to be processed picked up by a microphone array;
carrying out distributed calculation on the voice data to be processed through a plurality of conference call devices to obtain target voice data; and a plurality of conference call devices communicate with each other.
4. The distributed computing-based multi-party conference call method of claim 3, wherein prior to receiving the pending voice data picked up by the microphone array, the method further comprises:
a computational handshake mechanism between a plurality of conference call devices is established.
5. The distributed computing-based multi-party conference call method of claim 4, wherein the method further comprises:
calculating the actual calculation force of each conference call device;
and performing distributed calculation on the voice data to be processed according to the actual calculation power of each conference call device.
6. The multi-party conference call method based on distributed computing of claim 5, wherein the k-th conference call device has an actual computational power of:
Gsk+G*(Tk-Gsk)/(T1+T2+……+Tk+……+Tn-Gs1-……Gsk-……-Gsn)
where Gsk is the pre-calculation power of the kth device, Gsn is the pre-calculation power of the nth device, G is the calculation power requirement of the microphone array algorithm, Tk is the original calculation power of the kth conference call device, and Tn is the original calculation power of the nth conference call device.
7. A computing device comprising a processor, an input device, an output device, and a memory, the processor, the input device, the output device, and the memory being interconnected, wherein the memory is configured to store a computer program comprising program instructions, the processor being configured to invoke the program instructions to perform the steps of:
receiving voice data to be processed picked up by a microphone array;
calculating the voice data to be processed by adopting actual calculation power to obtain target voice data;
wherein the actual computational power is obtained by computing according to computational power requirements of a plurality of conference call devices and a microphone array; the actual computational power that the kth conference call device possesses is:
Gsk+G*(Tk-Gsk)/(T1+T2+……+Tk+……+Tn-Gs1-……Gsk-……-Gsn)
where Gsk is the pre-calculation power of the kth device, Gsn is the pre-calculation power of the nth device, G is the calculation power requirement of the microphone array algorithm, Tk is the original calculation power of the kth conference call device, and Tn is the original calculation power of the nth conference call device.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110656110.3A CN113450821B (en) | 2021-06-11 | 2021-06-11 | Multi-party conference call system, method and computing device based on distributed computing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110656110.3A CN113450821B (en) | 2021-06-11 | 2021-06-11 | Multi-party conference call system, method and computing device based on distributed computing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113450821A true CN113450821A (en) | 2021-09-28 |
| CN113450821B CN113450821B (en) | 2024-05-07 |
Family
ID=77811319
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110656110.3A Active CN113450821B (en) | 2021-06-11 | 2021-06-11 | Multi-party conference call system, method and computing device based on distributed computing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113450821B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1543181A (en) * | 2003-04-30 | 2004-11-03 | 华为技术有限公司 | A distributed mix processing method |
| US20060062368A1 (en) * | 2004-09-20 | 2006-03-23 | International Business Machines Corporation | N-ways conference system using only participants' telephony devices without external conference server |
| CN101252452A (en) * | 2007-03-31 | 2008-08-27 | 红杉树（杭州）信息技术有限公司 | Distributed type tone mixing system in multimedia conference |
| CN101573955A (en) * | 2006-12-27 | 2009-11-04 | 诺基亚公司 | Distributed teleconference multichannel architecture, system, method, and computer program product |
| CN105068048A (en) * | 2015-08-14 | 2015-11-18 | 南京信息工程大学 | Distributed microphone array sound source positioning method based on space sparsity |
| CN106027946A (en) * | 2015-03-27 | 2016-10-12 | 阿尔卡特朗讯企业通信国际公司 | Method for allocating video conferencing task to processing device |
| CN108712584A (en) * | 2018-05-16 | 2018-10-26 | 中国电子科技集团公司第二十八研究所 | A kind of distributed sound mixing method for videoconference |
| US20200099792A1 (en) * | 2018-09-21 | 2020-03-26 | Dolby Laboratories Licensing Corporation | Audio conferencing using a distributed array of smartphones |
-
2021
- 2021-06-11 CN CN202110656110.3A patent/CN113450821B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1543181A (en) * | 2003-04-30 | 2004-11-03 | 华为技术有限公司 | A distributed mix processing method |
| US20060062368A1 (en) * | 2004-09-20 | 2006-03-23 | International Business Machines Corporation | N-ways conference system using only participants' telephony devices without external conference server |
| CN101573955A (en) * | 2006-12-27 | 2009-11-04 | 诺基亚公司 | Distributed teleconference multichannel architecture, system, method, and computer program product |
| CN101252452A (en) * | 2007-03-31 | 2008-08-27 | 红杉树（杭州）信息技术有限公司 | Distributed type tone mixing system in multimedia conference |
| CN106027946A (en) * | 2015-03-27 | 2016-10-12 | 阿尔卡特朗讯企业通信国际公司 | Method for allocating video conferencing task to processing device |
| CN105068048A (en) * | 2015-08-14 | 2015-11-18 | 南京信息工程大学 | Distributed microphone array sound source positioning method based on space sparsity |
| CN108712584A (en) * | 2018-05-16 | 2018-10-26 | 中国电子科技集团公司第二十八研究所 | A kind of distributed sound mixing method for videoconference |
| US20200099792A1 (en) * | 2018-09-21 | 2020-03-26 | Dolby Laboratories Licensing Corporation | Audio conferencing using a distributed array of smartphones |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113450821B (en) | 2024-05-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10546593B2 (en) | Deep learning driven multi-channel filtering for speech enhancement | |
| CN111951819A (en) | Echo cancellation method, device and storage medium | |
| CN108076226B (en) | Method for adjusting call quality, mobile terminal and storage medium | |
| US20200396329A1 (en) | Acoustic echo cancellation based sub band domain active speaker detection for audio and video conferencing applications | |
| US9143862B2 (en) | Correlation based filter adaptation | |
| WO2013009949A1 (en) | Microphone array processing system | |
| US20160133269A1 (en) | System and method for improving noise suppression for automatic speech recognition | |
| CN110769352B (en) | Signal processing method and device and computer storage medium | |
| CN112489670B (en) | Time delay estimation method, device, terminal equipment and computer readable storage medium | |
| CN110060696A (en) | Sound mixing method and device, terminal and readable storage medium storing program for executing | |
| US20220020386A1 (en) | Intelligent noise cancellation system for video conference calls in telepresence rooms | |
| CN110503973B (en) | Audio signal transient noise suppression method, system and storage medium | |
| CN113450821A (en) | Multi-party conference call system, method and computing device based on distributed computing | |
| CN110996208B (en) | Wireless earphone and noise reduction method thereof | |
| WO2024017110A1 (en) | Voice noise reduction method, model training method, apparatus, device, medium, and product | |
| CN113329372A (en) | Method, apparatus, device, medium and product for vehicle-mounted call | |
| CN112289336A (en) | Audio signal processing method and device | |
| CN110517682A (en) | Audio recognition method, device, equipment and storage medium | |
| CN113436636A (en) | Acoustic echo cancellation method and system based on adaptive filter and neural network | |
| CN114038452A (en) | Voice separation method and device | |
| CN107170461B (en) | Voice signal processing method and device | |
| CN111681666A (en) | Backup of filter coefficient, device and computer storage medium | |
| CN113409802B (en) | Method, device, equipment and storage medium for enhancing voice signal | |
| CN114023347A (en) | Directional sound pickup method and device, electronic equipment and storage medium | |
| US20230050621A1 (en) | Information transmission device, information reception device, information transmission method, recording medium, and system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |