CN111050262A - Intelligent voice-enhanced real-time electronic cochlea debugging system - Google Patents
Intelligent voice-enhanced real-time electronic cochlea debugging system Download PDFInfo
- Publication number
- CN111050262A CN111050262A CN202010024316.XA CN202010024316A CN111050262A CN 111050262 A CN111050262 A CN 111050262A CN 202010024316 A CN202010024316 A CN 202010024316A CN 111050262 A CN111050262 A CN 111050262A
- Authority
- CN
- China
- Prior art keywords
- module
- debugging
- algorithm
- voice
- speech
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 210000003477 cochlea Anatomy 0.000 title claims abstract description 62
- 238000004422 calculation algorithm Methods 0.000 claims abstract description 195
- 238000012545 processing Methods 0.000 claims abstract description 52
- 230000000694 effects Effects 0.000 claims abstract description 21
- 230000005540 biological transmission Effects 0.000 claims description 22
- 230000000638 stimulation Effects 0.000 claims description 22
- 238000004891 communication Methods 0.000 claims description 20
- 238000004088 simulation Methods 0.000 claims description 19
- 230000002860 competitive effect Effects 0.000 claims description 16
- 230000006870 function Effects 0.000 claims description 14
- 238000007781 pre-processing Methods 0.000 claims description 13
- 239000003990 capacitor Substances 0.000 claims description 12
- 230000003321 amplification Effects 0.000 claims description 8
- 238000006243 chemical reaction Methods 0.000 claims description 8
- 238000000034 method Methods 0.000 claims description 8
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 8
- 230000008676 import Effects 0.000 claims description 6
- 230000005236 sound signal Effects 0.000 claims description 6
- WHXSMMKQMYFTQS-UHFFFAOYSA-N Lithium Chemical compound [Li] WHXSMMKQMYFTQS-UHFFFAOYSA-N 0.000 claims description 5
- 229910052744 lithium Inorganic materials 0.000 claims description 5
- 238000004364 calculation method Methods 0.000 claims description 4
- 230000010354 integration Effects 0.000 claims description 4
- 230000000873 masking effect Effects 0.000 claims description 4
- 239000011159 matrix material Substances 0.000 claims description 4
- 230000004936 stimulating effect Effects 0.000 claims description 4
- 208000016444 Benign adult familial myoclonic epilepsy Diseases 0.000 claims description 3
- 208000016427 familial adult myoclonic epilepsy Diseases 0.000 claims description 3
- 235000019387 fatty acid methyl ester Nutrition 0.000 claims description 3
- ZGNITFSDLCMLGI-UHFFFAOYSA-N flubendiamide Chemical compound CC1=CC(C(F)(C(F)(F)F)C(F)(F)F)=CC=C1NC(=O)C1=CC=CC(I)=C1C(=O)NC(C)(C)CS(C)(=O)=O ZGNITFSDLCMLGI-UHFFFAOYSA-N 0.000 claims description 3
- 230000003595 spectral effect Effects 0.000 claims description 3
- 238000011161 development Methods 0.000 abstract description 3
- 238000011897 real-time detection Methods 0.000 abstract description 2
- 238000010586 diagram Methods 0.000 description 12
- 239000007943 implant Substances 0.000 description 6
- 230000037433 frameshift Effects 0.000 description 4
- 238000012360 testing method Methods 0.000 description 4
- 206010011878 Deafness Diseases 0.000 description 2
- 231100000895 deafness Toxicity 0.000 description 2
- 238000011156 evaluation Methods 0.000 description 2
- 238000001914 filtration Methods 0.000 description 2
- 208000016354 hearing loss disease Diseases 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 238000007476 Maximum Likelihood Methods 0.000 description 1
- 102100026436 Regulator of MON1-CCZ1 complex Human genes 0.000 description 1
- 101710180672 Regulator of MON1-CCZ1 complex Proteins 0.000 description 1
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000000338 in vitro Methods 0.000 description 1
- 230000002452 interceptive effect Effects 0.000 description 1
- 238000011084 recovery Methods 0.000 description 1
- 238000011160 research Methods 0.000 description 1
- 238000012827 research and development Methods 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/30—Monitoring or testing of hearing aids, e.g. functioning, settings, battery power
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61N—ELECTROTHERAPY; MAGNETOTHERAPY; RADIATION THERAPY; ULTRASOUND THERAPY
- A61N1/00—Electrotherapy; Circuits therefor
- A61N1/18—Applying electric currents by contact electrodes
- A61N1/32—Applying electric currents by contact electrodes alternating or intermittent currents
- A61N1/36—Applying electric currents by contact electrodes alternating or intermittent currents for stimulation
- A61N1/36036—Applying electric currents by contact electrodes alternating or intermittent currents for stimulation of the outer, middle or inner ear
- A61N1/36038—Cochlear stimulation
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L21/00—Speech or voice signal processing techniques to produce another audible or non-audible signal, e.g. visual or tactile, in order to modify its quality or its intelligibility
- G10L21/02—Speech enhancement, e.g. noise reduction or echo cancellation
- G10L21/0208—Noise filtering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/35—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception using translation techniques
- H04R25/356—Amplitude, e.g. amplitude shift or compression
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04R—LOUDSPEAKERS, MICROPHONES, GRAMOPHONE PICK-UPS OR LIKE ACOUSTIC ELECTROMECHANICAL TRANSDUCERS; DEAF-AID SETS; PUBLIC ADDRESS SYSTEMS
- H04R25/00—Deaf-aid sets, i.e. electro-acoustic or electro-mechanical hearing aids; Electric tinnitus maskers providing an auditory perception
- H04R25/50—Customised settings for obtaining desired overall acoustical characteristics
- H04R25/502—Customised settings for obtaining desired overall acoustical characteristics using analog signal processing
Landscapes
- Health & Medical Sciences (AREA)
- Engineering & Computer Science (AREA)
- Otolaryngology (AREA)
- General Health & Medical Sciences (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Signal Processing (AREA)
- Neurosurgery (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Biomedical Technology (AREA)
- Life Sciences & Earth Sciences (AREA)
- Animal Behavior & Ethology (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Computational Linguistics (AREA)
- Quality & Reliability (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Multimedia (AREA)
- Telephone Function (AREA)
Abstract
The invention provides an intelligent voice-enhanced real-time electronic cochlear debugging system which can be applied to actual cochlear front-end algorithm development and can help developers to detect and debug novel electronic cochlear front-end enhancement algorithms and speech coding strategies in real time. In the technical scheme of the invention, the performance of the electronic cochlea front-end algorithm to be tested in different auditory scenes is debugged by debugging an upper computer software module based on an off-line mode; after downloading the algorithm and the parameters of the front end of the electronic cochlea to be tested to the embedded processor module, switching the algorithm to be debugged in the portable debugging device based on the online mode, and transmitting the parameters for debugging to the portable debugging device; in the portable debugging device, the embedded processing module realizes real-time detection and debugging on the effect of the electronic cochlea front-end algorithm to be tested based on the voice signals acquired by the signal acquisition and processing module in real time.
Description
Technical Field
The invention relates to the technical field of electronics, computers and signal processing, in particular to an intelligent voice-enhanced real-time electronic cochlear debugging system.
Background
The cochlear implant is an important electronic hearing auxiliary device and can help patients with severe deafness and deafness to recover hearing. For researchers and engineers, the development of cochlear implant debugging platforms is very important for facilitating the study of cochlear implants.
A wireless data transmission-based artificial cochlea debugging and programming method and system (with the patent publication number of CN101744670A) are invented by people such as Shanghai Guanxin electronic technology Co., Ltd, and mainly achieve the function of connecting a speech processor and a programming debugger by a wireless transmission mode to exchange debugging parameters, so that a debugger is not limited by the distance and the range of motion of an electronic cochlea wearer when debugging programs. However, the invention can only modify and transmit parameters of a fixed algorithm, and cannot research and develop a novel algorithm.
Shenyang Hongdong kang medical instrument limited company has invented a debugging instrument for cochlear implant external devices (patent publication No. CN106075720A), an external machine and a micro control unit of an electronic cochlear implant can be connected through a connector, parameters such as frequency, carrier wave and gain are set for the external machine, and a storage unit is reversely debugged; and the running condition of the equipment with the modified parameters is monitored and observed in real time by using a display screen, an earphone and the like. However, the debugging instrument mainly faces to electronic cochlea wearers, achieves the effect of improving performance by adjusting the external equipment of the electronic cochlea wearers, and is not suitable for developing new algorithms in scientific researchers.
The invention relates to an evaluation system for testing the speech strategy performance of an electronic cochlea (patent publication number is CN104783928A) invented by Feng Hai hong et al of the institute of Acoustics of Chinese academy of sciences, wherein the evaluation system compares electrode signals collected in an electronic cochlea device with electrode signals simulated by a standard audio module on an image and checks whether a speech coding strategy generates normal stimulation pulse coding information or not, so that the purpose of evaluating the speech strategy performance of the electronic cochlea is achieved.
The invention discloses an electronic cochlea in-vitro debugging platform based on a local area network (with the patent publication number of CN102670331A) by the Gongqin and the like of the Qinghua university, and provides a debugging platform comprising an electronic cochlea algorithm experiment platform, an embedded processor module, a wireless transmitting coil and other modules. The platform can be used for setting parameters of a speech coding strategy on a PC (personal computer) and forming electrode coding information, and data are transmitted to the embedded equipment in a local area network communication mode to control the wireless transmitting coil to be coupled with the electronic cochlear implant so as to transmit data and energy. The platform integrates a mainstream electronic cochlear speech coding strategy, but does not realize the processing of real-time voice data, and can not effectively debug the electronic cochlear front-end enhancement algorithm.
Disclosure of Invention
The invention provides an intelligent voice-enhanced real-time electronic cochlear debugging system, which can be applied to actual cochlear front-end algorithm development and can help developers to detect and debug novel electronic cochlear front-end enhancement algorithms and speech coding strategies in real time.
The technical scheme of the invention is as follows: an intelligent voice-enhanced real-time electronic cochlear debugging system comprises a debugging upper computer software module, a signal acquisition processing module and an embedded processor module;
the debugging upper computer software module provides a selectable working mode for a user to support the user to debug the electronic cochlea front-end algorithm to be tested; the working modes comprise: the off-line mode allows a user to download the electronic cochlea front-end algorithm to be tested into the debugging upper computer software module after being started, then sets auditory scenes with different parameters according to the selection of the user, and respectively carries out off-line debugging on the performance of the electronic cochlea front-end algorithm to be tested, which is specified by the user, by using pre-stored voice data;
the method is characterized in that:
the operating mode further includes: in the online mode, after the debugging upper computer software module downloads the electronic cochlea front-end algorithm and the speech coding strategy to be debugged to be tested into the embedded processor module, the debugging upper computer software module transmits a command to switch the content to be debugged in the embedded processor module based on a USB protocol and transmits parameters used for debugging to the embedded processor module; the user uses a portable debugging device consisting of the embedded processing module and the signal acquisition processing module to finish the work of debugging the algorithm based on real-time dialogue voice, and further confirms the effect of the electronic cochlea front-end algorithm to be tested;
the embedded processor module is responsible for debugging the front-end algorithm of the electronic cochlea to be tested in real time, performing intelligent voice signal enhancement processing, processing of different speech coding strategies and communicating with the debugging upper computer software module in real time; it includes: a speech processor, an audio codec; the real-time signals collected by the signal collecting and processing module are preprocessed, transmitted to the audio coder-decoder for coding and then transmitted to the speech processor; the speech processor is integrated with different electronic cochlea front end enhancement algorithms and speech coding strategies, receives an instruction transmitted by the debugging upper computer software module in real time, selects a corresponding algorithm and a speech strategy, sets corresponding parameters, performs speech signal processing on a real-time speech signal transmitted by the signal acquisition processing module, performs off-LINE auditory simulation, controls the audio codec to convert a digital signal output in real time into an analog signal, and outputs the processed speech signal in a LINE-OUT mode so as to facilitate a user to confirm an effect in real time;
the signal acquisition processing module comprises: the system comprises a microphone structure, a signal preprocessing module and a power management module; the microphone structure is used for acquiring real-time sound signals in an actual scene; the signal preprocessing module is electrically connected with the microphone structure, performs signal processing on the sound signals collected by the microphone structure, and transmits the processed signals to the audio codec for decoding; the power management module is used for supplying power to the microphone structure and the signal preprocessing module;
the signal processing module and the embedded processor module form the portable debugging device, and are detachably in communication connection with the debugging upper computer software module through a USB port.
It is further characterized in that:
the signal acquisition processing module provides a voice enhancement function, acquires signals in real time by using the microphone array according to the wish of a called person, intelligently performs voice enhancement on the sound in the receiving direction, and removes competitive voice and background noise; in the signal acquisition and processing module, the microphone structure comprises two micro microphones, and the distance between the microphones is consistent with the arrangement of front and back microphones of the electronic cochlea; the two microphones convert the analog sound source signals into weak electric signals; the signal preprocessing module amplifies and filters the signals acquired by the microphone structure through a signal amplifying circuit and a power frequency filter circuit, limits the frequency of the acquired signals to 160 Hz-12000 Hz, and amplifies the analog signals by about 20 times to enable the analog signals to be directly accessed to a LINE-IN audio port of the audio codec; the signal amplification circuit is also provided with a patch adjustable resistor to realize fine adjustment of the amplification factor of the analog signal; the power management module comprises a battery, a charging circuit and a USB interface, wherein the voltage of the battery is converted into other voltages through a voltage conversion circuit, and the charging circuit can be used for charging through the USB interface when the electric quantity of the lithium battery is consumed;
the embedded processor module provides a speech coding function, integrates an electronic cochlear front-end algorithm and an electronic cochlear speech coding strategy, performs off-line auditory simulation, receives an instruction transmitted by the debugging upper computer software module by using a USB protocol, and performs parameter setting and algorithm switching of the electronic cochlear speech coding strategy in real time; in the embedded processor module, the speech processor is realized based on a digital signal processor chip, and the audio codec is realized based on an audio chip; the audio codec is mainly responsible for analog-digital/digital-analog conversion of signals, transmits data to the speech processor through the McASP interface, processes the data through an algorithm arranged in the speech processor, communicates with the debugging upper computer software module through the USB interface, receives an instruction from the debugging upper computer software module, and performs parameter setting and algorithm switching operation;
the connection circuit of the speech processor and the audio codec comprises: an audio chip U1, a digital signal processor chip U2; the 37 pin of the audio chip U1 is connected with the 43 pin of the digital signal processor chip U2, the 38 pin of the audio chip U1 is connected with the 37 pin of the digital signal processor chip U2, the 39 pin of the audio chip U1 is connected with the 33 pin of the digital signal processor chip U2, the 40 pin of the audio chip U1 is connected with the 26 pin of the digital signal processor chip U2, the 41 pin of the audio chip U1 is connected with the 24 pin of the digital signal processor chip U2, the 1 pin of the audio chip U1 is connected with the 9 pin of the digital signal processor chip U2, and the 2 pin of the audio chip U1 is connected with the 11 pin of the digital signal processor chip U2;
the embedded processor module also comprises a USB transmission module, the USB transmission module realizes parameter setting and algorithm switching instruction transmission between the debugging upper computer software module and the embedded processor module through a USB transmission circuit, and the USB transmission circuit comprises: the current-limiting power distribution circuit comprises a current-limiting power distribution switch U3, an ESD protection diode U4, a MINI USB interface CON1, a triode Q1, resistors R1-R4, capacitors C1 and C2; one end of the resistor R1 is connected to the USB0_ DRWBUSn pin of the dsp chip U2, the other end of the resistor R2, one end of the resistor R2, and the base of the transistor Q1 are connected to each other, the other end of the resistor R2 and the emitter of the transistor Q1 are connected to each other and then grounded, the collector of the transistor Q1, one end of the resistor R3, and 4 pins of the current-limiting distribution switch U3 are connected to each other, the other end of the resistor R3 is connected to a power supply, the 5 pin of the current-limiting distribution switch U3 is connected to the power supply, the 2 pin of the current-limiting distribution switch U3 is grounded, the 1 pin and 3 pin of the current-limiting distribution switch U3, one end of the resistor R4, one end of the capacitor C1, one end of the capacitor C2, the 5 pin of the ESD protection diode U4, the 1 pin of the MINI USB interface CON1 is connected to the power supply, and the other corner of the resistor 46r 45 is connected to the, the other ends of the capacitors C1 and C2 are connected to the ground, the other end of the capacitor C1 and C2 is connected to the ground, the pin 1 of the ESD protection diode U4 is connected to the pin 3 of the MINI USB interface CON1, the pin 2 of the ESD protection diode U4 and the pin 5 of the MINI USB interface CON1 are connected to the ground, the pin 3 of the ESD protection diode U4 is connected to the pin 4 of the MINI USB interface CON1, the pin 6 of the ESD protection diode U4 is connected to the pin 2 of the MINI USB interface CON1, and the pins 6, 7, 8 and 9 of the MINI USB interface CON1 are connected to the ground;
the sub-modules of the off-line mode in the debugging upper computer software module comprise: module selection, data import, database selection, scene simulation, algorithm selection and effect display;
the module selection module defines a main working mode of the offline mode, and the main working module comprises: the system comprises a voice enhancement algorithm debugging module, a speech coding strategy debugging module and a hardware communication module;
the voice enhancement algorithm debugging module supports a user to debug by using a preset algorithm and simultaneously provides a set of standard interface form to support the user to introduce a new algorithm for debugging; the speech coding strategy debugging module is preset with the existing speech coding strategy and simultaneously supports a user to download a new speech coding strategy to the module for debugging;
the debugging mode supported by the module selection module comprises the following steps: monotone and joint tone; the monotony refers to that the algorithm in the voice enhancement algorithm debugging module and the speech strategy in the speech coding strategy debugging module are respectively and independently debugged, and in the monotony mode, a quiet mode is selected in the scene simulation module when the speech coding strategy is independently debugged; the joint debugging refers to that an algorithm in the voice enhancement algorithm debugging module and a speech strategy in the speech coding strategy debugging module are simultaneously debugged, in the joint debugging mode, a mode with noise is selected in the scene simulation module when the speech coding strategy is debugged, and then the voice enhancement algorithm and the speech coding algorithm are simultaneously set;
the hardware communication module transmits the amplitude information and the rate information of the electrical cochlea stimulating current generated after the algorithm and the speech strategy are debugged in the voice enhancement algorithm debugging module and the speech coding strategy debugging module to the speech processor in a file form based on a USB protocol;
the data import module provides services for importing the existing voice enhancement algorithm and the speech coding strategy which meet the requirements of the standard interface form and the voice enhancement algorithm and the speech coding strategy designed by the user; the imported existing and self-defined voice enhancement algorithm and the language coding strategy are provided for the user to select in the algorithm selection module;
the database selection module stores preset voice packets for off-line debugging, wherein the voice packets comprise voice packets of different language types;
setting a specific debugging scene in the scene simulation module, and dividing an application scene of a voice enhancement algorithm into a strong noise mode, a conference mode and a daily mode according to different noise signal combination types received by a microphone; the strong noise mode only sets background noise, namely the target voice is only interfered by the background noise; the conference mode only sets competitive voice noise, namely, the target voice is influenced by preset competitive voice but has no influence of background noise; the daily mode simultaneously sets background noise and competitive voice, namely, the target voice is influenced by the competitive voice and is interfered by the background noise;
in the algorithm selection module, a user selects an algorithm to be tested, a speech coding strategy and corresponding parameter configuration during debugging, which are debugged in the voice enhancement algorithm debugging module and the speech coding strategy debugging module, according to the needs of the user; the specific setting content in the algorithm selection module comprises: the method comprises the following steps of voice enhancement algorithm, speech coding strategy, coding channel number, stimulation channel number, minimum electric stimulation intensity T value generated by auditory sense, maximum comfort threshold C value of a wearer, stimulation rate, pulse width and stimulation mode; after the algorithm selection module selects the content to be debugged, debugging is carried out based on the voice enhancement algorithm debugging module and the speech coding strategy debugging module;
the effect display module supports the performance of displaying the algorithm in different forms, and the display mode comprises the following steps: a spectrogram, a gray scale image, a time domain oscillogram, audio playing and performance index calculation; the results of the debugging of the voice enhancement algorithm debugging module and the speech coding strategy debugging module are transmitted to the effect display module to be displayed in different forms from different angles;
the preset algorithm in the voice enhancement algorithm debugging module comprises: spectral subtraction, beam integration, masking matrix; the preset speech coding strategy in the speech coding strategy debugging module comprises the following steps: a CIS strategy, an ACE strategy, an FFC strategy, a FAME strategy and a 9-degree coding strategy;
the online mode in the debugging upper computer software module comprises a data communication module, an algorithm switching module and a parameter setting module, and the data communication module supports the real-time debugging of the algorithm to be tested, the speech coding strategy and the electronic cochlea stimulation current amplitude information and speed information generated during the debugging in the offline mode to the embedded processor module in the portable debugging device based on the real-time voice environment in the portable debugging device through real-time voice data; the algorithm switching module and the parameter setting module transmit commands and parameters to the portable debugging device based on a USB protocol, switch algorithms and speech coding strategies which need to be debugged in the portable debugging device, and set debugging parameters.
The invention provides an intelligent voice-enhanced real-time electronic cochlea debugging system, which debugs the performance of an electronic cochlea front-end algorithm to be tested in different auditory scenes by debugging an upper computer software module based on an off-line mode; after downloading the algorithm and the parameters of the front end of the electronic cochlea to be tested to the embedded processor module, switching the algorithm to be debugged in the portable debugging device based on the online mode, and transmitting the parameters for debugging to the portable debugging device; in the portable debugging device, the embedded processing module realizes real-time detection and debugging on the effect of the electronic cochlea front-end algorithm to be tested based on the voice signal acquired by the signal acquisition processing module in real time; based on the technical scheme of the invention, technicians can realize the debugging of the cochlear front-end algorithm in different scenes, and a portable debugging device is used for detecting and debugging a novel electronic cochlear front-end enhancement algorithm and a speech coding strategy in real time; the portable debugging device in the technical scheme of the invention can be carried by a user, and further provides a real-time research and development environment for developers to develop a voice enhancement algorithm and a new speech coding strategy.
Drawings
FIG. 1 is a block diagram of the components of the debugging system of the present invention;
FIG. 2 is a functional block diagram of a debug system according to the present invention;
fig. 3 is a block diagram of a microphone array module according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of an audio codec and a DSP according to an embodiment of the present invention;
FIG. 5 is a circuit diagram of a USB-OTG interface according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a software module of a debugging upper computer according to an embodiment of the present invention;
FIG. 7 is a diagram illustrating an offline mode interface of a debugging host computer according to an embodiment of the present invention;
FIG. 8 is an on-line mode display diagram of a debugging host computer according to an embodiment of the present invention;
FIG. 9 is a diagram showing an interface of test results of off-line debugging of the system according to the embodiment of the present invention;
fig. 10 is a diagram illustrating a test effect of real-time debugging of the system according to the embodiment of the present invention.
Detailed Description
As shown in fig. 1 of the attached drawings of the specification, the intelligent voice-enhanced real-time electronic cochlear debugging system comprises a debugging upper computer software module 1, an embedded processor module 2 and a signal acquisition processing module 3;
the debugging upper computer software module 1 provides a selectable working mode for a user to support the user to debug the electronic cochlea front-end algorithm to be tested; the working modes comprise: the off-line mode is started, the user is allowed to download the electronic cochlea front-end algorithm to be tested into the debugging upper computer software module, then auditory scenes with different parameters are set according to the selection of the user, and the performance of the electronic cochlea front-end algorithm to be tested specified by the user is respectively debugged off-line by using pre-stored voice data;
the working mode further comprises: in the online mode, after the debugging upper computer software module downloads the electronic cochlea front-end algorithm and the speech coding strategy to be debugged to the embedded processor module, the debugging upper computer software module transmits a command to switch the content to be debugged in the embedded processor module based on the USB protocol and transmits parameters used for debugging to the embedded processor module 2; a user uses a portable debugging device consisting of the embedded processing module 2 and the signal acquisition processing module 3 to complete the work of debugging the algorithm based on real-time dialogue voice, and further confirm the effect of the algorithm at the front end of the electronic cochlea to be tested;
the embedded processor module 2 is responsible for debugging the front-end algorithm of the electronic cochlea to be tested in real time, carrying out intelligent voice signal enhancement processing, processing of different speech coding strategies and communication with the debugging upper computer software module 1 in real time; it includes: a speech processor 2-2 and an audio codec 2-1; the real-time signals acquired by the signal acquisition and processing module 2 are subjected to preprocessing such as amplification and filtering, transmitted to the audio codec 2-1 for encoding, and then transmitted to the speech processor 2-2; different electronic cochlea front end enhancement algorithms and different speech coding strategies are integrated in the speech processor, instructions transmitted by a debugging upper computer software module are received in real time, corresponding algorithms and speech strategies are selected, corresponding parameters are set, then the real-time voice signals transmitted by the signal acquisition processing module 3 are subjected to voice signal processing, off-LINE auditory simulation is performed, meanwhile, the audio codec 2-1 is controlled to convert digital signals output in real time into analog signals, and the processed voice signals are output through the earphone 4 in a LINE-OUT mode, so that a user can confirm effects in real time;
the signal acquisition processing module 3 includes: the device comprises a microphone structure 3-1, a signal preprocessing module 3-2 and a power management module 3-3; the microphone structure 3-1 is used for collecting real-time sound signals in an actual scene; the signal preprocessing module 3-2 is electrically connected to the microphone structure, and performs signal processing on the sound signal collected by the microphone structure 3-1, such as: enhancing the voice, removing competitive voice and background noise, and transmitting the processed signal to an audio codec 2-1 for decoding; the power management module 3-3 is used for supplying power to the microphone structure 3-1 and the signal preprocessing module 3-2;
the signal processing module 3 and the embedded processor module 2 form a portable debugging device which is detachably in communication connection with the debugging upper computer software module 1 through a USB port.
As shown in fig. 2 of the attached drawings of the specification, the electronic cochlea real-time debugging system based on the microphone array provided by the invention realizes the function of voice enhancement through the signal acquisition and processing module 3, acquires the field voice signals in real time, performs processing such as amplification and filtering on the signals, then performs voice signal processing on the real-time voice based on the electronic cochlea front-end enhancement algorithm to be tested and the speech coding strategy which are arranged in the embedded processor module 2, realizes auditory simulation, and ensures that a user can confirm the effect of algorithm debugging in real time through earphone output; moreover, the embedded processor module 2 and the debugging upper computer software module 1 can carry out real-time interactive communication based on the USB protocol; the user can confirm the debugging results of the enhanced algorithm and the speech coding strategy at the front end of the electronic cochlea according to the voice effect output by the real-time earphone 4, and the parameters in the embedded processor module 2 and the signal acquisition processing module 3 are adjusted in real time by debugging the upper computer software module 1, so that the implementation and debugging work of the algorithm is realized.
As shown in fig. 3 of the attached drawings of the specification, the signal acquisition processing module 3 provides a voice enhancement function, and acquires signals in real time by using a microphone array according to the wish of a called person, so that voice enhancement is intelligently performed on the sound in the receiving direction, and competitive voice and background noise are removed; in the signal acquisition and processing module, the microphone structure includes two micro microphones mic1, mic2, in this embodiment, the two microphones use an omnidirectional silicon microphone, the distance between the microphones is consistent with the arrangement of the front and back microphones of the cochlea, in this embodiment: 15 mm; the microphone is recorded as a front microphone close to the sound source, and the rest microphones are recorded as rear microphones, so that the real-time sound data of the site can be completely collected; the two microphones convert the analog sound source signals into weak electric signals; the signal preprocessing module amplifies and filters signals acquired by the microphone structure through a signal amplifying circuit and a power frequency filter circuit, limits the frequency of the acquired signals to 160 Hz-12000 Hz, and amplifies the analog signals by about 20 times to enable the analog signals to be directly accessed to a LINE-IN audio port of an audio codec; the signal amplification circuit is also provided with a patch adjustable resistor to realize the fine adjustment of the amplification factor of the analog signal; the power management module comprises a battery, wherein 4.2V lithium batteries are used for supplying power in the embodiment, the voltage of the battery is converted into other voltages through a voltage conversion circuit, and in the embodiment, 4.2V is converted into 5V, 3.3V and 1.8V through the voltage conversion circuit; the charging circuit is also included, and when the electric quantity of the lithium battery is completely consumed, the charging can be carried out through the USB interface; the voltage conversion circuit and the charging circuit in this embodiment can be implemented by using existing and known circuits.
The embedded processor module 2 provides a speech coding function, integrates an electronic cochlear front-end algorithm and an electronic cochlear speech coding strategy, performs off-line auditory simulation, receives an instruction for debugging the upper computer software module 1 by utilizing a USB protocol, and performs parameter setting and algorithm switching of the electronic cochlear speech coding strategy in real time; in the embedded processor module 2, a speech processor 2-2 is realized based on a digital signal processor chip, and an audio codec 2-1 is realized based on an audio chip; the audio codec 2-1 is mainly responsible for analog-to-digital/digital-to-analog conversion of signals, data are transmitted to the speech processor 2-2 through the McASP interface and processed through an algorithm arranged in the McASP interface, and the speech processor 2-2 can communicate with the debugging upper computer software module 1 through the USB interface, receive an instruction from the debugging upper computer software module, and perform parameter setting and algorithm switching operations.
The embedded processor module 2 comprises: the audio chip U1 and the digital signal processor chip U2 are externally assisted by an audio decoding circuit and a USB transmission circuit, and have the functions of signal processing and data transmission, the audio chip U1 and the digital signal processor chip U2 carry out control instruction communication through an IIC protocol, and carry out audio data transmission with the DSP through a McASP bus transmission form; in the embodiment, the U1 is realized by using an audio chip with the model of AIC3106, and the U2 is realized by using chips of TMSC67X series; in the audio decoding circuit, a pin 37 of an audio chip U1 is connected with a pin 43 of a digital signal processor chip U2, a pin 38 of an audio chip U1 is connected with a pin 37 of a digital signal processor chip U2, a pin 39 of the audio chip U1 is connected with a pin 33 of a digital signal processor chip U2, a pin 40 of an audio chip U1 is connected with a pin 26 of the digital signal processor chip U2, a pin 41 of the audio chip U1 is connected with a pin 24 of a digital signal processor chip U2, a pin 1 of the audio chip U1 is connected with a pin 9 of the digital signal processor chip U2, and a pin 2 of the audio chip U1 is connected with a pin 11 of the digital signal processor chip U2.
The embedded processor module 2 also comprises a USB transmission module which realizes parameter setting and algorithm switching instruction transmission between the debugging upper computer software module 1 and the embedded processor module 2 through a USB transmission circuit; during data transmission, the DSP will detect the status of the ID pin via USB0_ ID: if the ID pin is grounded at the moment, the DSP needs to be set to be in a host mode, and the power supply specification in the USB 2.0 is 5V @500mA, so that the current limiting operation needs to be carried out on the power supply; in the embodiment, the current is limited to 500mA by enabling the current-limiting power distribution switch TPS2041BDBVT chip through the USB0_ DRWBUSn pin and the triode switch circuit; in consideration of the characteristic of USB hot plug, an integrated ESD protection diode PRTR5V0U4Y chip is used at the USB interface, so that the damage of the USB interface caused by electrostatic discharge is prevented; USB transmission circuit based on USB-OTG interface referring to fig. 5 of the accompanying drawings of the specification, comprising: the current-limiting power distribution circuit comprises a current-limiting power distribution switch U3, an ESD protection diode U4, a MINI USB interface CON1, a triode Q1, resistors R1-R4, capacitors C1 and C2; one end of a resistor R1 is connected with a USB0_ DRWBUSn pin of a digital signal processor chip U2, the other end of a resistor R2, one end of a resistor R2 and the base of a triode Q1 are connected with each other, the other end of a resistor R2 and the emitter of a triode Q1 are connected with each other and then grounded, the collector of a triode Q1, one end of a resistor R3 and 4 pins of a current-limiting distribution switch U3 are connected with each other, the other end of a resistor R3 is connected with a power supply, a 5 pin of a current-limiting distribution switch U3 is connected with the power supply, a 2 pin of a current-limiting distribution switch U3 is grounded, a 1 pin and a 3 pin of a current-limiting distribution switch U3, one end of a resistor R4, one end of a capacitor C1, one end of a capacitor C2, a 5 pin of an ESD protection diode U4 and a 1 pin of a MICON interface CON1 are connected and then connected with the power supply, the other corner of a resistor R4 is connected with the power supply, the other end of an ESD, a pin 2 of the ESD protection diode U4 and a pin 5 of the MINI USB interface CON1 are connected and then grounded, a pin 3 of the ESD protection diode U4 is connected to a pin 4 of the MINI USB interface CON1, a pin 6 of the ESD protection diode U4 is connected to a pin 2 of the MINI USB interface CON1, and a pin 6, a pin 7, a pin 8, and a pin 9 of the MINI USB interface CON1 are connected and then grounded;
the USB circuit which is the simplest hardware circuit and the smallest occupied area of components is selected in the embodiment, so that the minimum size of the portable debugging device is ensured, and the portable debugging device is further convenient for a user to carry.
As shown in fig. 6 and 7 of the attached drawings of the specification, the debugging upper computer software module comprises an online mode module and an offline mode module; the sub-modules of the offline mode include: module selection, data import, database selection, scene simulation, algorithm selection and effect display;
the module selection module defines a main working mode of the offline mode, and the main working module comprises: the system comprises a voice enhancement algorithm debugging module, a speech coding strategy debugging module and a hardware communication module;
the voice enhancement algorithm debugging module supports a user to debug by using a preset algorithm and simultaneously provides a set of standard interface form to support the user to introduce a new algorithm for debugging; the preset algorithm comprises the following steps: spectral subtraction, beam integration, masking matrix; the speech coding strategy debugging module is preset with the existing speech coding strategy and simultaneously supports downloading of a new speech coding strategy into the module, and the preset speech coding strategy comprises the following steps: a CIS strategy, an ACE strategy, an FFC strategy, a FAME strategy and a 9-degree coding strategy;
the debugging mode supported by the module selection module comprises the following steps: monotone and joint tone; the monotony is that the algorithm in the voice enhancement algorithm debugging module and the speech strategy in the speech coding strategy debugging module are respectively debugged independently, and in the monotony mode, a quiet mode is selected in the scene simulation module when the speech coding strategy is debugged independently; joint debugging refers to simultaneously debugging the algorithm in the voice enhancement algorithm debugging module and the speech strategy in the speech coding strategy debugging module, in a joint debugging mode, selecting a mode with noise in the scene simulation module during the speech coding strategy debugging, and then simultaneously setting the voice enhancement algorithm and the speech coding algorithm;
the hardware communication module transmits the amplitude information and the rate information of the electrical cochlea stimulating current generated after the algorithm and the speech strategy are debugged in the voice enhancement algorithm debugging module and the speech coding strategy debugging module to a speech processor in a file form based on a USB protocol;
the data import module provides services for importing the existing voice enhancement algorithm and the speech coding strategy which meet the requirements of the standard interface form and the voice enhancement algorithm and the speech coding strategy designed by the user; the imported existing and self-defined voice enhancement algorithm and the language coding strategy are provided for the user to select in an algorithm selection module;
in this embodiment, in the offline mode, a new algorithm may be introduced to the debugging upper computer, as long as the input and output of the loaded algorithm meet the standard interface form of the system, the currently defined interface form is:
function xfinal＝MyAlg(x,Srate,FrameL,FrameS,FLOOR)
wherein, the function name MyAlg can be freely defined, and is input into the parameters: x represents time domain data of a signal with noise, Srate represents a sampling rate, FrameL represents a frame length, FrameS represents a frame shift, FLOOR represents an over-subtraction factor, and an output parameter xfinal represents the time domain data of the processed signal;
similarly, speech coding is one of the debugging contents of the technical scheme of the invention, and a speech coding strategy can convert acoustic signals into amplitude information and rate information of stimulating current of the electronic cochlea; in an off-line mode, a user also supports the user to load a self-defined speech coding strategy, only the input and output of a loaded algorithm meet the standard interface form of the system, and the currently defined interface form is as follows:
function data_out＝UserDefine(freshdata,framelength,frameshift,
channel_num,channel,fs,C,T)
wherein, the function name UserDefine can be freely defined, and is input into the parameters: the freshdata represents time domain data of a signal to be detected, fs represents a sampling rate, framelength represents a frame length, frameshift represents frame shift, channel _ num is a selected channel number, C and T respectively represent preset C and T values, wherein the T and the C values are integers of 0-255, and the C value is larger than the T value. The output parameter data _ out represents the stimulation coding of the signal;
after a user loads a self-defined algorithm or a speech coding strategy into a system, the user needs to select self-definition in an algorithm selection module, and then the self-defined algorithm and the coding strategy are debugged through an algorithm and a speech coding strategy debugging module in a voice enhancement algorithm debugging module;
the database selection module stores preset voice packets for off-line debugging, wherein the voice packets comprise voice packets of different language types; the user can select the off-line voice packet used in off-line debugging by himself, and a Chinese corpus and an English corpus are used in the embodiment;
setting a specific debugging scene in a scene simulation module, and dividing an application scene of a voice enhancement algorithm into a strong noise mode, a conference mode and a daily mode according to different noise signal combination types received by a microphone; the strong noise mode only sets background noise, namely the target voice is only interfered by the background noise; only setting competitive voice noise in the conference mode, namely the target voice is influenced by preset 1-2 competitive voices but has no influence of background noise; setting background noise and competitive voice at the same time in a daily mode, namely the target voice is influenced by the competitive voice and is interfered by the background noise;
in the algorithm selection module, a user selects an algorithm to be tested, a speech coding strategy and corresponding parameter configuration during debugging, which are debugged in the voice enhancement algorithm debugging module and the speech coding strategy debugging module, according to the needs of the user; the specific setting content in the algorithm selection module comprises the following steps: the method comprises the following steps of voice enhancement algorithm, speech coding strategy, coding channel number, stimulation channel number, minimum electric stimulation intensity T value generated by auditory sense, maximum comfort threshold C value of a wearer, stimulation rate, pulse width and stimulation mode; after the algorithm selection module selects the content to be debugged, debugging is carried out based on the voice enhancement algorithm debugging module and the speech coding strategy debugging module; in the two modules of the voice enhancement algorithm and the speech coding strategy, besides the existing algorithm and the speech coding strategy, a self-defined option is also respectively set for supporting a user to select the algorithm and the speech coding strategy loaded by the user;
the specific settings in this embodiment are shown in table 1 below:
table 1: algorithm selection module parameter configuration table
The effect display module supports the performance of the algorithm displayed in different forms, and the display mode comprises the following steps: a spectrogram, a gray scale image, a time domain oscillogram, audio playing and performance index calculation; and the results of the debugging of the voice enhancement algorithm debugging module and the speech coding strategy debugging module are transmitted to the effect display module to be displayed in different forms from different angles.
The online mode in the debugging upper computer software module comprises a data communication module, an algorithm switching module and a parameter setting module, wherein the data communication module supports the real-time debugging of the algorithm to be tested, the speech coding strategy and the electronic cochlea stimulation current amplitude information and speed information generated during the debugging in the offline mode to the embedded processor module in the portable debugging device based on the real-time voice environment in the portable device; the algorithm switching module and the parameter setting module transmit commands and parameters to the portable debugging device based on the USB protocol, switch algorithms and speech coding strategies which need to be debugged in the portable debugging device, and set debugging parameters.
In this embodiment, VC + +, the function of the real-time mode in the debugging upper computer software module is realized based on the MFC framework, the user directly switches the algorithm and the configuration parameters on the interface of the debugging upper computer software module, then transmits the instruction to the portable debugging device through USB transmission, performs real-time signal processing, and finally directly listens to the voice signal after algorithm processing through the earphone output end of the portable debugging device. Now, referring to fig. 8, the interface shown in the figure is a real-time mode interface, i.e. an operation interface for debugging an online mode in an upper computer software module, and an algorithm implemented by the game system according to the present invention includes: a masking matrix algorithm, a maximum likelihood estimation denoising algorithm based on beam integration, and CIS and ACE speech coding strategies. After selecting the speech coding strategy, the user can also set the output channel and the stimulation channel by parameters, and the specific parameter settings are shown in table 2 below.
Table 2: parameter setting list in online mode module
In the technical scheme of the invention, when a user downloads a self-defined algorithm and a speech coding strategy to the portable debugging device, the program needs to be rewritten, an MATLAB program is rewritten into a C language program, and the algorithm of the C language version is downloaded to the embedded portable debugging device.
The system is tested in an "offline mode" and an "online mode" respectively. Under an off-line mode, a daily conversation scene is simulated, namely a plurality of people speak at the same time and background noise interference exists, an enhancement algorithm to be tested is selected on the system, an ACE (adaptive communication interface) speech coding strategy is used for carrying out pulse coding on the enhanced speech, and a coding gray-scale image and a time-domain waveform image of the simulated speech as shown in figure 9 can be obtained. As can be seen from fig. 9, due to the interference of the speech noise and the background noise, the phenomenon of wrong coding occurs in the stimulus coding of the mixed speech, or the phenomenon of wrong selection of the stimulus channel occurs, and it can be obviously found by reflecting on the coding gray scale map: the gray level image of the mixed voice is not consistent with the gray level image of the original voice, the voice which is subjected to voice enhancement and then subjected to ACE coding has high recovery degree of the stimulation coding, the stimulation coding is modulated by a vocoder to synthesize simulated voice, and the performance of a novel enhancement algorithm can be compared by observing the time domain waveform of the simulated voice.
In an 'online mode', a microphone in equipment is directly used for acquiring voice signals in the environment, algorithm switching and parameter setting of a portable debugging device are controlled by a debugging upper computer software module, a sound card is connected to an audio output port of the portable debugging device, data output by a system in real time are acquired, a time domain waveform diagram shown in fig. 10 can be obtained, time points 1,2 and 3 of algorithm switching can be clearly seen in the diagram, and the real-time performance of the system operation is also proved.
By using the technical scheme of the invention, a user can debug the electronic cochlea front-end algorithm and the electronic cochlea speech coding strategy to be debugged based on the preset language packet in the off-line mode of debugging the upper computer software module, and after the debugging is finished, the debugged electronic cochlea front-end algorithm, the electronic cochlea speech coding strategy and the electronic cochlea stimulation current amplitude information and speed information generated in the off-line mode during debugging are transplanted into the portable debugging device and are debugged in real time based on real-time voice data; meanwhile, the portable debugging device is connected with the upper computer software module based on the USB interface, so that the connection between the portable debugging device and the upper computer software module can be released, the portable debugging device is carried to enter different actual environments, and further debugging is carried out on the electronic cochlea front-end algorithm and the electronic cochlea speech coding strategy to be debugged.
In the technical scheme of the invention, a debugging upper computer software module is provided, and the debugging upper computer software module has two debugging modes of an off-line mode and an on-line mode, under the off-line mode system, a user designs different auditory scenes, tests the performance of an algorithm under different conditions, also provides a display interface and the calculation of common indexes, and displays a time domain graph, a spectrogram graph and a gray scale graph of stimulation codes of processed voice; under the 'online mode', a user can communicate with the portable debugging device through a USB protocol by using the debugging upper computer software module to perform algorithm switching and parameter control, parameter adjustment can be performed in real time through the debugging upper computer software module, under the condition that the connection environment of the debugging upper computer software module is not available, the electronic cochlea front-end algorithm to be debugged and the electronic cochlea speech coding strategy can be downloaded and debugged in various complex environments through the portable debugging device independently, the algorithm can be debugged based on real-time voice data by the user, and the user can also be ensured to debug the customized electronic cochlea front-end algorithm.
Claims (8)
1. An intelligent voice-enhanced real-time electronic cochlear debugging system comprises a debugging upper computer software module, a signal acquisition processing module and an embedded processor module;
the debugging upper computer software module provides a selectable working mode for a user to support the user to debug the electronic cochlea front-end algorithm to be tested; the working modes comprise: the off-line mode allows a user to download the electronic cochlea front-end algorithm to be tested into the debugging upper computer software module after being started, then sets auditory scenes with different parameters according to the selection of the user, and respectively carries out off-line debugging on the performance of the electronic cochlea front-end algorithm to be tested, which is specified by the user, by using pre-stored voice data;
the method is characterized in that:
the operating mode further includes: in the online mode, after the debugging upper computer software module downloads the electronic cochlea front-end algorithm and the speech coding strategy to be debugged to be tested into the embedded processor module, the debugging upper computer software module transmits a command to switch the content to be debugged in the embedded processor module based on a USB protocol and transmits parameters used for debugging to the embedded processor module; the user uses a portable debugging device consisting of the embedded processing module and the signal acquisition processing module to finish the work of debugging the algorithm based on real-time dialogue voice, and further confirms the effect of the electronic cochlea front-end algorithm to be tested;
the embedded processor module is responsible for debugging the front-end algorithm of the electronic cochlea to be tested in real time, performing intelligent voice signal enhancement processing, processing of different speech coding strategies and communicating with the debugging upper computer software module in real time; it includes: a speech processor, an audio codec; the real-time signals collected by the signal collecting and processing module are preprocessed, transmitted to the audio coder-decoder for coding and then transmitted to the speech processor; the speech processor is integrated with different electronic cochlea front end enhancement algorithms and speech coding strategies, receives an instruction transmitted by the debugging upper computer software module in real time, selects a corresponding algorithm and a speech strategy, sets corresponding parameters, performs speech signal processing on a real-time speech signal transmitted by the signal acquisition processing module, performs off-LINE auditory simulation, controls the audio codec to convert a digital signal output in real time into an analog signal, and outputs the processed speech signal in a LINE-OUT mode so as to facilitate a user to confirm an effect in real time;
the signal acquisition processing module comprises: the system comprises a microphone structure, a signal preprocessing module and a power management module; the microphone structure is used for acquiring real-time sound signals in an actual scene; the signal preprocessing module is electrically connected with the microphone structure, performs signal processing on the sound signals collected by the microphone structure, and transmits the processed signals to the audio codec for decoding; the power management module is used for supplying power to the microphone structure and the signal preprocessing module;
the signal processing module and the embedded processor module form the portable debugging device, and are detachably in communication connection with the debugging upper computer software module through a USB port.
2. The intelligent voice-enhanced real-time electronic cochlear debugging system of claim 1, wherein: the signal acquisition processing module provides a voice enhancement function, acquires signals in real time by using the microphone array according to the wish of a called person, intelligently performs voice enhancement on the sound in the receiving direction, and removes competitive voice and background noise; in the signal acquisition and processing module, the microphone structure comprises two micro microphones, and the distance between the microphones is consistent with the arrangement of front and back microphones of the electronic cochlea; the two microphones convert the analog sound source signals into weak electric signals; the signal preprocessing module amplifies and filters signals acquired by the microphone structure through a signal amplifying circuit and a power frequency filter circuit, limits the frequency of the acquired signals to 160 Hz-12000 Hz, and amplifies analog signals by about 20 times to enable the analog signals to be directly accessed to a LINE-IN audio port of the audio codec; the signal amplification circuit is also provided with a patch adjustable resistor to realize fine adjustment of the amplification factor of the analog signal; the power management module comprises a battery, converts the voltage of the battery into other voltages through a voltage conversion circuit, and also comprises a charging circuit, and when the electric quantity of the lithium battery is consumed, the lithium battery can be charged through a USB interface.
3. The intelligent voice-enhanced real-time electronic cochlear debugging system of claim 1, wherein: the embedded processor module provides a speech coding function, integrates an electronic cochlear front-end algorithm and an electronic cochlear speech coding strategy, performs off-line auditory simulation, receives an instruction transmitted by the debugging upper computer software module by using a USB protocol, and performs parameter setting and algorithm switching of the electronic cochlear speech coding strategy in real time; in the embedded processor module, the speech processor is realized based on a digital signal processor chip, and the audio codec is realized based on an audio chip; the audio codec is mainly responsible for analog-digital/digital-analog conversion of signals, transmits data to the speech processor through the McASP interface, processes the data through an algorithm arranged in the speech processor, communicates with the debugging upper computer software module through the USB interface, receives an instruction from the debugging upper computer software module, and performs parameter setting and algorithm switching operation.
4. The intelligent voice-enhanced real-time electronic cochlear debugging system of claim 1, wherein: the connection circuit of the speech processor and the audio codec comprises: an audio chip U1, a digital signal processor chip U2; the 37 feet of the audio chip U1 are connected with the 43 feet of the digital signal processor chip U2, the 38 feet of the audio chip U1 are connected with the 37 feet of the digital signal processor chip U2, the 39 feet of the audio chip U1 are connected with the 33 feet of the digital signal processor chip U2, the 40 feet of the audio chip U1 are connected with the 26 feet of the digital signal processor chip U2, the 41 feet of the audio chip U1 are connected with the 24 feet of the digital signal processor chip U2, the 1 foot of the audio chip U1 is connected with the 9 feet of the digital signal processor chip U2, and the 2 feet of the audio chip U1 are connected with the 11 feet of the digital signal processor chip U2.
5. The intelligent voice-enhanced real-time electronic cochlear debugging system of claim 1, wherein: the embedded processor module also comprises a USB transmission module, the USB transmission module realizes parameter setting and algorithm switching instruction transmission between the debugging upper computer software module and the embedded processor module through a USB transmission circuit, and the USB transmission circuit comprises: the device comprises a current-limiting distribution switch U3, an ESD protection diode U4, a MINI USB interface CON1, a triode Q1, resistors R1-R4, capacitors C1 and C2; one end of the resistor R1 is connected to the USB0_ DRWBUSn pin of the dsp chip U2, the other end of the resistor R2, one end of the resistor R2, and the base of the transistor Q1 are connected to each other, the other end of the resistor R2 and the emitter of the transistor Q1 are connected to each other and then grounded, the collector of the transistor Q1, one end of the resistor R3, and 4 pins of the current-limiting distribution switch U3 are connected to each other, the other end of the resistor R3 is connected to a power supply, the 5 pin of the current-limiting distribution switch U3 is connected to the power supply, the 2 pin of the current-limiting distribution switch U3 is grounded, the 1 pin and 3 pin of the current-limiting distribution switch U3, one end of the resistor R4, one end of the capacitor C1, one end of the capacitor C2, the 5 pin of the ESD protection diode U4, the 1 pin of the MINI USB interface CON1 is connected to the power supply, and the other corner of the resistor 46r 45 is connected to the, the other ends of the capacitors C1 and C2 are connected to the ground after being connected, a pin 1 of the ESD protection diode U4 is connected to a pin 3 of the MINI USB interface CON1, a pin 2 of the ESD protection diode U4 and a pin 5 of the MINI USB interface CON1 are connected to the ground after being connected, a pin 3 of the ESD protection diode U4 is connected to a pin 4 of the MINI USB interface CON1, a pin 6 of the ESD protection diode U4 is connected to a pin 2 of the MINI USB interface CON1, and a pin 6, a pin 7, a pin 8 and a pin 9 of the MINI USB interface CON1 are connected to the ground after being connected to each other.
6. The intelligent voice-enhanced real-time electronic cochlear debugging system of claim 1, wherein: the sub-modules of the off-line mode in the debugging upper computer software module comprise: module selection, data import, database selection, scene simulation, algorithm selection and effect display;
the module selection module defines a main working mode of the offline mode, and the main working module comprises: the system comprises a voice enhancement algorithm debugging module, a speech coding strategy debugging module and a hardware communication module;
the voice enhancement algorithm debugging module supports a user to debug by using a preset algorithm and simultaneously provides a set of standard interface form to support the user to introduce a new algorithm for debugging; the speech coding strategy debugging module is preset with the existing speech coding strategy and simultaneously supports a user to download a new speech coding strategy to the module for debugging;
the debugging mode supported by the module selection module comprises the following steps: monotone and joint tone; the monotony refers to that the algorithm in the voice enhancement algorithm debugging module and the speech strategy in the speech coding strategy debugging module are respectively and independently debugged, and in the monotony mode, a quiet mode is selected in the scene simulation module when the speech coding strategy is independently debugged; the joint debugging refers to that an algorithm in the voice enhancement algorithm debugging module and a speech strategy in the speech coding strategy debugging module are simultaneously debugged, in the joint debugging mode, a mode with noise is selected in the scene simulation module when the speech coding strategy is debugged, and then the voice enhancement algorithm and the speech coding algorithm are simultaneously set;
the hardware communication module transmits the amplitude information and the rate information of the electrical cochlea stimulating current generated after the algorithm and the speech strategy are debugged in the voice enhancement algorithm debugging module and the speech coding strategy debugging module to the speech processor in a file form based on a USB protocol;
the data import module provides services for importing the existing voice enhancement algorithm and the speech coding strategy which meet the requirements of the standard interface form and the voice enhancement algorithm and the speech coding strategy designed by the user; the imported existing and self-defined voice enhancement algorithm and the language coding strategy are provided for the user to select in the algorithm selection module;
the database selection module stores preset voice packets for off-line debugging, wherein the voice packets comprise voice packets of different language types;
setting a specific debugging scene in the scene simulation module, and dividing an application scene of a voice enhancement algorithm into a strong noise mode, a conference mode and a daily mode according to different noise signal combination types received by a microphone; the strong noise mode only sets background noise, namely the target voice is only interfered by the background noise; the conference mode only sets competitive voice noise, namely, the target voice is influenced by preset competitive voice but has no influence of background noise; the daily mode simultaneously sets background noise and competitive voice, namely, the target voice is influenced by the competitive voice and is interfered by the background noise;
in the algorithm selection module, a user selects an algorithm to be tested, a speech coding strategy and corresponding parameter configuration during debugging, which are debugged in the voice enhancement algorithm debugging module and the speech coding strategy debugging module, according to the needs of the user; the specific setting content in the algorithm selection module comprises: the method comprises the following steps of voice enhancement algorithm, speech coding strategy, coding channel number, stimulation channel number, minimum electric stimulation intensity T value generated by auditory sense, maximum comfort threshold C value of a wearer, stimulation rate, pulse width and stimulation mode; after the algorithm selection module selects the content to be debugged, debugging is carried out based on the voice enhancement algorithm debugging module and the speech coding strategy debugging module;
the effect display module supports the performance of displaying the algorithm in different forms, and the display mode comprises the following steps: a spectrogram, a gray scale image, a time domain oscillogram, audio playing and performance index calculation; and the results of the debugging of the voice enhancement algorithm debugging module and the speech coding strategy debugging module are transmitted to the effect display module to be displayed in different forms from different angles.
7. The smart voice-enhanced real-time electronic cochlear debugging system of claim 6, wherein: the preset algorithm in the voice enhancement algorithm debugging module comprises: spectral subtraction, beam integration, masking matrix; the preset speech coding strategy in the speech coding strategy debugging module comprises the following steps: CIS policy, ACE policy, FFC policy, FAME policy, 9 degree encoding policy.
8. The intelligent voice-enhanced real-time electronic cochlear debugging system of claim 1, wherein: the online mode in the debugging upper computer software module comprises a data communication module, an algorithm switching module and a parameter setting module, and the data communication module supports the real-time debugging of the algorithm to be tested, the speech coding strategy and the electronic cochlea stimulation current amplitude information and speed information generated during the debugging in the offline mode to the embedded processor module in the portable debugging device based on the real-time voice environment in the portable debugging device through real-time voice data; the algorithm switching module and the parameter setting module transmit commands and parameters to the portable debugging device based on a USB protocol, switch algorithms and speech coding strategies which need to be debugged in the portable debugging device, and set debugging parameters.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010024316.XA CN111050262B (en) | 2020-01-10 | 2020-01-10 | Intelligent voice-enhanced real-time electronic cochlea debugging system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010024316.XA CN111050262B (en) | 2020-01-10 | 2020-01-10 | Intelligent voice-enhanced real-time electronic cochlea debugging system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111050262A true CN111050262A (en) | 2020-04-21 |
| CN111050262B CN111050262B (en) | 2021-04-13 |
Family
ID=70244295
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010024316.XA Active CN111050262B (en) | 2020-01-10 | 2020-01-10 | Intelligent voice-enhanced real-time electronic cochlea debugging system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111050262B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111768802A (en) * | 2020-09-03 | 2020-10-13 | 江苏爱谛科技研究院有限公司 | Artificial cochlea voice processing method and system |
| CN113663216A (en) * | 2021-08-12 | 2021-11-19 | 中国人民解放军总医院第一医学中心 | Artificial cochlea debugging auxiliary system and auxiliary debugging method |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101251819A (en) * | 2007-12-11 | 2008-08-27 | 浙江大学 | Debug method suitable for multi-processor core system chip |
| CN101310695A (en) * | 2007-05-21 | 2008-11-26 | 香港中文大学 | Artificial electronic cochlea based on mobile telephone and voice tone information |
| US20090079612A1 (en) * | 2007-09-21 | 2009-03-26 | Siemens Energy & Automation, Inc. | Systems and/or Devices for Providing an Isolated Analog Output or Analog Input |
| CN101744670A (en) * | 2008-12-16 | 2010-06-23 | 上海冠芯电子科技有限公司 | Artificial cochlear debugging programming method and system adopting wireless data transmission |
| CN102670331A (en) * | 2012-06-04 | 2012-09-19 | 清华大学 | Electronic cochlear in-vitro debugging platform based on local area network (LAN) |
| EP2942009A1 (en) * | 2013-04-27 | 2015-11-11 | Jiangsu Betterlife Medical Co., Ltd. | Device for hearing diagnosis and treatment |
| CN108665904A (en) * | 2017-03-31 | 2018-10-16 | 上海北京大学微电子研究院 | A kind of external speech signal processing system of cochlear implant and cochlear implant |
| CN109036569A (en) * | 2018-09-17 | 2018-12-18 | 重庆大学 | The verification method of the time domain fine structure novel coding of artificial cochlea's tone language |
| US20180361143A1 (en) * | 2014-06-18 | 2018-12-20 | Northwestern University | Systems and methods for neuromodulation device coding with trans-species libraries |
| WO2019076382A1 (en) * | 2017-10-20 | 2019-04-25 | 捷开通讯(深圳)有限公司 | Mobile terminal-based hac system and implementation method thereof |
| CN110111769A (en) * | 2019-04-28 | 2019-08-09 | 深圳信息职业技术学院 | A kind of cochlear implant control method, device, readable storage medium storing program for executing and cochlear implant |
-
2020
- 2020-01-10 CN CN202010024316.XA patent/CN111050262B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101310695A (en) * | 2007-05-21 | 2008-11-26 | 香港中文大学 | Artificial electronic cochlea based on mobile telephone and voice tone information |
| US20090079612A1 (en) * | 2007-09-21 | 2009-03-26 | Siemens Energy & Automation, Inc. | Systems and/or Devices for Providing an Isolated Analog Output or Analog Input |
| CN101251819A (en) * | 2007-12-11 | 2008-08-27 | 浙江大学 | Debug method suitable for multi-processor core system chip |
| CN101744670A (en) * | 2008-12-16 | 2010-06-23 | 上海冠芯电子科技有限公司 | Artificial cochlear debugging programming method and system adopting wireless data transmission |
| CN102670331A (en) * | 2012-06-04 | 2012-09-19 | 清华大学 | Electronic cochlear in-vitro debugging platform based on local area network (LAN) |
| EP2942009A1 (en) * | 2013-04-27 | 2015-11-11 | Jiangsu Betterlife Medical Co., Ltd. | Device for hearing diagnosis and treatment |
| US20180361143A1 (en) * | 2014-06-18 | 2018-12-20 | Northwestern University | Systems and methods for neuromodulation device coding with trans-species libraries |
| CN108665904A (en) * | 2017-03-31 | 2018-10-16 | 上海北京大学微电子研究院 | A kind of external speech signal processing system of cochlear implant and cochlear implant |
| WO2019076382A1 (en) * | 2017-10-20 | 2019-04-25 | 捷开通讯(深圳)有限公司 | Mobile terminal-based hac system and implementation method thereof |
| CN109036569A (en) * | 2018-09-17 | 2018-12-18 | 重庆大学 | The verification method of the time domain fine structure novel coding of artificial cochlea's tone language |
| CN110111769A (en) * | 2019-04-28 | 2019-08-09 | 深圳信息职业技术学院 | A kind of cochlear implant control method, device, readable storage medium storing program for executing and cochlear implant |
Non-Patent Citations (3)
| Title |
|---|
| S. PARRIS, M. TORLAK AND N. KEHTARNAVAZ: "Real-time implementation of cochlear implant speech processing pipeline on smartphones", 《2014 36TH ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY》 * |
| 刘托，关添，宫琴: "无线便携式人工耳蜗调试系统的设计", 《北京生物医学工程》 * |
| 蒋涛，宫琴: "基于嵌入式脉冲控制方式的电子耳蜗调试平台的体内系统的研发", 《仪器仪表学报》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111768802A (en) * | 2020-09-03 | 2020-10-13 | 江苏爱谛科技研究院有限公司 | Artificial cochlea voice processing method and system |
| WO2022048041A1 (en) * | 2020-09-03 | 2022-03-10 | 江苏爱谛科技研究院有限公司 | Voice processing method and system for cochlear implants |
| CN113663216A (en) * | 2021-08-12 | 2021-11-19 | 中国人民解放军总医院第一医学中心 | Artificial cochlea debugging auxiliary system and auxiliary debugging method |
| CN113663216B (en) * | 2021-08-12 | 2023-12-01 | 中国人民解放军总医院第一医学中心 | Artificial cochlea debugging auxiliary system and auxiliary debugging method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111050262B (en) | 2021-04-13 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Yip et al. | A fully-implantable cochlear implant SoC with piezoelectric middle-ear sensor and arbitrary waveform neural stimulation | |
| US20180326212A1 (en) | Low power cochlear implants | |
| US7937155B1 (en) | Envelope-based amplitude mapping for cochlear implant stimulus | |
| CN111050262B (en) | Intelligent voice-enhanced real-time electronic cochlea debugging system | |
| US20040082980A1 (en) | Programmable neurostimulator | |
| BR112021016503A2 (en) | IMPLANTABLE COCHLEAR SYSTEM WITH INTEGRATED COMPONENTS AND CONDUCTOR CABLE SPECIFICATION | |
| CN102670331B (en) | Electronic cochlear in-vitro debugging platform based on local area network (LAN) | |
| EP0555278A1 (en) | Bimodal speech processor | |
| US11979715B2 (en) | Multiple sound source encoding in hearing prostheses | |
| CN105263450A (en) | Single channel cochlear implant artifact attenuation in late auditory evoked potentials | |
| CN104783928A (en) | Performance evaluation system for testing language strategy of electronic cochlea | |
| Ghosh et al. | Portable smart-space research interface to predetermine environment acoustics for cochlear implant and hearing aid users with CCi-MOBILE | |
| Yip et al. | 18.2 A fully-implantable cochlear implant SoC with piezoelectric middle-ear sensor and energy-efficient stimulation in 0.18 μm HVCMOS | |
| CN106075721B (en) | Impedometer | |
| Kaiser et al. | Using a personal computer to perform real-time signal processing in cochlear implant research | |
| AU2009101375A4 (en) | Spectral tilt optimization for cochlear implant patients | |
| CN108814583A (en) | A kind of multi-functional physiological signal generator based on smart phone | |
| Lobo et al. | A PDA-based research platform for cochlear implants | |
| Karrenbauer et al. | A High-Performance, Low Power Research Hearing Aid featuring a High-Level Programmable Custom 22nm FDSOI SoC | |
| Seligman | Speech-processing strategies and their implementation | |
| Tan et al. | Speech-enabled pure tone audiometer | |
| Kuczapski et al. | Real-time interfacing for fault detection and auralization with MED-EL cochlear implant processors | |
| CN218298396U (en) | Artificial cochlea simulation rapid integration test system | |
| Yousheng et al. | Simulation System of Cochlear Implant | |
| US11463829B2 (en) | Apparatus and method of processing audio signals |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |