TW200304119A - Voice activity detection (VAD) devices and methods for use with noise suppression systems - Google Patents

Abstract

Voice Activity Detection devices, systems and methods are described for use with signal processing systems to denoise acoustic signals. Components of a signal processing system and/or VAD system receive acoustic signals and voice activity signals. Control signals are automatically generated from data of the voice activity signals. Components of the signal processing system and/or VAD system use the control signals to automatically select a denoising method appropriate to data of frequency subbands of the acoustic signals. The selected denoising method is applied to the acoustic signals to generate denoised acoustic signals.

Description

200304119 发明, Description of the invention ^ 丽 g ^ Tg 明 Technical field, prior art, content, implementation and drawings are briefly explained) This application claims priority from the following US patent applications, including: March 5, 2002 Filed an application with application number 60 / 362,162 for "PATHFINDER-BASED VOICE ACTIVITY DETECTION (PVAD) USED WITH PATHFINDER NOISE" using path finding noise suppression "; March 5, 2002 Filed an application with application number 60 / 362,170 "ACCELEROMETER-BASED VOICE ACTIVITY DETECTION (PVAD) WITH PATHFINDER NOISE) using path finding noise suppression; filed on March 5, 2002 , Application No. 60 / 361,981 "ARRAY-BASED VOICE ACTIVITY DETECTION (AVAD) AND PATHFINDER NOISE); Application was made on March 5, 2002, and the application number is 60 / 362,161 "PATHFINDER NOISE USING AN EXTERNAL VOICE ACTIVITY DETECTION (VAD) DEVICE)"; March 5, 2002 Issued application, application number 60 / 362,103 "Accelerometer-Based Voice Activity Detection" and application filed on March 27, 2002, application number 60 / 368,343 " "Dual microphone type frequency-based voice activity detection." The above applications are currently under review. In addition, this application is related to the following U.S. patent applications, including an application filed on July 12, 2001, Application No. 09 / 905,361: Method and Apparalxis for Removing Noise from Electronic Sign "; Application was filed on May 30, 2002, with application number 10 / 159,770," DETECTING VOICED AND UNVOICED SPEECH USING BOTH ACOUSTIC AND NONACOUSTIC SENSORS " , And an application filed on November 21, 2002, with Application No. 10 / 301,237, “Method and Apparatus for Removing Noise from (Method and Apparatus for Removing Noise from □ Continued on the next page) Please note and use the continuation page) 200304119 Description of the Invention Continuation page of Lectronic Signals) "-[Technical Field to which the Invention belongs] The embodiments disclosed in the present invention are used for detecting and processing in the presence of sound noise The system and method of the desired signal are related. C Previous technology] Over the years, the industry has developed many noise suppression algorithms and technologies. The noise suppression systems used in sound communication systems are based on the single microphone spectral subtraction technique developed in the 1970s. For example, S. F. Boll in IEEE Trans, on ASSP, ρρ 113-120, 1979, "Suppression of Acoustic noise in Speech using Spectral Subtraction" published by these technologies. These technologies have been improved for many years, but the basic operating principles remain unchanged. For example, U.S. Patent No. 5,687,243, the inventor is McLaughlin, etc., and U.S. Patent No. 4,811,404, the inventor is Vilmui, etc. are examples. Generally, these technologies are voice activity detection using a single microphone Voice Activity Detector (VAD) is used to determine the specificity of background noise. Voice—words contain human voices, or mixed human voices and unvoiced voices. VAD has been applied to digital honeycomb systems ( digital cellular system), such as US Patent No. 6,453,291, and the inventor is Ashley, which describes how VAD is configured in the digital Nested front end system. In addition, some Code Division Multiple Access (CDMA) systems use VAD to minimize the effective radio frequency spectrum (radio spectrum) used, so that the system capacity can be increased. Similarly, VAD can be added to the Global System for Mobile Communications (GSM) to reduce co-channel interference and reduce the battery power consumption of the client or payer device. This type of typical single-microphone VAD system uses a single microphone to receive sound information and analyze it using typical signal processing techniques. Therefore, its functions are greatly limited by the analysis results. In particular, this type of mono] (Please note and use the continuation sheet when using it.) 200304119 Description of the invention _ Page Microphone VAD system has been pointed out, in low-noise ratio (SNR) signals' and background noise changes quickly in the environment, performance is not good, so This kind of single-microphone VAD application will have similar restrictions on noise suppression systems. [Content] The following will describe a variety of voice activity detection (VAD) devices and methods for adaptive noise suppression systems. In addition, the following VAD devices and methods used as noise suppression system components will also be used, especially according to the path-finding noise suppression provided by the California San Francisco Elif Corporation (website at http://www.aliph.com) The experimental results obtained by the system (Pathfinder Noise Suppression System) will be described, but the embodiment of the present invention is not limited thereto. In the following description, if there is a case of a path finding noise suppression system, it should be noted that the noise waveform can be estimated and subtracted from the signal, and VAD information can be used at the same time to provide reliable operation. The noise suppression system is also included in the reference example. Here is to provide a practical example to describe the system that can process the vocal signals and noises to be extracted, so the path finding system is used as a reference. The VAD device and method used in conjunction with the noise suppression system described here, in which the VAD signal is processed independently of the noise suppression system, so the actions of receiving and processing VAD information are also independent of the related process of noise suppression. The embodiments of the invention are not limited thereto. This independence is through physical 'that is, different hardware is used to receive and process VAD-related signals and perform noise suppression; and through processing methods, that is, the same hardware is used to receive signals to noise. Suppression systems are only achieved by using independent technologies (software, algorithms, routines); and through a combination of different hardware and software. In the following description, acoustic (acoustic) is generally defined as sound waves transmitted through space. Sound waves transmitted by media other than air will be pointed out separately. Voice or voice—generally, human voices that include human voices, unvoiced voices, and / or a combination of the two are distinguished when needed. The term "noise suppression" usually refers to the electronic signal □ continuation page (when the description page of the invention is insufficient, please note and use the continuation page) 8 Description of the invention _ stomach 200304119 is used to reduce or eliminate noise Any method. In addition, VAD — words are usually defined as vectors, or array signals, data, or information presented in some way, in the digital or analogue category of speech. A general-purpose VAD information is represented by a one-bit digital signal, and the sampling rate is the same as the corresponding audio signal ', where the zero number 値 represents within the corresponding sampling time. No speech is generated, and the unity of unity indicates that speech was generated at the corresponding sampling time. Although the embodiments described below are generally in the digital domain ', the same description can be applied to the analogy category. The VAD devices / methods described herein generally include, but are not limited to, vibration and motion sensors, sound sensors, and manual VAD devices. In one embodiment, the accelerometer is placed on the human skin, and | is used to detect the vibration of the skin surface related to human voice. The recorded vibration is used to calculate the VAD signal. This VAD. Signal can be used to provide adaptive noise. The signal suppression algorithm is used to suppress the ambient sound noise recorded with the sound signal (within a few microseconds ms), and the sound signal contains sound and noise. Another embodiment of the VAD device / method disclosed herein includes a modified microphone with a membrane (membrane), which prevents the microphone from effectively detecting sound vibrations in the air, but this layer of membrane can detect Vibration generated by contacting objects, such as human skin (good mechanical impedance matching). In other words, the sound microphone is modified so that it cannot detect sound vibrations in the air (that is, it cannot achieve good physical impedance matching), but can only detect vibrations of objects and bodies in contact with it. Doing so makes the microphone and Accelerometer-like, it can detect the vibration generated by the human body, but it will not detect the ambient sound noise in the air. After the detected vibration is processed, it will generate a VAD signal for use. The noise suppression system will be described in detail below. The VAD disclosed in another embodiment uses an electromagnetic vibration sensor, such as a radio frequency vibrometer (RFvibrometer) for detecting skin vibration. In addition, the radio frequency vibrometer can detect Measure the internal tissues of the body such as the movement of the cheeks or the inner surface of the trachea. Both the epidermis and the internal tissue vibrations related to sound generation can be used to generate VAD signals for use in noise suppression systems, which will be described in detail below.] Continued (When the description page of the invention is insufficient, please note and use the continuation page) 200304119 Invention description_ The VAD disclosed in another embodiment includes a vocal cord vibration measuring device (electrglottograPh, EGG), which can directly detect the vocal cord (vocai fold) activity, EGG is a method using alternating current, which can be used to measure the vocal fold area. When EGG detects a sufficient amount of vocal fold activity, it is assumed that a vocal action is in progress, and a corresponding VAD signal will be generated. To represent someone, and the VAD signal can be used in noise suppression systems, which will be described in detail below. Similarly, there is an additional VAD embodiment to use The signal system detects the movement of a person's vocal organs, and it does show that sound is generated. Another VAD device / method described below uses one or more sound microphones to receive signals, and uses corresponding signal processing technologies. In most cases, VAD signals are accurately and reliably generated in the presence of environmental noise. Such embodiments include a simple array and a combination of omnidirectional and single-directional sound microphones placed together. The simplest of these VAD embodiments The configuration is to use a single microphone, placed near the speaker's mouth, in order to record the signal at a relatively high signal-to-noise ratio. This microphone can be a gradient microphone, or a microphone for close-range radio reception. Other configuration methods include the use of a single pointing Omnidirectional and omnidirectional microphones to achieve various directions and configurations. The signals received by such microphones and related signal processing tasks can be used to calculate VAD signals for use in the noise suppression system described below. Similarly, the following will describe the VAD system initiated by humans, such as simple Transceiver (walkie-talkie), or a system started by a system observer. As mentioned earlier, the VAD devices and methods disclosed here are used in noise suppression systems, such as Airy by San Francisco, California Pathfinder Noise Suppression System by Aliph ^ i ^ T * # iS ^ M (Pathfinder system) ° VAD m settings can be found in the path finding noise suppression system, familiar with this The skilled artisan should understand that such VAD devices and methods can be used in various conventional noise suppression systems and methods. The path finding system is a digital signal processing (DSP) -based sound noise suppression and echo-cancellation system. The path-finding system can be connected to the front end of the speech processing system, use VAD information and received sound information to estimate the noise waveform, and subtract it from the signal containing speech and noise. When the page is inadequate, please note and use the continuation page.) 119 Description of the Invention Continuation pages are used to reduce or eliminate noise in the sound signal. The following detailed description and related applications will further describe the fun-seeking system. [Methods] FIG. 1 is a block diagram of a signal processing system 100 according to an embodiment, which includes a path finding noise suppression system 铳 101 and a VAD system 102. The signal processing system 100 includes two microphones, MIC 1 110 and MIC 2 112, for receiving signals or information from at least one human voice signal source 120 and at least one noise source 122. The path s⑻ from the human voice signal source 120 to the MIC 1 and the path n (n) from the noise source 122 to the MIC 2 are considered to be unity. In addition, ft⑵ represents a path from the noise source 122 to the MIC 1, and H < z) substitutes a path from the human voice signal source 120 to the MIC 2. For comparison with the signal processing system 100 including the path finding system 101, FIG. 2 is a block diagram of the signal processing system 200, which includes a conventional adaptive noise cancellation system 202. The components of the signal processing system 100, such as the noise suppression system 101, are connected to the microphones MIC 1 and MIC 2 through a combination of wireless, wired, and wireless and wired connections. Similarly, the VAD system 102 and the noise suppression system 101 are connected to the components of the signal processing system 100 through a wireless link, a wired link, and / or a combination of wireless and wired links. For example, the components of the VAD system 102, the VAD device and the microphone, can communicate using the Bluetooth wireless specification and other components of the signal processing system, but this is not a limitation. Please refer to FIG. 1. The VAD signal 104 generated by the VAD system 102 controls the action of eliminating signal noise regardless of the noise type, amplitude, and / or direction of the signal received by the system. When the VAD signal 104 shows that no one is speaking, the path finding system 101 uses the MIC 1 and MIC 2 signals to calculate a model of the transfer function 下 in the pre-specified subband specified in the received signal. Coefficient. When the VAD signal 104 shows that someone is speaking, the path finding system 101 stops updating H «z and starts to calculate the pre-specified subband D pre-specified in the received signal. (Please note and use the continuation page when using it.) Description of the Invention Continuation page can continue to update the coefficient of the transfer function H2 (z) under this pair of 200304119. If 畐! The signal-to-noise ratio (SNR) of the 1 band is low, and the ft coefficient of the band is low. In the embodiment, the path-finding system uses the Least Mean Square Root (LMS) technology. For this technology, please refer to the book "Adaptive Signal Processing" by B. Widrow and S. Steams. Published by Hall Corporation, ISBN book number 0-13-004029-0, but the present invention is not limited thereto. The transfer function can be calculated based on the time domain, frequency domain, or a time / frequency domain combination. The path-finding system 101 then uses the combination of the transfer function 乩 ⑵ and ft (z). The noise in the desired sound signal is removed, so at least one noise signal stream is generated after noise removal. The path finding system can be accomplished in many different ways, but in these embodiments all rely on accurate and reliable VAD devices and / or methods. The reason why the VAD device / method must be accurate is because the path-finding system needs to update its filter coefficients when there is no human voice. If someone's vocal energy appears during the process of updating the coefficients, the subsequent vocals will be suppressed if they have similar spectral characteristics, which is not a good thing. VAD devices / methods need powerful functions to be able to achieve high accuracy under different environmental conditions. 'Obviously there are no VAD devices / methods that can provide satisfactory results in some situations' but under normal circumstances, VAD devices / methods It should be able to provide the largest noise suppression effect, while not having too much negative effects on human voice. When a VAD device / method is used in a noise suppression system, the VAD signal is processed independently of the noise suppression system, so the process of receiving and processing the VAD signal is independent of the noise suppression processing process. However, the embodiment of the present invention does not Not in this limit. This independence is through physical, that is, different hardware is used to receive and process VAD-related signals and perform noise suppression; through processing, that is, the same hardware is used to receive signals to the noise suppression system , But using independent technologies (software, algorithms, routines); and using a combination of different hardware and software to achieve 'as described below. Fig. 1A is a block diagram of a VAD system 102A according to an embodiment, which includes hardware for receiving and processing VAD-related signals. The VAD system 102A includes a VAD device 130 connected to the corresponding VAD continuation page (when the description page of the invention is insufficient, please note and use the continuation page) 200304119 ^ --- description page of the invention Algorithm 140 for providing information. Please note that in other embodiments, those skilled in the art can use various known methods to allow the noise suppression system to integrate some or all of the VAD algorithms and noise suppression processing functions. FIG. 1B is a block diagram of a VAD system 102B in another embodiment, which uses a related noise suppression system 101 to receive VAD information 164. The VAD system 102B includes a VAD algorithm 150 that receives data 164 from MIC 1 and MIC 2 or other components in the signal processing system 100. In other embodiments, those skilled in the art can use various known methods to let the noise suppression system integrate some or all of the VAD algorithms and noise suppression processing functions. VAD devices / methods that use vibration / action as the cattle VAD devices that use vibration / action as the main component include physical hardware devices to receive signals related to processing and VAD and noise suppression. When a talker or user makes a human voice, the generated vibration is transmitted through the talker's body tissue, so various ways can be used to detect the vibration on or under the skin. This type of vibration can be a good source of VAD information, because they have a close relationship with the sounds of people uttering and no sound (however, the vibration is very weak and difficult to detect when there is no sound), and it is generally affected by environmental sounds. The effect of environmental noise is not large (some or no effect on some devices / methods), for example, an electromagnetic vibration sensor as described below. These tissue vibrations or movements can use several VAD devices, including accelerometer-based devices, skin surface microphone (SSM) devices, and electromagnetic vibrometers (EM vibrometers) including radio frequency (RF) vibrometers and laser vibrometers. ), Direct vocal cord motion measurement device and video detection device. Accelerometer-based VAD device / method The accelerometer can detect skin vibrations related to human voice. For example, in Figure 1 and Figure 1A, the VAD system 102A 5 allows an embodiment to include The VAD device 130 is used to provide the skin vibration [□ Continued pages ('Notes and use of continuation pages when the Instruction Sheet is not enough) 13 200304119 Inventive Instructions Continuation Page' to the relevant algorithm 140. The algorithm used in the embodiment uses energy calculation technology and compares thresholds, which will be described below, but not limited to this. It should be noted that those skilled in the art can use more complex energy calculation methods. FIG. 3 is a flowchart 300 of an embodiment, a method for determining whether or not a human voice is produced using an accelerometer-based VAD. Generally, the energy is defined by the standard window size, and then the sum of the squared amplitudes over a period of time is calculated as follows:

Energy = J] x12, i where i is the label of the digital sample and starts at the beginning of the range and ends at the end. 0 Referring to FIG. 3, the accelerometer data is initially received in block 302. The VAD-related processing at block 304 includes filtering the data from the accelerometer to eliminate aliasing in advance, and digitizing and filtering the data. In block 306, the data is divided into intervals of 20 microseconds (ms), and the data is presented in steps of δ microseconds. The divided data in the interval is further processed in block 308 to remove spectrum information or other unwanted information that is disturbed by noise. In block 310, the sum of the squares of the calculated amplitudes is used to calculate the energy in each interval. The calculated energy 値 can be divided by the interval length for normalization, but this involves additional calculations. The length does not change, so there is no need to deal with it. The calculated or normalized energy chirp is compared with a threshold value at block 312. In Block 314, when the energy of the accelerometer data is equal to or higher than the critical threshold, the sound corresponding to the accelerometer data is marked as someone speaking. Similarly, in block 316, when the energy of the accelerometer data is equal to or lower than the critical threshold, the sound corresponding to the accelerometer data is marked as unvoiced. In other embodiments, the noise suppression system may use multiple critical thresholds to indicate the relative strength and accuracy of the human voice signal, but not limited to this. In addition, you can also use multiple sub-band processing methods to improve accuracy. Figure 4 shows an output chart in an embodiment, which is a noise sound signal (live recording) 402 plus a corresponding accelerometer-based VAD signal 404, a corresponding accelerometer output signal 412, and The path system uses the VAD signal 404 to eliminate noise from the audio signal 422. In this example, when the 500 to [] continuation page is inadequate (please note and use the continuation page when the invention description page is insufficient) 14 200304119-invention description, the accelerometer data between 2500 Hz on the continuation page is bandpassed ( bandpass) to remove sound noise below 500Hz that would couple to the accelerator. The audio signal 402 was recorded in a 6-foot-long, 8-foot ceiling room using Aliph's microphone set and a standard accelerometer. There was noisy noise in the environment. The path finding system uses a real-time processing method with a delay of about 10ms. It can be seen from the difference between the original sound signal 402 and the noise-removed sound signal 422 that the noise suppression is about 25 to 30 dB. The distortion of the human voice signal is not large, therefore, providing VAD information with an accelerometer can effectively eliminate noise. The VAD device / method of the skin surface (SSM) is bovine. Back to FIG. 1 and FIG. 1A, in one embodiment, the VAD system 102A includes an SSM VAD device 130, which is used to provide information to the relevant algorithm 140. SSM is a modification of the traditional microphone, which can prevent the sound information transmitted through the air from running into the microphone's detection element. The microphone has a layer of silicone gel or other attached substances that can change the impedance of the microphone to avoid air-borne sound. The message was clearly detected. Therefore, the microphone will not be affected by the energy transmitted by the air, but it can still detect the sound waves transmitted through the medium other than air, as long as it is always in contact with the medium. In order to effectively detect the sound energy on the human skin, the impedance of the silicone (gd) will match the mechanical impedance characteristics of the skin. When speaking, the SSM is placed on the cheek or neck, so the SSM can easily detect the vibration caused by the sound, and the SSM cannot easily detect the sound data from the air. After the sound signal transmitted by the organization is detected by the SSM, it is used to generate a VAD signal, which is used to process and eliminate the noise of the sound signal. As mentioned above, the VAD signal and map generated by the accelerometer can be referred to; 3 〇 ' Figure 5 shows an example of a noisy sound signal (live recording) 502 plus a corresponding SSM-based VAD signal 504, a corresponding SSM output signal 512, and a path finding system using a VAD signal 504 to eliminate the noise of the sound signal 522 graph. In this example, the accelerometer data between 500 and 2500 Hz is bandpass filtered to remove acoustic noise below 500 Hz that would couple to the accelerator. The audio signal 502 uses Aliph's microphone, which is installed in a room 6 feet long and the ceiling height is 8] Continued pages (when the description page of the invention is insufficient, please note and use the continued page) 15 200304119-Description of the invention _ page feet Recording in the room, there is noisy noise in the environment. The path-finding system uses a real-time processing method with a delay of about 10ms. It can be seen from the difference between the original sound signal 502 and the noise-removed sound signal 522 that the noise suppression is about 25 to 30 dB, and the human voice signal The distortion is small, so VAD information provided by SSM can effectively eliminate noise. VAD devices / methods based on thunder (EM) vibrometers Back to FIG. 1 and FIG. 1A, in one embodiment the VAD system 102A includes a VAD device 130 of an electromagnetic vibrometer (EM vibrometer) for providing information The relevant algorithm is 140. The electromagnetic vibration meter can also detect tissue vibration, but can detect tissue vibration at a distance without touching the tissue. In addition, the electromagnetic vibration meter can detect the internal tissue of the human body. vibration. The electromagnetic vibrometer is immune to sound noise _ and is therefore suitable for use in highly noisy environments. In this embodiment, the path finding system receives information from an electromagnetic vibrometer. The electromagnetic vibrometer includes an RF vibrometer and a laser vibrometer, but it is not limited to this, which will be described one by one below. The operating range of the RF vibrometer is between the radio frequency and the microwave of the electromagnetic spectrum. It can measure the relative movement of the internal tissues of the human body during vocalization. The internal human tissues include trachea, cheeks, jaw, and / or nose / nasal cavity, but not limited to this. . The RF vibrometer can use low-power radio waves to detect actions, and the data provided by these devices match the adjusted target. Since the RF vibrometer does not detect sound noise, the path finding in the embodiment is The system uses the signals provided by these devices to construct the VAD by referring to the accelerometer-based VAD and the energy / critical threshold method shown in Figure 3. The vocal cord electromagnetic motion sensing (GEMS) radio vibrometer (radiovibrometer) introduced by Aliph Corporation of San Francisco, California, is an RF vibrometer. For other RF vibrometers, please refer to relevant applications and the University of California, Davis Gregory C. Burnett's doctoral dissertation published in January 1999: `` The Physiological Basis of Glottal Electromagnetic Micropower Sensors (The Physiological Basis of Glottal Electromagnetic Microsensors) GEMS) and 〇 Continued pages (If the description page of the invention is insufficient, please note and use the continued page) 200304119-Π Page of the description of the invention

Their Use in Defining an Excitation Function for the Human Vocal Tract). " The operating range of the laser vibrometer is close to visible light or visible light, so it can only detect surface vibrations, similar to the accelerometer or SSM described above. The laser vibrometer and the RF vibrometer will not detect sound noise, so the path finding system in the embodiment uses the signals provided by these devices, refer to the VAD based on the above-mentioned elocimeter and the energy of FIG. 3 / Critical Martingale method for constructing VAD. Figure 6 shows a diagram in an embodiment, with a noisy sound signal (live recording) 602 plus a corresponding GEMS VAD signal 604, a corresponding GEMS output signal 612, and a path finding system using VAD Signal 604 to eliminate noise from sound signal 622. Provided by Aliph Company of San Francisco, California, the GEMS radio vibrometer mounted on the trachea emits a VAD signal 604, and the sound signal 602_ is a 6-foot-long, 8-foot ceiling height installation using Alih's microphone Recording in the room, there is noisy noise in the environment. The path finding system uses a real-time processing method with a delay of about 10ms. As can be seen from the difference between the original sound signal 602 and the noise-removed sound signal 622, the noise suppression is about 20 to 25 decibels (dB). The distortion of the human voice signal is not large, so using VAD information based on GEMS can effectively eliminate noise. Obviously, even if GEMS does not detect unvoiced speech, the VAD signal and the action of eliminating noise are still quite effective. Since the unvoiced sound energy is quite low, it does not significantly affect the cricket. Convergence and vocal quality after noise removal. VAD device / method for direct vocal cord motion measurement Refer to FIG. 1 and FIG. 1A. In one embodiment, the VAD system 102A includes a direct glottal motion measurement VAD device 130 for providing data to related Algorithm 140. In an embodiment, the VAD device for direct vocal cord motion measurement of the path finding system includes a vocal cord vibration detector (EGG), and any device that can directly measure vocal cord movement or position. EGG uses two or more electrodes placed on the side of the thyroid cartilage to return the corresponding signal of the vocal cord contact area 'a small amount of AC current transmitted from one or more currents will pass through the neck tissue (including Vocal cord) to the neck □ continuation sheet (if the description sheet of the invention is insufficient, please note and use the continuation sheet) 17 200304119 invention description on other electrodes on the other side of the continuation sheet part, The amount of current flowing from one electrode to the other electrode will increase. If the vocal folds do not touch each other, the current will decrease. For example, the electromagnetic vibrometer and SSM are the same. The signal of EGG is not affected by noise. Therefore, in this embodiment, the path finding system receives information from the EGG, and refers to the above-mentioned accelerometer-based VAD and the energy / critical threshold method of FIG. 3 to construct the VAD. FIG. 7 shows an embodiment In the output picture, the recorded English voice male voice data 702 plus digitally added noise, the corresponding EDG-based VAD signal 704, and the corresponding high-pass filtered EGG output signal 712. If you compare the sound data 702 with the EGG output signal 712, it can be seen that EGG is quite accurate in detecting the sound of someone making a sound, but when the vocal folds don't touch each other, such as when no one speaks. Cannot be detected. However, in the experiment, the problem of unvoiced or very soft speech (both of which have low energy) cannot be detected, and it will not significantly affect the ability of the system to eliminate human noise in the general environment. For more information, see "A Critical Review of Electroglottography," published by DG Childers and AK Krishnamurthy in the journal CRC Crit Rev Biomedical Engineering in 1985, No. 12, pages 131-161. • VAD device / method for detecting video as cattle Refer to FIG. 1 and FIG. 1A. In one embodiment, the VAD system 102A includes a VAD device 130 for video detection, which is used to provide data to related algorithms 140. In the embodiment Video cameras and processing systems detect movements of vocal organs including the jaw, lips, teeth, and tongue. The video and computer systems currently under development support two-dimensional computer images, so VAD systems based on video detection can be provided. Construction tools for this type of system can be accessed at http: // wwwjnteLcom / research / mrl / research / opencv / query. In the embodiment, the path-finding system can use the components of the video system to detect the movement of the vocal organs, and continue to the next page (when the description page of the invention is insufficient, please note and use the continued page) 18 200304119 Invention Description Generate VAD information. FIG. 8 shows a flowchart 800 of a method for determining a voice of a person using a video-based VAD in one embodiment. In block 802, the components of the video system will find the user's face and sound generator, and in block 804, calculate the actions of the starting sound organ. At block 806, the components of the video system and / or the path-finding system determine whether the calculated vocal organ moves faster than the critical speed and is vibrating (the vocal organ moves back and forth, it can be discerned that it is not a simple accidental action). If the action is slower than the critical speed and / or not vibrating, and if the action is slower than the critical speed and / or not vibrating, return to the action of block 802. If in block 806, the action is faster than the critical speed and is vibrating, then in block 808, the components of the video system φ and / or the path finding system will determine whether the vocal organ action is greater than the critical rate. If the action is greater than the critical threshold, in block 810, the components of the video system and / or the path finding system determine that the target is sounding, and then in block 812, the information is passed to the path finding system. This video-based VAD device / method is not affected by acoustic noise, and can be used at a distance from the user or talker, so it is suitable for monitoring applications. Acoustic information-based VAD setup / method Refer to the description of Figures 1 and 1B above. When using VAD in a noise suppression system, the VAD signal is processed independently of the noise suppression system, so it is received and processed. The action of VAD information is also independent of related processing of noise suppression, but the embodiment of the present invention is not limited thereto. The independence of the VAD device mainly based on sound information is through the processing method, that is, it can use the same hardware to receive signals to noise. The suppression system, but uses independent technology (software, algorithms, or routines) Processes the received signal, but in some cases, the sound microphone is used to construct the VAD function, not to suppress noise. In the embodiment, the audio information-based VAD device / method needs to rely on one or more traditional voice microphones to detect the human voice to be received, so it is relatively susceptible to interference from ambient sound noise, and it cannot be used in all noise. Reliable operation in the environment, however, VAD mainly based on audio information is simple and convenient] Continued page (When the description page of the invention is insufficient, please note and use the continued page) 19 200304119-Description of the advantages And the same microphone can be used for VAD and receiving sound signals. Therefore, this type of VAD solution will be more popular in situations where the cost is more important than the performance of noise removal. In the embodiment, the voice information is mainly VAD devices / methods include single-microphone VAD, path-finding VAD, stereo VAD (SVAD), array VAD (AVAD), and other traditional single-microphone VAD devices / methods, but not limited to this, which will be described in detail below. Single microphone-based VAD device / method This may be the easiest way to detect whether the user is speaking. Please refer to FIG. 1 and FIG. 1B. In one embodiment, the VAD system 102B includes a VAD algorithm 150 for Receive signal processing from the corresponding signal. Data 164 from the microphone of the system 100, the microphone (usually a "close-talk" or gradient microphone) is relatively close to the user's mouth, sometimes directly touch Touch your lips. Gradient microphones are very insensitive to sounds more than a few centimeters away from the microphone (and the frequency range received is usually less than 1KΗζ), so they can be used to record signals with a fairly high SNR. Of course, the performance achieved by a single microphone and the distance from the user's mouth to the microphone, the harshness of environmental noise, and whether the user is willing to put things as close to the lips will have an impact. Since at least a part of the frequency spectrum of the data or signal recorded from a close-range single microphone has a relatively high SNR, in the embodiment, the path-finding system receives a signal from a single microphone, referring to the accelerometer-based VAD and The energy / critical threshold method of Figure 3 is used to construct the VAD. Figure 9 shows a diagram in an embodiment. Noisy sound signal 902 (live recording) plus-corresponding single (gradient) microphone VAD signal 904, corresponding gradient microphone output signal 912, and the path finding system uses VAD signal 904 to eliminate noise from sound signal 922. Provided by Aliph Company of San Francisco, California, the GEMS radio vibrometer installed on the trachea emits VAD signal 604, and the sound signal 902 uses Aliph's microphone set and accelerometer. Recording in the room, there is noisy noise in the environment. [Find] Continued pages (Note when the invention description page is inadequate, please note and use the continuation page) 20 200304119 Invention description The page diameter system uses a real-time processing method, with a delay of about 10ms' from the original sound signal 902 and noise reduction The difference between the sound signal 922 can be seen, the noise suppression is about 25 to 30 dB, the distortion of the human voice signal is not large. These results show that the use of single microphone-based VAD information can effectively eliminate noise. The VAD (PVAD) device / method based on the path-finding system returns to FIG. 1 and FIG. 1B again. In one embodiment, the PVAD system 102B includes a PVAD algorithm 150 to receive a microphone from the corresponding signal processing system 100. 164 from the array. The microphone φ array contains two microphones, but this is not a limitation. In this embodiment, the PVAD operates in the time domain, and the two microphones are a few centimeters apart. At least one of the microphones is a unidirectional microphone. FIG. 10 shows a single cardioid curve unidirectional microphone 1002 in an embodiment, and a unidirectional microphone 1002 related to the spatial response curve 1010 on the force port is also called Noise microphone 1002, or MIC 2, its direction is to allow the user's mouth to be within or near the null zone 1012 of the frequency response 1010 of the noise microphone 1002, but the system is not limited to using a heart shape Curved directional microphone. FIG. 11 shows a microphone array 1100 of a PVAD system according to an embodiment. The microphone array 1100 includes two cardioid curve single-directional microphones MIC 1002 and MIC 2 1102, and their space The response curves are 1010 and 1110, respectively. There are no restrictions on the type of microphone used as the vocal microphone MIC 1 in the microphone array 1 100 '. However, when the vocal microphone MIC 1 is a unidirectional microphone, and its direction is such that the user's mouth is at or near the frequency response curve 1 When the maximum time is 10, the best effect will be achieved. This will ensure that the signals from the two microphones are significantly different when someone is speaking. In one embodiment, the microphone configuration includes Μία and MIC 2, and these microphones are placed close to the human ear. This configuration is directed to the vocal microphone MIC 1 towards the user's mouth, and the noise microphone [I continued page (When the description page of the invention is not enough, please note and use the continuation page) 21 200304119 Description of the page continued MIC 2 is away from the user's head, so that the spatial response curve of each microphone is about 90 degrees from each other The angle difference is so that the noise microphone MIC 2 can fully capture the noise at the front end of the head, but at the same time will not capture too much signal. The other two embodiments are to adjust the pointing positions of the microphones 1102 and 1002 'so that the maximum 値 of the spatial response curve of each microphone is 75 degrees and 135 degrees apart, respectively. The configuration of these PVAD systems is to put the microphones together as much as possible to simplify the calculation of Hl (z), and then place the microphone as a human microphone. MIC 1 can detect most human voices, while the noise microphone MIC 2 can detect Most noise is detected (that is, H2 (z) is relatively small), and the angular difference between the maximum chirps of the spatial response curve of each microphone can be up to about 180 degrees, but should not be less than About 45 degrees. The PVAD system uses a path finding method, which calculates the differential path between the vocal microphone and the noise microphone (referred to as Hi in the path finding method, described below) to calculate the VAD. The path finding system does not use this information for noise suppression. Instead, it uses chirp gain to determine when to eliminate noise. Comparing the signal energy ratio of a vocal microphone with the signal energy ratio of a noise microphone ’, the VAD gain (hereinafter referred to as the gain) can be obtained by the following formula

Gain = | Hj (z) | =

Energy of speech mic Energy of noise mic

Where & is the i-th sample of the digitized signal of the vocal microphone, and y. Is the i-th sample of the digitized signal of the noise microphone. In this VAD application, there is no need to adjust the calculation of ft, although this example is It is described digitally, but it is also effective for analogy, and the gain can also be calculated in the time or frequency domain. In the frequency domain, the gain parameter is the sum of the square of the ft coefficient. As mentioned above, the length of the interval (calculation interval) does not appear in the calculation of energy, because it will be offset when calculating the energy ratio. Finally, this example It is used in a single sub-band, but can be applied to any number of sub-bands. Returning to FIG. 11, the gain of the spatial response curves 1010 and 1110 of the microphone array 1100 in the first hemisphere 1120 is greater than 1 and the gain in the second hemisphere 1130 is less than 1, but this is not a limitation. This feature □ Continued (please note and use the continuation page when the invention description page is insufficient) 22 200304119

Description of the invention The closeness of the MM and the vocal microphone MIC 1 to the user's mouth helps to distinguish human voice from noise. The microphone array 1100 in the PVAD embodiment enables the path-finding system to achieve the best performance. At the same time, two identical microphones can be used to process VAD and eliminate noise, so the system cost can be reduced. 'Providing additional benefits . However, in order to achieve the best VAD performance, the two microphones should be oriented in opposite directions in order to take advantage of the very large gain variation in this configuration. In another embodiment, the PVAD also includes a third single directional microphone MIC 3 (not shown), but is not intended to be limiting. The third microphone, MIC 3, is facing the opposite direction of MIC 1. It is only used for VAD purposes, while MIC 2 is only used for noise suppression, and MIC 1 is used for VAD and noise suppression. In this way, Lu will improve the overall system Performance, but at the cost of a microphone, and more than 50% more sound. Data. In one embodiment, the path finding system receives signals from PVAD, and refers to the above-mentioned accelerometer-based VAD and the energy / critical threshold method of FIG. 3 to construct a VAD. Because there may be considerable noise components in the microphone data, accelerometer-based energy / critical 値 VAD detection methods cannot be used in every case. Another VAD embodiment uses the past gain (when there is only noise) to detect whether someone is talking, which will be described below. FIG. 12 shows a flowchart 1200 of a method for determining the presence or absence of a voice by using gain 値 in another PVAD embodiment. At the beginning, the data of the system microphone was received in block 1202. The components of the PVAD system at block 1204 filter the data to eliminate aliasing in advance, and digitize the filtered data. In block 1206, the data is divided into intervals of 20 microseconds (ms), and the data is presented in steps (_) every 8 microseconds. In block 1208, further process the segmented data in the interval, remove the unnecessary spectrum information, calculate the standard deviation (SD) of the last 50 gains in the interval with only noise, and call it the vector OLD jTD, and calculate 〇 LD_STD average 値 (AVE). In block 1210, the AVE and SD 値 will be compared with the pre-specified minimum 値. If they are smaller than the minimum 値, they will be increased to the minimum 分别 respectively. In block 1212, the components of the PVAD system then add AVE to the multiple of SD to calculate someone ’s voice. Continued page (when the description page of the invention is insufficient, please note and use the continued page) 23 200304119 The critical point of the continued page of the invention description 値. The low critical value is AVE plus 1.5 times the SD, and the high critical value is AVE plus four times the SD. At block 1214, the squared amplitude can be added to obtain the energy in each interval. In addition, In block 1214, the ratio of the energy of MIC 1 to the energy of MIC 2 may be calculated to obtain the gain. A small cutoff value may be added to the amount of soap in MIC 2 to ensure stability, but this The embodiment is not limited thereto. In block 1216, comparing the obtained gain with the critical threshold, there are 3 possible results. When the gain is less than the low critical threshold, then it can be determined that there is no human voice in this interval, and the OLD jTD vector is updated to new If the gain is greater than the low critical 値 and lower than the high critical 値, it can be determined that there is no human voice in this interval, but it is suspected that someone is vocal, so the OLD jTD is not updated to the new gain 値; when the gain Above the low and high critical thresholds, it can be determined that someone is speaking in this interval, and 〇LD_STD is not updated to the new gain threshold. No matter how this method is implemented, its spirit is to use the gain of H <z) = M <z) / M2 (z) when using human voice to isolate background noise. According to the configuration of the microphone, the calculated gain when there is human voice should be relatively large, because the human voice received by the vocal microphone MIC 1 is much louder than the noise received by the noise microphone MIC 2; otherwise, due to noise The noise is likely to spread with the terrain, so the noise received by MIC 2 is usually louder than that of MIC 1. However, if an omnidirectional microphone is used as the vocal microphone, the above statement is not necessarily correct, and the ability of the system to control noise is also limited at this time. Please note that methods that use only sound to eliminate noise are more susceptible to environmental noise. However, some test results show that the above configuration with a single directional microphone and a single directional microphone can reach an SNR of MIC 1 slightly less than OdB, which is a satisfactory result. Therefore, this PVAD-based noise suppression system has a large Most users will be able to operate effectively in the noisy environment. Similarly, if necessary, the microphone can be moved closer to the user's mouth to improve the SNR of MIC 1. FIG. 13 is a chart in an embodiment, a noise sound signal 1302 (live recording) plus a corresponding single (gradient) microphone-based PVAD signal 1304, and a corresponding gradient microphone (gradient ϋcontinued on the next page) (When the description page of the invention is insufficient, please note and use the continuation page.) 24 200304119 Description of the page continued (microphone) Output signal 1312, and the path finding system uses the PVAD signal 1304 to eliminate the noise of the sound signal 1322. The sound signal 1302 is made using Aliph's microphone set and accelerometer. The device is recorded in a 6-foot-long room with a ceiling height of 8 feet. There is noisy noise in the environment. The path-finding system uses a real-time processing method with a delay of about 10 ms. From the difference between the original sound signal 1302 and the noise signal 1322 after noise cancellation, it can be seen that noise suppression is between 20 and 25 dB. The distortion of the vocal signal is small. These results show that the use of single microphone-based PVAD information can effectively eliminate noise. Stereo VAD (SVAD) device / method returns to FIG. 1 and FIG. 1B. In one embodiment, the SVAD system 102B includes an SVAD algorithm 150 ′ to receive frequency-based dual signals from the corresponding signal processing system 100. Information 164 from the microphone array. The theory of the SVAD algorithm is that in the frequency spectrum, the received human voice and noise can be distinguished. Therefore, the SVAD device / method includes a comparison of the average FFT (fast Fourier transform) between microphones. SVAD uses 2 microphones For directions, please refer to the PVAD and Figure 11 above, and also rely on the noise data in the previous section to determine whether the current section contains human voice. As described above for the PVAD device / method, the vocal microphone is called MIC 1 here. The noise microphone is called MIC 2. Please refer to Figure 1. This path finding noise suppression system uses dual microphones to capture the characteristics of the signal (MIC 1) and noise (MIC 2). Naturally, both microphones will receive mixed human voice and noise signals. However, in the embodiment, it is assumed that the SNR of MIC 1 is greater than that of MIC 2, which means that MIC 1 is closer to the signal source (user) or better in orientation than MIC 2, and the distance between any noise source and MIC 1 and MIC 2 is better than The signal is far away. However, the same effect can be achieved with omnidirectional and single-directional or similar microphones. The SNR difference between the two microphones can be used in the time or frequency domain. In order to separate the noise from the human voice, it is necessary to calculate the average spectrum of the noise over a period of time. We can use the following exponential function average method (exponential averaging method) to calculate: Two] Continued pages (When the description page of the invention is insufficient, please note and use the continued page) Invention description sequel 200304119 L (i, k) = aL (i -1, k) + (1 -a) S (i, k), where a controls the smoothness of the average, 0.999 means that the average is very smooth, and 0.9 is not very smooth. The variables L (i, k) and S (i, k) are the average and instantaneous variables, respectively, i represents the discrete-time samples, and k represents the frequency bin (frequency bin) 'The number 値 is determined by the length of the FFT The decision 'These numbers can also be calculated using the traditional average or moving average (movinSaveraSe) method. Fig. 14 shows a flowchart 1400 of an embodiment of a method for determining the presence or absence of sound by using stereo VAD (SVAD). In this example, you can refer to the description in Figure 1, which uses two microphones to record data at 8KHz, and has been carefully processed in advance to remove ghosting. The interval used is 20ms long ’each step _ level 8ms 0 · At first, two microphones were used to receive data in block 1402. The signal from the microphone is properly filtered to eliminate ghosting in advance, and the filtered data is digitized for processing. In addition, in block 1404, the previous 160 samples collected by MIC 1 and MIC 2 were allocated in each interval using the Hamming interval (window), and then in blocks 1406 and 1408, the components of the SVAD system calculated and allocated in the interval. FFT size of the data to obtain FFT1 and FFT2. In block 1410, the above exponential function averaging method is used, and the number α of α is specified to be 0.85. FFT1 and FFT2 are exponentially averaged to generate MF1 and MF2. At block 1412, the system uses MF1 and MF2 ® to calculate the ratio of MF1 to MF2 plus cutoff, and then calculates the average and marks it as VAD_det. The formula is as follows: 1 (MF1,

VAD det1 = —V — ^.-128 V ^ MF2ik + cutoff J where i represents the i-th interval and k is the frequency interval, and the cutoff 値 is used to reasonably adjust the ratio to prevent the amplitude of the MIC 2 frequency interval from being too small. The situation happened. Since the length of the FFT is 128, divide the result by 128 to get the average 値 of the ratio. In block 1414, the components of the path-finding system compare VAD_det with the voicing threshold 値 Vjhresh. After comparison, the following page (continued if the description page is insufficient, please note and use the continued page) If the number of values is lower than Vjhresh, the system component sets VAD_state to 0. If the number of VAD_det is higher than Vjhresh, then set VAD_state to 1. In block 1416, it is determined whether VADjtate is equal to 1. When VAD_state is equal to 1, in block 1417, the components of the path finding system will update the parameters and record the largest VAD_det in the counter that records the continuous voicing section. , And then enter the operation of block 1420. If an unvoiced section appears after a vocalized section, Bay (1 checks the maximum VAD_det recorded in the previous continuous vocal section (which can contain one or more sections), see The vocalization shows whether the result is wrong. If the largest VADjet in the segment is lower than a preset critical threshold (for example, the difference between the low critical threshold and the high and low critical threshold @ 40%), the vocalization state of the interval is set. It is -1, which can be used to remind the algorithm of eliminating noise. ”In the previous vocal segment, it is actually impossible for someone to make a sound, and the path finding system can also modify the calculation of the coefficient. _ At block 1416, when the SVAD system decides VAD_state is equal to 0. Then in block 1418, the components of the SVAD system will reset the parameter containing the maximum value of VAD_det. Similarly, if someone made a sound in the previous interval, the system Check whether the previously vocalized segment was misjudged. In block 1420, the components of the path-finding system will then update the high and low determinant levels to calculate the critical threshold 声 Vjhresh, and then return to the block. Operation of 1402. In this embodiment, the low and high decision levels are both calculated using an exponential average method, where the number φ 値 is higher or lower than the low and high decision levels according to the current VAD_det, as described below. For low decision levels, if the number of VAD_det is higher than the preset low decision level, then the number of α is set to 0.999, otherwise it is set to 0.9. For high decision levels, a similar method is used, only However, when the 値 of VAD_det. Is lower than the current high decision level, α is set to 0.999, and when the 値 of VAD_det is higher than the current high decision level, α is set to 0.9. In other embodiments These levels can be determined using traditional average or moving average methods. In one embodiment, the critical threshold is generally set to a low determination level plus a 15% difference between the high and low determination levels, and will Absolute minimum critical value, but this embodiment is not limited to it. Absolute minimum critical value] Continued page (When the description page of the invention is insufficient, please note and use the continued page) 27 200304119 Invention description β stomach should not be in VAD It is set arbitrarily in a quiet environment. --- In other embodiments, in the case of using SVAD to determine whether there is a vocalization, different parameters can be used, including the interval size, FFT size, cutoff number 値, and α number 値Etc. to compare the average FFT 値 between microphones. As long as the SNR difference between the microphones is large enough, the SVAD device / method can be used to handle any noise. The SNR of the two microphones is definitely not more important than the relative SNR. Therefore, if you configure the microphones, The relative SNR difference becomes larger, and generally a better VAD performance can be obtained. The SVAD device / method has been successfully used in many different microphone configurations, noise types, and noise levels. Take Figure 15 as an example, which shows a diagram in an embodiment, with a noisy sound signal-1502 (live recording) plus a corresponding single (gradient) microphone SVAD signal 1504, and _ seek The path system uses the SVAD signal 1504 to eliminate noise from the audio signal 1522. The sound signal 1502 is a microphone set using Aliph. The device is recorded in a 6-foot long room with a ceiling height of 8 feet. There is noisy noise in the environment. The path finding system uses a real-time processing method with a delay of about 10ms. From the difference between the original sound signal 1502 and the noise signal 1522, it can be seen that the noise suppression is about 25 to 30 dB. The distortion of the vocal signal obtained by the SVAD signal 1504 is not great. • With the VAD (AVAD) device / method, return to FIG. 1 and FIG. 1B. In one embodiment, the AVAD system 102B includes an AVAD algorithm 150 for receiving the information transmitted by the microphone array of the corresponding signal processing system 100.来 的 资料 164. AVAD-based systems include an array of 2 or more microphones to distinguish the user's voice from ambient noise, but not limited to this. In one embodiment, the two microphones are at a predetermined distance from each other, so that a sound source in a particular direction can be emphasized, for example, on the axis connecting the microphones, or at the midpoint of the line. In another embodiment, 'beamforming or source tracking can be used to find the signals to be taken out within the field of view of the array, and to construct relevant adaptive noise suppression—I continued page (Explanation of the invention When the page is insufficient, please note and use the continuation page) 28 200304119-Description of the invention Continuation page system, such as the VAD signal used by the path finding system. Those familiar with this technique should refer to the 2001 book "Microphone Arrays" by M. Brandstein and D. Ward under the ISBN 3_540-41953_5 for the practice of other embodiments. In one embodiment, the AVAD includes an array of dual microphones. A unidirectional microphone from Panasonic is used. The unidirectionality of the microphone allows the array to detect sound sources placed in front of or in front of the array. However, it is not necessary to use a unidirectional microphone. When the array is placed, only one end of the array will emit sound, for example, placed on a wall. In this case, there is no need to use a unidirectional microphone. The linear distance between the two microphones is about 30.5 cm. The low-noise amplifier can amplify the data from the microphone. It was recorded on a personal computer (PC) using Labview 5.0 software of National Instruments, but This is not intended to limit the embodiment. The components of the AVAD system can use this array to record microphone data at 12 bit and 32KHz, and then use digital filtering to down-convert the data to 16KHz. In other embodiments, a much lower resolution can be used (maybe 8 Bit) and sampling frequency (down to a few KHz), plus sufficient pre-analog filtering, because the facsimile of audio data is usually not the focus of consideration. When the target signal source (human talker) is about 30 cm away from the center line of the microphone array, this configuration allows the MIC 1 and MIC 2 to record the sound of the target signal source. The delay between the two is zero. For other signal sources, there is a non-zero delay. In other embodiments, several different configurations can be used. The number of delays varies, and each delay defines an active area where the target signal source can be placed. area). -In this experiment, two speakers were used to provide noise, one of which was about 50 cm from the right of the microphone array, and the second speaker was located about 150 cm from the right behind the speaker. The noise of street and truck noise is 2 to 5dB. In addition, some recordings do not include noise, which is to facilitate the adjustment of the relationship. FIG. 16 shows a flowchart of a method for determining the presence or absence of utterance by using AVAD in one embodiment] Continued page (when the invention description page is insufficient, please note and use the continuation page) 29 200304119-Invention Description_Page 1600 . At the beginning, two microphones were used to receive data in block 1602. At block 1604, the signal from the microphone is properly filtered to remove ghosting in advance, and the filtered data is digitized for processing. In block 1606, the digitized data is allocated in intervals of 20ms in length, and the data is 8ms per step. In block 1608, the data allocated in the interval is further filtered in order to remove the spectrum information interfered by noise or other unwanted signals. In block 1610, the interval data from MIC 1 is added to the data from MIC 2, and the result is then squared, as shown in the following formula: M12 = (M! + M2) 2. The action of summing the microphone data will emphasize the result. The data component of zerodday in the data will have an additive effect on microphone data with the same phase, and an offsetting effect on data with inconsistent phase; because the target signal source is in-phase at all frequencies (in phase), so that adds up to constructive benefits, and the phase relationship of the noise source changes with frequency, so there is usually a destructive offset. Then, after the obtained signal is squared, the zero-delay signal component in the signal will be greatly aggravated. The calculated signal can use a simple energy / critical algorithm to detect whether someone is making a sound (please refer to the above accelerometers for details) VAD and the description of Figure 3), and at this time the zero delay signal component has actually been increased. Next, in block 1612, the squares of the amplitudes are added to calculate the energy of the resulting vector. In block 1614, calculate the standard deviation (SD) of the last 50 noise-only intervals, LD_STD, and its average AVE. In block 1616, the numbers AVE and SD are compared with the pre-specified minimum number 如果. If it is less than the minimum 値, J [J increases to the minimum 分别 respectively. Then in block 1618, the components of the path-finding system add AVE to a multiple of SD to calculate the critical threshold 出发. The low criticality is AVE plus 1.5 times SD, and the high criticality is AVE plus 4 times SD. In block 1620, comparing the obtained gain and criticality, there are 3 possible results. When the gain is less than the low critical threshold, then it can be determined that there is no human voice in this interval, and the 0LD_STD vector is updated to [□ Continued page (if the description page of the invention is insufficient, please note and use the continued page) 30 200304119 发明 说明 _ Page new gain 値; if the gain is greater than low critical 値 and lower than high critical 同时, you can decide that there is no human voice in this interval, but it is suspected that someone is vocal, so LD_STD is not updated to the new gain 値; When the gain is greater than the low and high thresholds, it can be determined that someone is speaking in this interval, and OLD_STD is not updated to the new gains. FIG. 17 is a graph showing sound signals 1710 and 1720 from different microphones in the AVAD system, and corresponding VAD signals 1712 and 1722 in an embodiment. The figure also shows the signal 1730 obtained by adding the audio signals 1710 and 1720 together. The speaker is placed about 30 cm from the center line of the microphone array. The noise used is truck noise, and the SNR of both microphones is less than OdB. VAD signals 1712 and 1722 can be used as input signals for path finding systems or other noise suppression systems. Conventional single-microphone VAD device / method In a system in which the noise suppression system of an embodiment uses dual microphones, the signal of one microphone is used to generate VAD information, but this is not a limitation. FIG. 18 is a block diagram of a signal processing system 1800 including a path finding noise suppression system 101 and a single microphone VAD system 102B in an embodiment. System 1800 includes a main microphone, MIC 1, or a vocal microphone, and a reference microphone, MIC 2, or a noise microphone. The main microphone MIC 1 provides signals to the VAD system 102B and the path finding system 101. The reference microphone MIC 2 provides a signal to the path finding system 101. Therefore, the signal from the main microphone MIC 1 provides vocal and noise data to the path finding system 101, and provides data to the VAD system 102B for generating VAD information. The VAD system i02B contains a VAD algorithm, as described in the inventions of U.S. Patent Nos. 4,811,404 and 5,687,243, for calculating the VAD signal, and the obtained data 104 is provided to the path finding system 101, but this The embodiment is not limited thereto, and the signal received through the reference microphone MIC 2 is only used for noise suppression. FIG. 19 is a flowchart of a method for generating audio information by using a single microphone in an embodiment. Continued page (When the description page of the invention is insufficient, please note and use the continued page) 31 200304119-Description of the invention continued page 1900. Initially, the signal was received with the main microphone in block 1902. In block 1904, the processing work related to VAD includes filtering the signal received by the main microphone to eliminate ghosts in advance, digitizing the filtered data, and processing at an appropriate sampling frequency (generally 8KHz). In block 1906, the digitized data was segmented and filtered to meet traditional VAD processing needs. In block 1908, the VAD information is calculated using the VAD algorithm, and then the VAD information is provided to the path finding system in block 1910 to eliminate noise. VAD settings / methods derived from airflow VAD devices / methods based on airflow use the airflow sent from the user's mouth and / or nasal cavity to construct VAD signals. Users can use various known methods to measure airflow. Separate airflow from breathing and gross motion flow in order to obtain the correct VAD information. Because breathing and action airflow are mostly composed of low-frequency (less than 100Hz) energy, high-pass filtering can be used to filter airflow data and separate airflow from breathing and action airflow. The equipment that can be used to measure airflow can refer to the Pneumotach Masks of Glottal Enterprise. For further information, please refer to http://www.glottal.com ° VAD devices / methods based on airflow are unlikely Affected by sound noise, because it is necessary to be very close to the mouth and nose when detecting airflow, you can refer to the above-mentioned accelerometer-based VAD and the energy / critical calculation algorithm in Figure 3 to detect the presence of human voice and Construct a VAD signal. In other embodiments of airflow-based VAD devices and / or related noise suppression systems, those skilled in the art may use other energy-based methods to generate VAD signals. FIG. 20 shows a flowchart 200 of a method for determining the presence or absence of a vocalization using a VAD based on airflow in an embodiment. Initially in block 2002, airflow data was received. In block 2004, VAD-related processing includes filtering airflow data to eliminate ghosting in advance, and digitizing the filtered data. In block 2006, the digitized data was allocated in a 20ms interval, and the data was 8ms per step. Next in block 2008, the process includes filtering the data allocated in each section to remove it.] Continued pages (when the invention description page is insufficient, please note and use the continuation page) 32 200304119

Description of the Invention The artificial sounds produced by MM's low frequency activities and breathing, among other things, contain other unwanted spectral information. In block 2010, the sum of the squared amplitudes is calculated to calculate the energy in each interval. In block 2012, the calculated energy 値 is then compared with the critical ，. In block 2014, when the inter-regional energy is equal to or higher than the critical 値, then the section corresponding to the airflow data is designated as someone speaking. In block 2016, a voiced data message was transmitted to the path finding system as VAD information. In other embodiments of the noise suppression system, multiple critical thresholds can be used to indicate the relative strength and accuracy of the human voice signal, but not here limit. Manual VAD device / method · The manual VAD device in one embodiment includes a VAD device that can be manually activated by a user or an observer, such as a VAD device activated by a button or a switching device, activating such a manual VAD device, or manually forcing modification automation VAD device can generate VAD signal. FIG. 21 shows the noise signal 2102 with corresponding manual start / calculated VAD signal 2104 and the path finding system using the manual VAD signal 2104 to eliminate the noise of the sound signal 2122 in one embodiment. chart. The sound signal 2102 is a microphone set using Aliph. The device is recorded in a 6-foot-long room with a ceiling of 8 feet. There is noisy noise in the environment. The path-finding system uses immediate processing, with a delay of about 10ms. It can be clearly seen from the difference between the original sound signal 2102 and the noise-removed sound signal 2122 that the noise suppression is about 25 to 30. Between dB, the distortion of the human voice signal is not large, so it is effective to use manual VAD information to eliminate noise. Those skilled in this art should understand that the electronic system used to process mixed sound information and noise in the signal can take advantage of the above VAD device / method to achieve its effectiveness. For example, if the earphone or headphone has one of the above VAD devices, it can be connected to a handset, such as a mobile phone, through a wired and / or wireless connection. Corresponding VAD algorithm. In particular, 'for example, a headset plugged in the ear or a head may include the above-mentioned skin surface microphone] Continued page (If the description page is not enough, please note and use the continuation page) 33 Invention description Continued 200304119 ( SSM), which transmits signals wirelessly to the device's path finding VAD on the handheld unit. The path finding noise suppression system is as described above. Figure 1 is a block diagram of the signal processing system 100 in one embodiment, including Path finding noise suppression system 101 and VAD system 102. The signal processing system 100 includes two microphones, MIC 1 110 and MIC 2 112, for receiving signals or information from at least one human voice signal source 120 and at least one noise source 122. The path s (n) from the human voice signal source 120 to the MIC 1 and the path η⑻ from the noise source 122 to the MIC 2 are considered to be unity. In addition, Η (ζ) represents a path from the noise source 122 to the MIC 1, and H2 (z) represents a path from the human voice signal source 120 to the MIC 2. The VAD signal 104 derived in some way is a method for controlling noise removal. The sound information received by MIC 1 is represented as n ^ n), and the sound information received by MIC 2 is represented as m2 (n ), In the field of z (digital frequency), we can represent them as Mi⑵ and M2⑵, so: MJz ^ S ^ + nWhJz) (1) M2 (z) = N (z) + S (z) H2 (z ) This situation can be applied to all practical dual microphone systems. MIC 1 will always receive some noise, and some signals will also go to MIC 2. There are 4 unknown terms in equation 但是, but there are only two relations. Obviously The answer cannot be solved directly. However, there may be some way to solve some unknown terms in Equation ⑴. Take the case where no signal is generated as an example, that is, VAD shows that no one is speaking. In this case, equation (1) can be further expressed as:

Mln = M2n (z) = N (z) where n under the M variable means that only noise is received and calculated. Continued page (If the description page of the invention is insufficient, please note and use the continued page) 34 200304119

Mln (z) = M2n (2) ^ (2)

Mln (z) M2n (z) Description of the invention_page (2) In the presence of noise only, we can use any available system identification algorithms and the output of the microphone to calculate Ηι (ζ), This calculation should be done in an adaptive way so that the system can track any changes in noise. After finding an unknown term in equation (1), you can use VAD to find out the number of cases where someone utters sound but not much noise. When VAD shows someone to utter, but recently (about 1 second) Or similar) The microphone record shows a low noise level. At this time, it can be assumed that n (s) = N⑵ is approximately 0, so equation (1) can become:

Mls (z) = S (z) M2s (z) = S (z) H2 (z) After calculation M2s (z) = M1s (z) H2 (z) H2 (z) = M2s (z)

Mls (z) Although the equation for calculating ft⑵ looks like the equation is reversed when calculating 迚 ⑵, remember that the input signals used are different. It should be noted that H2 相当 should be quite fixed, because there will always be only A single signal source (user), and the relative position of the user and the microphone should also be fairly fixed. 'If a small amount of adaptive gain is added when calculating the pressure, the calculation can be made more efficient' and the noise It can also play a powerful role in the existing conditions. After calculating Η! ⑵ and H2 (z), then use them to remove noise from the signal. 'Rewrite equation ⑴ as · S (z) = M1 (z) -N (z) H1 (z) N (z) = M2 (z) -S (z) H2 (z) S (z) = Mj (z)-[Μ2 (z)-S (z) H2 (z) ^! (Z) S (z ) [l-H2 (z ^ H, (z)] = M, (z)-M2 (z) □ Next page (when the description page of the invention is insufficient, please note and use the continued page) 35 200304119 Description of the invention $ Selling M can calculate s⑵ (2) i-h2 (z) Hi (z) Generally speaking, H2⑵ is quite small 'and Hi⑵ is less than 1, so for most frequencies in most cases: H2 (z) H1 (z) «1, and the signal can be obtained using the following equation: So it is assumed here that B⑵ is not calculated, and 乩 ⑵ is the only transfer function that needs to be calculated. Although we can still find 値 for B (z), but if the microphone If the position and direction of the signal are well placed, the need to calculate h2 (z) can be omitted. When processing sound signals, multiple sub-bands can be used to achieve significant noise suppression. This is because of the adaptive filter used to calculate the transfer function (adaptive filter) is mostly FIR type, this type of filter uses only zero (zero) instead of pole (count) A system containing zeros and poles is as follows: Η (ζ) _ &gt; Β (ζ) iUl〇DELS &gt; Α (:) This model can be accurate when there is sufficient input information, but this can also be significant Improve computing cost and convergence time. In energy-based adaptive filtering systems, such as the minimum root mean square (LMS) system, the system can be calculated to be fairly close to a small frequency range containing more energy. Amplitude and phase, which allows the LMS system to perform its maximum function, minimizing the energy of error, but this result may increase the noise outside the matched frequency range and reduce the noise. Signal suppression. The effectiveness of the system. The use of subbands can solve this problem. The signals from the primary and secondary microphones are filtered into multiple subbands, and the data calculated for each subband (possibly a shift Frequency and down-converted data, depending on user needs) are sent to individual adaptive filters, so the adaptive filter must find a way to match in its own sub-band Data, not just finding where in the signal the D continuation page (when the invention description page is insufficient, please note and use the continuation page) 36 200304119 The invention description has the highest energy, and the signal suppression results obtained in each sub-band Can sum up to get the signal after the last noise is eliminated. It is not easy to make each signal match the time and compensate the filter drift, but the result is better for the system, but only in the memory and processing The demand on the device is high. At first glance, the path finding algorithm seems to be very close to other algorithms, such as traditional ANC (Adaptive Noise Reduction), as shown in Figure 2. However, after careful comparison, we can see that there are differences in noise suppression effects in several places, including the use of VAD information to control the adjustment of the received signal by the noise suppression system, and the use of multiple sub-bands to ensure that the target spectrum To achieve proper convergence, and the reference microphone of the support system to operate the target sound signal, the following will be explained one by one. Regarding the use of VAD information to control the adjustment of the received signal by the noise suppression system, traditional ANC does not use VAD information, because in the process of someone speaking, the reference microphone will receive the signal, so adjust a ( z) (the path from noise to the main microphone) will cause the vocal energy of the target signal to be largely removed, with the result that the signal is distorted and eliminated (de-signalized). Therefore, in the various methods described above, the VAD information is used to construct a fairly accurate VAD in order to inform the path finding system when to adjust &amp; (contains only noise) and H2 (when someone speaks, it can be used if necessary). coefficient. As mentioned above, a significant difference between the path finding system and the conventional ANC is the action of cutting sound data into sub-bands. The path-finding system uses many sub-bands, and the LMS algorithm is used to calculate information in individual sub-bands. Therefore, it can ensure that the data in all sub-bands can be properly converged after being summed up, allowing the path-finding system to play a good role in the target spectrum efficacy. Since the ANC algorithm usually uses the LMS adaptive filter to build the 迚 model, and this model uses all the zeros to construct the filter, an "actual" usable system is unlikely to use this model to achieve accurate results . Almost all available systems contain poles and zeros, and therefore are quite different from the frequency response calculated by the LMS filter. Generally, LMS can only match the phase and amplitude of a real system at a single frequency, and the results obtained outside this frequency are poor, and noise in other regions will also increase. Therefore, using the LMS algorithm for the entire spectrum of the target signal often results in] Continued pages (when the description page of the invention is inadequate, please note and use the continued page) 37 200304119 _ Description of the continued page The target signal does not match in amplitude / phase Signal degradation occurs in the best place. Finally, the reference microphone of the path finding system support system operates the target sound signal so that the reference microphone can receive the sound signal, which means that the positions of the microphones can be closer, which is different from the traditional ANC microphone configuration. Closer distances simplify adaptive filtering calculations and make microphone configuration / solution easier. At the same time, there are special microphone configurations that can minimize signal distortion and de-signalization, and can establish a signal path model between the target signal source and the reference microphone. In one embodiment, the use of a directional microphone can ensure that the transfer function will not approach 1. However, even with a directional microphone, the noise microphone will still receive some signals. If you ignore it, and false _ fixed ft (z) = 0, then assuming VAD is ideal, there will be some distortion. We can refer to the * equation ⑵ and find the following equations without H2 (z): S (z) [l-H2 (z) H1 (z)] = M, (z )-M2 (z) ^ (z). (4) This means that the signal will have a distortion of [1-H2 (z) ft⑵] factor, so the type and amount of distortion will be affected by the noise environment. In the environment with very little noise, H <z is about 0, and the distortion is not large. When there is noise, the amount of distortion will be changed by the type, location and intensity of the noise source. Good microphone configuration The design minimizes this type of distortion. When VAD shows that no one is speaking, or someone is talking but the SNR of the sub-band is quite low, we can calculate ft. Similarly, when VAD shows that someone is talking and the SNR of the sub-band is quite high, H2 can be calculated. However, with proper microphone placement and processing, the problem of signal distortion can be minimized, and we only need to calculate the 値 of B. This can greatly reduce the processing required and simplify the construction of the path finding algorithm. The traditional ANC method is to not allow any signal to go to MIC 2. The path-finding algorithm can allow the MIC 2 to receive the signal under the condition of proper microphone and microphone configuration. An example of a suitable microphone configuration is shown in Fig. 11, where two cardioid curve unidirectional microphones MIC 1 and MIC 2 are used. This configuration allows the MIC 1 to point at the user's mouth. In addition, this configuration allows the MIC 2 to be as close to the MIC 1 as possible, and makes the MIC 2 pointing 90 degrees away from the MIC 1. 〇 Continued pages (If the description page of the invention is insufficient, please note and use the continuation page) 38 200304119 --- Π The description page of the invention may prove that the best way to suppress noise related to VAD is in the case of VAD errors. , Check the effect of VAD error on noise elimination action. Two types of errors that can occur include false positives (FP), that is, when no vocalization occurs, VAD shows that someone is talking; and false negatives (FN), that is, VAD is not detected Someone made a noise. If the FP occurs too often, it will become very troublesome. This is because the accidental FP will only temporarily stop the system from updating the coefficient of IL, so it will not significantly affect the performance of the noise suppression system. From another perspective See, FN can cause problems, especially when undetected human voices have high SNR. Assume that the microphone in the system has received human voice and noise, but VAD fails to detect the signal, and _ returns an FN. All systems only detect noise, then the signal received by MIC 2 is :-M2 = h1n + h2s, where z is not marked for clarity. Because VAD shows that only noise exists, the system will try to represent the above system as a single noise and a single noise according to the following equation. Model of the transfer function: TF model = 1¾ ^. Path finding systems use LMS algorithms to calculate 氐, but LMS algorithms are generally best used to build time-invariant (all-zero) systems On the model. Because noise and human voice signals are unlikely to be related, in general, the system will build the ability of the 乩 and H2 ® models based on the SNR of the data received by MIC 1, and also the non-temporal change of 乩 to create the human voice. Models with related transfer functions, or models with noise and related transfer functions, are described below. Regarding the SNR of MIC 1, the very low SNR (smaller than zero) tends to converge the path-finding system to a noise transfer function. In contrast, the high SNR (smaller than (0)) tends to converge the path-finding system. Is a vocal transfer function. As for the ability to build the model with H2, if it is easier to build ft or fast with LMS (all-zero model), the path-finding system tends to build such individual transfer functions. The model of the system is related to the non-time-varying nature of the pressure, and the LMS system performs best in building the non-time-varying system. However, empirically, if the time-varying system, the system change is more convergent than the LMS. Continued The next page (please note and use the continuation page when the description page of the invention is insufficient) 39 200304119 The description page is slow, so you can also use the LMS system to build a time-varying system model. Therefore, the path-finding system usually converges to a, because a usually changes at a slower rate than To. If the LMS chooses to use the noise transfer function to establish a vocal transfer function, then the vocal will be classified as noise, and as long as the coefficient of the LMS filter remains constant or similar, the vocal will be removed. Therefore, when the path-finding system converges to the ft model of the vocal transfer function (which can occur within a few ms), any subsequent vocals (even if the vocals are detected under normal VAD function), will Its transfer function is similar to the transfer function model established when VAD fails, so that energy is removed. In this case, the H2 model is mainly established, so noise is not affected, or only a part is removed. The final result of this process is that the volume becomes smaller, and the original clean human voice will be distorted. The degree of distortion is related to the variables discussed above. If the system tends to converge, the subsequent gain loss and vocal distortion will not be too obvious. If the system tends to converge to H2, then there may be severe distortion. VAD error analysis is not used to describe the details of using the sub-band, or the position, type, and orientation of the microphone. It is mainly to convey the importance of VAD for eliminating noise. The results obtained above can be used in a single sub-band or arbitrary The number of sub-bands, because the action in each sub-band is actually the same. In addition, in the above VAD error analysis description, the dependence on VAD and the problems caused by VAD errors are not limited to the path finding noise suppression system. Any adaptive filter noise suppression using VAD to determine how to remove noise The system is also affected. The content disclosed here is a reference path suppression system, but remember that all noise suppression systems that can use multiple microphones to estimate the noise pattern and remove the noise from the signal including human voice and noise, There are also clutter suppression systems that rely on VAD to provide reliable operation, which are all within the scope of the present invention. The path finding system is just an actual system that is referenced for convenience. Aspects of the invention can be programmed into functions in various circuits, including programmable logic devices (PLDs), such as field programmable gate arrays (FPGAs), programmable gate array logic (PAL) devices, electronic programmable logic, and Memory devices and standard cell devices, others also include application-specific integrated circuits (ASIQ 〇 Other devices that can meet certain aspects of the present invention include: Microcontroller with memory [□ Continued page (Invention page When not enough, please note and use the continuation sheet) 200304119 Description of the invention Continuation sheet (microcontroller), such as electronic erasable programmable stomach (EEPRQM), processor, firmware, software, etc. in case). If at least one step of some aspects of the invention is implemented in software during the manufacturing process, such as embedded in firmware or placed in PLD, then the software should be able to use any computer-readable medium, such as magnetic or Optically readable disc (fixed or software), using carrier signal modulation or other transmission methods. In addition, various aspects of the present invention can also be implemented using a microprocessor to simulate software circuits, discrete logic (sequential or combined), custom devices, fuzzy (neural) logic, quantum devices, and a mixture of the various device types described above. Of course, the technology used in these devices can also be used in various components, such as metal-oxide-semiconductor field-effect transistor (MOSFET) technology, such as complementary metal-oxide-semiconductor; bipolar technology such as emitter-coupled logic (ECL); polymer technology such as Silicon-conjugated polymer and metal-polymer-metal structure; mixed analog and digital circuits. Unless specifically mentioned, the terms "comprise, comprising" and other similar terms used in the detailed description and the scope of the patent application are used to include possible forms, not to exclude other possibilities, that is, It should be "include, not limited to" usage. The use of the singular or plural nouns also includes the plural and singular forms. In addition, "herein", "hereunder", "above", "below" or other similar narrative terms are applied in this patent application, and this case should be referred to The whole rather than a specific narrative. If "or (or)" is used in this patent application to refer to 2 or more items, it means that there are the following possibilities, any item in the list, all items in the list, and various items in the list The combination. The above description of the embodiments of the present invention is not intended to be exhaustive or to limit the present invention within the disclosed scope. Various specific embodiments of the present invention and the examples given are provided to provide explanations. Those skilled in the art will understand that Various equivalent modifications can be made in the present invention, and the description of the invention herein can be applied to other processing systems and communication systems, and is not necessarily limited to the above-mentioned processing systems. The elements and actions mentioned in each of the above embodiments can be combined to derive further implementations.] Continued pages (when the invention description page is insufficient, please note and use the continuation page) 200304119-invention description sequel, Various changes can be made to the present invention based on the above detailed description. --- The present invention refers to the above description and related US patent applications for reference. If necessary, various aspects of the present invention can also be modified in order to apply the systems, functions, and ideas proposed in the above patents and applications, to provide A further embodiment. In general, the terms used in the scope of the subsequent patent applications should not be used to infer the invention to be limited to the specific embodiments disclosed in the detailed description and the scope of the patent application, but should be used to infer that all A system operating under the scope of the patent application to provide the purpose of compressing and decompressing data files or streams. Therefore, the present invention is not limited to what is disclosed, and the scope of the present invention should be determined entirely by the scope of patent application. Although certain aspects of the present invention have been described by the following patent applications, the inventors have considered that the present invention can be expressed in the form of any number of patent applications, for example, although only one aspect of the invention Mentioned the use of computer-readable media, but other aspects can also be implemented using computer-readable media. Therefore, the inventor reserves the right to increase the scope of additional patent applications, so that after filing a patent application, It also filed additional patent applications. [Brief description of the drawings] FIG. 1 is a block diagram of a signal processing system including a path finding noise suppression system and a VAD system in an embodiment. FIG. 1A illustrates a VAD system according to an embodiment, which includes hardware for receiving and processing VAD-related signals. FIG. 1B illustrates a VAD system in another embodiment, which uses the hardware of the related noise suppression system to receive VAD information. FIG. 2 is a block diagram of a signal processing system in which a conventional conventional adaptive noise cancellation system is applied. □ Continued pages (note the insufficient pages of the invention, please note and use the continuation pages) 42 200304119 Continued pages of the invention Fig. 3 is an embodiment of the method of using the accelerometer-based VAD to determine the presence or absence of vocal pronunciation flow chart. Figure 4 shows the noise signal (live recording) plus the corresponding accelerometer-based VAD signal, the corresponding accelerometer output signal, and the path finding system using the VAD signal to eliminate Chart of noise of sound signal. Figure 5 shows an example of a noisy sound signal (live recording) plus the corresponding SSM VAD signal, the corresponding SSM output signal, and the path finding system uses the VAD signal to eliminate the sound signal. Noise chart. Figure 6 shows an example of a noisy sound signal (live recording) plus a corresponding GEMS VAD signal, a corresponding GEMS output signal, and a path finding system using the VAD signal to eliminate the sound signal Noise chart. FIG. 7 shows the recorded human voice data plus digitally added noise in one embodiment, the corresponding EDG-based VAD signal, and the corresponding high-pass filtered EGG output signal. FIG. 8 is a flowchart 800 of a method for determining a voice of a person using a video-based VAD in an embodiment. FIG. 9 shows a noise signal (live recording) plus a corresponding single (gradient) microphone-based VAD signal, and a corresponding gradient microphone output signal. The path system uses the VAD signal to eliminate noise from the sound signal. FIG. 10 is a diagram showing an array of a single cardioid curve and a single directional microphone, plus related spatial response curves in an embodiment. FIG. 11 shows a microphone array of a PVAD system in an embodiment. Fig. 12 is a flowchart of a method for determining whether or not a person makes a sound by using gain 値 in another PVAD embodiment. Continue &amp; Stomach (Note if the Instruction Sheet is inadequate, please note and use the Continue Sheet) 43 200304119-Description of the Invention Continued Figure 13 shows the noise signal (live recording) with noise in one embodiment The corresponding microphone is the main PVAD signal, the corresponding PVAD gain is relative to the time signal, and there is a graph of the path finding system using the PVAD signal to eliminate the noise of the sound signal. Fig. 14 is a flow chart showing a method for determining the presence or absence of sound by using a stereo VAD in one embodiment. FIG. 15 is a diagram showing noise signals (live recording) with corresponding SVAD signals and a path finding system using SVAD signals to eliminate noise from the sound signals in an embodiment. FIG. 16 is a flowchart of a method for determining the presence or absence of utterance using AVAD in an embodiment. FIG. 17 is a diagram showing a corresponding combined energy signal on the sound signal port of each microphone of the AVAD system in an embodiment. FIG. 18 is a block diagram of a signal processing system including a path finding noise suppression system and a single microphone VAD system in an embodiment. FIG. 19 is a flowchart of a method for generating voice information using a single microphone in an embodiment. Fig. 20 is a flowchart showing a method for determining the presence or absence of utterance using a VAD dominated by airflow in one embodiment. FIG. 21 is a diagram showing a noise signal with noise and a corresponding manually activated / calculated VAD signal, and a path finding system using a manual VAD signal to eliminate noise of the sound signal in one embodiment. In the drawings, the same reference numbers indicate the same or substantially similar elements or actions. In order to facilitate the discussion and description of any specific element or action, the most obvious number or reference number refers to the figure number where the element first appears, for example, the element 104 first appears in Figure 1 and discuss. [Explanation of component symbols] 100 signal processing system 1402 receives the signal □ Continued page (when the description page of the invention is insufficient, please note and use the continued page) 44 200304119-Description of the invention continued page 102VAD system 104VAD signal 110 microphone MIC 1 112 microphone MIC 2 120 voice signal source 122 noise source 102 AVAD system 130VAD device 140VAD algorithm 102BVAD system 150VAD algorithm 164 VAD information 302 receive accelerometer data 304 filter accelerometer data and digitization 306 segmentation and stepwise digitization data 30δ removed by The spectral information of noise interference 310 is calculated. The energy in each interval is calculated. 312 compares the energy 値 with the critical value. 314 is higher than the critical value. The energy is shown as someone ’s voice. 310 The energy below the critical value is shown as unvoiced. 802 The vocal organ 804 calculates whether the motion of the starting vocal organ 806 is faster than the critical speed and is vibrating? Is the 808 action greater than the critical threshold? 810 The target is sounding 1404 Allocating samples of MIC 1 and MIC2 1406 Calculating FFT1 and FFT2 1408 Calculating the size of FFT1 and KFT2 1410 Calculating the exponential average of FFT1 and FFT2 1412 Calculating the ratio (FFT_mtio) and its average 値 1414 of the previous step Average 値 and critical 値, set the VAD state 1416 to determine whether VAD_state is equal to 1 1417. Update the parameters. Remember the highest average 续 1418 to reset the parameters. If the first unvoiced zone appears after the voiced zone, Be [J checks if someone's previous voice was misjudged. 1420 Calculate the high and low energy levels and calculate the new vocal critical 値, plus the absolute minimum critical 値 if appropriate. 1602 Receive microphone signal / data 1604 Filter microphone data and digitize 1606 Segmentation and stepwise digitize data 1608 Filter the data allocated in the interval 1610 Add the data in the interval, and then square the result to calculate the energy of the vector 1612 □ Continued page (If the description page of the invention is not enough, please note and use the continued page) 45 200304119-Description of the invention Stomach 812 passes information to the path finding system 1002 Unidirectional microphone MIC 1 1010 MIC 1 Spatial response curve 1012 None Frequency response area 1014 値 1100 microphone array 1102 unidirectional microphone MIC 2 1110 MIC 2 spatial response curve 1120 first hemisphere 1130 second hemisphere 1202 receiving system microphone data 1204 filtering signal and digitization 1206 segmentation, stepping and filtering Digitized data 1208 Calculate standard variation and past gain 値 average 1210 Check standard variability and average based on minimum 値 Check and adjust 1212 Calculate critical threshold for vocalization 1212 Calculate energy and gain in the interval 1216 Compare critical 値 and gain to determine interval The information in it is voiced or unvoiced. Measure the standard variation of the number 値 and average 1616. Take the standard variation and the average and the minimum number 値 and adjust it. 1618 Calculate the voicing threshold. 1620 Compare the energy and the critical 値. Determine whether the data in the interval is a person or no one. 1902 Receive the signal from the main microphone 1904 Filter the received signal and digitize 1906 Divide and filter the digitized data 1908 Calculate VAD information 1910 Provide VAD information to the noise suppression system 2002 Receive airflow data 2004 Filter airflow data to eliminate ghosting and Digitize 2006 collect data for the next 20ms interval 2008 filter out unwanted airflow spectrum 2010 calculate the energy in the interval 2012 whether the energy in the interval 2012 is greater than the threshold 値 2014 someone vocalize 2016 send information to the path finding noise suppression system

46

Claims (1)

200304119 Scope of patent application: A system for removing noise from sound signals, including: A denoising subsystem including at least one connected receiver for providing an environmental sound signal to the noise canceling system Components of a voice detection system; a voice detection subsystem is connected to the noise reduction subsystem, the voice detection subsystem receives a voice activity signal containing human voice activity information, and the voice detection subsystem Using the information of the voice activity signal to automatically generate a control signal; wherein the components of the noise reduction sub-system use the control signal to automatically select at least one frequency subband data removal method suitable for sound signals; And the components of the noise reduction sub-system use the selected noise reduction method to process the sound signal to generate a noise canceled sound signal. 2. The system described in item 1 of the patent application scope, wherein the receiver is connected to at least one microphone array capable of detecting the sound signal. 3. The system according to item 2 of the scope of patent application, wherein the microphone array includes at least two closely spaced microphones. 4. The system according to item 1 of the scope of patent application, wherein the voice detection sub-system receives the voice activity signal through a sensor, wherein the sensor is selected from at least one of the following: a Accelerometer, skin surface microphone in contact with human skin, human tissue vibration detector, human body vibration detector, RF vibration detector, laser vibration detection Device (laser vibration detector), an electroglottograph (EGG), and a computer vision tissue vibration detector ° 5. The system according to item 1 of the scope of patent application, wherein The voice detection sub-system receives the voice activity signal through a microphone array connected to the] continuation page (note when the patent application page is insufficient, please note and use the continuation page) 47 200304119 The microphone array includes at least a microphone, a gradient microphone, and a pair of unidirectional i〇nal) One of the microphones. 6. The system according to item 1 of the scope of patent application, wherein the voice detection sub-system receives the voice activity signal through a microphone array connected to the receiver, wherein the microphone array includes a first unidirectional microphone and A second unidirectional microphone in the same position, wherein the position of the first unidirectional microphone is set to a spatial response curve maximum of the first unidirectional microphone and the second unidirectional microphone The maximum azimuth difference of a spatial response curve of a sexual microphone is between 45 degrees and 180 degrees. 7. The system according to item 1 of the scope of patent application, wherein the voice detection sub-system receives the voice activity signal through a microphone array connected to the receiver, wherein the microphone array includes a first unidirectional microphone and One of the second unidirectional microphones that is positioned colinearly. 8 · —Method for eliminating noise of sound signals, including: receiving sound signals and voice activity signals; automatically generating control signals from the data of the voice activity signals; using the control signals to automatically select at least one at least one suitable for sound signals A 'noise canceling method for subband data; and the sound signal using the selected noise canceling method and generating noise canceling. 9. The method according to item 8 of the scope of patent application, wherein selecting at least one noise reduction method and further comprising selecting a first noise reduction method for frequency sub-bands including voiced speech. 10. The method as described in item 9 of the scope of patent application, wherein selecting at least one noise reduction method and further comprising selecting a second noise reduction method for unvoiced speech is continued on the next page (When the patent application page is insufficient, please note and use the continuation page) 200304119 _ A Please request the patent application page to renew the frequency sub-band. 11. The method according to item 8 of the scope of patent application, wherein selecting at least one noise canceling method and further comprising selecting a noise canceling method for a frequency sub-band including a void of speech. 12. The method according to item 8 of the scope of patent application, wherein selecting at least one noise reduction method further comprises selecting a noise reduction method in response to noise information of the received sound signal, wherein the noise information includes At least one of a noise amplitude, a noise type, and a noise direction associated with a speaker. 13. The method as described in item 8 of the scope of patent application, wherein the at least one noise reduction method is selected to further include a step of φ, in response to the noise information of the received sound signal, and a noise reduction method is selected, in which the noise The signal information includes a noise source motion related to a speaker. 14. A method for eliminating noise of a sound signal includes: receiving a sound signal; receiving a signal related to a human voicing activity Generating at least one control signal for controlling noise removal from the sound signal; responding to the control signal and automatically generating at least one transfer function for processing the sound signal in at least one sub-band; Applying the generated transfer function to the sound signal; and removing noise from the sound signal. 15. The method as described in item 14 of the scope of patent application, and further comprising dividing the received sound signal. Into a plurality of frequency sub-bands. 16. The method according to item 14 of the scope of patent application, wherein the method for generating the transfer function further comprises, when the control signal shows that there is no voicing information in the sound signal of a sub-band, adjusting at least one The coefficient of the first transfer function representing a sub-band sound signal. 17. The method as described in item 14 of the scope of patent application, in which the method of generating the transfer function is further included] Continued page (If the page of patent scope is insufficient, please note and use the continued page) 49 200304119 --3 The scope of patent application_page includes, when the control signal shows that there is sound information in the sound signal of a sub-band, adjusting the coefficient of at least one second transfer function representing the sound signal of a sub-band. 18. The method according to item 14 of the scope of patent application, wherein the method of applying the generated transfer function further comprises: generating a noise waveform estimate related to the noise of the sound signal; And when the sound signal includes human voice and noise, the noise wave pattern is estimated to be subtracted from the sound signal. 50