KR102184932B1 - Voice Recognition Method using Multiple Channels - Google Patents

Abstract

In the speech recognition method using a multi-channel according to the present invention, in the speech recognition method using a speech recognition sensor having a plurality of frequency channels, two or three frequency channels with high sensitivity among the plurality of frequency channels are selected according to frequencies. It is characterized in that speaker recognition is performed using a signal combined from a selected and selected frequency channel.

Description

Voice Recognition Method using Multiple Channels

The present application relates to a voice recognition method using multiple channels.

The speech recognition sensor refers to a sensor that extracts linguistic information from acoustic information contained in a human voice, recognizes it, and makes it react. In today's need for a natural user interface (UI) that can be used easily and conveniently, voice conversation is considered the most natural and convenient method among numerous human and machine information exchange media in the future IoT era. However, in order to communicate with the machine by voice, human voice must be converted into a format that can be processed by the machine, and this process is voice recognition.

Voice recognition, represented by Apple's Siri, consists of a combination of a microphone, ADC (Analog to Digital Converter), and DSP (Digital Signal Processing), and it consumes high power to be used as a mobile standby. It is operating by pressing the end button. This is one of the biggest challenges for realizing voice recognition based IoT (Internet of Things, Internet of Things), and it is expected to open endless IoT applications when developing a low-power, always-on voice recognition system.

The voice recognition system, which can be easily used without additional learning or training, is a promising technology that will lead the future industry in the IoT era when the demand for UI development and construction for innovative next-generation IT products is increasing. Since it can be input and the input speed is faster than typing, it has the advantage of being able to process information at high speed or in real time.

In recent years, the evolution of smartphone terminal performance, advancement of artificial intelligence and knowledge search technology, and large-capacity data processing through cloud-based voice recognition system enable users to find the answers they want accurately and quickly as an intelligent agent. Despite the possibility, voice recognition technology still has the following limitations.

First, from a hardware point of view, the existing voice recognition technology using a combination of microphone, ADC, and DSP consumes very high power, so it is practically impossible to recognize voice in the normal standby state without a separate power source. Moreover, it is applied to a voice recognition sensor for mobile. Is very limited due to energy problems. In addition, preliminary actions such as pressing the voice recognition start button are required, and its accuracy, reliability, and speed are poor. In other words, high sensitivity is essential for application to IoT-based smartphones, TVs, automobiles, and other wearable devices, and it must be able to recognize the user's voice with super power by maintaining the standby state at all times without consuming large power even in the sleep state do.

Next, from the perspective of acoustics and linguistics, there is a limit in recognizing natural conversational objects because the voice recognition of the current combination of microphone, ADC, and DSP is based on a complex algorithm.

On the other hand, the human cochlear effectively processes a complex language through a simple algorithm after frequency separation. Despite the various devices using the principle of the cochlear, there is a precedent for applying it to a cochlear implant by simulating it, but there are still no cases used as low-power voice recognition sensors for IoT.

Additionally, a conventional voice sensor may generally consist of a sensor unit and readout integrated circuits (ROIC), and a cap type voice sensor must always provide a bias to the sensor unit.

Application examples of flexible piezoelectric thin-film cochlear implants are described in H. Lee et. al's Journal of Advanced Functional Materials, Vol. 24, No. 44, pg 6914, 2014. Three piezoelectric elements were attached on a trapezoid-shaped thin silicon membrane to separate the audio signal in the audible frequency band according to the frequency. In the above document, three separate piezoelectric elements were attached on a silicon membrane to separate the frequencies and applied to the cochlear implant, but the algorithm and circuit design were not considered as a low-power voice sensor for IoT.

On the other hand, even in the case of pre-registered patent No. 10-1718214 by the present applicant, the voice detected using a plurality of frequency separation channels in a trapezoidal shape is separated through a plurality of channels according to frequencies, and the separated voice signal is piezoelectric. Although the technical content of recognizing by converting a mechanical vibration signal into an electrical signal through an element is disclosed, there is a limitation that the response performance sensed through a plurality of frequency channels cannot be smoothly implemented in a certain area.

In addition, as a conventional document that proposes a piezoelectric device that outputs a haptic feedback effect using a plurality of resonance frequencies, reference may be made to Patent Publication No. 10-2012-0099036 (2012.09.06). Meanwhile, although the document provides a haptic feedback technology based on tactile sensation, force, and motion, there is a limitation in that a method for recognizing a recognized voice in a state in which it is divided into a plurality of frequencies is not disclosed separately.

(Thesis) H. Lee et. al, Advanced Functional Materials, 24(44), 6914, 2014

(Patent Document 1) KR10-1718214 B

(Patent Document 2) KR10-2012-0099036 A

The present invention is to solve the above conventional problems, by using a flexible piezoelectric thin film embodied as a single element, separating voices sensed through a plurality of frequency separation channels in a curved shape through the plurality of channels according to frequencies, and The purpose of this is to provide a method of recognizing a speaker using a machine learning algorithm.

In other words, in the existing structure in which a plurality of frequency separation channels are arranged in a trapezoidal shape, the problem that the sensitivity to respond in a specific region's frequency band is significantly reduced in a curve shape, and the overall high sensitivity is maintained in the human audible frequency band. The purpose of this is to provide a voice recognition sensor and realize a high speaker recognition rate through a machine learning algorithm suitable for it.

In addition, the present invention detects and detects an acoustic signal in a form separated by frequency before performing digital sampling and processing of an acoustic signal on the spectrum of a human voice, so that it is more sound than a high-power voice recognition sensor based on a conventional microphone, ADC, and DSP circuit. It is an object to provide a piezoelectric voice recognition sensor that greatly reduces power consumption through simplification of the recognition circuit, and to provide a method of recognizing a speaker based on this.

In addition, the present invention provides a next-generation low-power voice recognition sensor that can replace the microphone sensor unit of a conventional voice sensor composed of a combination of a microphone, ADC, and DSP using a flexible inorganic piezoelectric material with high sensitivity. Its purpose is to provide a way to recognize.

In addition, the present invention selects two or three channels with high responsiveness to a voice frequency among a plurality of channels included in the voice recognition sensor, and performs voice recognition using only the signals sensed in the selected channel, thereby preventing errors due to resonance. It is an object to provide a speech recognition method that increases the recognition rate by reducing it.

In order to solve the above problems, an embodiment of the present invention provides a voice recognition method.

The voice recognition method is a voice recognition method using a voice recognition sensor having a plurality of frequency channels, in which two or three frequency channels with high sensitivity are selected from among the plurality of frequency channels according to frequencies and combined in the selected frequency channel. It is characterized in that speaker recognition is performed using one signal.

In addition, the solution to the above-described problem does not enumerate all the features of the present invention. Various features of the present invention and advantages and effects thereof may be understood in more detail with reference to the following specific embodiments.

The voice recognition sensor according to the present invention separates voices sensed through a plurality of frequency-separating channels in a curved shape through the plurality of channels according to frequencies, thereby maintaining a high overall sensitivity in a human audible frequency band.

In the present invention, the frequency response value through a plurality of frequency separation channels decreases linearly as the frequency response value moves from the low frequency region to the high frequency region. Show response characteristics.

The present invention uses a plurality of frequency separation channels in a curved shape to convert a separated voice signal from a mechanical vibration signal to an electrical signal through a piezoelectric element to be recognized, and employs a sound transmission mechanism of the cochlea in the human body. , A flexible piezoelectric voice recognition sensor capable of frequency separation and a sensor module compatible with it are manufactured to realize a low-power voice UI for realizing the Internet of Things that can always be driven.

In addition, by making a flexible piezoelectric material using a multi-channel structure and separating frequencies by channels for voice recognition, the machine can identify the language and speaker in the standby state, which reduces the consumption of power to the maximum. , It is possible to implement an embedded voice recognition sensor and module capable of two-way communication and correspondence.

The present invention enables faster and more accurate acoustic signal processing and high-sensitivity recognition by separating the speech spectrum for each frequency and digital sampling, and simplifies the acoustic analysis module to reduce cost. This enables speaker identification in spite of variability such as ambient noise.

In addition, in the present invention, even in the sleep state, there is little power consumption, so that it is always on standby and enables voice recognition.

The present invention makes it possible to easily and conveniently recognize a speaker and basic commands without preliminary operations such as operating a voice recognition start and end button.

In addition, the present invention selects two or three channels with high responsiveness to a voice frequency among a plurality of channels included in the voice recognition sensor, and performs voice recognition using the average value of signals sensed in the selected channel. Recognition rate can be increased by reducing errors.

1 is a comparison diagram showing differences between a conventional speech recognition system and the present invention.
2 to 10 are step-by-step cross-sectional views illustrating a method of manufacturing a piezoelectric voice recognition sensor according to an embodiment of the present invention.
11 is a schematic diagram of a piezoelectric voice recognition sensor according to an embodiment of the present invention.
In FIG. 12, a horizontal axis indicates a frequency range set through a plurality of frequency separation channels, and a vertical axis indicates sensitivity, which is a relative response (dB).
13 is a diagram specifically illustrating a voice recognition sensor having a plurality of frequency channels according to the present invention.
14 is an image showing a fabrication form of a voice recognition sensor according to an embodiment of the present invention.
15 is a diagram schematically showing the concept of speaker recognition.
16 is a diagram showing a voice recognition sensor and an output signal characteristic thereof according to an embodiment of the present invention.
17 is a graph showing the sensitivity of each channel in the voice recognition sensor according to an embodiment of the present invention.
18 is a graph showing a channel used for speaker recognition according to an embodiment of the present invention.
19 is a comparison graph showing sensitivity according to the number of channels used for speaker recognition.
20 is a diagram for comparing an original voice and a voice that has passed through f-PAS.
21 is a diagram showing a data processing algorithm.
22 is a diagram showing a probability distribution of trained data.
23 is a graph showing speaker recognition results.
24 to 26 are graphs comparing an error rate with a speaker recognition rate according to the prior art and the present invention.

Hereinafter, preferred embodiments will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art may easily implement the present invention. However, in describing a preferred embodiment of the present invention in detail, if it is determined that a detailed description of a related known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description thereof will be omitted. In addition, the same reference numerals are used throughout the drawings for portions having similar functions and functions.

In addition, throughout the specification, when a part is said to be'connected' to another part, it is not only'directly connected', but also'indirectly connected' with another element in the middle. Include. In addition, "including" a certain component means that other components may be further included rather than excluding other components unless specifically stated to the contrary.

1 is a comparison diagram showing differences between the existing speech recognition system and the present invention.

The existing voice recognition system shown on the left of FIG. 1 receives a micro voice signal in an analog form, converts it into a digital signal through an analog to digital converter (ADC), and processes the digital signal through digital signal processing (DSP). In this case, the frequency is separated, but there is a disadvantage in that high power is consumed.

Specifically, the conventional speech recognition technology has a characteristic of maintaining a constant low sensitivity state according to a frequency domain in a state based on a non-resonant method. Conventionally, there is a limitation in that only one output signal is provided through a capacitance method requiring an external DC power supply and a dielectric film is used.

On the other hand, the low-power voice recognition sensor having a plurality of frequency channels in the present invention is a piezoelectric sensor having a resonance type, and has the advantage of being able to directly recognize voice and thus low-power driving. First, a voice signal is separated from a plurality of electrode channels according to frequencies. At the same time, mechanical motion is converted into an electrical signal in a thin film made of a piezoelectric element, so that an electrical signal is detected in each frequency band.

That is, in the case of a conventional microphone, a frequency band filter, ADC, and DSP are used, which consumes high power, but the present invention uses a piezoelectric element that generates current separated by frequency, so the power required for the band filter, ADC, and DSP. Can be reduced. In addition, since the sensitivity of the microphone is high, the voltage gain to be used in the circuit unit can be lowered, thereby reducing power consumption and improving circuit stability.

In addition, according to the present invention, a plurality of frequency separation channels are arranged in a row, but are generally formed in a curve shape, so that the response characteristic linearly decreases linearly as they move from the low frequency region to the high frequency region over the audible frequency region. In the above process, it is characterized in that the overall high sensitivity is maintained from the low frequency 200 Hz band to the high frequency 4 kHz band.

The present invention uses a piezoelectric method capable of self-driving the sensor unit, and generates a plurality of output signals having different regions in one chip. In addition, it uses a unique technology that utilizes a flexible inorganic piezoelectric material.

2 to 10 are step-by-step cross-sectional views illustrating a method of manufacturing a piezoelectric voice recognition sensor according to an embodiment of the present invention.

2, a silicon substrate 100 that is a sacrificial substrate is disclosed. In the present invention, the sacrificial substrate 100 provides a stress deviation from the metal layer to be deposited later, but is not directly bonded to the nanogenerator device. In one embodiment of the present invention, the compressive stress of the silicon substrate 100 is inconsistent with the tensile stress of the metal layer bonded to the top of the device, and a separate buffer layer bonded to the silicon substrate 100 by external energy applied thereafter. (Silicon oxide layer in an embodiment of the present invention) is cracked, the horizontal crack of the buffer layer will be described in more detail below. In particular, the present invention can control and control the cracked portion according to a difference in stress between the metal layer and the sacrificial substrate.

A buffer layer 200 such as silicon oxide is deposited on the silicon substrate 100. In the present invention, the buffer layer 200 is bonded to the nanogenerator device at a level that can be dropped according to the physical force generated according to the stress difference. In an embodiment of the present invention, a silicon oxide layer is used as the buffer layer 200, and the bonding force between the silicon oxide layer and the nanogenerator is that the nanogenerator element can be effectively separated by a difference in stress between the lower substrate and the metal layer. Level.

Referring to FIG. 3, a PZT thin film 300 as a piezoelectric material layer is deposited on the buffer layer 200 through a sol-gel process, which is a known technique. In order to remove organic components from the thin film of the sol-gel solution, a 0.4M PZT sol-gel solution (Zr:Ti in a 52:48 molar ratio of PbO exceeding 10 mol%) was thermally decomposed in an air atmosphere at 450 °C for 10 minutes. With spin-cast on the wafer at 2500rpm.

The deposition and pyrolysis steps are repeated several times to form a 2 μm-thick PZT thin film. Crystallization of the PZT thin film was performed in air at 650° C. for 45 minutes. Rapid heat treatment (RTA) is used for pyrolysis and crystallization processes.

Referring to FIG. 4, a nickel layer 400 as a metal layer is deposited on the upper surface of the PZT thin film 300. According to an embodiment of the present invention, the nickel layer 400 may be laminated through a conventional semiconductor process such as sputtering or PVD process, and may also be laminated according to a conventional metal coating method. A nickel layer 400 bonded on the PZT thin film 300 is formed according to the stacking.

Referring to FIG. 5, mechanical energy (eg, physical impact) or thermal energy is applied to the nickel layer 400, which is a metal layer having residual tensile stress. As a result, residual tensile stress of nickel 400 occurs, and a mismatch between the residual compressive stress and the residual tensile stress of the silicon substrate 100 indirectly bonded to the nanogenerator element through the buffer layer 200 or An asymmetry effect occurs, and as a result, a phenomenon in which the bonding between the two layers falls at the interface between the buffer layer 200 of silicon oxide and the PZT thin film 300 occurs. The present invention is a metal layer having a tensile stress different from the residual compressive stress of a silicon substrate. After laminating a desired device and a substrate, energy is applied from the outside to separate the device from the weak bonding surface. In particular, since the separation surface that causes the separation of the device is set as the interface between the PZT thin film 300 and the buffer layer bonded with the weakest force, there is an advantage that the device manufactured on the silicon substrate can be separated and transferred as it is. In addition, the device isolation position may be controlled according to a difference in stress between the metal layer and the sacrificial substrate.

Referring to FIG. 6, the PZT thin film 300, which is not bonded due to the residual tensile stress mismatch of the metal layer in contact with the silicon substrate, is separated from the silicon oxide buffer layer 200 (see FIG. 7 ).

On the other hand, the process of separating the PZT thin film 300 from the silicon oxide buffer layer 200 may also be possible by a laser lift off (LLO) process. That is, in order to separate the PZT thin film 300 from the buffer layer 200, irradiation to the rear surface of the silicon oxide buffer layer 200 through an XeCl-pulse excimer laser, for example, the photon energy (4.03 eV) of the XeCl laser is the buffer layer ( Since it is smaller than the band-gap energy of 200) and larger than that of the PZT thin film 300, it enables the PZT thin film to be transferred to the flexible plastic substrate. As a result, the laser beam penetrates the silicon oxide buffer layer, followed by local melting and dissociation of the PZT at the boundary with the buffer layer.

As described above, a laser lift off (LLO) process for converting the PZT thin film into a plastic substrate occurs.

Referring to FIG. 8, the separated PZT thin film 300-nickel 400 layer is physically moved and bonded to a flexible plastic substrate 600. This completes the flexible nanogenerator transferred onto the flexible plastic substrate 600.

Referring to FIG. 9, the nickel layer 400 is removed through etching, which is a typical chemical etching process. For example, the nickel layer 400 may be removed by immersing the upper portion of the device bonded to the plastic substrate 600 in a specific etchant for etching the nickel layer 400. However, in addition to this, the nickel layer 400 may be selectively removed according to various conventional metal layer removal methods, and this also falls within the scope of the present invention.

Next, referring to FIG. 10, an electrode 500 is stacked on the PZT thin film 300, whereby in the form of a plastic substrate 600, a PZT thin film 300, and an electrode 500 as a flexible thin film from the bottom. Are stacked. Here, the electrode 500 forms a plurality of frequency separation channels.

The electrode 500 may be any one of electrode materials including Ti/Au, Ti/Pt, Cr/Au, and Cr/Pt.

The plastic substrate 600 may be any one of substrate materials including PET, PEN, Parylene, and Kapton.

Meanwhile, referring to FIG. 11, the piezoelectric speech recognition sensor according to the present invention may selectively add a passivation layer to cover the electrode 500 as a whole. The protective layer may be Parylene or SU-8.

An adhesive layer is disposed between the plastic substrate 600 and the PZT thin film 300, and the adhesive layer may be Norland or PU.

The basic concept of the present invention will be described with reference to FIG. 12.

In FIG. 12, a horizontal axis indicates a frequency range set through a plurality of frequency separation channels, and a vertical axis indicates sensitivity, which is a relative response (dB).

As an example, frequency channels corresponding to four channels shown in FIG. 12 sense a frequency domain of an overlapping area. That is, on the graph, the leftmost channel 1 shows high response characteristics, and the rightmost channel 4 shows relatively low response characteristics.

The present invention can retain higher sensitivity in a specific frequency domain by using a resonant element. In other words, when compared to the conventional microphone (Ref. Mic) that maintains a constant sensitivity of about -100dB, it can be seen that all the overlapping regions of the frequency channel far exceed the sensitivity of the microphone (Ref. Mic).

Conventional microphones consist of a sensor and a read-out IC (ROIC), and when the membrane vibrates due to sound pressure, an electrical signal due to a potential difference enters the ROIC. ROIC consists of an amplifier and an impedance converter, and the sensitivity is determined by the gain of the amplifier.

12 shows the sensitivity of the sensor part excluding ROIC in ref.mic, and shows response characteristics corresponding to each frequency when white noise of 20-20 kHz is input as 94SPL (sound-level pressure).

For example, in the resonant frequency channel 1 (Ch1), the 3dB bandwidth of the channel 1 is widened to more than 50 Hz and the Q value is set in a direction of lowering the Q value to less than 35, thereby uniformly sensing 200 Hz to 4 kHz.

Hereinafter, a voice recognition sensor having a plurality of frequency channels according to the present invention will be described in detail with reference to FIG. 13.

The voice recognition sensor has a plurality of frequency channels having different lengths. The plurality of frequency channels may be in the form of six electrode channels arranged to have predetermined intervals on one chip. The plurality of frequency channels are arranged side by side in the order of ch1 to ch6 from left to right.

In the case where the plurality of frequency channels are connected to both edge sides in succession, the overall shape forms a curved surface.

Specifically, when going from ch1 to ch2 and ch3, the slope increases to a fairly gentle state, but when going from ch3 to ch4, ch5, and ch6, the slope increases to a state with a significantly larger slope.

That is, the slope value of the curved surface tends to increase gradually from ch1 as the shortest channel to ch6 as the longest channel.

13 shows the tendency of the frequency channel based on the highest peak point. Here, the vibration signal and the electric signal coincide in almost all frequency channels.

Among the six electrode channels, a frequency channel on one side for sensing a low frequency is set as a first sensing unit, and a frequency channel on the other side for sensing a high frequency relative to a first sensing unit is set as a second sensing unit.

The length ratio between the second sensing unit corresponding to the shortest-length channel sensing the highest frequency region and the first sensing unit corresponding to the longest-length channel sensing the lowest frequency region is in the range of 1:1.5 to 6.5. .

Preferably, the ratio between the shortest length channel and the longest length channel may be in the range of 1:4. This is due to the fact that in the case of 1:3, the frequency coverage may be biased toward the high frequency side, and in the case of 1:5, the frequency coverage may be biased toward the lower frequency side.

When the ratio at which the length of the electrode channel decreases on the first sensing unit is set to a and the ratio at which the length of the electrode channel decreases on the second sensing unit is set to b, b/a, which is the ratio between a and b, is It is preferably determined in a range less than 1.

Through this, the performance of sensing a low frequency in the first sensing unit in which the ratio at which the length of the electrode channel decreases is set to a may be further improved.

Since the voice sensor according to the present invention is a piezoelectric type resonance element, a quality factor corresponds to an important feature.

Q (Quality factor) in resonance means the frequency selection characteristic quality. The difference between the frequencies at the point at which the resonance frequency is attenuated by 3 dB, ie, half, is called the so-called 3 dB bandwidth, and the Q value is the resonance frequency divided by the 3 dB bandwidth. That is, the sharper the resonance characteristic, the narrower the 3dB bandwidth will be, and eventually the Q value will increase. On the one hand, if Q is low, it may mean that the band is wide.

In the present invention, the 3dB bandwidth of each channel is set in the direction of lowering the Q value by increasing the bandwidth to 50Hz or more, so that sensing is performed evenly with higher sensitivity than the conventional MEMS-based microphone within the range of 200Hz to 4kHz. The existing MEMS-based microphone is the performance of a pure microphone sensor excluding the effect of the ROIC part, and means a performance of -100dB under a white noise condition of 20~20000Hz of about 94SPL, and the flexible piezoelectric voice recognition sensor according to the present invention is 200~ It means that it has a performance higher than -100dB, which represents the performance value of the existing MEMS-based microphone in the range of 90% or more of 4kHz.

There are electromechanical coupling factor K and electromechanical quality factor Q that show more effective performance than piezoelectric coefficient.

There are two types of quality factors: electrical quality factor (Qe) and mechanical quality factor (Qm).

Qe is the reciprocal of the electrical loss (tanδ), while Qm is the concentration of the displacement due to the mechanical vibration damping of the vibrating body. Piezoelectric materials for resonators require a Qm value of 1500 or more, whereas for filters, a Qm value of 400 to 600, and piezoelectric speakers have a Qm value of 80 or less, and the material development focuses on high dielectric constant. have.

In the case of the voice sensor according to the present invention, it is possible to have a total of 7 channels as a multi-channel, and the Q value has a value between 18 and 28. Based on the above results, it may be desirable to maintain the Q value of 35 or less.

The mechanical quality factor above represents the ratio of the energy accumulated during the exchange between electrical energy and mechanical energy, and is due to the phase difference between the applied voltage generated when the permanent dipoles move, and the losses are mostly dissipated in the form of thermal energy, The piezoelectric material determines the sharpness of the resonance at the resonance frequency.

If the value of the mechanical quality factor is low, degradation generally occurs quickly and has a different value from the electrical quality factor.

14 is an image showing a fabrication form of a voice recognition sensor according to an embodiment of the present invention.

Referring to FIG. 14, a piezoelectric material PZT is laminated using PET on a PCB in which an empty space is formed. The piezoelectric material forms a plurality of frequency channels in a curved shape and is formed to correspond to the empty space. Meanwhile, the Au electrode is formed on the upper surface of the PCB through a process of being electrically connected to both ends of the piezoelectric material.

PZT and PU adhesives are located on PET, which is a square-shaped transparent plastic substrate, and electrical energy generated from PZT is collected through Au electrodes. In addition, a passivation layer that protects it is additionally deposited to protect the device.

Meanwhile, PET, UV-sensitive PU adhesive, PZT, and protective layer may be made of a transparent material. In the Au electrode, the color of the electrode may be seen as gold to the naked eye.

15 is a diagram schematically showing the concept of speaker recognition.

Referring to FIG. 15, the flexible piezoelectric voice recognition sensor f-PAS vibrates by the voice of a speaker, and an output voltage is generated due to the vibration.

The output voltage signal can be used to generate a database of training utterances through a machine learning process.

Thereafter, when a test speech is input, a speaker recognition can be performed by selecting a similar person from the trained database.

As an example, by applying the speech recognition method of the present invention to be described later, as training data, 40 speakers uttered 70 times each to perform a total of 2800 learning (90% of the total data), and 40 speakers as test data As a result of conducting a total of 280 tests by firing 7 times (10% of the total data), it was confirmed that a recognition rate of 97.5% was achieved.

16 is a diagram showing a voice recognition sensor and an output signal characteristic thereof according to an embodiment of the present invention.

Referring to FIG. 16, the right side shows a voice recognition sensor including 7 channels as a voice recognition sensor manufactured according to the present invention, and the right side shows an output signal for each channel included in the voice recognition sensor.

17 is a graph showing the sensitivity of each channel in the voice recognition sensor according to an embodiment of the present invention.

Referring to FIG. 17, it can be seen that a plurality of channels included in the voice recognition sensor have different sensitivities, that is, relative responses, over a frequency domain.

Accordingly, in the present invention, two or three channels with high sensitivity are selected from among a plurality of channels according to a frequency domain, and voice recognition may be performed using only a signal sensed in the selected channel.

18 is a graph showing a channel used for speaker recognition according to an embodiment of the present invention.

Referring to FIG. 18, it is shown that two channels having the highest relative response (dB) sensitivity are selected as the frequency changes in an audible frequency region ranging from 100 Hz to 4000 Hz on the horizontal axis. For example, about 120 Hz components use Ch3 and Ch6, about 510 Hz components use Ch1 and Ch2, about 1 kHz components use Ch4 and Ch5, about 2 kHz components use Ch2 and Ch7, and about 4 kHz. Components Ch1 and Ch7 can be used. As described above, according to an embodiment of the present invention, as the frequency changes, it is possible to use two channels with high sensitivity at the corresponding frequency.

19 is a comparison graph showing sensitivity according to the number of channels used for speaker recognition.

Referring to FIG. 19, the comparison graph shows the sensitivity of the channel with the highest sensitivity among a plurality of channels (The highest), the sensitivity of the channel with the second highest sensitivity, and the sensitivity as the frequency changes. It shows the average of the two high channels (Two-averaging) and the average of all channels (eg, 7 channels) included in the voice recognition sensor (Seven-averaging).

20 is a diagram for comparing an original voice and a voice that has passed through f-PAS.

Referring to FIG. 20, from the left, the upper graph shows an original sound signal, a fast Fourier transform (FFT) result of the original sound signal (Original Sound FFT), and a short-time Fourier transform (STFT) of the original sound signal, respectively, from the left. transform) shows the result (Original Sound STFT).

In addition, the graph at the bottom is the flexible piezoelectric voice recognition sensor (f-PAS) signal (f-PAS Signal) according to the present invention from the left, the FFT result of the f-PAS signal (f-PAS FFT), and the f-PAS signal. The STFT result (f-PAS STFT) is shown.

In FIG. 20, when comparing the original voice signal at the top and the f-PAS signal at the bottom, it can be seen that the frequency response is not flat, so there is a slight difference, but the large flow is similar.

21 is a diagram showing a data processing algorithm.

Referring to FIG. 21, data processing for speech recognition according to the present invention may largely include a speaker training process 2110 and a speaker testing process 2120.

In the speaker training process 2110, when a training signal is input through the acoustic sensor 2111, STFT is performed for each channel included in the voice recognition sensor 2111 (2112), and then STFT Features are extracted (2113), and a GMM score is calculated through training Gaussian Mixture Modeling (GMM) 2114 (2115). However, the algorithm for speaker recognition is not necessarily limited to GMM, and various machine learning-based speaker recognition algorithms such as HMM (Hidden Markov Model) or SVM (Support Vector Machine) techniques can be applied.

n). 여기서, 복수의 주파수 분리 채널은 100Hz 내지 8Khz 대역의 주파수를 가지며, 복수의 주파수 분리 채널은 채널의 길이가 증가할수록 낮은 주파수를 감지하는 것일 수 있다.Here, the speaker training process 2110 may perform speaker training using m frequency separation channels representing the maximum sensitivity at a corresponding frequency among n frequency separation channels (n, m are natural numbers, m

n). Here, the plurality of frequency separation channels may have a frequency in the 100Hz to 8Khz band, and the plurality of frequency separation channels may sense a lower frequency as the length of the channel increases.

In the speaker test process 2120, when a test signal is input through the acoustic sensor 2121, STFT is performed for each channel included in the voice recognition sensor 2121 (2122), and then STFT A speaker can be recognized by extracting features (2123) and calculating a GMM score (2115).

In addition, in the speaker test process 2120, when an untrained test signal is input, a person having the highest likelihood may be recognized as a speaker.

For example, as training data, 40 speakers speak 70 times each, and a total of 2800 speaker training processes are performed to build a database (90% of the total data). Referring to FIG. 22, a diagram showing the characteristics of the utterances as a distribution through a speaker training process for 2800 utterances (t-SNE plot).

In addition, as test data, 40 untrained speakers speak 7 times each, and a total of 280 speaker test processes are performed to recognize the speaker (10% of the total data). Referring to FIG. 23, a horizontal axis of the graph represents a frame obtained by dividing a speech into a specific time section, and a vertical axis represents a speaker recognition result (Estimated Label) according to a frame in a speaker recognition process. Here, it can be seen that as the number of frames increases, the speaker can be accurately recognized.

The technology for performing speaker recognition according to the above-described voice recognition method can be widely applied to a voice sensor system to which the Internet of Things (IoT) based on voice recognition is applied, a smart home home appliance, and a portable terminal device.

24 to 26 are graphs comparing an error rate with a speaker recognition rate according to the prior art and the present invention.

24 to 26, in the case of a conventional microphone (Commercial MEMS Mic.), whereas the speaker recognition rate is only about 90% (that is, the error rate is 10%), including a plurality of channels according to the present invention In the case of a flexible piezoelectric voice recognition sensor, a higher speaker recognition rate can be achieved compared to conventional microphones.

In addition, in the case of using signals from all channels included in the flexible piezoelectric speech recognition sensor (seven-averaging), the speaker recognition rate is about 92.7% (that is, the error rate is 7.3%), and signals of two channels with high sensitivity are used. In the case (two-averaging), the speaker recognition rate is 97.5% (that is, the error rate is 2.5%), and the speaker recognition rate can be greatly improved when signals of two channels are used.

In addition, referring to FIG. 26, compared to a case where a speaker is recognized using only a signal of a channel with the highest sensitivity or a signal of a channel with the second highest sensitivity among a plurality of channels included in the flexible piezoelectric speech recognition sensor, It can be seen that a higher speaker recognition rate can be achieved when using both signals.

The present invention focuses on implementing speech recognition by simulating the cochlear, which is a human auditory organ, and a simple circuit based on a flexible piezoelectric voice sensor, rather than a conventional microphone, ADC, and DSP combination method for frequency separation, can significantly reduce power consumption. have. In addition, if an efficient recognition algorithm compatible with this is implemented, human natural language can be recognized with high selectivity, sensitivity, detection speed and stability.

This technology can be applied in real life. For example, it is possible to safely use the vehicle information system by voice while driving, only by voice while in a standby state. Through this, TV, vacuum cleaner, washing machine, air conditioner, etc. Low-power control can only be performed. In particular, it is possible to use facilities such as elevators more conveniently by registering caring or voices of disabled and patients with discomfort in hands and feet.

This technology is a fusion technology that enriches human life by taking inspiration from nature as a theme that encompasses the entire IT-NT-BT-material. Through the speaker's voice, it is possible to grasp the identity, psychology, health status, language ability, etc. in the normal standby state with little power, enabling personalized service provision, and all fields of sensors ranging from fields such as security, finance, and medical education. It can be used for

In particular, it can be applied to mobile healthcare, such as analyzing and storing voice patterns in big data, analyzing and storing emotional states, and bringing out psychological stability through a feedback system, and a security system through voice authentication and speaker identification. It is expected to be strengthened to help protect personal information and privacy.

The present invention can implement a voice recognition-based Internet of Things (IoT) and a ultra-small voice sensor system for mobiles through the above features.

The voice sensor according to the present invention can be used in smart home appliances including TVs and refrigerators, voice assistants, and voice security applications.

In the present invention, a voice recognition sensor made of a high-efficiency inorganic piezoelectric material on a flexible substrate separates the mechanical vibration energy from the voice at different positions for each frequency by using the piezoelectric material before digital sampling and processing the sound signal. After converting it into an electrical signal, the voice signals are processed in parallel for each frequency.

In the present invention, a plurality of frequency separation channels form a shape of an artificial cochlear resembling a xylophone shape, and as the sizes of the plurality of frequency separation channels are changed, the positions at which high-frequency sounds and low-frequency sounds resonate are changed. To separate. Here, each separated sound is amplified through an analog circuit for each frequency, filtered, and converted into a digital signal for processing. This process significantly reduces power consumption compared to the method using a conventional microphone, ADC, and DSP combination.

The present invention provides a piezoelectric voice recognition sensor coupled to a flexible thin film, and can be used even in a state attached to clothing or the like. In other words, it can be applied to a technology that harvests physical energy in the area of sound waves and ultrasonic waves that are easily generated around while attached to clothing and converts them into electrical energy.

In general, the existence of a ubiquitous power source that exists and works everywhere is indispensable for the realization of a ubiquitous network that exists anywhere. On the other hand, the power of ubiquitous network components that exist everywhere should be in a self-sufficient form that does not require charging. That is, both power generation capability and power storage capability must be provided.

As described above, the piezoelectric speech recognition sensor according to the present invention separates voices detected using a plurality of frequency separation channels in a trapezoidal shape through the plurality of channels according to frequencies and simultaneously piezoelectrically separates the separated speech signals. Through the device, the mechanical vibration signal is converted into an electrical signal to be recognized.

Although the preferred embodiment of the present invention has been described above, the present invention is not limited to the specific embodiment described above. That is, a person of ordinary skill in the technical field to which the present invention pertains can make a number of changes and modifications to the present invention without departing from the spirit and scope of the appended claims, and all such appropriate changes and modifications It should be considered that equivalents are also within the scope of the present invention.

Claims (11)

In the voice recognition method using a voice recognition sensor having a plurality of frequency channels,
Depending on the frequency, two or three frequency channels with high sensitivity are selected among the plurality of frequency channels, and speaker recognition is performed using a signal combined from the selected frequency channel,
The plurality of frequency channels form a curved surface when both edge sides are connected continuously, and the slope value of the curved surface increases from the shortest length frequency channel to the longest frequency channel,
The plurality of frequency channels sense a frequency region overlapping each other,
A voice recognition method, characterized in that the 3dB bandwidth of at least one of the plurality of frequency channels is increased to 50Hz or more to lower the quality factor of the channel to less than 35 to sense a frequency range of 200Hz to 4kHz with high sensitivity.
The method of claim 1,
The voice recognition method, characterized in that the voice recognition sensor is a flexible piezoelectric-based voice recognition sensor.
The method of claim 2,
The voice recognition sensor includes a piezoelectric material and an electrode formed on the piezoelectric material,
The electrode includes a plurality of frequency separation channels having different lengths, the plurality of frequency separation channels have a frequency in the 100Hz to 8Khz band, and the plurality of frequency separation channels detect a lower frequency as the length of the channel increases. Voice recognition method, characterized in that.
The method of claim 2,
A speech recognition method that performs speaker recognition using the average of two or three channels with high sensitivity.
The method of claim 2, wherein the voice recognition method,
A speaker training process for performing speaker training using a training signal input through the speech recognition sensor; And
A speech recognition method comprising a speaker test process of performing speaker recognition using a test signal input through the speech recognition sensor.
The method of claim 5,
In the speaker training process, speaker training is performed using m frequency separation channels representing maximum sensitivity at a corresponding frequency among n frequency separation channels, n and m are natural numbers, and m

Voice recognition method, characterized in that n.
The method of claim 6,
In the speaker test process, when an untrained test signal is input, a person having the highest likelihood is recognized as a speaker.
The method of claim 5,
The speaker training process is a speech recognition method for performing speaker training by applying a Gaussian Mixture Modeling (GMM), Hidden Markov Model (HMM), or Support Vector Machine (SVM) technique.
delete delete delete

Images