JP2019128531A - Voice analysis apparatus and voice analysis method - Google Patents

Abstract

To provide a voice analyzing technology capable of estimating dry mouth and the like with good precision.SOLUTION: In a preferred embodiment of the present invention, a voice analysis apparatus includes: a voice receiving section for receiving an utterance voice; a voice analyzing section for analyzing voice data received by the voice receiving section to calculate a voice feature quantity; a data storing section for storing a second voice feature quantity formed with analyzed results of second voice data; a feature comparing section for determining the difference between the voice feature quantity calculated by the voice analyzing section and the second voice feature quantity; and an output section for outputting results determined by the feature comparing section. The analyzing target of the voice analyzing section is a predetermined vowel in the utterance voice, and a procedure for obtaining a fundamental frequency and resonant frequency as the voice feature quantity.SELECTED DRAWING: Figure 3

Description

  The present invention relates to a voice analysis device and a voice analysis method.

  In Patent Document 1, an object is to “provide a monitoring system that can detect an individual affected with a disease at an early stage in a poultry house”. “In chicken diseases, the respiratory system is damaged. Therefore, at least three or more microphones 10, a filter 24 that separates a predetermined frequency component from the output of each of the microphones 10, and the frequency component from the output of the filter 24 and the position of the microphone 10 are obtained. A poultry house monitor that has a control device 30 that calculates the point of occurrence and a pointing device 40 that points to the calculated position, detects a specific sound generated by a disorder in the respiratory system, and identifies the position where the sound is generated System is disclosed technology.

  In Patent Document 2, “The speech analysis apparatus of the present invention includes a speech acquisition unit, a frequency conversion unit, an autocorrelation unit, and a pitch detection unit. The frequency conversion unit converts the speech signal captured by the speech acquisition unit into a frequency spectrum. The autocorrelation unit obtains the autocorrelation waveform while shifting the frequency spectrum on the frequency axis, and the pitch detection unit obtains the pitch frequency from the local peak-to-peak or valley-valley interval of the autocorrelation waveform. Is disclosed.

Japanese Patent Application Laid-Open No. 2017-000062 International Publication No. WO2006 / 132159

  Regarding the detection of abnormalities in the human throat, it has been the mainstream that doctors have made subjective judgments based on their auditory impressions while interacting directly with patients. Can be mentioned.

  As a proposal of a technical solution for the purpose of estimating a speech state and characteristics from speech characteristics, for example, Patent Document 1 discloses at least three microphones (microphones) and respective outputs of the microphones. A filter that separates a predetermined frequency component from the above, a control device that calculates a point where the frequency component is generated from the output of the filter and the position of the microphone, and a pointing device that indicates the calculated position, and a respiratory system A technology for a poultry house monitoring system that detects a specific sound caused by a failure of the scrambler and identifies a position where the sound is generated is disclosed.

  Patent Document 2 includes a voice acquisition unit, a frequency conversion unit, an autocorrelation unit, and a pitch detection unit. The frequency conversion unit converts a voice signal captured by the voice acquisition unit into a frequency spectrum, and the autocorrelation unit has a frequency. There has been disclosed a technology of a speech analysis apparatus that obtains an autocorrelation waveform while shifting a spectrum on the frequency axis, and that a pitch detection unit obtains a pitch frequency from the interval between local peaks and peaks or valleys and valleys of the autocorrelation waveform.

  According to the above prior art documents, there is a possibility that the voice feature can be quantitatively grasped from the uttered voice. However, in Patent Document 1, the method of estimating from only the monotonic frequency analysis result is complicated because it targets chicken calls. It is considered difficult to adapt to human speech that includes various phoneme changes. Moreover, in patent document 2, although the technique which estimates a pitch with sufficient accuracy is shown, the objective is to estimate an emotion and it is inadequate only with a pitch as a parameter which estimates abnormalities in the mouth, such as a voice and dry mouth. it is conceivable that.

  Therefore, in order to easily detect a state of a person's throat, for example, an abnormality, a method for accurately estimating the dryness in the oral cavity for a person's speech is required. The present invention has been made in view of such circumstances, and it is an object of the present invention to provide a speech analysis technology that enables dry mouth and the like to be accurately estimated.

  A preferred aspect of the present invention includes a voice receiving unit that receives speech speech, a voice analysis processing unit that analyzes voice data received by the voice receiving unit to calculate a voice feature, and a second voice data analysis result A data storage unit for storing the second audio feature amount, a feature amount comparison unit for determining a difference between the audio feature amount calculated by the audio analysis processing unit and the second audio feature amount, and a feature amount comparison unit And an output unit for outputting a determination result, wherein the voice analysis processing unit performs processing for obtaining a fundamental frequency and a resonance frequency as voice feature amounts, with a specific vowel in the utterance voice as an analysis target. Voice analysis device.

  Another preferred aspect of the present invention includes: a voice receiving step for receiving uttered voice; a voice analyzing step for analyzing voice data of the received uttered voice to calculate an evaluation target voice feature quantity; and voice data as a reference The reference acquisition step for acquiring the reference speech feature amount, the feature amount comparison step for determining the difference between the evaluation target speech feature amount and the reference speech feature amount, and the determination result of the feature amount comparison step are output. The voice analysis step includes a result output step, and in the voice analysis step, processing for obtaining a fundamental frequency and a resonance frequency is performed by using a specific vowel in the voice as voice data to be analyzed.

  The techniques of the present invention provide for early detection of conditions in the oral cavity, such as dry condition (dry mouth). Other problems, configurations, effects, and the like will be made clear by the description of the embodiments below.

BRIEF DESCRIPTION OF THE DRAWINGS The schematic diagram which shows an example of an external appearance and system configuration | structure of the speech analyzer 1 of 1st embodiment of this invention. The functional block diagram which shows an example of a structure of the audio | voice analysis apparatus 1 of 1st embodiment. The flowchart which shows an example of the dry mouse detection process in 1st embodiment. 6 is a flowchart showing an example of speech analysis processing in the first embodiment. The schematic diagram which shows an example of the frequency analysis result (waveform) in 1st embodiment. The table which shows an example of the audio | voice analysis result in 1st embodiment. The flowchart which shows an example of the dryness estimation process in 1st embodiment. The flowchart which shows an example of the feature-value comparison process in 1st embodiment. The schematic diagram which shows an example of the display screen of the dry mouse detection result in 1st embodiment. The flowchart which shows an example of the process which changes the feature-value comparison processing method in 1st embodiment. The schematic diagram which shows an example of an external appearance and system configuration | structure of the speech analyzer 3 of 2nd embodiment of this invention. The functional block diagram which shows an example of a structure of the audio | voice display apparatus 3 of 2nd embodiment. The flowchart which shows an example of the dry mouse detection process in 2nd embodiment. The schematic diagram which shows an example of the display screen of the dry mouse detection result in 2nd embodiment.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following, the same reference numerals may be given to the same configuration throughout all the drawings, and redundant description may be omitted.

  The present invention should not be construed as being limited to the description of the embodiments below. Those skilled in the art can easily understand that the specific configuration can be changed without departing from the spirit or the spirit of the present invention.

  In the present specification etc., the notations such as "first", "second", "third" etc. are added to identify the constituent elements, and the number, order, or contents thereof are necessarily limited. is not. In addition, the identification numbers of components are used for each context, and the numbers used in one context do not necessarily indicate the same configuration in other contexts. In addition, it does not prevent that a component identified by a certain number doubles as a feature of a component identified by another number.

  The positions, sizes, shapes, ranges, and the like of the components shown in the drawings and the like may not indicate actual positions, sizes, shapes, ranges, and the like in order to facilitate understanding of the invention. For this reason, the present invention is not necessarily limited to the position, size, shape, range, and the like disclosed in the drawings and the like.

  In the embodiment described below, a voice analysis technique that enables a dry mouse to be estimated with high accuracy using the frequency characteristics of a speech voice as a main variable will be described. For this purpose, for example, the voice reception unit that receives the speech voice, the voice analysis processing unit that analyzes the voice data received by the voice reception unit and calculates the voice feature amount, and the analysis result of the second voice data A data storage unit that stores the second voice feature value, a feature value comparison unit that determines a difference between the voice feature value calculated by the voice analysis processing unit and the second voice feature value, and a determination result of the feature value comparison unit And a voice analysis unit that performs a process of obtaining a fundamental frequency and a resonance frequency by using a specific vowel in a speech as an analysis target. The

  The number of ambulances transported to hospitals due to heat stroke has been increasing in correlation with the trend of increasing outdoor temperatures in summer. It is often the case that an elderly person falls at home as a condition for transportation. There is also a research result that thirst becomes insensible as we get older, and as a countermeasure method to prevent this emergency transportation, we catch the change that heat stroke has on the body in daily life quickly and notice the person Is meaningful.

  In addition, for example, certain therapeutic drugs for kidney disease have a strong diuretic effect, and neglecting frequent water intake during regular use not only causes dehydration, but also increases blood concentration. There is a risk of increasing the risk of co-occurring. Therefore, it is meaningful to reduce or eliminate the effects of drug side effects by detecting thirst at an early stage and prompting the person to rehydrate.

  In addition, there is a causal relationship between stress and thirst, and in situations where it is difficult to solve the fundamental problem of removing the stress factor, eliminating thirst is one of the measures to prevent worsening symptoms that may occur simultaneously. As mentioned.

  In the following, a voice analysis device and a voice analysis method for detecting a tendency of dry mouth from human's voice will be described to solve the above-mentioned social problem.

  FIG. 1 shows an example of the appearance of a voice analysis device. This is because, for example, the voice analysis apparatus 1 having the appearance of a doll with an animal motif is used for monitoring and monitoring abnormal behavior of the user in daily life, and by acquiring and analyzing voice data emitted by the user. It achieves the above purpose. The voice analysis device 1 incorporates electronic circuit components including a microphone and a speaker necessary for conducting a simple conversation with the user, and transmits and receives data to and from the communication device 2 by connection of a wireless network. Make it possible. Only the microphone (microphone) 10 and the speaker 11 are shown on the external appearance of the voice analysis device 1, and other electronic circuits and the like necessary for voice analysis and voice display are described. Note that the external shape need not be limited to animals, and may be placed close to the user, or an electronic circuit component other than a microphone or the like may be exposed.

  The communication device 2 corresponds to a so-called smartphone or tablet personal computer, and displays a message to notify the user on the display screen 20. In addition, by installing application software for operating the voice analysis device 1, it is possible to send an operation command to the voice analysis device 1 or grasp the operating state in accordance with, for example, a user operation. To do. The operation button 21 performs an operation such as calling a home screen of the communication device 2. Note that the operation method of the communication device 2 is not limited to the use of the operation button 21, and a sensor that detects a tactile sensation may be mounted on the surface of the display screen 20, and may be performed by a screen touch operation.

  FIG. 2 is a block diagram showing a functional configuration related to the voice analysis function, which is built in the voice analysis device 1. The configuration basically includes a computer including an input device, an output device, a processing device, and a storage device.

  The voice receiving unit 101 digitizes the analog voice input from the microphone 10 so that the processing unit 102 can handle it. The processing unit 102 is a functional unit that performs the entire digital processing of the voice analysis device 1. The voice receiving unit 101 and the processing unit 102 can be configured by electronic components, for example, a microcomputer chip or a CPU (Central Processing Unit), which are processing devices that perform various processes based on a program. The processing unit 102 performs analysis processing of audio data, reading / writing of data to / from the data storage unit 103 and the memory 104, and data transmission / reception with other functional units. For example, control data from the communication device 2 is received via the communication unit 106, or audio data is transmitted to the audio output unit 105.

  The data storage unit 103 and the memory 104 are storage devices. The data storage unit 103 has a nonvolatile memory, and controls reading and writing of data on the nonvolatile memory in accordance with an instruction from the processing unit 102. For example, a program that is read at startup and used by the voice receiving unit 101 and the processing unit 102, voice feature value data (normal data 2001) associated with the individual user, and threshold data necessary for determination processing described later A threshold value 2002, an intensity threshold value 2003, a determination threshold value 2004, and the like are recorded. The memory 104 is a volatile memory, and expands the series of data (program, feature amount data, threshold data) necessary for processing in the processing unit 102 or writes data that needs to be temporarily stored. Used in applications such as reading.

  The audio output unit 105 which is an output device (output unit) receives, for example, audio data recorded in advance in the data storage unit 103 or the memory 104 or audio data on which the processing unit 102 has performed audio synthesis processing. Perform audio output processing. A communication unit 106 which is an input and output device has an antenna for performing near field communication, and controls data transmission and reception with the communication device 2.

  In the above description, the functions such as processing and control are realized by the CPU stored in the data storage unit 103 being executed by the CPU of the microcomputer to cooperate with other hardware to execute the determined processing. I decided to

  However, all of these functions may be implemented as hardware circuits or implemented as programs. For example, in this embodiment, a function equivalent to a function configured by software can be realized by hardware such as an FPGA (Field Programmable Gate Array) and an ASIC (Application Specific Integrated Circuit).

  The above configuration may be configured by a single computer built in the voice analysis device 1 as described above, or any part of the input device, output device, processing device, and storage device may be a network. It may be constituted by other computers connected by, for example. For example, the audio signal acquired by the audio receiving unit 101 can be transmitted from the communication unit 106 to a remote server via a network and processed by the processing unit 102, the memory 104, or the like provided in the server. Alternatively, similar processing may be performed by the communication device 2.

  FIG. 3 shows a processing sequence when the voice analysis device 1 receives voice input from the user. The user's voice input is, for example, a user's utterance action or calling phrase with a single sentence structure. After the voice analyzer 1 detects a voice, it detects a silent section of a certain length. It recognizes that the input is completed, and repeats this processing sequence every time one input is completed.

  In the voice acquisition process of step (hereinafter referred to as S) 301, voice data of the user is acquired by the voice reception unit 101 through the microphone 10. Performs digital discretization (sampling) processing, quantization processing (conversion from analog to digital data), and noise removal processing using general techniques as necessary to obtain a digital signal of the user's speech Then, the process is transferred to the processing unit 102, and the process proceeds to S302. The accuracy of sampling processing and quantization processing may be in accordance with a generally-used audio format, but as processing becomes finer, more precise processing becomes possible, but the required memory amount increases, so It is good because you do the suitable setting.

  As the timing of the voice acquisition processing S301, for example, the processing can be started at the timing at which the input of the voice is detected by a separately provided voice sensor. Alternatively, the processing may be started by issuing a message prompting utterance from the speaker 11 or the communication device 2 of the voice analysis device 1 at an arbitrary or regular timing.

  In the vowel part extraction process of S302, the processing unit 102 performs phoneme analysis of the input audio digital signal, and the process proceeds to S303. For decomposition into phonemes, the phoneme set differs depending on the language. For example, in Japanese, five vowels are generally vowels (/ a /, / i /, / u /, / e /, Detect / o / from the speech waveform. In the case of English, as an example of what has various theories, there are 15 sounds (phonetic symbols are shown in Table 1).

  Among the fifteen sounds, the same phoneme (see, for example, Table 2) having the same tongue surface high and low position with respect to the tongue surface (for example, see Table 2) may be regarded as a detection target.

  As a method of detecting phonemes, for example, a general waveform of each vowel is recorded in advance in the data storage unit 103 as a part of a program, and vowels are detected by judging based on the similarity to the input waveform. is there. In recent years, there are software and API (Application Programming Interface) that decompose a sentence of input voice data into phonemes, and it can be easily realized by incorporating these into functions.

  In the analysis target presence / absence determination process in S303, the processing unit 102 determines that there is data to be analyzed if the vowel is included in the result of phoneme decomposition, and proceeds to S304. It is judged that there is no data of and the process is ended. For example, if the acquired utterance content is “Good morning” in Japanese, this sentence includes “o (/ o /)” and “i (/ i /)” as vowels, and is analyzed. It is determined that there is an object. By extracting the vowel part without using the entire utterance, privacy can be considered, and at the same time, the characteristics of the utterance can be easily analyzed and the state of the utterer can be easily determined.

  In the speech analysis process of S304, the beginning of the vowel data is set as the analysis start point for all detected vowels, the process shown in FIG. 4 is performed, and the process proceeds to S305.

  FIG. 4 shows details of the voice analysis processing in S304 of FIG. The speech analysis process of FIG. 4 is performed on each of the data of vowel parts extracted in S302. The speech analysis processing is performed by the speech analysis processing unit included in the processing unit 102.

  In the data length determination process of S401, the processing unit 102 determines whether the data length (frame length) from the analysis start point has sufficient length for analysis with respect to the selected vowel data. To do. The length threshold 2002 for determination may be expressed using a time length expression such as 10 milliseconds or a data length expression such as 882 bytes, which is recorded in the data storage unit 103 in advance as part of the program and activated. At the same time, it is expanded and used in the memory 104. If the length is sufficient, the process proceeds to step S402. If the length is insufficient, the process proceeds to step S407.

  In the sound intensity analysis process of S402, the processing unit 102 processes the average value of the waveform amplitude with respect to the sound intensity (intensity or volume for data of a predetermined frame length such as 10 milliseconds in S401). ) And record it in the memory 104 and proceed to S403.

  In the frequency analysis process of S403, the processing unit 102 performs Fourier transform on data of a predetermined frame length to calculate a frequency spectrum, and the process proceeds to S404. When Fourier transformation is performed, a window function may be used in consideration of the discontinuity of the end points of data. However, a Hamming window or a Hanning window may be used as a general window function.

  FIG. 5 shows an example of the frequency analysis result. When the horizontal axis represents frequency (kHz) and the vertical axis represents intensity (power) (arbitrary unit), the speech waveform is generally a comb-shaped waveform (solid line in the figure).

  Returning to FIG. 4, in the fundamental frequency (pitch) estimation process of S404, the processing unit 102 calculates the fundamental frequency, and stores the fundamental frequency in the memory 104, and the process proceeds to S405. As a method of calculating the fundamental frequency, for example, a method of selecting the lowest frequency peak value of the comb-shaped waveform obtained in S403, or an interval between peaks of the comb-shaped waveform in a predetermined frequency band is determined. There is a method of obtaining as a value. In the example of FIG. 5, the difference between the lowest frequency peak and the second lowest frequency peak is indicated as F0.

  In the resonance frequency estimation process of S405, the processing unit 102 calculates a spectral envelope by applying a method such as a cepstrum method or a linear prediction analysis method (Linear Prediction Coding) to the waveform of the frequency characteristic obtained in S403, The lowest frequency peak value and the second lowest frequency peak value among the lines are recorded in the memory 104 as the first resonance frequency and the second resonance frequency, and the process proceeds to S406. In FIG. 5, the spectral envelope is indicated by a broken line, indicated by F1 at the point of the first resonance frequency and F2 at the point of the second resonance frequency. Peak values such as F1 and F2 are called formants and take specific values depending on the speaker, but generally change with time even during speech. It also changes depending on the state of the speaker's dry mouth, etc., and this change can be detected by calculating changes in the average value and dispersion value of the fundamental frequency and resonance frequency.

  In the analysis place movement process of S406, the processing unit 102 changes the start position of analysis shown in S402 to S405 by a predetermined time interval (frame update period) such as 10 milliseconds, and proceeds to S401. That is, with respect to the range of the audio signal extracted as the vowel, the analysis position is repeatedly shifted by a predetermined frame update period and the analysis process is repeatedly performed.

  In the result storage process of S407, the processing unit 102 calculates a feature amount from the result of repeating S403 to S405 (all combinations of time, volume level, fundamental frequency, and resonance frequency) and records it in the memory 104. The process ends. In a specific example, an average value and a dispersion value of the fundamental frequency, the first resonance frequency, and the second resonance frequency in a time when the volume size is a certain level or more (for example, 60 dB or more) are determined. Record as feature quantity.

  FIG. 6 is an example of data indicating a feature amount as a voice analysis result. In FIG. 6, the average and the variance of each of the fundamental frequency, the first resonance frequency, and the second resonance frequency are recorded for each vowel. Note that, in the data storage unit 103, a feature amount serving as a reference of, for example, the same tabular format corresponding to the above is recorded in advance as the normal data 2001 in advance. This is, for example, a feature value acquired from a normal utterance of the user.

  When the volume level is low, there is a concern that the voice volume is small and the fundamental frequency cannot be detected. By setting the volume level to a certain level or more, there is an effect that a clear acoustic feature can be obtained. In this case, only the sound signal of sufficient intensity is used for analysis using the volume intensity threshold 2003. However, since there is an individual difference in the setting of the volume intensity threshold 2003 for obtaining a clear feature, for example, the initial setting is set to a low value in advance, and the setting value is changed little by little depending on the success / failure rate of detection. The intensity threshold 2003 may be variable.

  FIG. 7 shows details of the dryness estimation process in S305 of FIG. In the dryness degree estimation process of S305, the process shown in FIG. 7 is performed on the feature amount data for all vowels obtained in S304, and the process of FIG. 3 is ended.

  In FIG. 7, in the normal data acquisition process of S <b> 501, the processing unit 102 acquires normal data 2001 that is characteristic amount data of each vowel in the normal state of the user, which is acquired and stored in advance from the data storage unit 103. Then, the process proceeds to S502.

  In the feature amount comparison process in S502, the processing unit 102 compares the feature amount data of each vowel data acquired this time with the normal feature amount data acquired in S501.

  FIG. 8 shows an example of the feature amount comparison processing procedure S502 of FIG. The feature amount comparison process is performed by the feature amount comparison unit included in the processing unit 102. In the vowel type determination process in S601, the processing unit 102 determines that the target vowel is analysis target if the target vowel is predetermined vowel information determined in advance by a program or the like, and proceeds to S602. In the case of other vowels, the processing is performed. Not implemented. For example, if the acquired vowel data is “i (/ i /)” and “e (/ e /)”, the analysis target is set. Otherwise, the analysis target is not set. As for the target vowel, a feature amount serving as a reference corresponding to the data storage unit 103 is recorded, and the data is acquired in S501.

  In the fundamental frequency comparison process of S602, the processing unit 102 compares the average values of the fundamental frequencies, and if there is a difference greater than or equal to a predetermined value, it is determined that there is a significant difference and the process proceeds to S603, and the difference is smaller than the predetermined value. It is determined that there is no significant difference (invalid), and the process proceeds to S605. For this determination, the determination threshold value 2004 stored in the data storage unit 103 is used. The value of the determination threshold 2004 used for the determination may be represented by a constant frequency such as 30 Hz, or represented by a ratio such as 20%.

  In the resonance frequency comparison process of S603, the processing unit 102 compares the dispersion values of the first resonance frequency, and if there is a difference greater than or equal to a predetermined value, the processing unit 102 determines that there is a significant difference and proceeds to S604. If it is smaller, it is determined that there is no significant difference (invalid), and the process proceeds to S605. For this determination, the determination threshold value 2004 stored in the data storage unit 103 is used. As for the value of the determination threshold 2004 used for the determination, for example, a ratio such as an expression with a value such as 50, or a probability distribution of the distribution of the target vowel data up to that point, which falls within 80% of the distribution. There is an expression by.

  The processing in S602 and S603 is processing to compare the calculated average value of the fundamental frequency and the dispersion value of the resonance frequency with those in the steady state. The data on the average value of the fundamental frequency and the dispersion value of the resonance frequency is obtained by calculating the average value and the dispersion value of the plurality of data in S407 from the plurality of data acquired by shifting the analysis place little by little in S404, S405, and S406. It is calculated, and the result is recorded as a feature amount, and is used in S602 and S603.

  In the above, using the average value of the fundamental frequency of normal voice data as a reference, it is determined whether or not the difference between the average value of the fundamental frequency obtained by the speech analysis processing unit and the reference is greater than a predetermined threshold. Further, using the dispersion value of the resonance frequency of the normal audio data as a reference, it was determined whether or not the difference between the resonance frequency dispersion value obtained by the sound analysis processing unit and the reference is larger than a predetermined threshold. In this example, when both results are YES, it is determined that the mouse is dry. According to such a determination method, it can be efficiently determined whether it is a dry mouse.

  In the present embodiment, the determination threshold 2004 is set for each of the average value of the fundamental frequency and the dispersion value of the resonant frequency. In the above example, the determination is performed using the average value of the fundamental frequency and the dispersion value of the resonance frequency as parameters, but this does not prevent the addition of other parameters.

  In the difference validity determination process of S604, the processing unit 102 finally determines that there is a significant difference (valid) for the compared vowels, and the process proceeds to S605.

  In the result storage process of S605, the processing unit 102 stores the result of the significant difference between valid and invalid in the memory 104, and ends the process.

  The above processing in FIG. 8 is repeated for each vowel, and if it is determined that the difference is valid at a certain ratio or more among the latest predetermined number of feature amount comparison processing results recorded in the memory 104, the dry mouse It is determined that the symptom is detected, and the process proceeds to S503. For example, the latest 10 feature value comparison processing results are always stored in the memory 104, and detection is performed when it is determined that more than 80% of the results are valid.

  In the message display process of S503, the processing unit 102 performs a process of transmitting a message suggesting the possibility of a dry mouse to the user via the communication unit 106 to the communication apparatus 2, and the process proceeds to S504.

  In FIG. 9, the example of the message display screen in the display screen 20 of the communication apparatus 2 is shown. This will be explained again later.

  In the audio output process of S504, the processing unit 102 reads out audio data information to be reproduced in this case from the audio data recorded in advance in the data storage unit 103 or previously developed in the memory 104, and the audio output unit 105 Transfer to the end of the process. The voice output unit 105 reproduces the received voice data information and outputs it to the speaker 11 to notify the user of the message.

  According to the above implementation contents, the effect of discovering dryness in the oral cavity at an early stage and noticing the user is obtained, and the user promotes actions such as rehydrating himself to recover from the dry state, or worsening the deterioration. An effect to prevent is obtained. Further, according to the above implementation contents, since only vowels are extracted for analysis, there is no need to grasp the entire utterance contents, and an effect of realizing processing in consideration of privacy can be obtained. Using the average value for comparison of the fundamental frequencies has the effect of well reflecting the change of the fundamental frequency due to the change of the stiffness of the glottis and vocal folds due to dryness. In addition, although a dispersion value is used for comparison of resonance frequencies, it is difficult to judge only the average value for a change in the resonance phenomenon in the oral cavity, and an effect that the change can be detected well by using the dispersion value is obtained.

  In the feature amount comparison process shown in FIG. 8, the vowels to be compared in the vowel type determination process are, for example, “a (/ a /)” and “i (/ i /)” and “e (/ e)”. They may be different as in “/)”, or the beginning and end of the utterance may be excluded from the processing target because they have a large change.

  In addition, in the fundamental frequency comparison process, the dispersion value of the fundamental frequency may be compared to evaluate whether the difference between the average value and the both is an individual fixed value or more, or the fluctuation of the fundamental frequency is significantly large ( If the variance is large), there is a possibility that it is a user's intentional timbre change, so processing may be performed so as not to be compared.

  Further, in this embodiment and FIG. 6, information on the fundamental frequency is recorded for each vowel. However, since the fundamental frequency is a feature amount of a part that does not change greatly due to sounds such as glottis and vocal tract, One value may be recorded. Further, in the resonance frequency comparison process S603, the dispersion values of the second resonance frequency are also compared, and the difference between the dispersion value of the first resonance frequency and the dispersion value of the second resonance frequency is greater than or equal to an individual fixed value. You may evaluate something. Alternatively, the third resonance frequency or more may be calculated similarly to the first and second resonance frequencies, and the difference comparison results of the feature quantities may be combined.

  As in such an example, with respect to the feature amount comparison processing, it is possible to expect the effect of improving the accuracy of dry mouse detection by various algorithms, and it is possible to select a method that can realize the best detection according to individual characteristics.

  Various methods can be considered for acquiring the normal data 2001 which is the normal audio feature value described in FIG. 7. For example, as a part of the initial setting performed immediately after the user obtains the voice analysis device 1, a function test of the voice acquisition unit or the microphone may be required. At the time of this functional test, a fixed sentence including one or more vowels to be analyzed is uttered to the speech analysis apparatus 1, and the feature quantity extraction processing described in FIG. 3 to FIG. 4 is performed, The analysis result can be recorded in the data storage unit 103 as a speech feature (normal data 2001) in the normal state. Alternatively, after the speech analysis apparatus 1 is activated, an average value can be calculated for all or a predetermined number of vowel feature value extraction processing results to obtain a normal speech feature value.

  It is known that formants change gradually in the process of gender differences and personal growth, assuming that speech analyzers are used over the long term. For this reason, it may be necessary to change the feature amount data in normal times. As the method, for example, with regard to the feature amount data of the vowel when it is determined that there is no difference continuously a fixed number of times in the feature amount comparison process, the processing unit 102 uses the average value as the new feature amount data of the normal. Overwrites the normal data 2001 recorded in the data storage unit 103. By incorporating such processing, it is possible to obtain an effect of realizing a dry-mouse detection method corresponding to a slowly changing individual characteristic.

  FIG. 9 shows an example of a method of displaying a message notified on the screen of the communication device 2 as a result of the message display process described in S503. On the display screen 20 of the communication apparatus 2, a message text display field 801 for displaying a message and a message evaluation unit 802 for evaluating whether the message is appropriate are displayed. The communication device 2 notifies the voice analysis device 1 of the evaluation result when the user presses the appropriate button or the improper button. The display process shown in FIG. 9 is an example, and other information transmission means such as voice, vibration, light, and sound can be used or used together to explain the situation to the user.

  FIG. 10 shows a procedure for changing the feature amount comparison processing method S502 in the voice analysis device 1 that has received the determination result.

  In the evaluation information acquisition process of S901, the processing unit 102 reads from the memory 104 the determination threshold value 2004 used for feature amount comparison, for example, the determination threshold value of the fundamental frequency comparison process and the determination threshold value of the resonance frequency comparison process, and proceeds to S902.

  In the feature amount correction process of S902, the processing unit 102 updates the determination threshold 2004 with, for example, a value that is increased or decreased at a constant rate, updates the values held by the memory 104 to the data storage unit 103, and ends the process. For example, even if the determination of the voice analysis device 1 is a dry mouse, if the evaluation of the user is "inappropriate", the threshold is increased. At this time, since the determination criteria for the previous analysis results and the future analysis results are different, the past vowel feature value comparison results stored in the memory 104 may be discarded.

  In the above embodiment, the process is described on the assumption that there is one user for the sake of simple explanation. For example, even when there are a plurality of users, any user's utterance is at the beginning or during the voice analysis process. In addition, the voice feature data (normal data 2001) and threshold data (determination threshold 2004) used for comparison processing for each speaker are recorded in the data storage unit 103 for all users. Similar analysis processing can be realized, and by specifying the target user name on the display message, it is possible to realize an effect of detecting the dry mouse early and notifying the user. Even when the utterances of a plurality of persons are input simultaneously, the above process can be realized by performing the process of separating the utterance contents at the beginning of the voice analysis process. As a general method of separating speech contents, there is a method that applies DNN (Deep Neural Network).

  In addition, the voice analysis device 1 described in the present embodiment may further have a function such as providing a camera as a part of the function of watching the user. By using camera images, it is possible to determine who is speaking from what distance, so it is possible to estimate the volume more reliably, and to determine who is to be analyzed when there are multiple users An effect such as facilitating can be obtained.

  The processing order of the processes in the processing sequences of FIGS. 3, 4, 5, and 8 described above can be changed as appropriate. For example, the order of the fundamental frequency estimation S404 and the resonance frequency estimation S405 in FIG. 4 may be switched.

  In FIG. 11, as a second embodiment, a voice analysis device 3 (for example, for controlling a home appliance by voice and collecting information on a network and answering an inquiry from a user as a second embodiment (for example, An example of the appearance of an artificial intelligence loading speaker and a home control terminal is shown.

  The voice analysis device 3 has the following configuration. Reference numeral 30 denotes a power button, which operates activation / termination of the voice analysis device 3. Reference numeral 31 denotes an operation button (up), which greatly changes the sound output volume when the speaker function is used, or moves the selection cursor upward when an operation menu or the like is displayed on the main body display screen 35. Reference numeral 32 denotes an operation button (down), which moves the selection cursor downward when the volume of the audio output when using the speaker function is changed or when the operation menu or the like is displayed on the main body display screen 35. Reference numeral 33 denotes a selection / decision button that performs an operation of selecting a menu item pointed to by the cursor when the home screen at the time of operation is called or when an operation menu or the like is displayed on the main body display screen 35.

  A plurality of microphones (microphone arrays) 34 can detect the user's speech from any direction. Reference numeral 35 denotes a main body display screen, which displays a message to be transmitted to the user from the operation menu or the voice analysis device 3 in a sentence. An indicator lamp 36 notifies the user of the operation status of the voice analysis device 3 and the presence / absence of an unread message by color. For example, it switches to no light when the power is turned off, blue light after startup, and green light when speech is accepted. The shape of the indicator lamp is not limited to the example shown in the figure, and may be a shape that can recognize the color development from any direction of 360 degrees, for example, arranged around the entire periphery of the speaker. Reference numeral 37 denotes a speaker, which outputs conversations and messages with the user as voices. A plurality of speakers may be combined to have a shape such that sound can be clearly transmitted in any direction of 360 degrees.

The communication device 2 is the same as in the description of the first embodiment, and transmits / receives data to / from the voice analysis device 3 to notify the user of a message. FIG. 12 shows a functional configuration of the voice analysis device 3. The voice receiving unit 101 converts the data received by the microphone 34 into digital data and transmits the digital data to the processing unit 102. When the microphones have a plurality of configurations (microphone arrays), a speech source from a distance from the voice analysis device 3 is also introduced by introducing a publicly available sound source position detection technology, a noise suppression technology, and the like. It is possible to reliably acquire the utterance content.

  In addition to the first embodiment, the processing unit 102 has a function of acquiring information from the operation receiving unit 107, the temperature sensor 108, and the humidity sensor 109, and a function of transmitting message data to notify the user to the display unit 110. . When data is received from the temperature sensor 108 and the humidity sensor 109, the latest value is recorded in the memory 104. When the user's device operation information is received from the operation receiving unit 107, appropriate processing is performed according to the type of the pressed button. For example, the power is turned on / off, or the display unit 110 is instructed to update the display content.

  The data storage unit 103 and the memory 104 are the same as in the first embodiment. The audio output unit 105 outputs audio to the user through the speaker 37. As in the first embodiment, the communication unit 106 has an antenna for performing near-field communication, and controls data transmission and reception with the communication device 2. The operation receiving unit 107 notifies the processing unit 102 when the button group (the power button 30, the operation button 31, the operation button 32, and the selection / decision button 33) illustrated in FIG. 11 is pressed.

  The temperature sensor 108 and the humidity sensor 109 have a function of detecting the temperature and humidity of the external environment and periodically transmitting it to the processing unit 102. The detection and transmission interval is set to 1 second, for example. The display unit 110 controls to display the character string data received from the processing unit 102 on the main body display screen 35, and controls lighting and coloring of the indicator lamp 36 according to the device status information received from the processing unit 102.

  FIG. 13 shows a processing sequence when the voice analysis device 3 receives voice input from the user. The definition of the user's voice input is the same as in the first embodiment.

  In the sensor data acquisition process of S1201, the processing unit 102 periodically acquires the temperature / humidity measurement results of the temperature sensor 108 and the humidity sensor 109 updated from the memory 104, and proceeds to S1202.

  In the execution determination process of S1202, the processing unit 102 determines whether the dry mouse detection process is necessary from the temperature / humidity measurement result. As a judgment method, when either temperature or humidity exceeds a predetermined value (for example, the temperature is 27.0 ° C. or more, the humidity is 70% or more), it is judged to be necessary. Alternatively, the heat index value (Wet-Bulb Globe Temperature) is estimated from the correlation between temperature and humidity, and is necessary when the result is a certain value (for example, 25 degrees Celsius) or more, and is unnecessary when the result is less than a certain value. Good. When it is determined that the dry mouse detection process is necessary, the process proceeds to the analysis process of S1203, and when it is determined that the process is unnecessary, the process ends.

  In the drying degree estimation process of S1203, the dry mouse detection process described using FIGS. 3 to 7 of the first embodiment is performed, and the process is ended. However, in the message display process in S503, the processing unit 102 performs a message display process for the user on the main body display screen 35 in accordance with the above-described processing contents, and a lighting method (for example, red color) that notifies the indicator lamp 36 that there is a message. Perform processing to change to blinking etc.).

  FIG. 14 shows an example of the display screen. On the main body display screen 35, a message display unit 1301 is displayed, and a message for prompting the user of the risk of heatstroke and prompting rehydration is described. Further, a button operation explanation display unit 1302 is displayed to urge the user to perform an appropriate button operation.

  According to the above embodiment, it is possible to obtain an effect of avoiding the risk of heat stroke of the user at an early stage by detecting that the environment is likely to cause heat stroke in the home environment and performing the dry mouth detection process. In addition, by displaying a message on the display unit of the terminal, an effect of making the user aware of the dry state with the voice analysis device 3 alone without the communication device 2 can be obtained, and the user can replenish himself / herself. The effect is promoted to prevent the deterioration of dryness.

  In the above configuration example, the temperature sensor 108 and the humidity sensor 109 are described as separate components. However, the temperature sensor 108 and the humidity sensor 109 may be integrated and processed as one temperature / humidity sensor. In addition, the temperature and humidity information may be acquired by another method. For example, when there is a device capable of measuring temperature and humidity capable of data communication with the voice analysis device 3 via a network, the processing unit 102 communicates. It is also used for the above control by obtaining temperature and humidity information by inquiring to the device via the unit 106, or by regularly receiving and obtaining temperature and humidity information from the device at all times. Similar implementation contents can be realized, and similar effects can be obtained. Further, in the execution determination process of S1202, both information of temperature and humidity may be used, or only one of them may be used. Alternatively, other sensor data may be added and used.

  The present invention is not limited to the embodiments described above, but includes various modifications. Further, for example, the above-described embodiments are described in detail in order to explain the present invention in an easy-to-understand manner, and the present invention is not necessarily limited to those having all the described configurations. Further, part of the configuration of each embodiment can be added to, deleted from, or replaced with another configuration.

  Each of the above-described configurations, functions, processing units, processing means, and the like may be realized by hardware by designing a part or all of them with, for example, an integrated circuit. The present invention can also be realized by a program code of software that realizes the functions of the embodiment. In this case, a non-transitory storage medium storing the program code is provided to the computer, and a processor included in the computer reads the program code stored in the non-transitory storage medium. In this case, the program code itself read from the non-temporary storage medium realizes the functions of the above-described embodiments, and the program code itself and the non-temporary storage medium storing the program code constitute the present invention. Will do. As a storage medium for supplying such a program code, for example, a flexible disk, a CD-ROM, a DVD-ROM, a hard disk, an SSD (Solid State Drive), an optical disk, a magneto-optical disk, a CD-R, a magnetic tape, A non-volatile memory card, ROM or the like is used.

  Further, program code for realizing the functions described in the present embodiment can be implemented by a wide range of programs or script languages such as, for example, assembler, C / C ++, perl, shell, PHP, Java (registered trademark).

  Furthermore, by distributing the program code of the software that implements the functions of the embodiments via a network, the program code is stored in a storage means such as a hard disk or memory of a computer or a storage medium such as a CD-RW or CD-R. A processor included in the computer may read out and execute the program code stored in the storage unit or the storage medium.

  According to the above embodiment, the vowel part is extracted by phoneme analysis with respect to the voice data obtained by acquiring the user's utterance voice and converted into digital data, and the extracted vowel part for each data of a predetermined time length. Frequency analysis is performed to obtain acoustic feature quantities (volume, fundamental frequency, resonant frequency). The obtained acoustic feature quantity is compared with the normal acoustic feature quantity recorded in advance. If the time series of the comparison result is continuously in the state of large difference, it is estimated that the oral cavity is in the dry state (dry mouth) and the message is notified, whereby the effect of early detection is obtained, and the user can It has the effect of promoting actions such as hydration itself and recovering from dryness.

  1: voice analysis device 2: communication device 10: microphone 11: speaker 20: display screen 30: power button 31: operation button (upper) 32: operation button (lower) 33: selection / determination Button 34: microphone 35: main body display screen 36: indicator lamp 37: speaker 101: audio receiver 102: processor 103: data storage 104: memory 105: audio output 106: 106 Communication unit 107: operation reception unit 108: temperature sensor 109: humidity sensor 110: display unit 801: message text display field 802: message evaluation unit 1301: message display unit 1302: button operation explanation display unit

Claims (15)

A voice receiving unit for receiving a uttered voice;
A voice analysis processing unit that analyzes voice data received by the voice reception unit and calculates a voice feature amount;
A data storage unit for storing a second voice feature amount comprising the analysis result of the second voice data;
A feature amount comparison unit that determines a difference between the voice feature amount calculated by the voice analysis processing unit and the second voice feature amount;
An output unit that outputs a determination result of the feature amount comparison unit,
The speech analysis apparatus characterized in that the speech analysis processing unit analyzes a specific vowel in the speech speech and obtains a fundamental frequency and a resonance frequency as the speech feature amount. In the voice analysis device according to claim 1,
The data storage unit stores, as the second audio feature amount, an average value of the fundamental frequency of the second audio data as an average value reference,
The speech analysis apparatus characterized in that the feature amount comparison unit determines that a difference between an average value of the fundamental frequency obtained by the speech analysis processing unit and the average value reference is greater than a predetermined threshold. In the voice analysis device according to claim 1,
The data storage unit stores a dispersion value of a resonance frequency of the second sound data as a dispersion value reference as the second sound feature amount,
The feature amount comparison unit is a speech analysis device characterized in that a determination result is that a difference between a variance value of the resonance frequency obtained by the speech analysis processing unit and the variance value reference is larger than a predetermined threshold value. In the voice analysis device according to claim 1,
The data storage unit stores, as the second audio feature amount, an average value of the fundamental frequency of the second audio data as an average value reference,
The feature amount comparison unit has a first determination result that the difference between the average value of the fundamental frequency obtained by the speech analysis processing unit and the average value reference is greater than a predetermined threshold value,
The data storage unit stores the dispersion value of the resonant frequency of the second audio data as a dispersion value reference as the second audio feature value,
The feature amount comparison unit has a second determination result that a difference between the dispersion value of the resonance frequency obtained by the speech analysis processing unit and the dispersion value reference is larger than a predetermined threshold value.
A speech analysis apparatus characterized in that a final determination result is obtained when both the first determination result and the second determination result are satisfied. In the voice analysis device according to claim 1,
The voice analysis processing unit performs processing for obtaining a fundamental frequency and a resonance frequency in a plurality of temporally different analysis locations in a specific vowel in the utterance voice,
A voice analysis apparatus characterized in that an average value of fundamental frequencies in the plurality of analysis locations and a variance of a resonant frequency are obtained as the voice feature amount. In the voice analysis device according to claim 1,
The voice analysis device, wherein the voice analysis processing unit analyzes the voice data having a predetermined length or more. In the voice analysis device according to claim 1,
The voice analysis device, wherein the voice analysis processing unit analyzes the voice data having a predetermined strength or more. In the voice analysis device according to claim 1,
Has an input unit that accepts input from the user,
The input unit receives an input from the user corresponding to the determination result of the feature amount comparison unit output by the output unit.
And changing the value of the threshold used in the feature amount comparison unit according to the input. In the speech analysis apparatus according to claim 1, further,
A temperature detection unit that detects a temperature;
A humidity detector for detecting humidity;
The speech analysis apparatus characterized by starting the processing of the speech analysis processing unit and the feature amount comparison unit when detection results of the temperature detection unit and the humidity detection unit are greater than a predetermined threshold. A voice receiving step of receiving a uttered voice;
A speech analysis step of analyzing speech data of the received speech and calculating an evaluation target speech feature amount;
A reference acquisition step for acquiring a reference voice feature amount, which is composed of an analysis result of the reference voice data;
A feature amount comparison step for determining a difference between the evaluation target speech feature amount and the reference speech feature amount;
A result output step of outputting a determination result of the feature amount comparison step,
In the speech analysis step, the speech analysis method is characterized in that a process for obtaining a fundamental frequency and a resonance frequency is performed by using a specific vowel in the speech speech as speech data to be analyzed. The reference voice feature value includes an average value of fundamental frequencies and a variance value of resonant frequencies of the reference voice data,
In the feature amount comparison step, using an average value threshold value corresponding to the average value and a variance value threshold value corresponding to the variance value,
In the feature amount comparison step, the difference between the average value of the fundamental frequency obtained in the speech analysis step and the average value of the fundamental frequency included in the reference speech feature amount is compared with the average value threshold value,
In the feature amount comparison step, the difference between the dispersion value of the resonance frequency obtained in the speech analysis step and the dispersion value of the resonance frequency included in the reference speech feature amount is compared with the dispersion value threshold.
When the difference between the average values of the fundamental frequencies exceeds the average value threshold, and the difference between the dispersion values of the resonance frequencies exceeds the dispersion value threshold, the result output step is performed. The speech analysis method according to claim 10. In the voice analysis step, when performing processing for obtaining a fundamental frequency and a resonance frequency using a specific vowel in the speech voice as voice data to be analyzed,
A vowel part extraction step of extracting a vowel part including the specific vowel in the uttered voice;
A data length determination step of determining whether the vowel portion has a predetermined length;
The sound analysis method according to claim 10, further comprising: performing a sound intensity analysis step of determining whether or not the vowel part has a predetermined sound intensity. In the speech analysis step, when the speech data of the uttered speech is analyzed and the evaluation target speech feature value is calculated, the fundamental frequency and the analysis frequency at a plurality of different locations in the specific vowel in the uttered speech Process to determine the resonant frequency,
The speech analysis method according to claim 10, wherein an average value of fundamental frequencies and a variance of resonance frequencies at the plurality of analysis locations are obtained as the evaluation target speech feature quantities. Receiving input from the user corresponding to the determination result output in the result output step;
11. The speech analysis method according to claim 10, wherein the threshold value used in the feature amount comparison step is changed according to the input. further,
At least one of a temperature detection step of detecting a temperature and a humidity detection step of detecting a humidity;
The speech analysis method according to claim 10, wherein the speech analysis step is performed when a detection result of at least one of the temperature detection step and the humidity detection step is larger than a predetermined threshold value.

Images