KR102564412B1 - System for providing self dysphonia treatment service - Google Patents

Abstract

A self-speech therapy service providing system is provided, response data is entered into a self-report questionnaire to determine voice disorder, voice speech is recorded or uploaded in real time, and voice disorder analyzed by response data and voice speech and degree A user terminal outputting at least one corresponding voice treatment content, a self-report questionnaire for determining voice disorder, a criterion for determining whether or not there is a voice disorder according to response data, and at least one corresponding to the degree of voice disorder and whether or not and the degree of voice disorder A storage unit that converts voice therapy contents into a database, a questionnaire unit that receives response data from a user terminal, and when a voice speech is received from a user terminal, inputs a query to a previously built speech disorder determination algorithm to determine whether or not a voice disorder exists A voice treatment service providing server including a voice analysis unit that analyzes the degree, and a self-treatment unit that outputs at least one voice treatment content corresponding to the presence and degree of voice disorder analyzed by response data and voice speech from a user terminal. do.

Description

Self voice therapy service providing system {SYSTEM FOR PROVIDING SELF DYSPHONIA TREATMENT SERVICE}

The present invention relates to a system for providing a self-speech therapy service, and provides a system for providing a self-reported questionnaire, analyzing response data and speech utterances, and providing voice therapy contents to proceed with a self-speech therapy process.

Voice means when a sound generated through a human speech organ has a linguistic meaning, and is a concept different from sound. Pronunciation organs are divided into respiratory organs, vocal organs, resonance and articulatory organs, and voice disorders refer to organic or functional problems in these organs. Voice evaluation for patients with voice disorders is largely divided into auditory perceptual evaluation and instrumental evaluation. Auditory evaluation, such as the GRBAS scale rating method published by the Japanese Society of Speech and Linguistic Medicine or The Consensus Auditory-Perceptual Evaluation of Voice published by the American Speech-Hearing-Language Association, is an auditory evaluation in which the rater listens to the speech of a patient with or suspected of having a speech disorder and hears the quality of the speech. The instrumental evaluation is a method of analyzing the acoustic and aerodynamic aspects of the patient's voice using devices such as the Computerized Speech Lab's Multidimensional Voice Program or Phonatory Aerodynamic System.

At this time, a method of helping treatment for voice disorders by analyzing voice speech has been researched and developed. In this regard, prior art, Korea Patent Registration No. 10-1413598 (published on July 4, 2014) and Korea Patent Publication No. 2020 -0105110 (published on September 7, 2020) has a configuration that generates vibration corresponding to the pronunciation uttered by the person to be corrected through a vibration speaker in a vibration chair, and when a vocal training selection menu is selected, a vocal signal is received and vocalization is performed. Configurations for inducing ignition promotion are disclosed respectively.

However, in both the former and latter cases, training is conducted after being diagnosed with a voice disorder. Recently, the evaluation of communication disorders is gradually shifting from an assessor-centered to a subject-centered one. In other words, the evaluation target is no longer passively evaluated only according to the evaluator's instructions, but is changing so that they can actively present their opinions and reflect their contents in the evaluation. Voice evaluation is also changing its evaluation method to reflect this trend, and if the purpose of speech evaluation is to collect information to eliminate voice problems in patients with voice disorders, what problems do patients struggle with during the evaluation process and what problems are they? It is also very important to understand how it affects your quality of life. To this end, one of the methods used by voice disorder experts is to have patients report their voice problems from their own point of view by answering a checklist related to voice disorders. Therefore, research and development of a platform capable of diagnosing voice disorders using a self-reporting evaluation tool and performing self-voice therapy is required.

An embodiment of the present invention is a voice handicap index (VHI), a voice-related quality of life (V-RQOL), and a voice symptom scale (VoiSS) so that the user can check whether or not he or she has a voice disorder through a self-report questionnaire. , VFI (Voice Fatigue Index), and Voice Activity Participation Profile (Voice Activity Participation Profile) are provided, and the response data and voice speech answered by users are analyzed to classify the presence, type and degree of voice impairment, By providing a self-voice treatment process corresponding to the presence, type and degree of voice disorder, it is possible to provide a self-voice treatment service providing system that can help a user to identify and treat voice disorder by himself. However, the technical problem to be achieved by the present embodiment is not limited to the technical problem as described above, and other technical problems may exist.

As a technical means for achieving the above-described technical problem, an embodiment of the present invention inputs response data into a self-report questionnaire for determining voice disorder, records or uploads voice speech in real time, and responds to the response data and voice. A user terminal that outputs at least one voice treatment content corresponding to the presence and degree of voice disorder analyzed by speech, a self-report questionnaire for determining voice disorder, criteria for determining voice disorder according to response data, and degree of voice disorder, voice A storage unit that compiles at least one voice treatment content corresponding to the presence and degree of disability into a database, a questionnaire unit that receives response data from the user terminal, and a query to the previously built voice disability determination algorithm when voice speech is input from the user terminal A voice analysis unit that analyzes the presence and degree of voice disorder by inputting (Query), self-treatment that outputs at least one voice treatment content corresponding to the presence and degree of voice disorder analyzed by response data and voice speech from the user terminal and a voice therapy service providing server including a unit.

According to any one of the above-mentioned problem solving means of the present invention, to check whether the user himself has a voice disorder through a self-report questionnaire, VHI (Voice Handicap Index), V-RQOL (Voice-Related Quality of Life), Surveys such as VoiSS (Voice Symptom Scale), VFI (Voice Fatigue Index), Voice Activity Participation Profile (Voice Activity Participation Profile) are provided, and user response data and speech speech are analyzed to determine whether or not there is a voice disorder, type and By categorizing the degree and providing a self-voice treatment process corresponding to whether or not the voice disorder exists, the type, and the degree, the user can help the user to identify and treat the voice disorder himself.

1 is a diagram for explaining a self-voice therapy service providing system according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating a voice therapy service providing server included in the system of FIG. 1. Referring to FIG.
3 and 4 are diagrams for explaining an embodiment in which a self-voice therapy service is implemented according to an embodiment of the present invention.
5 is an operational flowchart illustrating a method for providing a self-speech therapy service according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail so that those skilled in the art can easily practice the present invention with reference to the accompanying drawings. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

Throughout the specification, when a part is said to be "connected" to another part, this includes not only the case where it is "directly connected" but also the case where it is "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, this means that it may further include other components, not excluding other components, unless otherwise stated, and one or more other characteristics. However, it should be understood that it does not preclude the possibility of existence or addition of numbers, steps, operations, components, parts, or combinations thereof.

As used throughout the specification, the terms "about", "substantially", etc., are used at or approximating that value when manufacturing and material tolerances inherent in the stated meaning are given, and do not convey an understanding of the present invention. Accurate or absolute figures are used to help prevent exploitation by unscrupulous infringers of the disclosed disclosure. The term "step of (doing)" or "step of" as used throughout the specification of the present invention does not mean "step for".

In this specification, a "unit" includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Further, one unit may be realized using two or more hardware, and two or more units may be realized by one hardware. On the other hand, '~ unit' is not limited to software or hardware, and '~ unit' may be configured to be in an addressable storage medium or configured to reproduce one or more processors. Thus, as an example, '~unit' refers to components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, and procedures. , subroutines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays and variables. Functions provided within components and '~units' may be combined into smaller numbers of components and '~units' or further separated into additional components and '~units'. In addition, components and '~units' may be implemented to play one or more CPUs in a device or a secure multimedia card.

In this specification, some of the operations or functions described as being performed by a terminal, device, or device may be performed instead by a server connected to the terminal, device, or device. Likewise, some of the operations or functions described as being performed by the server may also be performed by a terminal, apparatus, or device connected to the server.

In this specification, some of the operations or functions described as mapping or matching with the terminal mean mapping or matching the terminal's unique number or personal identification information, which is the terminal's identifying data. can be interpreted as

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a diagram for explaining a self-voice therapy service providing system according to an embodiment of the present invention. Referring to FIG. 1 , a self-speech therapy service providing system 1 may include at least one user terminal 100, a voice therapy service providing server 300, and at least one medical staff terminal 400. However, since the self-voice therapy service providing system 1 of FIG. 1 is only an embodiment of the present invention, the present invention is not limitedly interpreted through FIG. 1 .

At this time, each component of FIG. 1 is generally connected through a network (Network, 200). For example, as shown in FIG. 1 , at least one user terminal 100 may be connected to a voice therapy service providing server 300 through a network 200 . In addition, the voice therapy service providing server 300 may be connected to at least one user terminal 100 and at least one medical staff terminal 400 through the network 200 . In addition, at least one medical staff terminal 400 may be connected to the voice therapy service providing server 300 through the network 200 .

Here, the network refers to a connection structure capable of exchanging information between nodes such as a plurality of terminals and servers, and examples of such networks include a local area network (LAN) and a wide area network (WAN: Wide Area Network), the Internet (WWW: World Wide Web), wired and wireless data communications networks, telephone networks, and wired and wireless television communications networks. Examples of wireless data communication networks include 3G, 4G, 5G, 3rd Generation Partnership Project (3GPP), 5th Generation Partnership Project (5GPP), Long Term Evolution (LTE), World Interoperability for Microwave Access (WIMAX), Wi-Fi , Internet (Internet), LAN (Local Area Network), Wireless LAN (Wireless Local Area Network), WAN (Wide Area Network), PAN (Personal Area Network), RF (Radio Frequency), Bluetooth (Bluetooth) network, NFC ( A Near-Field Communication (Near-Field Communication) network, a satellite broadcasting network, an analog broadcasting network, a Digital Multimedia Broadcasting (DMB) network, etc. are included, but not limited thereto.

In the following, the term at least one is defined as a term including singular and plural, and even if at least one term does not exist, each component may exist in singular or plural, and may mean singular or plural. It will be self-evident. In addition, the singular or plural number of each component may be changed according to embodiments.

At least one user terminal 100 inputs response data to a self-report questionnaire for determining voice disorder using a web page, app page, program, or application related to a self-speech therapy service, and records or real-time voice speech. It may be a terminal that uploads and outputs at least one voice treatment content corresponding to the presence and degree of voice disorder analyzed by response data and voice speech.

Here, at least one user terminal 100 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a laptop, a desktop, a laptop, and the like equipped with a navigation system and a web browser. In this case, at least one user terminal 100 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one user terminal 100 is, for example, a wireless communication device that ensures portability and mobility, navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet ) may include all types of handheld-based wireless communication devices such as terminals, smartphones, smart pads, tablet PCs, and the like.

The voice therapy service providing server 300 may be a server that provides a self-speech therapy service web page, app page, program or application. In addition, the voice treatment service providing server 300 provides a self-report questionnaire for determining voice disorder, a criterion for determining whether or not there is a voice disorder according to the response data, and at least one voice treatment corresponding to the degree of voice disorder and the presence and degree of voice disorder. It may be a server that converts content into a database. When receiving response data from the user terminal 100 and receiving voice speech from the user terminal 100, the voice therapy service providing server 300 inputs it as a query to the previously built voice disorder determination algorithm, It may be a server that analyzes the presence and degree of voice disorder and outputs at least one voice treatment content corresponding to the presence and degree of voice disorder analyzed by response data and voice speech from the user terminal 100 .

Here, the voice therapy service providing server 300 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a laptop, a desktop, a laptop, and the like equipped with a navigation system and a web browser.

At least one medical staff terminal 400 may be a medical staff terminal connected to the user terminal 100 using a web page, app page, program, or application related to self-speech therapy service. In this case, at least one medical staff terminal 400 may be a terminal that runs timestamp or SBC (Server Based Computing) for personal information protection. The voice therapy service providing server 300 connected to the medical staff terminal 400 prevents leakage of personal information by preventing internal data and files from being stored in the medical staff terminal 400 . In addition, information input from at least one medical staff terminal 400 may be processed to prevent forgery or falsification after a timestamp or the like.

Here, at least one medical staff terminal 400 may be implemented as a computer capable of accessing a remote server or terminal through a network. Here, the computer may include, for example, a laptop, a desktop, a laptop, and the like equipped with a navigation system and a web browser. In this case, at least one medical staff terminal 400 may be implemented as a terminal capable of accessing a remote server or terminal through a network. At least one medical staff terminal 400 is, for example, a wireless communication device that ensures portability and mobility, and includes navigation, PCS (Personal Communication System), GSM (Global System for Mobile communications), PDC (Personal Digital Cellular), PHS (Personal Handyphone System), PDA (Personal Digital Assistant), IMT (International Mobile Telecommunication)-2000, CDMA (Code Division Multiple Access)-2000, W-CDMA (W-Code Division Multiple Access), Wibro (Wireless Broadband Internet ) may include all types of handheld-based wireless communication devices such as terminals, smartphones, smart pads, tablet PCs, and the like.

2 is a block diagram illustrating a voice therapy service providing server included in the system of FIG. 1, and FIGS. 3 and 4 describe an embodiment in which a self-speech therapy service is implemented according to an embodiment of the present invention. It is a drawing for

Referring to FIG. 2, the voice therapy service providing server 300 includes a storage unit 310, a questionnaire unit 320, a voice analysis unit 330, a self-treatment unit 340, a three-element analysis unit 350, It may include a treatment connection unit 360, a guide unit 370, a record providing unit 380, a medical staff connection unit 390, and a record protection unit 391.

The voice therapy service providing server 300 according to an embodiment of the present invention or another server (not shown) operating in conjunction with the self-voice therapy service is provided to at least one user terminal 100 and at least one medical staff terminal 400. When transmitting an application, program, app page, web page, etc., at least one user terminal 100 and at least one medical staff terminal 400 install the self-speech therapy service application, program, app page, web page, etc. or open Also, the service program may be driven in at least one user terminal 100 and at least one medical staff terminal 400 by using a script executed in a web browser. Here, the web browser is a program that allows users to use the web (WWW: World Wide Web) service, and means a program that receives and displays hypertext described in HTML (Hyper Text Mark-up Language). For example, Netscape , Explorer, Chrome, and the like. In addition, an application means an application on a terminal, and includes, for example, an app running on a mobile terminal (smart phone).

Referring to FIG. 2 , the storage unit 310 includes a self-report questionnaire for determining voice disorder, a criterion for determining whether or not there is a voice disorder according to response data, a degree of voice disorder, and at least one voice corresponding to the presence and degree of voice disorder. Treatment content can be databased. The self-report questionnaire may include a questionnaire for determining the Voice Handicap Index (VHI) and a questionnaire for determining the Korean Version of Voice-Related Quality Of Life (K-VRQOL). can The questionnaire unit 320 may receive response data from the user terminal 100 . At this time, since the above-described self-report questionnaire has already established a standard for assigning points for each item and a protocol for determining whether or not there is a voice disorder or identifying its type according to the score, it is possible to determine whether or not the voice disorder exists using this standard. database can be made. Of course, various self-report questionnaires may be used in addition to the methods described above.

When a voice speech is input from the user terminal 100, the voice analysis unit 330 may analyze the presence and degree of voice disorder by inputting it as a query to a pre-established voice disorder determination algorithm. The pre-constructed voice disorder judgment algorithm receives voice speech and evaluates voice disorder as an evaluation scale including GRBAS (Grade, Roughness, Breathiness, Asthenicity, Strain) or CAPE-V (Consensus Auditory-Perceptual Evaluation of Voice). It may be an algorithm that determines

<VRP>

Additionally, the Voice Range Profile (VRP) is one of the acoustic evaluations and is a test that can objectively measure the sound range of a specific subject. Quantitative measures measured through this can be used to suggest a starting point for singing training or to measure training performance, and are also used for diagnosing voice disorders, and are also useful for evaluating the progress through voice therapy. A recent discussion by an expert panel of the American Speech-Language-Hearing Association (ASHA) presented key tasks and measurements of the protocol for instrumental evaluation of vocal function, and it is also recommended to include a procedure for measuring the range as essential here. did That is, it is recommended to measure the minimum and maximum values of Vocal Sound Pressure Level and Vocal Frequency. They also suggested a task to hold for seconds.

VRP can be performed using a much more diverse test performance method than this, for example, from the lowest to the highest pitch, and from the smallest to the loudest by referring to individual sound stimuli in each semitone. There is a way to establish a full phonetogram by recording a range of sizes. Alternatively, in order to improve time efficiency, an abbreviated VRP method that measures the range of intensity in 7 notes for males and 9 notes for females by reflecting a wide range of voices can be used. , It is possible to use a method of starting from a comfortable pitch and volume, cascading (DeCrescendo) to the lowest and lowest sound, and then crescendo from the first sound to the largest and highest sound. In most cases, it is based on the fact that frequency and intensity are proportional, so that the phonetogram becomes an ellipse tilted upward.

Various types of VRP can evaluate the subject's vocal ability, track its change, and provide feedback to the subject. Specifically, when examining various studies related to VRP, the VRP range of patients with voice disorders decreases compared to normal groups with normal voice or increases after voice therapy using specific treatment techniques, so it has potential as a tool for diagnosis and intervention effect evaluation there is VRP is usually evaluated through vowel vocalization, which brings a limitation that it is difficult to see that various vocalization patterns are fully reflected when producing actual speech.

<SRP>

In this context, several studies may use a method to evaluate functional speech production ability through a speech range profile (SRP) or to examine voice changes before and after intervention. Due to the nature of using speech, SRP can be implemented using various tasks and materials such as counting, which is an automatic spoken language, monologue, reading sentences and paragraphs, singing, and shouting. For example, frequency and intensity measurements can be presented in a normal person using a paragraph-level new SRP speech task that includes statements, interrogative sentences, and loud shouting. Like VRP, there are differences between the patient group with voice impairment and the normal group with normal voice in terms of frequency and intensity-related variables. In addition, using the area, maximum and minimum intensity of the VRP, SF0 range of the SRP, and the maximum value of the intensity, it is possible to predict with more than 95% accuracy each voice disorder. As such, the measurement values of VRP and SRP suggest the possibility of being used as a test tool for screening voice disorders, so these two methods can be used more.

<EGG>

Average speech fundamental frequency measured by ElectroGlottoGraphy (EGG) during paragraph reading task using variables related to speech frequency measured through relatively simple VRP and SRP applied to normal adults It is also possible to build two models that can predict (Speaking Fundamental Frequency, SF0). In addition, SF0 predicted through each model of VRP and SRP is defined as ESF0VRP and ESF0SRP as estimated firing fundamental frequencies (Estimated SF0), and the absolute value of the difference between SF0EGG and ESF0 in normal people, that is, differences in firing fundamental frequencies (Differences in SF0, ΔSF0VRP and ΔSF0SRP), and the sum of the two ΔSF0s (ΔSF0SUM) can also be calculated. Through this, it is possible to present the data of each variable according to gender in normal adults. At this time, if there is a difference in ΔSF0VRP, ΔSF0SRP, and ΔSF0SUM between the patient group with voice disorder and the normal group, and if there is a difference, a database can be established to determine whether it can be used as a screening tool for voice disorder, and then it can be used for screening voice disorder. there is.

Therefore, after constructing data that can confirm whether there is a difference in ΔSF0VRP, ΔSF0SRP, and ΔSF0SUM according to gender between the patient group with voice disorder and the normal group, and whether there is a correlation between SF0EGG, ESF0VRP, and ESF0SRP, and the three ΔSF0 After confirming whether there is a correlation with the overall severity rating measured through auditory perceptual evaluation, and whether it has usefulness as a screening test tool for voice disorders, a cutoff score for distinguishing between the patient group and the normal group is established. settings and may be applied to the platform of the present invention.

The self-treatment unit 340 may output, from the user terminal 100, at least one voice treatment content corresponding to the presence and degree of voice disorder analyzed by response data and voice speech. The at least one voice therapy content may be at least one of video, animation, and cartoon content, and any one of the at least one content may be content using a teaching aid or a treatment tool. The user terminal 100 inputs response data into a self-report questionnaire for determining voice disorder, records or uploads voice speech in real time, and responds to at least One voice therapy content can be output.

The three-factor analysis unit 350 receives age and gender from the user terminal 100, and determines three factors of sound including intensity of sound, which is the intensity of voice speech input from the user terminal 100, and pitch and timbre of pitch. If the element is not included in the spectrum corresponding to the age and gender input from the user terminal 100, it can be analyzed as a voice disorder. In this case, a machine learning algorithm may be used, and feature extraction for detecting a speech impairment may be performed first. For speech recognition or speaker recognition, feature vectors representing spectral characteristics of input speech, such as MFCC (Mel-Frequency Cepstral Coefficient), are generally used, but different feature vectors are used for detection of impaired speech. In the case of dysphonia, it is difficult to maintain a constant pitch, and the change in pitch frequency tends to be severe. When pronouncing a vowel for a long time, if there is a disability, the amplitude change of the voice signal appears relatively severe because it is difficult to pronounce a certain volume. In the case of dysarthria, glottal closure is not performed periodically or accurately, so even when voiced sounds are uttered, voices similar to voiceless sounds may appear, harmonics may not be properly formed, and signals with characteristics similar to noise may be included.

In order to use these characteristics of impaired speech, Jitter calculates the rate of change of pitch, Shimmer measures the rate of change of the maximum amplitude of the voice signal, and HNR / HNR / Traditional characteristics based on NHR (Harmonics to Noise Ratio/Noise to Harmonics Ratio) and the like can be used. In addition to this, entropy measurement (Phase Permutation Entropy, PPE) for pitch stability measurement, autocorrelation measurement in non-stationary signals expressing low-frequency vibration similar to nonlinear vocal cord vibration (Detrended Fluctuation Analysis, DFA), Period shift can be measured to measure the ability of the vocal folds to maintain continuous vibration (RPDE), which can be used as a feature vector.

In one embodiment of the present invention, traditional features and recently proposed features can be used, and the features are shown in Table 1 below.

One

Jitter:

2 Jitter divided by the average value of F0 3 Perturbation Quotient for Jitter K Cycle 4 Mean value for the deviation of F0 5 Shimmer:

6 Shimmer divided by the average value of A0 7 Shimmer's Perturbation Quotient for K Cycle (K=3) 8 Shimmer's Perturbation Quotient for K Cycle (K=5) 9 Mean value for A0 deviation 10 HNR:

, Rxx (lmax): maximum autocorrelation 11 standard deviation of HNR 12 NHR:

13 standard deviation of NHR 14 Pitch Period Entropy (PPE) 15 Detrended Fluctuation Analysis (DFA) 16 Recurrence Period Density Entropy (RPDE)

1 to 4 are Jitter based features and 5 to 9 are Shimmer based features. In the following equation, Fo,i is the pitch frequency of the i-th frame, and Ao,i represents the amplitude (sum of absolute values of samples within a frame) of the i-th frame. <Feature vector scaling>

Feature vectors can be extracted through various algorithms, and therefore, each feature has a variety of ranges. For example, Jitter can be 016 to 130, and Shimmer can be 001 to 015. In many machine learning algorithms, a scaling process for features in these feature vectors is often required as a preprocessing process. In particular, in the case of machine learning in which the distance between two feature vectors is calculated as the Euclidean distance, if the distribution range of a specific feature among feature vectors is wide, the overall Euclidean distance can be determined by the feature, so the feature vector scaling process This will greatly affect performance. In addition, in the case of machine learning learning using gradient descent, whether or not each feature in the feature vector is scaled affects the convergence process of parameter learning. When learning a parameter using the gradient descent method, the feature is included in the equation for updating the parameter, so if the range between the features is different, the update rate of the parameter for each feature is different.

For a certain feature vector x={x1,x2,...,xi,...,xN}, scaling techniques that can be used as preprocessing of machine learning are as follows.

① Standardization is a process of normalizing the distribution of each feature so that the mean is 0 and the standard deviation is 1, and is converted as shown in Equation 1 below using the feature mean and standard deviation of the learning data.

② Min-max Scaling limits the range of feature distribution, and usually converts to limit the minimum value to 0 and the maximum value to 1.

③ Abs-Max Scaling restricts the distribution of the absolute value of a feature. Usually, the maximum value of the absolute value is 1. That is, it is normalized so that the maximum absolute value of the feature does not exceed 1. Unlike the previous two scaling techniques, since a constant is not subtracted from the feature, the sign of the feature does not change after scaling.

④ Normalization normalizes the feature vector so that its norm is 1. For this normalization, L2 Norm can be used when the dot product is used to measure the distance between two feature vectors, and L1 Norm can be used when measuring the distance from the feature representing the histogram to the Manhattan distance.

Which of the scaling techniques listed above should be used is not determined solely by machine learning methods. In general, the selection of a scaling technique is determined experimentally, since the effects of scaling techniques vary depending on the characteristics of features along with the type of machine learning.

<Machine learning method>

KNN (K-Nearest Neighbors) is a classification method that does not use a mathematical model. After selecting K feature vectors of the training data that are closest to the feature vectors of the test data, they are assigned to the class to which the largest number of data belongs. . Support Vector Machine (SVM) classifies two classes by finding the linear function that maximizes the margin between classes. The kernel method can be introduced to solve nonlinear problems. A decision tree is a method of classifying a class at the final node by branching into different branches according to the answer to the question. CART (Classification and Regression Trees) learning using statistical methods was applied. A random forest is an extension of the decision tree, which randomly extracts a data set from the entire training data to generate several trees, and then determines the classification result using the bootstrap aggregating (bagging) method. Adaboost is a method of combining simple classifiers to create a high-performance classifier, and learns weights according to classification performance. Naive Bayes is a classifier that creates and combines Bayes classifiers for each feature under the assumption that the features are independent of each other. Neural Networks is an artificial neural network classifier that uses a multilayer perceptron that applies a structure with a hidden layer in addition to an input/output layer. LDA (Linear Discriminant Analysis) determines a linear boundary surface by assuming that the two classes each follow a Gaussian distribution and that the variance matrix is a diagonal matrix. QDA (Qudratic Discriminant Analsysis), unlike LDA, does not apply the assumption that the variance matrix is a diagonal matrix. After performing modeling through learning and verification using all of the above-described machine learning, machine learning with the highest accuracy can be set as the algorithm of the present invention.

The treatment connection unit 360 builds at least one treatment institution and a pool of medical staff, and treats the voice disorder when it is classified as a voice disorder in response to the response data and voice speech input from the user terminal 100. Information on at least one treatment institution and medical staff may be provided or analysis results may be linked. When outputting at least one voice treatment content, the guidance unit 370 may receive voice speech input from the user terminal 100 in real time, analyze the voice speech, and then output voice speech instructions.

The recording provider 380 analyzes the response data and voice utterance of the user terminal 100 as a voice disorder, performs a self-voice treatment process with the voice treatment contents in the user terminal 100, and then records the user terminal 100 When requesting the analysis and treatment history, the analysis and treatment history may be transmitted to the user terminal 100 . When the user terminal 100 requests connection with at least one medical staff terminal, the medical staff connection unit 390 may connect the user terminal 100 and at least one medical staff terminal. If non-face-to-face care is possible, it can be provided. The record protection unit 391 may be configured to perform server-based computing or virtual desktop infrastructure when at least one medical staff terminal outputs the analysis and treatment history of the user terminal 100. there is.

Hereinafter, an operation process according to the configuration of the above-described voice therapy service providing server of FIG. 2 will be described in detail with reference to FIGS. 3 and 4 as examples. However, it will be apparent that the embodiment is only any one of various embodiments of the present invention, and is not limited thereto.

Referring to FIG. 3, (a) the voice therapy service providing server 300 builds a self-diagnosis questionnaire and a database for a program or algorithm for voice analysis. And, (b) when the user terminal 100 responds to the self-diagnostic questionnaire and utters voice speech, it is analyzed to provide a diagnosis result, and a voice treatment process or content is provided so that self-diagnosis can be performed. (c) When requested by the user terminal 100, the voice therapy service providing server 300 may link with at least one medical staff terminal 400 or a medical institution or treatment institution. A platform page as shown in FIG. 4a may be provided, and a self-diagnosis type questionnaire as shown in FIGS. 4b to 4q or a measurement tool capable of determining voice speech may be provided. Of course, the questionnaire or measurement tool is not limited to the above.

Matters not described in the self-voice therapy service provision method of FIGS. 2 to 4 are the same as or easily inferred from the description of the self-voice therapy service provision method through FIG. Omit the explanation.

5 is a diagram illustrating a process of transmitting and receiving data between components included in the self-voice therapy service providing system of FIG. 1 according to an embodiment of the present invention. Hereinafter, an example of a process of transmitting and receiving data between each component will be described through FIG. 5, but the present application is not limited to such an embodiment, and according to various embodiments described above, It is obvious to those skilled in the art that a process of transmitting and receiving data may be changed.

Referring to FIG. 5, the voice therapy service providing server provides a self-report questionnaire for determining voice disorder, a criterion for determining whether or not there is a voice disorder according to response data, a degree of voice disorder, and at least one voice corresponding to whether and how much of a voice disorder. The treatment contents are made into a database (S5100), and response data is input from the user terminal (S5200).

In addition, when the voice treatment service providing server receives voice speech from the user terminal, it is input as a query to the previously built voice disorder determination algorithm to analyze whether and how much the voice disorder exists (S5300), and the response data and voice At least one voice treatment content corresponding to whether and how much the speech disorder is analyzed is output from the user terminal (S5400).

The order between the above-described steps (S5100 to S5400) is only an example, and is not limited thereto. That is, the order of the above-described steps (S5100 to S5400) may be mutually changed, and some of the steps may be simultaneously executed or deleted.

Matters not explained about the self-voice therapy service provision method of FIG. 5 are the same as or easily inferred from the description of the self-voice therapy service provision method through FIGS. 1 to 4. Omit the explanation.

The self-speech therapy service providing method according to an embodiment described with reference to FIG. 5 may be implemented in the form of a recording medium including instructions executable by a computer, such as an application or program module executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. Also, computer readable media may include all computer storage media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data.

The above-described method for providing a self-speech therapy service according to an embodiment of the present invention can be executed by an application basically installed in a terminal (this may include a program included in a platform or operating system basically installed in the terminal), and , It may be executed by an application (that is, a program) that the user directly installs in the master terminal through an application providing server such as an application store server, an application or a web server related to the corresponding service. In this sense, the above-described method for providing a self-speech therapy service according to an embodiment of the present invention is implemented as an application (ie, a program) that is basically installed in a terminal or directly installed by a user, and is readable by a computer such as a terminal. It can be recorded on a recording medium.

The above description of the present invention is for illustrative purposes, and those skilled in the art can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, the embodiments described above should be understood as illustrative in all respects and not limiting. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is indicated by the following claims rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and equivalent concepts should be construed as being included in the scope of the present invention. do.

Claims (10)

Entering response data into a self-report questionnaire for determining voice disorder, recording or uploading voice speech in real time, and at least one voice treatment content corresponding to the presence and degree of voice disorder analyzed by the response data and voice speech a user terminal that outputs; and
A self-report questionnaire for determining voice disorder, a criterion for determining voice disorder according to response data and a degree of voice disorder, a storage unit that compiles at least one voice treatment content corresponding to the presence and degree of voice disorder into a database, the user terminal A questionnaire unit for receiving the response data from the user terminal, a voice analysis unit for inputting a query into a pre-established speech impairment determination algorithm to analyze the presence and degree of the speech impairment when receiving voice speech from the user terminal; A self-treatment unit that outputs at least one voice treatment content corresponding to the presence and degree of voice disorder analyzed by the response data and voice speech from the user terminal; receives input of age and gender from the user terminal; Three-factor analysis that analyzes as a voice disorder if the three elements of sound, including the intensity of the input voice speech, the pitch and tone, are not included in the spectrum corresponding to the age and gender input from the user terminal At least one treatment institution and a pool of medical staff are established, and if the voice disorder is classified in response to response data and voice speech input from the user terminal, the at least one treatment institution to treat the voice disorder, and A treatment linkage unit that provides information on medical staff or links analysis results, and when the user terminal requests a connection with at least one medical staff terminal, connects the user terminal and at least one medical staff terminal, enabling non-face-to-face treatment In this case, when outputting the analysis and treatment history of the user terminal in the medical staff connection unit that provides connection with the medical staff terminal, the at least one medical staff terminal proceeds with Server Based Computing or Desktop Virtualization (Virtual Desktop Infrastructure). a voice therapy service providing server including a record protection unit set to do so;
including,
The three element analysis unit,
When analyzing voice disorders, a machine learning algorithm is used, but feature extraction is performed to detect voice disorders,
In feature extraction, MFCC (Mel-Frequency Cepstral Coefficient), which is a feature vector representing the spectral characteristics of the input speech, is used,
In the case of dysphonia, it is difficult to maintain a constant pitch, so the change in pitch frequency is relatively strong compared to the case without the disorder. In the case of dysarthria, the glottal closure is not performed periodically, so even when voiced sounds are uttered, vocalizations similar to voiceless sounds appear, or harmonics are not formed and signals with characteristics similar to noise are included. In order to detect impaired voice using voice characteristics,
As feature vectors different from the MFCC, a jitter that calculates the rate of change of pitch, a shimmer that measures the rate of change of the maximum amplitude of the voice signal, and HNR that calculates the average and standard deviation by calculating the ratio of harmonic components and noise according to voiced speech A feature vector based on /NHR (Harmonics to Noise Ratio/Noise to Harmonics Ratio) is used, but additionally, phase permutation entropy (PPE) for measuring pitch stability and low-frequency vibration similar to nonlinear vocal cord vibration are used. The data obtained by measuring the autocorrelation in the non-stationary signal (Detrended Fluctuation Analysis, DFA) and measuring the ability to maintain continuous vibration of the vocal cords by measuring the periodic variation of vibration (RPDE) are used as feature vectors. ,
Thereafter, preprocessing is performed by selecting and applying a specific scaling technique among a plurality of scaling techniques to the features in the feature vectors extracted by a plurality of machine learning algorithms, and scaling according to the type of machine learning and the property of the feature at the time of selection. Considering that the effect of the application of the technique is different, preprocessing is performed by applying the experimentally determined scaling technique among a plurality of scaling techniques,
In the plurality of scaling techniques,
Standardization, the first technique that normalizes the distribution of each feature so that the mean is 0 and standard deviation is 1, and Min-max Scaling, the second technique that limits the range of the feature distribution so that the minimum value is 0 and the maximum value is 1. , Abs-Max Scaling, which is a third technique in which the absolute value distribution of the feature is restricted so that the maximum value of the absolute value is 1, and the sign of the feature does not change after scaling, and normalization so that the norm of the feature vector is 1, In normalization, L2 Norm is used when the dot product is used to measure the distance between two feature vectors, and L1 Norm is used when the distance is measured with the Manhattan distance from the feature representing the histogram. Normalization, which is a fourth technique, is provided,
In the plurality of machine learning algorithms,
As a classification method that does not use a mathematical model, the first algorithm KNN (K-Nearest Neighbors ),
Support Vector Machine (SVM), a second algorithm that classifies two classes by finding a linear function that maximizes the margin between classes;
A decision tree, which is a third algorithm that branches to different branches according to the answer to the question and classifies the class at the final node;
CART (Classification and Regression Trees), which is a fourth algorithm using statistical methods;
A fifth algorithm, Random Forest, which randomly extracts a data set from all training data to create several trees and then determines the classification result using a bagging (Bootstrap Aggregating, Bagging) method;
Adaboost, a sixth algorithm that learns weights according to classification performance;
Naive Bayes, the seventh algorithm, which is a classifier that creates and combines Bayes classifiers for each feature under the assumption that the features are independent;
Neural Networks, an eighth algorithm using a multilayer perceptron that applies a structure with a hidden layer in addition to an input/output layer as an artificial neural network classifier;
LDA (Linear Discriminant Analysis), which is a ninth algorithm for determining a linear boundary surface by assuming that the two classes each follow a Gaussian distribution and the variance matrix is a diagonal matrix, and
It includes a 10th algorithm, Quadratic Discriminant Analsysis (QDA), which does not apply the assumption that the variance matrix is a diagonal matrix,
The three-factor analysis unit performs modeling through learning and verification using all of the first to tenth algorithms, and then uses the machine learning algorithm that has the highest accuracy among the first to tenth algorithms for the voice therapy service. Set to the machine learning algorithm used in the providing server,
The self-report questionnaire,
Including a questionnaire for determining the Voice Handicap Index (VHI) and a questionnaire for determining the Korean Version of Voice-Related Quality Of Life (K-VRQOL),
It is databased to determine whether or not there is a voice disorder by using a protocol to determine whether or not there is a voice disorder or to identify the type according to the criteria and scores for assigning scores for each item that have been established,
The questionnaire for determining the voice-related quality of life is prepared so that responses to each question are made in consideration of how the voice has been for the past two weeks in order to confirm the effect of voice problems on daily life,
Item 1 corresponds to Difficulty speaking loudly in noisy situations because of the voice or it is difficult for others to understand me, Item 2 corresponds to shortness of breath when speaking and shortness of breath. Question 3 is "It is difficult to predict whether will come out", Question 4 is "Sometimes anxious or embarrassed because of my voice", Question 5 is "Sometimes I get depressed because of my voice", Question 5 is "I have difficulty talking on the phone because of my voice" Item 6 pertains, Item 7 pertains to I have difficulties while doing my job or professional work because of my voice, Item 8 pertains to I avoid social life, I repeat words so that others can understand. Including item 9 corresponding to I have to and item 10 corresponding to I am less extroverted because of my voice,
The medical staff terminal is a terminal that runs timestamp or SBC (Server Based Computing) for personal information protection,
The voice therapy service providing server connected to the medical staff terminal prevents leakage of personal information by preventing internal data and files from being stored in the medical staff terminal, but processing is performed to prevent forgery and alteration of information input from the medical staff terminal,
A system for providing a self-speech therapy service that links at least one medical staff terminal or medical institution or treatment institution upon request from a user terminal.
delete According to claim 1,
The pre-constructed voice disorder determination algorithm,
Characterized in that it is an algorithm that receives the speech speech and determines the voice disorder by the speech speech with an evaluation scale including GRBAS (Grade, Roughness, Breathiness, Asthenicity, Strain) or CAPE-V (Consensus Auditory-Perceptual Evaluation of Voice) A self-speech therapy service providing system.
delete According to claim 1,
The at least one voice therapy content,
At least one of video, animation, and cartoon content;
Self-speech therapy service providing system, characterized in that any one of the at least one content is content using a teaching aid or a treatment tool.
delete According to claim 1,
The voice therapy service providing server,
When outputting the at least one voice treatment content, a guidance unit that receives voice speech input from the user terminal in real time, analyzes the voice speech, and then outputs voice speech instructions;
Self-speech therapy service providing system further comprising a.
According to claim 1,
The voice therapy service providing server,
When the response data and voice speech of the user terminal are analyzed as a voice disorder, the user terminal performs a self-voice treatment process with the voice treatment contents, and then the user terminal requests analysis and treatment history, the user terminal a record providing unit that transmits the analysis and treatment history to;
Self-speech therapy service providing system further comprising a.
delete delete

Images