KR102290233B1 - System and Method for Analyzing Vocal Cord Condition - Google Patents

Abstract

The present invention relates to a method and system for analyzing the state of the vocal cords, and more particularly, to a method and system for analyzing the state of the user's vocal cords based on the user's biometric information.
The disclosed vocal cord state analysis method includes an operation of obtaining the user's biometric information and user information, an operation of storing the obtained biometric information and user information into a database, and determining the user's vocal cord state based on the obtained biometric information It can include actions.
According to the present invention, it is possible to reduce the user's use of the vocal cords by providing a notification to the user when the vocal cords are used a lot by analyzing the user's vocal cord use state.

Description

Vocal cord condition analysis method and system {System and Method for Analyzing Vocal Cord Condition}

The present invention relates to a method and system for analyzing vocal cord condition.

Teachers, instructors, singers, etc. who use the vocal cords professionally may have a disease called vocal cord nodules in which the vocal cords are formed due to voice abuse or excessive vocalization.

And for people who use their vocal cords professionally, there are studies that show that experiencing vocal cord problems not only affects their jobs, but can also be a source of stress.

These vocal cord-related problems can be prevented by reducing the use of the vocal cords by not speaking or whispering, and by taking rest.

However, for those who use vocal cords professionally, it is not easy to take preventive measures to reduce the use of the vocal cords or to take rest, and there are cases where they unconsciously use the vocal cords a lot.

Therefore, there is a need to suggest a way to reduce the use of vocal cords to professional vocal cord users.

Republic of Korea Patent Publication No. 10-1547990 (Registered on Aug. 21, 2015)

SUMMARY OF THE INVENTION It is an object of the present invention to provide a method and system for analyzing the use of the vocal cords so that users who use the vocal cords can continuously grasp the state of their vocal cords and manage them.

According to various embodiments of the present invention, the apparatus for analyzing a vocal cord state proposed by the present invention includes a communication module for acquiring biometric information and user information of a user, a data acquisition module for storing the acquired biometric information and user information into a database, and the acquired biometric information and user information. and a vocal cord state determination module configured to determine the user's vocal cord state based on biometric information.

According to various embodiments, the vocal cord state determination module may determine whether the user uses the vocal cords based on the biometric information, and determine the user's vocal cord state based on a result of determining whether or not to use the vocal cords a plurality of times.

According to various embodiments, the vocal cord state determination module determines whether the user's vocal cords are used based on the biometric information at a first time interval, and at a second time interval longer than the first time interval, the user's vocal cords A state parameter indicating the state of the user's vocal cords may be updated based on the results of determining whether or not to use the user.

According to various embodiments, the vocal cord state determination module adds 1 to the state parameter if more than 2/3 of the results of determining whether or not to use the vocal cords determined during the second time interval indicate that the vocal cords are used, and 2/3 or less In this case, the state parameter may be reset to 0.

According to various embodiments, the vocal cord state determination module may determine that the user's vocal cords have been used for a long time if the state parameter is greater than or equal to a preset value, and display a notification message to the user.

According to various embodiments, the vocal cord state determination module may determine whether the user uses the vocal cords by using a machine learning algorithm using the biometric information as learning data.

According to various embodiments, the machine learning algorithm may be at least one of Gaussian Naive Bayes (GNB), Support Vector Machine (SVM), Decision Tree (DT), and K-Nearest Neighbors (KNN).

According to various embodiments, the user's biometric information includes a photoplethysmography (PPG) sensor for measuring heartbeat, an electromyogram (EMG) sensor for measuring the user's electromyography, and a galvanic skin response (GSR) sensor for measuring the user's skin conductivity. ) sensor and the user's biometric information obtained by using at least one of a MIC sensor for measuring voice.

According to various embodiments, the method for analyzing a vocal cord state includes an operation of acquiring the user's biometric information and user information, an operation of storing the acquired biometric information and user information into a database, and the user based on the acquired biometric information It may include an operation of determining the state of the vocal cords.

According to various embodiments, the operation of determining the state of the user's vocal cords based on the obtained biometric information includes the operation of determining whether the user uses the vocal cords based on the biometric information and the result of determining whether the vocal cords are used a plurality of times. It may include an operation of determining the state of the user's vocal cords based on the.

According to various embodiments, the operation of determining the state of the user's vocal cords based on the obtained biometric information includes determining whether the user uses the vocal cords based on the biometric information at a first time interval, The operation of determining the state of the user's vocal cords based on the result of determining whether or not to use the vocal cords a plurality of times includes determining the state of the user's vocal cords based on the results of determining whether the user uses the vocal cords at every second time interval longer than the first time interval. It may include an operation of updating a state parameter indicating

According to various embodiments, the updating of the state parameter indicating the state of the user's vocal cords based on the results of determining whether the user has used the vocal cords at each second time interval longer than the first time interval may include the second time interval. The method may include an operation of adding 1 to the state parameter if more than 2/3 of the results of determining whether or not to use the vocal cords have been used, and resetting the state parameter to 0 if it is less than 2/3 of the results of determining whether or not to use the vocal cords.

According to various embodiments, the method may further include determining that the user's vocal cords have been used for a long time if the state parameter is equal to or greater than a preset value, and displaying a notification message to the user.

According to various embodiments, determining whether the user uses the vocal cords based on the biometric information includes determining whether the user uses the vocal cords using a machine learning algorithm using the biometric information as learning data can do.

According to various embodiments, the machine learning algorithm may be at least one of Gaussian Naive Bayes (GNB), Support Vector Machine (SVM), Decision Tree (DT), and K-Nearest Neighbors (KNN).

According to various embodiments, the user's biometric information includes a photoplethysmography (PPG) sensor for measuring heartbeat, an electromyogram (EMG) sensor for measuring the user's electromyography, and a galvanic skin response (GSR) sensor for measuring the user's skin conductivity. ) sensor and the user's biometric information obtained by using at least one of a MIC sensor for measuring voice.

According to various embodiments, the vocal cord state analysis system receives from the sensing device for obtaining the user's biometric information, the app device for generating an ontology combining the biometric information obtained from the sensing device and the user information of the user, and the app device It may include a server that extracts the biometric information and the user information from one ontology, stores it in a database, and determines the user's vocal cord state based on the biometric information.

According to various embodiments, the sensing device includes a photoplethysmography (PPG) sensor for measuring the user's heartbeat, an electromyogram (EMG) sensor for measuring the user's electromyography, and galvanic skin (GSR) for measuring the user's skin conductance. response) a sensor module including at least one of a sensor and a MIC sensor for measuring voice, and a communication module for communicating with the app device.

According to various embodiments, the app device generates an ontology by combining a communication module for communicating with the sensing device and the server, biometric information received from the sensing device, and user information generated by the app device. It may include a ridge generation module and a state notification module for notifying the user by receiving the state of the vocal cords of the user from the server through the communication module.

According to various embodiments, the server includes a communication module for communicating with the app device, a data acquisition module for acquiring the biometric information and user information of the user from the app device through the communication module, and a database for storing it; and a vocal cord state determination module configured to determine the user's vocal cord state based on the obtained biometric information.

According to the present invention, it is possible to provide a notification to the user when the vocal cords are used a lot by analyzing the user's vocal cord usage state.

According to the present invention, it is possible to reduce the user's use of the vocal cords by providing a notification to users who use the vocal cords a lot.

1 is a diagram illustrating a vocal cord condition analysis system according to various embodiments.
2 is a diagram illustrating a configuration of a sensing device according to various embodiments of the present disclosure;
3 is a diagram illustrating a configuration of an app device according to various embodiments of the present disclosure;
4 is a diagram illustrating a configuration of a server according to various embodiments of the present disclosure;
FIG. 5 is a diagram illustrating a data structure stored in a database of the data acquisition module 120 according to an exemplary embodiment.
6 is a flowchart illustrating a vocal cord state analysis method according to various embodiments.

In order to clearly explain the present invention, parts irrelevant to the description are omitted, and the same reference numerals are used for the same or similar elements throughout the specification.

Throughout the specification, when a part is "connected" with another part, this includes not only the case of being "directly connected" but also the case of being "electrically connected" with another element interposed therebetween. . In addition, when a part "includes" a certain component, this means that other components may be further included rather than excluding other components unless otherwise stated.

When a part is referred to as being “above” another part, it may be directly on top of the other part, or the other part may be involved in between. In contrast, when a part refers to being "directly above" another part, no other part is involved in between.

Terms such as first, second and third are used to describe, but are not limited to, various parts, components, regions, layers and/or sections. These terms are used only to distinguish one part, component, region, layer or section from another part, component, region, layer or section. Accordingly, a first part, component, region, layer or section described below may be referred to as a second part, component, region, layer or section without departing from the scope of the present invention.

The terminology used herein is for the purpose of referring to specific embodiments only, and is not intended to limit the present invention. As used herein, the singular forms also include the plural forms unless the phrases clearly indicate the opposite. The meaning of "comprising," as used herein, specifies a particular characteristic, region, integer, step, operation, element and/or component, and includes the presence or absence of another characteristic, region, integer, step, operation, element and/or component. It does not exclude additions.

Terms indicating a relative space such as “below” and “above” may be used to more easily describe the relationship of one part shown in the drawings to another part. These terms, along with their intended meanings in the drawings, are intended to include other meanings or operations of the device in use. For example, if the device in the figures is turned over, some parts described as being "below" other parts are described as being "above" other parts. Thus, the exemplary term “down” includes both the up and down directions. The device may be rotated 90 degrees or at other angles, and terms denoting relative space are interpreted accordingly.

Although not defined otherwise, all terms including technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the art to which the present invention belongs. Commonly used terms defined in the dictionary are additionally interpreted as having a meaning consistent with the related technical literature and the presently disclosed content, and unless defined, they are not interpreted in an ideal or very formal meaning.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those of ordinary skill in the art can easily carry out the embodiments of the present invention. However, the present invention may be embodied in various different forms and is not limited to the embodiments described herein.

1 is a diagram illustrating a vocal cord condition analysis system according to various embodiments.

Referring to FIG. 1 , the vocal cord state analysis system may include a server 100 , an app device 200 , and a sensing device 300 .

The sensing device 300 may acquire biometric information from a user who has attached the sensing device. In addition, a plurality of pieces of biometric information obtained may be transmitted to the app device 200 through the communication module.

The app device 200 may generate an HBT ontology based on a plurality of biometric information and user information received from the sensing device 300 . In addition, the app device 200 may deliver the generated HBT ontology to the server 100 .

The server 100 may detect the user's voice use state based on the biometric information received from the app device 200 . In addition, the server 100 may deliver a warning notification to the app device 200 when the user's voice use state continues for a predetermined time or more. Upon receiving this, the app device 200 may provide a message to the user to stop using voice and take a break as a push message.

2 is a diagram illustrating a configuration of a sensing device 300 according to various embodiments of the present disclosure.

The sensing device 300 may be in the form of a string or a band, and may be attached to the user's neck as shown in FIG. 1 to obtain biometric information. In addition, the sensing device 300 may be provided with a Velcro so that the size can be adjusted to fit the user's neck.

Referring to FIG. 2 , the sensing device 300 detects at least one of a photoplethysmography (PPG) sensor 311 , an electromyogram (EMG) sensor 313 , a galvanic skin response (GSR) sensor 315 and a MIC sensor 317 . It may include a sensor module 310 that includes, a communication module 320 and a processor 330 for communicating with the app device 200 .

The PPG sensor 311 is referred to as a pulse wave sensor, and may measure a user's heart rate by irradiating infrared or LED green light to blood vessels around the neck and detecting light reflected through a photodiode.

The EMG sensor 313 may measure a movement of a muscle that occurs when a voice is used around the user's neck.

The GSR sensor 315 measures the conductivity of the skin, and by using it, the user's sweat output and body temperature change can be measured. The measurement result of the GSR sensor 315 may determine whether the user is in an excited state.

The MIC sensor 317 may measure a user's sound.

The processor 330 controls at least one sensor provided in the sensor module 310 to obtain the user's biometric information, and may receive the acquired biometric information from the sensor module 310 . In addition, the processor 330 may transmit the collected biometric information to the app device 200 through the communication module 320 . The processor 330 may additionally include a memory to temporarily store biometric information.

The communication module 320 may establish a communication connection with the app device 200 and transmit biometric information to the app device 200 under the control of the processor 330 . The communication module 320 may be implemented as one communication method among various wired communication methods such as a USB communication method, a UART communication method, and an I2C communication method as well as a wireless communication method such as a Bluetooth communication method, a Zigbee communication method, and a WiFi communication method. .

3 is a diagram illustrating a configuration of an app device 200 according to various embodiments of the present disclosure.

Referring to FIG. 3 , the app device 200 may include a communication module 210 , an ontology generating module 220 , and a status notification module 230 .

The communication module 210 may include a first communication module 211 for communicating with the sensing device 300 and a second communication module 213 for communicating with the server 100 .

The first communication module 211 may support the same communication method as the communication method provided in the sensing device 300 , and thus not only wireless communication methods such as Bluetooth communication method, Zigbee communication method, and WiFi communication method, but also USB communication method and UART communication method. One communication method among various wired communication methods such as a communication method and an I2C communication method may be used.

The second communication module 213 may use at least one of a long-term evolution (LTE) communication method, a Wi-Fi communication method, and an Ethernet communication method to communicate with the server 100 .

The first communication module 211 may communicate with the sensing device 300 and receive the user's biometric information from the sensing device 300 . The first communication module 211 may transmit the received biometric information to the ontology generating module 220 .

The ontology generation module 220 may integrate the biometric information received from the sensing device 300 through the first communication module 211 with the user information to form an HBT. In this case, the user information may include location information of the user. The location information may be acquired using GPS additionally included in the app device 200 . Also, the ontology generating module 220 may transmit the ontology generated in the HBT format to the second communication module 212 in order to transmit it to the server 100 . The second communication module 212 may transmit the received ontology to the server 100 according to a communication method connected to the server 100 .

The status notification module 230 may visually or aurally output the current status of the user received from the server 100 through the second communication module 212 in order to recognize the user's current status. In an embodiment, the state notification module 230 may express the user's vocal cord state using a picture so that the user can intuitively know. That is, the status notification module 230 may display blue when the status is good and black or red when the status is bad so that the user can intuitively know the current vocal cord status. In another embodiment, the status notification module 230 displays a push message such as “Stop speaking and drink water” to the user on the screen of the app device 200 so that the user can recognize that the vocal cord condition has deteriorated. can do. In another embodiment, the status notification module 230 may notify the user of the current status status by providing a voice message such as “Stop speaking and drink water” to the user, a beat sound, or a preset sound.

The app device 200 may operate as an app device by installing the application app according to the present invention in a typical smart phone, but is not limited thereto, and may be a laptop, PDA, tablet, or PC capable of performing the above-described functions. It may also operate as the app device 200 .

4 is a diagram illustrating a configuration of the server 100 according to various embodiments of the present disclosure.

Referring to FIG. 4 , the server 100 may include a communication module 110 , a data acquisition module 120 , and a vocal cord state determination module 130 .

The communication module 110 is for communicating with the app device 200 . The communication module 110 is a counterpart of the second communication module 213 provided in the app device 200 and may use a wired communication method such as a local area network (LAN) or an Ethernet communication method as a lower layer protocol.

The communication module 110 may use a hypertext transfer protocol (HTTP) as an upper layer protocol, and may receive the HBT ontology generated by the app device 200 in a POST form. The communication module 110 may transmit the received HBT ontology to the data acquisition module.

The data acquisition module 120 may store the HBT ontology information received through the communication module 110 as a database. And the data acquisition module 120 may deliver the biometric information included in the HBT ontology according to the request of the vocal cord state determination module 130 .

The vocal cord state determination module 130 may determine whether the user uses the vocal cords based on the received biometric information. In an embodiment, the vocal cord state determination module 130 may determine whether the user uses the vocal cords using a learning algorithm. Vocal cord state determination module 130 collects data when the vocal cords are used and data when the vocal cords are not used, and determines whether to use the vocal cords through learning based on the data collected using a machine learning algorithm. can The vocal cord state determination module 130 may store the determination result in the database of the data acquisition module 120 .

In addition, the vocal cord state determination module 130 may determine the vocal cord state by analyzing the determination result. In an embodiment, the vocal cord state determination module 130 may determine that the vocal cord state should be managed when it appears that the user is continuously using the vocal cords for a predetermined time or more. The vocal cord state determination module 130 may deliver a notification message to the app device 200 to notify the user based on the result.

FIG. 5 is a diagram illustrating a data structure stored in a database of the data acquisition module 120 according to an exemplary embodiment.

Referring to FIG. 5 , the data structure stored in the database of the data acquisition module 120 includes an ID field 510 , an acquisition time field 520 , an acquisition location field 530 , a collection type field 540 , and biometric information. It may include a field 550 , a result field 560 , and an accuracy field 570 .

The data acquisition module 120 includes an ID field 510, an acquisition time field 520, an acquisition location field 530, a collection type field 540, and a biometric information field ( 550) can be extracted and stored.

The ID field 510 may be used to identify the app device 200 that has transmitted biometric information. When a plurality of app devices 200 are connected to the server 100 to track the user's vocal cord usage, it is necessary to classify data coming from each app device 200 . To this end, the value of the ID field 510 may be used so that the vocal cord state determination module 130 recognizes from which app device 200 the data of the corresponding line comes from. Accordingly, the ID recorded in the ID field 510 may correspond to the app device 200 and may also correspond to a user who attaches the sensing device 300 connected to the app device 200 . The ID recorded in the ID field 510 may be received by being included in the HBT ontology to recognize the transmitted app device 200, and the HBT ontology from the same app device 200 may include the same ID. .

In the acquisition time field 520 , a time at which biometric information is acquired from the sensor may be recorded. In one embodiment, since the acquisition of biometric information is performed periodically at regular time intervals, the biometric information acquisition time of biometric information having the same ID may increase at regular time intervals, and the biometric information acquisition time may be determined in the acquisition time field ( 520) can be recorded.

In the acquisition location field 530, a location where biometric information is acquired is recorded. According to an embodiment, the location may be obtained using GPS provided in the app device 200 .

The collection type field 540 may indicate the type of acquired biometric information. According to an embodiment, it may indicate which sensor the acquired biometric information is obtained by using. The sensing device 100 of FIG. 2 includes four sensors as an embodiment. All of the four sensors may acquire and transmit biometric information, or three sensors, two sensors, or one of the four sensors. It is also possible to deliver biometric information obtained from the sensor of Accordingly, the collection type field 540 determines whether the acquired biometric information 550 is biometric information collected from the PPG sensor 311 , biometric information collected from the EMG sensor 313 , or biometric information collected from the GSR sensor 315 . It may indicate whether it is information, whether it is biometric information collected by the MIC sensor 317 , or whether it is biometric information collected from all four sensors.

The biometric information field 550 records biometric information collected by sensors provided in the sensing device 300 . The length of the biometric information field 550 may vary depending on how many sensors are used. When the biometric information field 550 includes biometric information collected from a plurality of sensors, the biometric information field 550 includes a sensor name or a biometric information name (eg, heartbeat) obtained from measurement results of the sensor and the obtained biometric information field 550 . The sensor value may be recorded in the form of 'name', 'value', 'name' and 'value' as shown in FIG. 5 .

The data acquisition module 120 may receive a result indicating whether the vocal cords are being used, which is determined by the vocal cord state determination module 130 based on the biometric information recorded in the biometric information field 550 , and may be stored in the result field 560 . In an embodiment, when the vocal cords are used, it may be recorded as 1, and when the vocal cords are not used, it may be recorded as 0. Then, a value indicating the accuracy of the result in the result field 560 may be recorded in the accuracy field 570 . In one embodiment, when the value of the result field 560 is 1 and the value of the accuracy field 570 is 89, it indicates that the probability that the user is using the vocal cords is 89%. In another embodiment, when the value of the result field 560 is 0 and the value of the accuracy field 570 is 96, it indicates that the probability that the user is not using the vocal cords is 96%.

The vocal cord state determination module 130 may determine the vocal cord state by tracking the result value in the result field 560 . In an embodiment, if it is determined that the user is continuously using the vocal cords for a predetermined time or more based on the result value, it may be determined that the vocal cord state should be managed. In an embodiment, the vocal cord state determination module 130 may request the data acquisition module 120 for the value of the result field 560 every 5 minutes. Then, the data acquisition module 120 may transmit the value of the result field 560 within 5 minutes before the request time to the vocal cord state determination module 130 . The vocal cord state determination module 130 sets 1 to the state parameter when the result indicating that the vocal cords are used among the values of the received result field 560 (ie, having a value of 1 according to an embodiment) is 2/3 or more. In addition, if it is less than 2/3, the state parameter can be reset to zero. Alternatively, according to another embodiment, the vocal cord state determination module 130 adds 1 to the state parameter if the result indicating that the vocal cords were used is 2/3 or more, and resets the state parameter to 0 if it is less than 1/3, and the middle If this is the case, the state parameter can be maintained as it is. The vocal cord state determination module 130 may update the value of the above-described state parameter by requesting the value of the result field 560 every 5 minutes.

The vocal cord state determination module 130 may determine that the vocal cord state is not good if the state parameter is greater than or equal to N (one embodiment 5). According to an embodiment, if it is determined that the user has spoken continuously for 5N minutes (an embodiment: 25 minutes) or more, it may be determined that the vocal cord condition is not good. When it is determined that the vocal cord state is not good, the vocal cord state determination module 130 may deliver a notification message to the app device 200 . Then, the app device 200 may notify the user that the vocal cord condition is not good using visual and/or auditory effects.

6 is a flowchart 600 illustrating a method for analyzing a vocal cord state, according to various embodiments.

Referring to FIG. 6 , in operation 610 , in the vocal cord state analysis method, the user's biometric information may be acquired by the sensing device 300 . The sensing device 300 may include at least one of a PPG sensor 311 , an EMG sensor 313 , a GSR sensor 315 , and a MIC sensor 317 , using the sensors to obtain user's biometric information. can

According to various embodiments, in operation 630, the vocal cord state analysis method may determine whether the user uses the vocal cords based on the acquired biometric information of the user. In an embodiment, whether the user uses the vocal cords may be determined using a machine learning algorithm. To this end, the machine learning algorithm collects biometric information when the vocal cords are used and biometric information when the vocal cords are not used for learning, and determines whether or not the user uses the vocal cords based on the learning results and current biometric information. can

According to various embodiments, in operation 650 , the vocal cord state analysis method may determine the vocal cord state based on whether or not the vocal cords are used, which is determined based on the user's biometric information. In an embodiment, if it appears that the user is continuously using the vocal cords for a certain period of time or more, it may be determined that the vocal cord state requires management. Specifically, the result of determining whether to use the vocal cords based on the user's biometric information at a predetermined first time interval (eg, 10 seconds) may be stored. In addition, the state parameter indicating the state of the user's vocal cords may be updated based on the stored results of determining whether to use the vocal cords every second time interval (eg, 5 minutes) longer than the first time interval. As an embodiment, if more than 2/3 of the results of determining whether or not to use the vocal cords during the second time interval indicate that the vocal cords are used, 1 may be added to the state parameter, and if less than 2/3, the state parameter may be reset to 0. In another embodiment, among the results of determining whether or not to use the vocal cords during the second time interval, if more than 2/3 indicates that the vocal cords are used, 1 is added to the status parameter, and if less than 1/3 indicates that the vocal cords are used, the status parameter is set to 0 It is reset, and if it is in the middle, the state parameter can be maintained as it is. In addition, when the state parameter reaches a specific value (eg, 5), it may be determined that the state of the vocal cords is a state requiring management.

According to various embodiments, in operation 670 , the vocal cord state analysis method may generate a notification message informing the user that the vocal cord state requires management, when it is further determined that the vocal cord state requires management. This notification message may be generated by the server 100 and delivered to the app device 200 to be displayed by the app device 200 to the user.

Table 1 below shows the accuracy of the judgment results in an experiment in which the use of vocal cords was determined by applying various machine learning theories using the system presented in the present invention.

learning algorithm Average Standard Deviation GNB 0.93 0.55 SVM 0.62 0.11 DT 0.92 0.09 KNN 0.94 0.02

The results in Table 1 show the PPG sensor 311, EMG sensor 313, GSR sensor 315, and MIC sensor 317 obtained by the vocal cord state determination module 130 for determining whether or not to use the vocal cords from the sensing device 300. All of the biometric information obtained from the system is used as learning data, and four machine learning algorithms are used: Gaussian Naive Bayes (GNB), Support Vector Machine (SVM), Decision Tree (DT), and K-Nearest Neighbors (KNN). It is the mean and standard deviation of the accuracy of the result of determining whether or not to use.

Referring to the results in Table 1, when the other three learning algorithms except for the SVM learning algorithm were applied, the accuracy of the result of determining whether or not to use the vocal cords was very high. In particular, while performing cross-validation, the accuracy was uniform even though the training data and the training data were changed. In particular, the KNN learning algorithm showed the highest accuracy with 94%, and the standard deviation was the most consistent with 0.02.

Therefore, the accuracy was high when the vocal cord state determination module 130 was designed using all of the biometric information obtained from the PPG sensor 311 , the EMG sensor 313 , the GSR sensor 315 , and the MIC sensor 317 . In particular, the accuracy of GNN, DT, and KNN was equally high. When comparing the accuracies for each algorithm used, the KNN algorithm was the most accurate.

In other experimental results, when using three sensors, using biometric information obtained from the PPG sensor 311, EMG sensor 313, and MIC sensor 317, and applying the GNN learning algorithm, the learning accuracy is 95 %, indicating that it is not appropriate to use three sensors because other algorithms have low accuracy and the standard deviation of each algorithm is large. Therefore, it is more appropriate to use all four sensors than to use only three sensors.

In particular, it was found that the MIC sensor 317 and the PPG sensor 311 are important factors in accurately classifying whether or not voice is used as a learning factor by setting the biometric information measured by each sensor as a variable. However, although the MIC sensor is an important factor in determining whether or not voice is used, if the surrounding environment is noisy or there are two or more speakers, it may not be accurate to determine that the user spoke only with the biometric information measured by the MIC sensor. It may not be reasonable to use only biometric information of

Those skilled in the art to which the present invention pertains should understand that the present invention can be embodied in other specific forms without changing the technical spirit or essential characteristics thereof, so the embodiments described above are illustrative in all respects and not restrictive. only do The scope of the present invention is indicated by the following claims rather than the detailed description, and all changes or modifications derived from the meaning and scope of the claims and their equivalent concepts should be construed as being included in the scope of the present invention. .

Claims (20)

In the vocal cord condition analysis device,
a communication module for acquiring the user's biometric information and user information;
a data acquisition module for storing the acquired biometric information and user information into a database; and
Comprising a vocal cord state determination module for determining the user's vocal cord state based on the obtained biometric information,
The vocal cord state determination module,
A vocal cord condition analysis apparatus for determining whether or not the user uses the vocal cords based on the biometric information, and determining the user's vocal cord condition based on a result of determining whether or not to use the vocal cords a plurality of times. delete According to claim 1, wherein the vocal cord state determination module,
Determining whether or not the user uses the vocal cords based on the biometric information at a first time interval,
For each second time interval longer than the first time interval, the vocal cord state analysis apparatus for updating the state parameter indicating the user's vocal cord state based on the results of determining whether the user uses the vocal cords. According to claim 3, wherein the vocal cord state determination module,
Vocal cord condition analysis, wherein, among the results of determining whether or not to use the vocal cords determined during the second time interval, if 2/3 or more of the vocal cords are used, 1 is added to the state parameter, and if it is less than 2/3, the state parameter is reset to 0. Device. According to claim 4, wherein the vocal cord state determination module,
If the state parameter is greater than or equal to a preset value, it is determined that the user's vocal cords have been used for a long time, and a notification message is displayed to the user. According to claim 1, wherein the vocal cord state determination module,
Vocal cord state analysis apparatus for determining whether the user uses the vocal cords by using a machine learning algorithm using the biometric information as learning data. 7. The method of claim 6, wherein the machine learning algorithm comprises:
At least one of Gaussian Naive Bayes (GNB), Support Vector Machine (SVM), Decision Tree (DT), and K-Nearest Neighbors (KNN). The method of claim 1,
The user's biometric information includes a PPG (photoplethysmography) sensor for measuring heartbeat, an EMG (electromyogram) sensor for measuring the user's EMG, a GSR (galvanic skin response) sensor for measuring the user's skin conductance and voice A vocal cord state analysis device comprising the user's biometric information obtained by using at least one of the MIC sensors. In the vocal cord condition analysis method for analyzing the vocal cord condition using the vocal cord condition analysis device,
obtaining, by the communication module, biometric information and user information of the user;
storing, by a data acquisition module, the acquired biometric information and user information into a database;
Comprising the step of determining, by the vocal cord state determination module, the state of the user's vocal cords based on the obtained biometric information,
The step of determining, by the vocal cord state determination module, the state of the user's vocal cords based on the obtained biometric information,
determining, by the vocal cord state determination module, whether the user uses the vocal cords based on the biometric information; and
and determining, by the vocal cord state determination module, the state of the user's vocal cords based on a result of determining whether or not to use the vocal cords a plurality of times. delete 10. The method of claim 9,
The step of determining, by the vocal cord state determination module, the state of the user's vocal cords based on the obtained biometric information,
and determining, by the vocal cord state determination module, whether or not the user uses the vocal cords based on the biometric information at first time intervals,
The step of determining, by the vocal cord state determination module, the state of the user's vocal cords based on a result of determining whether to use the vocal cords a plurality of times,
Including the step of updating, by the vocal cord state determination module, a state parameter indicating the state of the user's vocal cords based on the results of determining whether the user has used the vocal cords every second time interval that is longer than the first time interval, the vocal cord state analysis A vocal cord condition analysis method for analyzing the condition of the vocal cords using a device. 12. The method of claim 11,
The step of updating, by the vocal cord state determination module, the state parameter indicating the user's vocal cord state, based on the results of determining whether the user has used the vocal cords at every second time interval longer than the first time interval,
If the vocal cord status determination module indicates that more than 2/3 of the vocal cords are used among the results of determining whether or not to use the vocal cords during the second time interval, 1 is added to the status parameter, and if it is 2/3 or less, the status parameter is set to 0 A vocal cord state analysis method for analyzing a vocal cord state using a vocal cord state analysis device, comprising the step of resetting. 13. The method of claim 12, wherein the method comprises:
The vocal cord state determination module determines that the user's vocal cords have been used for a long time if the state parameter is greater than or equal to a preset value, and further comprising the step of displaying a notification message to the user. How to analyze vocal cord condition. 10. The method of claim 9,
The step of determining, by the vocal cord state determination module, whether the user uses the vocal cords based on the biometric information,
Vocal cord condition analysis method for analyzing the condition of the vocal cords using a vocal cord condition analysis device, comprising the step of the vocal cord condition determination module determining whether or not the user uses the vocal cords using a machine learning algorithm using the biometric information as learning data . 15. The method of claim 14, wherein the machine learning algorithm,
A vocal cord state analysis method for analyzing the vocal cord state using a vocal cord state analyzer, which is at least one of Gaussian Naive Bayes (GNB), Support Vector Machine (SVM), Decision Tree (DT), and K-Nearest Neighbors (KNN). 10. The method of claim 9,
The user's biometric information includes a PPG (photoplethysmography) sensor for measuring heartbeat, an EMG (electromyogram) sensor for measuring the user's EMG, a GSR (galvanic skin response) sensor for measuring the user's skin conductance and voice A vocal cord state analysis method for analyzing a vocal cord state using a vocal cord state analysis device, comprising the user's biometric information obtained using at least one of the MIC sensors. As a vocal cord condition analysis system,
a sensing device for acquiring the user's biometric information;
an app device that generates an ontology combining the biometric information obtained from the sensing device and the user information of the user; and
A server that extracts the biometric information and the user information from the ontology received from the app device and stores it in a database, and determines the user's vocal cord status based on the biometric information,
The server is
a communication module for communicating with the app device;
a data acquisition module for acquiring the user's biometric information and user information from the app device through the communication module, converting it into a database, and storing it; and
and a vocal cord state determination module for determining the user's vocal cord state based on the obtained biometric information,
The vocal cord state determination module,
A vocal cord state analysis system for determining whether or not the user uses the vocal cords based on the biometric information, and determining the state of the user's vocal cords based on a result of determining whether to use the vocal cords a plurality of times. The method of claim 17, wherein the sensing device,
PPG (photoplethysmography) sensor for measuring the heart rate of the user;
an electromyogram (EMG) sensor for measuring the user's EMG;
a galvanic skin response (GSR) sensor for measuring the user's skin conductivity; and
A sensor module including at least one of the MIC sensors for measuring voice, and
Containing a communication module for communicating with the app device, vocal cord condition analysis system. The method of claim 17, wherein the app device,
a communication module for communicating with the sensing device and the server;
an ontology generation module for generating an ontology by combining the biometric information received from the sensing device and user information generated by the app device; and
and a state notification module for notifying the user by receiving the state of the vocal cords of the user from the server through the communication module. delete

Images