JP2019208629A - Swallowing function evaluation method and swallowing function evaluation apparatus - Google Patents

Abstract

To provide a swallowing function evaluation method and a swallowing function evaluation apparatus capable of determining the voluntary swallowing strength and a difference of a single swallowing amount, a difference of physical properties (hardness, viscosity, temperature, liquid, individual) of food and a food bolus, a difference of a swallowing state and a presence of aspiration, types (overt aspiration, occult aspiration, aspiration before swallowing, aspiration during swallowing, aspiration after swallowing, etc), and a risk (larynx inflow, etc).SOLUTION: A swallowing function evaluation method of the present invention is characterized by: detecting at least a biological signal from swallowing start to swallowing completion; extracting a feature amount from the detected biological signal; identifying a swallowing state from the feature amount by using machine learning, and evaluating a swallowing function; using a suprahyoid muscle group biological signal by the muscle activities of the suprahyoid muscle group and an infrahyoid muscle group biological signal by the muscle activities of the infrahyoid muscle group as the biomedical signal; and extracting the feature amount from the suprahyoid muscle group biological signal and the infrahyoid muscle group biological signal.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a swallowing function evaluation method and a swallowing function evaluation apparatus that identify a swallowing state and evaluate a swallowing function using machine learning.

Eating and swallowing is a series of actions that recognize food, carry it to the mouth, chew it in the oral cavity, and carry it through the pharynx and esophagus to the stomach. 5 in the early stage, preparation stage, oral stage, pharyngeal stage, and esophageal stage. Divided into periods. Swallowing refers to the period from the oral phase to the esophageal phase, and is realized by a complex physiological mechanism in which voluntary movement and reflex movement coexist. Voluntary swallowing refers mainly to the chewing of food through the oral cavity and guiding it to the pharynx, and reflexive movement refers to the passage of food through the pharynx. The swallowing reflex is a pattern movement with high reproducibility by the central pattern generator (CPG) in the medulla, and whether swallowing is possible depends largely on the presence or absence of oral movement problems and the failure of the swallowing reflex. Yes. On the other hand, although it is patterned by CPG, there are various variations in swallowing, and movements of various swallowing organs such as hyoid bone and larynx according to the amount of swallowing (amount of food) and the physical properties of food It has also been reported that the activity pattern of each muscle involved in swallowing changes. In addition, although it is known that it changes, since the variation | change_quantity has a large individual difference, the method of estimating the condition of swallowing reversely from the variation | change_quantity is not established.
Thus, “the ability to change how to swallow” is considered to be a swallowing reserve ability (responding to food) to prevent suffocation and aspiration. However, it is known that cerebrovascular disorder, neuromuscular disease, muscular strength, sensory function, cognitive function, etc. due to aging decrease, swallowing reserve ability decreases, and the risk of suffocation / aspiration increases. When the swallowing function declines, the hyoid bone and laryngeal position drop, the hyoid bone and laryngeal elevation and forward movement decrease, the laryngeal elevation speed slows down, and the swallowing reflex is delayed Arise. In addition, the functional evaluation of these swallowing organs is generally performed using a swallowing contrast examination that observes the movement of the bolus, hyoid bone, and larynx under fluoroscopy. It is risky and unsuitable for repeated examinations at medical institutions, use in home medical care, and regular assessment in daily life.
According to the Ministry of Health, Labor and Welfare's annual demographic statistics, “pneumonia” is the third leading cause of death in Japan in FY2016, and about half of its 1,199,300 deaths are “aspiration pneumonia due to dysphagia” It is the cause. Therefore, in order to extend the healthy life expectancy of the elderly, a new technology for safe and simple evaluation of swallowing reserve ability and early detection of dysphagia that is difficult to recognize and flail elderly (swallowing disorder reserve army) is necessary.
The inventor has proposed a method for estimating jaw and mouth movement using a multichannel surface electromyogram of the suprahyoid muscle group (Patent Document 1).
According to Patent Document 1, not only voluntary movement of the tongue but also swallowing can be detected.

Japanese Patent No. 5924724

  However, in Patent Document 1, it is difficult to accurately evaluate the ability to change how to swallow (swallowing reserve ability), for example, the difference between voluntary swallowing and the amount of swallowing (difference in swallowing reflex). This is because hyoid and laryngeal elevation, which plays an important role during swallowing, such as laryngeal closure by reversing the epiglottis, is closely related to the activity of the epihyoid muscles and the reflex activity of the subhyoid muscles to it. This is because, however, only the muscular activity of the suprahyoid muscle group is focused on in spite of being controlled.

  The present invention includes voluntary swallowing strength and swallowing amount, food and bolus physical property values (hardness, viscosity, temperature, liquid, individual, etc.) Providing a swallowing function evaluation method and swallowing function evaluation device that can discriminate types (eg, overt aspiration, overt aspiration, aspiration before swallowing, aspiration during swallowing, aspiration after swallowing) and risk (such as laryngeal inflow) The purpose is to do.

The swallowing function evaluation method of the present invention according to claim 1 detects at least a biological signal from the start of swallowing to the end of swallowing, extracts a feature amount from the detected biological signal, and swallows from the feature amount using machine learning A swallowing function evaluation method for identifying a state and evaluating a swallowing function, wherein the biosignal is based on a suprahyoid muscle biomedical signal of the suprahyoid muscle group and a muscular activity of the subhyoid muscle group The feature quantity is extracted from the suprahyoid muscle group biosignal and the subhyoid muscle group biosignal using a subhyoid muscle group biosignal.
The present invention according to claim 2 is characterized in that in the swallowing function evaluation method according to claim 1, a surface myoelectric potential signal is used as the biological signal.
According to a third aspect of the present invention, in the swallowing function evaluation method according to the second aspect, the feature amount is extracted from at least the surface myoelectric potential signal from the start of swallowing to the end of swallowing, and the extracted feature amount is The swallowing state is identified by converting into an image file and performing image recognition.
According to a fourth aspect of the present invention, in the swallowing function evaluation method according to the third aspect, the image recognition is performed by deep learning.
According to a fifth aspect of the present invention, in the swallowing function evaluation method according to any one of the first to fourth aspects, a plurality of the detected biosignals used for the machine learning are detected at different wearing positions. A biological signal is used.
The swallowing function evaluation device of the present invention according to claim 6 detects at least a biological signal from the start of swallowing to the end of swallowing, extracts a feature amount from the detected biosignal, and swallows the feature amount using machine learning A swallowing function evaluating apparatus for identifying a state and evaluating a swallowing function, the multichannel electrode 11 for the suprahyoid muscle group for detecting the muscular activity of the suprahyoid muscle group, and the muscle of the subhyoid muscle group A multichannel electrode 11 for the suprahyoid muscle group for detecting the activity, and a controller 40, wherein the controller 40 is a biosignal of the suprahyoid muscle group from the multichannel electrode 11 for the suprahyoid muscle group; And a feature extraction unit 41 that extracts a pre-feature amount from the subhyoid muscle group biological signal from the multi-channel electrode 12 for the subhyoid muscle group, and the extracted feature amount is converted into an image file, and an image obtained by deep learning The swallowing state by recognizing And having a identifying operation identifying unit.

  According to the swallowing function evaluation method of the present invention, the hyoid bone and laryngeal elevation controlled by the suprahyoid and subhyoid muscle groups, the accompanying epiglottis reversal and laryngeal closure, etc. with a high discrimination rate Differences in swallowing status, the presence / absence / type / risk of aspiration, such as differences in voluntary swallowing strength, swallowing volume, physical properties of food and bolus, and swallowing conditions suitable for individuals Can be evaluated accurately.

Explanatory diagram showing the hyoid and subhyoid muscle groups The whole block diagram of the swallowing function evaluation apparatus by one Example of this invention Photograph showing the appearance of the multi-channel electrode used in the device for evaluating swallowing function Circuit configuration diagram of myoelectric amplifier box used in the evaluation device for swallowing function Schematic diagram of swallowing motion estimation algorithm based on time series data used for the swallowing function evaluation device Diagram showing how frame shift is performed Schematic diagram of swallowing motion estimation algorithm by image recognition used for the swallowing function evaluation device As an example, a photograph of the surface myoelectric potential signal imaged based on the feature value obtained by fast Fourier transform Diagram showing an example of maximum pooling when the image size is 8 × 8 and the pooling range is 2 × 2. Explanatory drawing which shows the structure of Alex Net which is the convolutional neural network (Convolutional Neural Network: CNN) used in the present Example. Explanatory drawing which shows the method of identifying by CVM using CNN as a feature extractor. Explanatory drawing which shows the method identified by Fine-tuning CNN The figure which shows the relationship between the electrode arrangement of the multi-channel electrode used for the experiment and the channel number Photo showing the actual wearing Diagram showing measurement operation in experiment The figure which shows an example of the calculation method of the identification rate for one person The figure which shows the combination of the identification operation | movement at the time of performing 2 operation | movement identification The figure which shows the arrangement | positioning of the image based on the electrode number and arrangement | positioning which are shown in FIG. Photograph showing images for one motion created by each feature extraction method Diagram showing analysis conditions Graph showing identification results when learning / identification is performed using images created by each feature extraction method Diagram showing analysis conditions for examining the effect of discriminator on discrimination accuracy A graph showing the classification rate when learning and classification are performed by each classifier Diagram showing the structure when combining the features of the hyoid and subhyoid muscle groups The figure which shows the method of imaging the hyoid bone muscle group and the subhyoid muscle group into one image The figure which shows the analysis conditions used for examination of the discrimination rate by the electrode used Graph showing identification rate results for each electrode used Diagram showing the analysis conditions used to examine the discrimination rate between the hyoid and subhyoid muscle groups The graph which shows the result of 2 movement discrimination of the hyoid and subhyoid muscle group in 22 channels The figure which shows the verification result of the effectiveness of the result which is obtained with average discrimination rate The graph which shows the result of 2 movement discrimination of the hyoid bone submuscular group and the subhyoid muscle group in 16 channels

  In the swallowing function evaluation method according to the first embodiment of the present invention, as the biosignal, the suprahyoid muscle group biosignal by the muscle activity of the suprahyoid muscle group and the subhyoid bone by the muscle activity of the subhyoid muscle group A feature value is extracted from the suprahyoid muscle signal and the subhyoid muscle signal using the muscle group signal. According to the present embodiment, the hyoid bone and laryngeal elevation controlled by the suprahyoid and subhyoid muscle groups can be estimated with a high identification rate, such as reversal of the epiglottis and laryngeal closure, Accurately evaluate differences in swallowing status, presence / absence / type / risk of swallowing, such as differences in voluntary swallowing strength, amount of swallowing, and physical properties of food and bolus, as well as swallowing conditions suitable for individuals it can.

  The second embodiment of the present invention uses a surface myoelectric potential signal as a biological signal in the swallowing function evaluation method according to the first embodiment. According to the present embodiment, it is possible to measure muscle activity using a multi-channel electrode with a wide range as a detection target.

  According to the third embodiment of the present invention, in the swallowing function evaluation method according to the second embodiment, a feature amount is extracted from at least a surface myoelectric signal from the start of swallowing to the end of swallowing, and the extracted feature amount is an image file. The swallowing state is identified by performing the image recognition. According to the present embodiment, the identification rate can be increased by image recognition.

  The fourth embodiment of the present invention performs image recognition by deep learning in the swallowing function evaluation method according to the third embodiment. According to the present embodiment, the identification rate can be increased, and voluntary swallowing and swallowing reflex can be accurately evaluated.

  In the fifth embodiment of the present invention, in the swallowing function evaluation method according to any one of the first to fourth embodiments, a plurality of biological signals detected at different mounting positions are used as learning biological signals used for machine learning. Is used. According to the present embodiment, an accurate evaluation can be performed without performing a calibration operation immediately after mounting the electrode, which is necessary when performing the swallowing function evaluation.

  The swallowing function evaluation apparatus according to the sixth embodiment of the present invention includes a multichannel electrode for detecting the suprahyoid muscle group and the tongue for detecting the muscular activity of the subhyoid muscle group. A multichannel electrode for the subosseous muscle group and a controller, the controller from the multichannel electrode for the suprahyoid muscle group and the multichannel electrode for the subhyoid muscle group and the multichannel electrode for the subhyoid muscle group A feature extraction unit that extracts a feature amount from a subhyoid muscle group biological signal, and an operation identification unit that identifies the swallowing state by converting the extracted feature amount into an image file and performing image recognition by deep learning It is. According to the present embodiment, it is possible to estimate the hyoid bone and laryngeal elevation and reflex movement controlled by the suprahyoid and subhyoid muscle groups with a high discrimination rate, and to accurately detect voluntary swallowing and swallowing reflexes. Can be evaluated.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is an explanatory diagram showing an upper hyoid bone group and a subhyoid muscle group.
The time required for swallowing is said to be 1 to 1.5 seconds for the oral and pharyngeal phases. In this short period of time, 1) lip closure, 2) transfer of the bolus to the pharynx by the tongue, and 3) nasopharynx 5) Laryngeal closure by raising the larynx and reversing the epiglottis, 6) Opening of the lower pharynx by forward movement of the larynx, 7) Closure of the glottis and increase of expiratory pressure, 8) Relaxation of the sphincter at the entrance of the esophagus occurs continuously in a predetermined order.
The suprahyoid and subhyoid muscle groups shown in FIG. 1 are largely involved in these operations. The suprahyoid muscle group consists of geniohyoid muscle, maxillohyoid muscle, bigastric muscle, and stylohyoid muscle. The hyoid bone and larynx are lifted forward and upward as the suprahyoid muscles contract. The subhyoid muscle group is composed of thyroid hyoid muscle, scapulohyoid muscle, sternohyoid muscle, and sternothyroid muscle. The larynx is pulled up to the highest position by contraction of the thyroid hyoid muscle by reflex movement of the suprahyoid muscle group. These exercises invert the epiglottis, closing the larynx and opening the esophageal entrance, allowing the bolus to pass safely into the esophagus without aspiration or suffocation. Elucidating the function of these muscles also leads to the evaluation of swallowing reserve.

The swallowing function evaluation method according to the present embodiment detects at least a biological signal from the start of swallowing to the end of swallowing, extracts a feature amount from the detected biological signal, identifies a swallowing state from the feature amount using machine learning, and swallows Evaluate functionality. Aspiration includes pre-swallowing aspiration and post-swallowing aspiration, which may flow unconsciously or cough, and evaluate swallowing function by detecting biological signals before and after swallowing be able to.
The swallowing function evaluation method according to this example focuses on the cooperative movement of the suprahyoid and subhyoid muscle groups, and the swallowing pattern varies depending on the strength of voluntary swallowing from the surface myoelectric potential signal and the amount of swallowing Is to identify. The identification rate obtained by identifying the swallowing pattern indicates that the greater the value, the higher the reproducibility of the muscle activity pattern, and that the swallowing pattern can be changed with respect to the physical property value and quantity. Therefore, it was defined that there was a swallowing reserve if the discrimination rate was high. On the other hand, when the discrimination rate is low, at this time, there are two possible causes: a case where there is no swallowing reserve capability and a case where the discrimination performance is insufficient. Therefore, in order to eliminate the cause of insufficient discrimination performance, it is necessary to develop a discrimination method capable of discriminating with high discrimination accuracy for a healthy person who can be assumed to have a high swallowing reserve ability.

FIG. 2 is an overall configuration diagram of a swallowing function evaluation apparatus according to an embodiment of the present invention.
The swallowing function evaluation apparatus according to the present embodiment includes a multichannel electrode 11 (10) for the suprahyoid muscle group, a multichannel electrode 12 (10) for the subhyoid muscle group, a myoelectric amplifier box (electromyograph) 20, AD / DA converter 30 and controller 40.

Two myoelectric amplifier boxes 20 and AD / DA converters 30 are used to perform two types of measurement of the upper hyoid bone group and the subhyoid muscle group. The myoelectric amplifier box 20 and the AD / DA converter 30 may be configured as a single unit.
The surface myoelectric potential signal is measured by differentially amplifying the potential difference between the reference electrode 13 attached to the earlobe and each electrode constituting the multi-channel electrode 10 with reference to the GND electrode 14 attached to the other earlobe. did. The detected surface myoelectric potential signal is amplified 2052 times through the myoelectric amplifier box 20 and taken into the AD / DA converter 30. A part of the multi-channel electrode 10 may be used as the reference electrode 13 or the GND electrode 14, and the reference electrode 13 or the GND electrode 14 may be provided together with the multi-channel electrode 10, and is not necessarily attached to the earlobe. Also good. When the reference electrode 13 and the GND electrode 14 are provided together with the multichannel electrode 10, the reference electrode 13 and the GND electrode 14 are provided on at least one of the multichannel electrode 11 side for the suprahyoid muscle group and the multichannel electrode 12 side for the subhyoid muscle group.
As the AD / DA converter 30 used for data collection, NI USB-6218 (NATIONAL INSTRUMENTS) was used. AD / DA converter 30 is equipped with analog input (16 bits, 250 kS / sec, 32 ch), analog output (16 bits, 250 kS / sec, 2 ch), 8 digital inputs, 8 digital outputs, and two 32-bit counter functions doing. In this study, data is collected using analog input.
In addition, by synchronizing the two AD / DA converters 30, two types of measurements of the upper hyoid muscle group and the subhyoid muscle group are simultaneously performed.

FIG. 3 is a photograph showing the appearance of the multichannel electrode used in the swallowing function evaluation apparatus.
The multi-channel electrode 10 is connected to the myoelectric amplifier box 20. The suprahyoid muscle group electrode 11 has an inverted V shape that does not interfere with the larynx and can measure the left and right pedicle hyoid muscles that exist in the back of the bottom of the lower jaw. The electrode for subhyoid muscle group 12 does not interfere with the movement of the larynx and is U-shaped so that it can measure the left and right thyroid hyoid muscle, scapulohyoid muscle, sternohyoid muscle, and sternothyroid muscle. is there.
The substrate thickness of the multi-channel electrode 10 is 0.3 mm, and the whole is covered with silicon for substrate protection and electrical insulation, and a surface myoelectric potential signal is extracted through a silver electrode embedded on the silicon. The silver electrode is not covered with silicon.
The silver electrode has a diameter of 2 mm and a height of 2.5 mm.
The suprahyoid muscle group electrodes 11 were embedded at intervals of 8 mm in length and 12.5 mm in width, and 22 electrodes were arranged so as to cover the entire bottom of the mandible. The subhyoid muscle group electrodes 12 were embedded at intervals of 8 mm in length and 8 mm in width, and 22 electrodes were arranged so as to cover the front surface of the neck. Also, the GND electrode 14 and the bipolar reference electrode 13 are arranged on the left and right earlobe, respectively. At the time of measurement, in order to suppress contact resistance, a multi-channel electrode 10 having a paste (Elefix, Nihon Kohden) applied to the electrode surface is attached to the subject. The signal obtained by the multichannel electrode 10 is sent to the myoelectric amplifier box 20.

FIG. 4 is a circuit configuration diagram of a myoelectric amplifier box used in the swallowing function evaluation apparatus.
The myoelectric amplifier box 20 stores therein an instrumentation amplifier (AD8226BRMZ) and an operational amplifier (AD8622ARMZ), and is capable of measuring 22ch surface myoelectric potential. The frequency band is 14 to 440 Hz, the gain is 2052 times, and the power source is four AA batteries.
The differential amplifier circuit 21 performs differential amplification by removing in-phase noise between the signal obtained from each channel and the reference signal. The DC servo circuit 22 detects and removes the low frequency band signal from the signal output from the differential amplifier circuit 21. The signal output from the differential amplifier circuit 21 is amplified by the signal amplifier circuit 23, and unnecessary high frequency noise is removed by the anti-aliasing filter circuit 24. This also has a function of preventing band folding during AD conversion. Finally, the band pass / output buffer circuit 25 removes the offset signal and high frequency noise generated inside the circuit and outputs the result.

FIG. 5 is a schematic diagram of a swallowing motion estimation algorithm based on time series data used in the swallowing function evaluation apparatus.
The motion identification algorithm extracts features from the suprahyoid muscle group biosignal from the multichannel electrode 11 for the suprahyoid muscle group and the subhyoid muscle group biosignal from the multichannel electrode 12 for the subhyoid muscle group. And a motion learning / identification unit 42 that identifies the swallowing state by machine learning, and is processed by the controller 40.
The feature extraction unit 41 uses a surface myoelectric potential signal as a biological signal, and an upper hyoid muscle biomedical signal based on muscle activity of the upper hyoid muscle group and a subhyoid muscle biomedical signal based on muscle activity of the subhyoid muscle group. Extract feature values from
Then, using the extracted feature quantity, the operation is identified by the support vector machine.
The feature extraction unit 41 extracts a characteristic signal component (feature amount) related to the action from the surface myoelectric potential (EMG) before performing the action identification, and constructs a feature vector. The following were used for this feature amount.

Root Mean Square (RMS)
RMS is expressed by Equation 1, and a characteristic regarding the amplitude of the surface myoelectric potential signal is obtained.
(1)

Cepstrum coefficient (CC)
CC is expressed by Equation 2. It is a feature amount extracted from the frequency domain, and has a feature capable of separating the envelope shape and fine structure of the power spectrum. When the order is low, the characteristics of the envelope shape appear, and when the order is high, the characteristics of the microstructure appear.

(2)

For the RMS calculation, the surface myopotential of the past n samples is used. At this time, a frame shift method is used in which n samples are cut out and calculated as one frame, and the cut out range is shifted at a constant period. Here, n is the frame length.
FIG. 6 is a diagram illustrating a state in which the frame shift is performed.
Furthermore, in order to suppress the variation between the frames, the feature amount is smoothed by performing a moving average as shown in Equation 3. Here, p is a frame number, and M is a moving average score.
(3)

    Further, threshold processing is performed on the calculated RMS sum of all channels, and motion labels corresponding to each tongue motion are assigned to the swallowing motion range (portion exceeding the threshold). A feature vector is constructed using the motion label, RMS, and CC created in this way, and used for learning and identification.

The operation learning / identification unit 42 shown in FIG. 5 will be described below.
First, a discriminant function is constructed using learning data.
Here, as learning data used for machine learning, it is preferable to use a plurality of biological signals detected at different mounting positions. In order to obtain the swallowing function evaluation by the swallowing function evaluation method according to the present embodiment, calibration work is required after the electrode is mounted. Even if the subject is the same person, if the electrode mounting positions are different, the relative positions of the electrodes and the muscles change, and the detected muscle activity patterns will also be different. Therefore, the calibration work is required every time the electrode is mounted, and feels troublesome, so it is desirable to omit the learning work for each electrode mounting.
By using multiple biological signals detected at different wearing positions as learning data, it is possible to correlate surface myoelectric potential signal patterns and swallowing states at multiple positions, and perform accurate evaluation without performing calibration work. Yes.

A support vector machine (SVM) is one of pattern recognition models using supervised learning, and is known as a method applicable to classification and regression. In the SVM, there are an operation learning unit used for learning and an operation identification unit that performs identification based on the learning result.
The action learning unit obtains SVM hyperparameters from learning data in which feature vectors (characteristics arranged in order) and correct labels (labeled action classes) are paired to form an identification function. In the learning data, γ (kernel parameter) and C (cost parameter), which are hyper parameters, are determined by lattice search. search range of gamma and C ^{γ = {2 -3, 2 -2.5} , ··· 2 1}, C = {2 3, 2 3.5, ···, 2 6} combination of 63 kinds of And a combination showing the highest identification rate is searched from among the identification rates of the respective lattice points. The identification rate at this time is obtained from the correctness of the identified action label and correct answer label, and the generalization performance is evaluated by cross-validation (cross-validation).
The action identification unit identifies an action class (action label) from the feature vector based on an identification function created by learning. Thereafter, a majority decision is made on the identified past k action labels, and the motion state is finally determined. If the value of k is set small, real-time operation identification is possible. In addition, if the value of k is set large and the majority process is performed so as to cover all operation labels from the start to the end of swallowing, a series of operations from the start to the end of swallowing can be identified as one operation.

  Since SVM is a technique for identifying two classes, when identifying a large number of classes, extension to multi-class is necessary. In general, there are two kinds of this method, the one-against-one method and the one-against-all method. In this method, the one-against-one method is adopted. This is a method of constructing combinations of all O classes, that is, O (O-1) / 2 discriminant functions, and discriminating feature vectors using each discriminant function. The superiority of this method is shown by comparative experiments on learning time and identification accuracy of the two methods by Hsu et al.

FIG. 7 is a schematic diagram of a swallowing motion estimation algorithm based on image recognition used in the swallowing function evaluation apparatus.
In the swallowing function evaluation method according to the present embodiment, it is preferable to identify the swallowing state by converting the extracted feature amount into an image file and performing image recognition. For image recognition, a raw waveform graph of a surface myoelectric potential signal can be used as an image file. For image recognition, a signal obtained by analyzing surface myoelectric potential signals such as frequency analysis and cepstrum analysis can be used as an image file.
Furthermore, this image recognition is preferably performed by deep learning, but may be performed by a method other than deep learning. Further, the action learning / identification unit 42 shown in FIG. 5 can be processed by deep learning without converting the extracted feature quantity into an image file. Deep learning has a feature amount extraction function and a feature amount identification function. Image recognition may be performed only by deep learning, or after extracting feature amounts from images by deep learning, other machine learning is used. The feature amount may be identified.
For identification of the swallowing state, after extracting the feature quantity and performing image recognition, the feature quantity can be extracted again, and the feature quantity can be used again.
In the swallowing motion estimation algorithm based on image recognition, after the surface myoelectric potential signal measured as preprocessing is imaged, the swallowing motion is identified using a convolutional neural network (CNN) (hereinafter referred to as CNN). I do.
However, in this embodiment, since there is no large-scale learning data, it is noted that CNN has two roles of a feature extractor and an action discriminator. (1) CNN is a feature extractor and SVM is an action discriminator. (2) A method of using, as a feature extractor and an action discriminator, a CNN obtained by fine-tuning a pre-learned CNN using swallowing movement learning data, and (3) a CNN subjected to fine-tuning in (2) Are identified by three methods: a feature extractor and a method using SVM as an action identifier.

A feature extraction method for converting a surface myoelectric potential signal into an image file will be described below.
Characteristic signal components (features) related to the motion are extracted from the surface myoelectric potential signal and imaged. The following was used for the feature quantity extraction method.

fast Fouriertrans form (FFT)
The FFT is expressed by Equation 4, and the power spectrum (Equation 5) of the surface myoelectric potential signal can be obtained. A Hanning window (Equation 6) was used as the window function.

(4)
(5)

(6)

Cepstrum coefficient (CC)
CC is omitted because it is the same as Equation 2.

wavelet transformation
The wavelet transform is expressed by Expression 7, and the wavelet is expressed by Expression 8. Moreover, a scalogram can be obtained by Equation 9. In the wavelet transform, it is possible to examine in detail the characteristic time using the mother wavelet and the wavelet that has been scaled and translated. In order to analyze the high frequency region, the time resolution is increased by using a short base, and the frequency resolution is increased by a long rule in the analysis of the low frequency region.

(7)

(8)

(9)

The imaging of the surface myoelectric potential signal will be described below.
The obtained feature quantity is imaged by color mapping for each channel. The maximum value and the minimum value of the color map are obtained by averaging the maximum value and the minimum value of the feature amount of each channel.
FIG. 8 is a photograph of the surface myoelectric potential signal imaged based on the feature amount obtained by the fast Fourier transform as an example.
The images created for each channel shown in FIG. 8 (a) are combined into one image according to the number of channels used in the analysis, and as shown in FIG. It was.

CNN is a feedforward neural network with a structure in which two types of layers called a convolution layer and a pooling layer are alternately stacked, and is one of the deep learning methods showing excellent performance in the field of image recognition. It is.
In the convolution layer, the product-sum calculation of the input image and the filter is performed, and the characteristic grayscale structure represented by the filter is extracted from the image. The size of the input image is W × W pixels, the pixel index of the input image is (i, j) (i = 0,..., W−1, j = 0,. Xij, the filter size is H × H, and the pixel index of the filter is (p, q) (p = 0,..., H−1, q = 0,..., H−1). Assuming that the pixel value of the filter is hpq, the calculation of convolution is expressed by Equation 10.

(10)

In the pooling layer, a specific range of the image is selected, and one pixel value is obtained therefrom. There are several methods for obtaining the pixel value. In the maximum pooling, the maximum pixel value is selected. In average pooling, the average value of the range is calculated. Thereby, even when the position of the target feature amount in the image is slightly changed, the output of the pooling layer remains unchanged.
FIG. 9 shows an example of maximum pooling when the image size is 8 × 8 and the pooling range is 2 × 2.

There are several difficulties in using CNN. One of them is network construction. Building a network from scratch requires a large amount of data per task, and also requires time to learn that large amount of image data. Therefore, swallowing exercises by using the already learned network for this task without using the network construction from scratch, making the most of the CNN's characteristic that the learned features are common between recognition tasks that differ to some extent. Identify.
FIG. 10 shows the structure of AlexNet, which is a pre-learned convolutional neural network (CNN) used in this embodiment.
AlexNet is a Deep CNN composed of five convolutional layers and three all-coupling layers proposed by Alex Krizhevsky et al. A normalization layer is used after the first and second convolution layers, and a maximum pooling layer is used after each normalization layer and after the fifth convolution layer.

In this embodiment, this AlexNet is diverted by the following three methods.
FIG. 11 is an explanatory diagram showing a method for identifying (1) CNN as a feature extractor and SVM as an action identifier.
In the method of performing identification by SVM using CNN as a feature extractor, the input image is feed forward with the network weight fixed, and the output of an appropriate intermediate layer is directly used as a feature vector. Then, SVM learning / identification is performed using the obtained feature vectors. The resulting feature vector is 4096 dimensions per image.
When CNN is used as a feature extractor, it is necessary to consider from which layer the feature extraction is performed. In CNN, it is known that as the layer closer to the input approaches the discriminating layer, it gradually becomes structured from a low-order visual feature to a semantic feature specialized for a data set. Therefore, taking features of a layer that is too low cannot benefit from a discriminatory structure with high CNN, and conversely, selecting features of a layer that is too high results in too much specialization in the learning dataset, There is a risk of performance degradation. Empirically, it is often the case that all the coupling layers before the identification layer are used. In this study, we treat the output from the last fully connected layer of Alex Net as a feature vector.
For the action learning / identification unit by SVM, the hyperparameters γ and C are determined by lattice search. search range of gamma and C ^{γ = {2 -16, 2 -15.5} , ··· 2 -12}, C = {2 7, 2 7.5, ···, 2 10} 63 Street A combination showing the highest identification rate is searched from among the identification rates of the respective lattice points. Note that the discrimination rate at this time is obtained from the correctness of the discrimination result and the action class of the data used for learning.
The action identification unit identifies a feature vector based on an identification function created by learning and assigns an action class.

FIG. 12 is an explanatory diagram showing a method of identifying by a fine-tuned CNN.
(2) In the method of fine-tuning the CNN, only the Alex Net identification unit is replaced with the target task, and the rest of the network is re-learned using the Alex Net parameters as initial values. Then, identification is performed using the re-learned network. In this study, the weight from the input layer to the second fully connected layer is fixed, and the remaining fully connected layer, softmax layer, and output layer are replaced with new ones and retrained. Parameters used in Fine-tuning were optimized by Bayesian optimization.

  (3) In the method of identifying a fine-tunned CNN as a feature extractor by SVM, the method of identifying by using the fine-tuned CNN shown in FIG. After performing Fine-tuning of Net, a feature vector is created by using Fine Net-tuned Alex Net as a feature extractor in the same manner as the method for identifying CNN as a feature extractor shown in FIG. Do.

Below, the verification result of the swallowing movement identification method by image recognition is demonstrated.
The subjects were 6 healthy adult men with normal swallowing function (age 22.7 ± 1.2 years, height 172.7 ± 5.5 cm, weight 60.0 ± 5.5 kg, mean ± SD). A, B, C, D, E, and F are distinguished.
FIG. 13 is a diagram showing the relationship between the electrode arrangement of the multi-channel electrodes used in the experiment and the channel numbers.
In the experiment, a 22-channel flexible electrode for the upper hyoid muscle group was attached to the lower jaw, a 22-channel flexible electrode for the lower hyoid muscle group was attached to the neck, and an ear electrode was attached to the earlobe. The suprahyoid muscle group was mounted at a position where the distance from the No. 1 and No. 2 electrodes in FIG. 13 (a) to the chin guy was between 25 mm and 30 mm and the electrode did not hit the jawbone. The subhyoid muscle group was mounted so that the electrodes Nos. 5 and 6 in FIG. 13B were located at the portion where the thyroid cartilage (throat Buddha) protruded most forward. The surface myoelectric potential signal was derived by differentially amplifying the potential difference between each surface electrode and the reference electrode attached to one earlobe with reference to the GND electrode attached to the other earlobe. As a result, a potential difference between any two points can be taken out like a normal differential amplification surface muscle potential.
FIG. 14 is a photograph showing a state of actual wearing.
In order to prevent the position of the electrode from being shifted during measurement, the 22-channel flexible electrode for the suprahyoid muscle group is the cap shown in FIG. 14B, and the 22-channel flexible electrode for the subhyoid muscle group is shown in FIG. 14C. Each band was fixed.

FIG. 15 is a diagram illustrating a measurement operation in the experiment.
Swallowing is a complex movement consisting of voluntary movement and reflex movement. Therefore, in this experiment, in order to examine the difference in the strength of voluntary swallowing, two types of swallowing as usual and swallowing with maximum effort (operation to swallow consciously and powerfully) were targeted. Moreover, in order to examine the difference in reflex movement, the amount of swallowing at one time was 3 ml and 15 ml of water. Then, a total of 4 actions (3 ml (normal: NS3), 3 ml (maximum effort: ES3), 15 ml (normal: NS15), 15 ml (maximum effort: ES15)) are combined, and 20 sets are measured. It was. 3 ml, which is a swallowing amount, is a prescribed amount for a revised drinking test, which is one of the simple tests (screening) of swallowing function, and 15 ml is a general swallowing amount that can be swallowed at once with the tongue. The sample water was inserted into the floor of the mouth with a syringe and swallowed according to the subsequent instructions. The posture during the experiment was a sitting position, and the head was fixed to the wall so that the neck angle did not change during the experiment. Swallowing within 2 seconds after resting for 2 seconds and resting for 2 seconds for a total of 6 seconds, 3 ml (normal: NS3), 3 ml (maximum effort: ES3), 15 ml (normal: NS15), 15 ml (maximum Effort: Measurement was performed one operation at a time in the order of ES15). Considering muscle fatigue due to continuous swallowing, a break of 30 seconds was sandwiched between 15 ml (maximum effort: ES15) for 10 seconds between 3 ml (maximum effort: ES3) and 15 ml (normal: NS15). The surface myoelectric potential was measured at an amplification factor of 2,052 times and a sampling frequency of 2,000 Hz.

FIG. 16 is a diagram illustrating an example of a method for calculating the identification rate for one person.
Learning / identification was performed by the method of FIG. 16 (cross-validation) so as not to be affected by the bias of the measurement data. The measurement data is assigned to A, B, C, and D in order from 1. Next, learning and identification are performed using two pieces of learning data and the remaining two pieces of test data. This is performed for all combinations in a total of six ways to calculate the identification rate. The identification rate is given by Equation 11.

(11)

The average of the obtained six discrimination rates was used as the final discrimination rate. The average recognition rate finally obtained was used as an index for evaluating the estimation accuracy.
Unless otherwise specified, the calculation result of the identification rate is expressed as mean ± SD for six subjects.

FIG. 17 is a diagram illustrating combinations of identification operations when performing two-operation identification (two-class classification).
The numbers i to vi shown in FIG. 17 are numbers corresponding to the combinations. For example, i is set to i in the comparison between NS3 and ES3.

The influence of the feature quantity extraction method for imaging on the identification accuracy will be described below.
The difference in the identification accuracy when the image is created by extracting the feature value using the three methods of fast Fourier transform (FFT), Cepstrum coefficient (CC), and wavelet transform. Consider comparison.

In the FFT and wavelet transform, spectrogram and scalogram were created respectively. The vertical axis represents frequency, the horizontal axis represents time, and the frequency is represented by 0 to 1,000 Hz. The CCs were imaged by arranging them from the bottom to the top in order from the low-order CC. The vertical axis is the order of the cepstrum coefficient, the horizontal axis is time, and the order of the cepstrum coefficient is 1-5.
FIG. 18 is a diagram showing the arrangement of images based on the electrode numbers and arrangement shown in FIG.
As for the image of each channel, the table is regarded as an image for one operation, and an image for one channel enters each square. The number on each square represents the electrode number.
FIG. 19 is a photograph showing an image for one operation created by each feature quantity extraction method.
FIG. 21 is a graph showing identification results when learning / identification is performed using images created by each feature extraction method. Here, in order to examine the contribution of each muscle group to the swallowing movement identification rate, the identification rate when using the surface myoelectric potential of the upper hyoid bone group or the surface myoelectric potential of the subhyoid muscle group is shown. Asked.
In 2 motion identification, although there are some differences depending on the combination of motions to be identified, when using the surface myoelectricity of either the hyoid or subhyoid muscle group, It was possible to obtain a high identification rate more stably than the feature extraction method. In the four-motion classification, both the upper hyoid muscle group and the subhyoid muscle group were identified using CC as a feature extraction method, sometimes showing the highest classification rate. In addition, CC was the only result exceeding 80% among the three conditions.
The reason for this is that all the three feature extraction methods compared this time are feature extraction methods in the frequency domain, but the CC can express spectral features with a small amount of information of five dimensions compared to other feature extraction methods. . For this reason, it is considered that when the image size was resized to 227 × 227 pixels, which is the input image size of Alex Net, there was little decrease in resolution for identification.
From the above results, it was suggested that CC is most suitable as a feature amount extraction method under the present analysis conditions for the present measurement data. In addition, it was shown that the swallowing motion could be identified by imaging the surface myoelectric potential and using CNN as the feature extractor and identifying with SVM.

The influence of the discriminator on the discrimination accuracy will be described below. Here, in order to examine the contribution of each muscle group to the swallowing movement identification rate, the identification rate when using the surface myoelectric potential of the upper hyoid bone group or the surface myoelectric potential of the subhyoid muscle group is shown. Asked.
There are two methods of using a network that has been learned in advance: a method of using as a feature extractor and a method of performing fine-tuning. Therefore, learning and identification are performed using an image created based on the optimized parameters, and (1) a method of identifying with SVM using CNN as a feature extractor, and (2) a fine-tuning CNN. The discrimination accuracy by the discrimination method and (3) the discrimination method by the SVM using the fine-tuned CNN as a feature extractor are compared, and which discriminator is optimal is examined.
FIG. 22 shows analysis conditions for examining the influence of the discriminator on the discrimination accuracy.
Since there are few images that can be used for Fine-tuning, 10 sets of identification data (test data) were used as verification data in the learning process. For this reason, the discrimination rate by the fine-tuned CNN cannot be said to be a pure discrimination rate, and this time, the comparison is performed as an identification rate of a reference level.
FIG. 23 is a graph showing an identification rate when learning / identification is performed by each classifier.
In 2-motion identification, (2) when a fine-tuning CNN was used for the classifier, depending on the combination, (1) the identification rate was higher than when CNN was used as a feature extractor. Looking at the average identification rate of the combination, (1) the identification rate was lower than when CNN was used as a feature extractor. (3) When a fine-tuning CNN was used as a feature extractor, the discrimination rate was lower than the other two in almost all combinations of operations, and the average discrimination rate was the lowest. In the 4-motion discrimination, (1) the feature discrimination by CNN + motion discrimination by SVM has a higher discrimination rate than the other two.
The reason for this is considered that the re-learning did not proceed sufficiently by Fine-tuning because the number of data that can be used for learning / identification was insufficient. 2 There is a combination in which the identification rate has increased in the operation identification. Since the same data is used for the verification data (test data) and the identification data, the information of the test data is obtained when the fine-tuning CNN is used. It is thought that the learning rate was included and the recognition rate increased.
This time, the discriminator had the highest discrimination rate. (1) The method of discriminating with SVM using CNN as a feature extractor was judged to be optimal.
However, with regard to Fine-tuning, there is a possibility that the identification rate is improved by increasing the number of data.

  When only the upper hyoid muscle group is used, when only the subhyoid muscle group is used, or when the features of the upper hyoid muscle group and the subhyoid muscle group are combined, the upper hyoid muscle group and the subhyoid muscle group In the following, the results of comparison and examination of the identification accuracy when learning / identification is performed by the four methods will be described below.

FIG. 24 is a diagram showing a structure in the case of synthesizing feature amounts of the upper hyoid bone group and the subhyoid muscle group.
When synthesizing the features of the suprahyoid and subhyoid muscle groups, first the image file created from the surface myoelectric signal of the upper hyoid muscle group and the surface myoelectric signal of the subhyoid muscle group were created. 4096-dimensional feature vectors are created for each image file using CNN as a feature extractor. Then, the two feature quantities are combined to create an 8192-dimensional feature vector, and learning identification is performed using SVM.
FIG. 25 is a diagram illustrating a method of imaging the upper hyoid bone group and the subhyoid muscle group into one image.
When imaging into one image, for example, any 16 channels are selected from 22 channels of surface myoelectric potential signals, or the potential difference between any two channels is calculated, and a new surface myoelectric potential signal is sent to 16 channels. It is assumed that there is a 16-channel surface myoelectric potential signal created or a combination thereof. At this time, the images of the channels of the upper hyoid bone group and the subhyoid muscle group are arranged in a 6 × 6 matrix as shown in FIG. 25 and converted into one image. Thus, without using the channel itself used for detection, it is possible to calculate the potential difference between the electrodes between the channels, create a new myoelectric potential signal, and arbitrarily select the number of channels used for imaging and the surface myoelectric potential signal. it can.

FIG. 26 shows analysis conditions used for examining the identification rate by the electrodes used.
FIG. 27 is a graph showing the result of the identification rate for each electrode used.
Compared to the case of learning and identification only in the suprahyoid muscle group and the subhyoid muscle group in both the 2 movement identification and 4 movement identification, the surface myoelectric signal of both the hyoid and subhyoid muscle groups is obtained. The identification rate was higher when the identification was performed. In addition, the identification rate is higher when the feature amounts are synthesized than when the image is formed into one image.
This is because the image size (227 × 227 pixels) used for image recognition is determined in the method of imaging into a single image, so if many channels of images are included, the amount of information that one channel has (( It is considered that the features necessary for identification could not be extracted by the feature amount extraction because the resolution for so-called identification) dropped. On the other hand, in the method of synthesizing features, the features were extracted from the suprahyoid and subhyoid muscle groups, so it was thought that they could be learned and identified with feature vectors that could capture both features well. .
Therefore, under this analysis condition for this measurement data, it was determined that the method of combining the features of the superior hyoid bone group and the subhyoid muscle group was the best for identifying swallowing movements.

The results of studies on elucidating the functions of the suprahyoid and subhyoid muscle groups for voluntary swallowing and swallowing reflex are described below.
Focusing on the muscular functions of the hyoid and subhyoid muscle groups, we examined how the function of each muscle group appears in the discrimination accuracy when the voluntary swallowing strength and swallowing amount are changed. .

FIG. 28 shows analysis conditions used for examining the discrimination rate between the hyoid and subhyoid muscle groups.
The number of channels of the surface myoelectric potential signal to be used is 22 channels and 16 channels so as not to be affected by how to calculate the potential difference.
FIG. 29 is a graph showing the results of two motion discrimination results for the upper hyoid bone group and the subhyoid muscle group in the 22 channel.
First, when the swallowing amount is equal and the swallowing strength is changed, that is, when 3 ml (normal: NS3) and 3 ml (maximum effort: ES3) are identified (i), 15 ml (normal: NS15) and 15 ml (maximum) Effort: When ES15) was identified, (vi) showed a higher discrimination rate in the suprahyoid muscle group than in the subhyoid muscle group. The suprahyoid muscle group has the role of stably supporting the hyoid bone located at the base of the tongue, and the hyoid bone is supported by the strength of swallowing, that is, the force that feeds water and bolus from the mouth to the pharynx. It is considered that a high discrimination rate was obtained with the surface myoelectric potential of the suprahyoid muscle group.
Next, when the swallowing strength is constant and the swallowing amount is changed, that is, when 3 ml (normal: NS3) and 15 ml (normal: NS15) are identified (ii) and 3 ml (maximum effort: ES3 ) And 15 ml (maximum effort: ES15), (v) showed a higher discrimination rate in the subhyoid muscle group than in the upper hyoid muscle group. The subhyoid muscle group has a role of raising the larynx to the highest position and closing the larynx as a reflex movement accompanying the contraction of the suprahyoid muscle group. Since it is known that the timing and closing time of the larynx change as the mouthful volume increases, it is considered that the difference in the swallowing amount appears in the surface myoelectric potential of the subhyoid muscle group. However, in (v), the discrimination rate for both the hyoid and subhyoid muscle groups was about 85%, which was lower than the others. This is thought to be due to the fact that not only changes in surface myoelectric potential due to changes in amount but also changes in surface myoelectric potential due to strong swallowing appeared as features, making it difficult to understand the features due to changes in amount when imaged.
These results suggest that the intensity of voluntary swallowing is likely to appear in the activity pattern of the suprahyoid muscle group, and the difference in swallowing amount is likely to appear in the activity pattern of the subhyoid muscle group. This knowledge is expected to be used for early detection of swallowing function decline and decision of training guidelines.

FIG. 30 shows a verification result of the validity of the result obtained by the average discrimination rate.
Although the surface myoelectric potential signal obtained by each electrode by the average discrimination of 6 subjects was considered, since there are few subjects this time, the average discrimination rate greatly depends on the discrimination result of one person. Therefore, for each subject, the identification rate of the superior hyoid bone group and the subhyoid muscle group is compared, and the one with the higher identification rate is counted, and the superior hyoid muscle group is determined for each movement according to the number of people, From the results, the effectiveness of the results obtained by the average discrimination rate was examined. If the two identification rates are compared and are the same, neither is counted.
FIG. 30 shows a result of comparing and counting the results of the two motion identifications of the upper hyoid bone group and the subhyoid muscle group for each subject. The number on the left side of each square is the number of people whose upper hyoid muscle group exceeds the recognition rate of the subhyoid muscle group, and the value on the right side of each square shows the identification rate of the subhyoid muscle group to identify the upper hyoid muscle group Indicates the number of people exceeding the rate.
When 3 ml (normal: NS3) and 3 ml (maximum effort: ES3) are identified (i), 15 ml (normal: NS15) and 15 ml (maximum effort: ES15) are identified (vi) The identification rate of the subhyoid muscle group exceeded that of the subhyoid muscle group, and the identification rate of the subhyoid muscle group exceeded that of the epihyoid muscle group. From this result, it can be said that the intensity of voluntary swallowing tends to appear in the activity pattern of the suprahyoid muscle group, and the difference in swallowing amount per time tends to appear in the activity pattern of the subhyoid muscle group.
From the above, the effectiveness of the results obtained by the average identification rate was confirmed.

First, when the swallowing amount is equal and the swallowing strength is changed, that is, when 3 ml (normal: NS3) and 3 ml (maximum effort: ES3) are identified (i), 15 ml (normal: NS15) and 15 ml (maximum) Effort: When ES15) is identified, looking at (vi), in (i) the subhyoid muscle group has a higher discrimination rate than in the hyoid muscle group, and in (vi) the hyoid bone is higher than the subhyoid muscle group The muscle group had a higher discrimination rate.
Next, when the swallowing strength is constant and the swallowing amount is changed, that is, when 3 ml (normal: NS3) and 15 ml (normal: NS15) are identified (ii) and 3 ml (maximum effort: ES3 ) And 15 ml (maximum effort: ES15), (v) showed a higher discrimination rate in the subhyoid muscle group than in the upper hyoid muscle group.
In (vi), (ii), and (v), the same tendency was observed as when 22 channels were identified, but in (i), the same tendency was not seen. The reason for this is that the discrimination rate in the 16 channels of the subhyoid muscle group is generally higher than that in the 22 channel, whereas it is only partially increased in the suprahyoid muscle group. . That is, it is considered that there is a difference in the change rate of the discrimination rate in the part of calculating the potential difference between the electrodes, and the conditions of the upper hyoid bone group and the subhyoid muscle group were not the same.

  Based on the results of examination of the discrimination rate between the hyoid and subhyoid muscle groups, how the function of each muscle group when the swallowing strength and swallowing volume are changed appears in the discrimination accuracy It was suggested that the intensity of voluntary swallowing tends to appear in the activity pattern of the suprahyoid muscle group, and the difference in swallowing amount per time tends to appear in the activity pattern of the subhyoid muscle group.

  The present invention can also be applied as a swallowing function meter for checking swallowing function on a daily basis.

DESCRIPTION OF SYMBOLS 10 Multichannel electrode 11 Multichannel electrode for epihyoid muscle group 12 Multichannel electrode for subhyoid muscle group 13 Reference electrode 14 GND electrode 20 Myoelectric amplifier box (electromyograph)
DESCRIPTION OF SYMBOLS 21 Differential amplifier circuit 22 DC servo circuit 23 Signal amplifier circuit 24 Anti-aliasing filter circuit 25 Band pass / output buffer circuit 30 AD / DA converter 40 Controller 41 Feature extraction part 42 Operation learning and identification part

Claims (6)

A swallowing function evaluation method that detects at least a biosignal from the start of swallowing to the end of swallowing, extracts a feature amount from the detected biosignal, identifies a swallowing state from the feature amount using machine learning, and evaluates swallowing function Because
As the biological signal, using the suprahyoid muscle group biological signal due to the muscular activity of the suprahyoid muscle group and the subhyoid muscle group biological signal due to the muscle activity of the subhyoid muscle group,
A method for evaluating a swallowing function, wherein the feature amount is extracted from the biosignal of the hyoid bone muscle group and the biosignal of the subhyoid muscle group. The swallowing function evaluation method according to claim 1, wherein a surface myoelectric potential signal is used as the biological signal. Extracting the feature quantity from the surface myoelectric potential signal at least from the start of swallowing to the end of swallowing,
Convert the extracted feature quantity into an image file,
The swallowing function evaluation method according to claim 2, wherein the swallowing state is identified by performing image recognition. The swallowing function evaluation method according to claim 3, wherein the image recognition is performed by deep learning. The swallowing function evaluation method according to any one of claims 1 to 4, wherein a plurality of the biological signals detected at different mounting positions are used as learning biological signals used for the machine learning. A swallowing function evaluation device that detects at least a biological signal from the start of swallowing to the end of swallowing, extracts a feature amount from the detected biosignal, identifies a swallowing state from the feature amount using machine learning, and evaluates a swallowing function Because
A multichannel electrode for the suprahyoid muscle group for detecting muscle activity of the suprahyoid muscle group;
A multichannel electrode for the subhyoid muscle group for detecting muscle activity of the subhyoid muscle group;
With a controller,
The controller is
A feature extraction unit for extracting pre-features from the suprahyoid muscle group biosignal from the suprahyoid muscle group multichannel electrode and the subhyoid muscle group biosignal from the subhyoid muscle group multichannel electrode; ,
A swallowing function evaluation device comprising: an operation identifying unit that identifies the swallowing state by converting the extracted feature quantity into an image file and performing image recognition by deep learning.