JP2022027304A - Swallowing function evaluation/training method and system therefor, using time series data prediction - Google Patents

Abstract

To provide an eating swallowing function evaluation technology that is non-invasive and with a low risk and enables simple prediction of hyoid bone movement even bedside or home medical care.SOLUTION: An eating swallowing function evaluation method includes a VF moving image step, a pre-treatment step, a signal measurement step, a feature amount extraction step, a learning step, a prediction step, and a motion prediction step. In the signal measurement step, synchronization with the videofluoroscopic moving image step is performed. In the learning step, a motion section is determined by fitting a feature amount to a teacher signal and movement of a hyoid bone is learned and predicted on the basis of the teacher signal and the feature amount using an LSTM, a recurrence type neural network architecture for learning long-term time dependence and short-term time dependence. In the prediction step, hyoid bone movement is predicted with respect to a subject's biological signal newly detected after the prediction step, by using the subject's hyoid bone movement learned in the learning step.SELECTED DRAWING: Figure 15

Description

The present invention detects biometric signals related to swallowing movements in and around the anterior neck during swallowing, extracts feature quantities from the detected biometric signals, and swallows the oral cavity, pharynx / larynx, esophagus, etc. The present invention relates to a swallowing function evaluation / training method and a swallowing function evaluation / training system for evaluating / training the swallowing function by predicting the movement of an organ and the movement of a swallowing object (bolus).

If swallowing function declines due to cerebrovascular accidents, neuromuscular diseases, age-related muscle weakness, etc., the risk of pharyngeal residue, laryngeal invasion, aspiration, and choking of the bolus increases, as shown in FIG. This includes hyoid / laryngeal drooping, accompanying decrease in hyoid and laryngeal elevation and anterior movement, delayed laryngeal elevation due to slowed laryngeal elevation, delayed swallowing reflex, and laryngeal closure. This is due to the deterioration of the motor and sensory functions of the swallowing organs, such as the timing shift of the larynx. Therefore, in order to evaluate swallowing function, medical institutions evaluate the movements of various swallowing organs related to swallowing such as the tongue, hyoid bone, larynx, epiglottis, and esophageal entrance, as well as the movement of the bolus. ..

For example, the gold standard for a detailed examination of swallowing function is the Videofluoroscopic examination of swallowing (VF). As shown in FIGS. 5 (a) to 5 (f), VF swallows a bolus containing a contrast medium under fluoroscopy, and moves the swallowing organs during the preparation / oral cavity, pharynx, and esophagus, as well as the bolus. It evaluates the movement of. In the preparation / oral phase, it is possible to evaluate the process and function of bolus formation by chewing, feeding into the pharynx by tongue movement, and residual in the oral cavity. In the pharyngeal stage, the swallowing reflex is induced by feeding the bolus, and the tongue and larynx are raised, the larynx is closed, the larynx is closed due to the epiglottis reversal, the esophageal entrance is dilated, and the pharynx remains in the bolus. , Pharyngeal invasion, aspiration, etc. can be evaluated in detail while observing the movements of swallowing organs such as the tongue bone, epiglottis, and esophageal entrance and the movements of the bolus. On the other hand, since this VF has risks such as radiation exposure and aspiration of contrast medium, there is a problem that the number, time, frequency of examinations, examination place, examination conditions, etc. are limited.

Recently, swallowing endoscopy (VE) is also widely used as a method of observing the movements of swallowing organs and bolus at bedside or at home. VE has the advantage of being able to assess the condition of the bolus and pharyngeal residue without the risk of exposure, but because the endoscope is inserted through the nasal cavity, there is a risk of mucosal damage and pain, and swallowing is not always natural. There are some aspects that cannot be said to be observing. In addition, there is a problem that the moment of swallowing in the pharyngeal period, preparation / oral cavity, and movement in the esophageal period cannot be observed.

Image processing is used to quantitatively evaluate the movements of swallowing organs and bolus, and the hyoid bone is one of the important swallowing organs that are often noted. The hyoid bone is the only adjacent bone in the human body, or a very peculiar bone floating in the air that does not exhibit the morphology of cartilage and joints. As shown in FIG. 1, the hyoid bone is located between the tongue and the larynx, and many muscles involved in swallowing movement are attached to it. For example, there are jaw hyoid muscle, stylohyoid muscle, jaw hyoid muscle, otogai hyoid muscle, thoracic hyoid muscle, thyroid hyoid muscle, scapulohyoid muscle, pharyngeal hyoid muscle, mesopharynic contractile muscle, etc. .. By the cooperative activity of these muscle groups, clever movements such as mastication, swallowing, and vocalization are performed.

As shown in FIG. 2, it is known that the hyoid bone draws a movement locus that resembles a triangle when swallowing. First, the raising movement is started relatively slowly, but this is often accompanied by a slight backward movement (1: raising and backward movement). Second, the hyoid bone rises sharply and moves forward rapidly (2: lift forward movement). Then, after staying at the maximum raising position and the maximum forward position, a third movement, that is, a backward movement and a downward movement are performed to restore the original position (3: downward backward movement). The suprahyoid muscles and the infrahyoid muscles are largely involved in these movements. As shown in FIG. 3, during this series of exercises, the suprahyoid muscles contract together with the pharyngeal muscles and the hyoid muscles, and the hyoid bones move upward and anteriorly. Then, the larynx is raised by the contraction of the infrahyoid muscle group following the hyoid bone, and the cricopharyngeal muscle is continuously relaxed and contracted, and the bolus passes through the esophageal entrance. In addition, the timing and time of elevation of the hyoid bone are closely related to the timing and time of closing the larynx, which prevents the bolus from invading the larynx.

As a technique for evaluating the eating and swallowing function focusing on the muscle activity of the suprahyoid muscle group and the infrahyoid muscle group, the feeding and swallowing function evaluation technique disclosed in Patent Document 1 is known. The swallowing function evaluation technique of Patent Document 1 detects a biological signal from the start of swallowing to the end of swallowing, extracts a feature amount from the detected biological signal, and uses machine learning to feed from the feature amount. It is a swallowing function evaluation method that identifies swallowing movements and evaluates swallowing function. As the biological signal, the suprahyoid muscle group biological signal due to the muscle activity of the suprahyoid muscle group and the infrahyoid muscle group biological signal due to the muscle activity of the infrahyoid muscle group are used to obtain the suprahyoid muscle group biological signal. The feature amount is extracted from the biological signal of the infrahyoid muscle group. However, Patent Document 1 describes differences in swallowing conditions and aspiration, such as differences in voluntary swallowing strength and single swallowing volume, and differences in physical properties (hardness, viscosity, temperature, liquid, individual, etc.) of food and bolus. Existence / type (explicit aspiration, invisible aspiration, pre-swallowing aspiration, swallowing aspiration, post-swallowing aspiration, etc.) It is a device and cannot directly predict the movements of swallowing organs and bolus that can be observed by VF or VE as time-series data. If these can be predicted, non-invasive and simple evaluation of swallowing function and swallowing training that can be used at bedside or at home can be realized.

Japanese Unexamined Patent Publication No. 2019-208629

In view of the above points, the present invention provides a non-invasive, low-risk, swallowing function evaluation / training technique capable of easily predicting the movement of swallowing organs and bolus even in bedside or home medical care. The task is to do.

[1] A swallowing imaging process in which the subject captures the movement of the bolus and the movements of the swallowing organs that the subject has eaten.
A pretreatment step of acquiring the positions of the swallowing organs and the bolus from the moving image of the swallowing imaging step and quantifying them as coordinates to create a teacher signal for the movement of the swallowing organs and the bolus.
A biological signal detection step of detecting a biological signal during swallowing by a sensor unit arranged on a predetermined skin surface of the subject in synchronization with the swallowing imaging step.
The feature amount extraction step of extracting the feature amount from the biological signal in the analysis unit,
RNN (Recurrent Neural Network) and LSTM (Long Short-Term Memory), GRU (Gated Recurrent Unit), LSTNet (Long- and Short-term Time-series Network), AR (Autoregressive) model derived from the RNN and the above. Based on the teacher signal and features using time-series data prediction methods including ARMA (Autoregressive Moving Average), ARIMA (Autoregressive Integrated Moving Average), and SARIMA (Seasonal AutoRegressive Integrated Moving Average) models derived from AR models. A learning step of learning the movements of the swallowing organs and the bolus and generating a model capable of predicting the movement of at least one of the swallowing organs and the bolus from the features.
Using the prediction model generated in the learning step, a prediction step of predicting the movement of at least one of the swallowing organs and the bolus from the feature amount,
Evaluation / training process for evaluating / training eating and swallowing function using the results of the prediction process, and
It is characterized by having.

According to such a configuration, a swallowing imaging step, a pretreatment step, a biological signal detection step synchronized with the swallowing imaging step, a feature amount extraction step, a learning step, a prediction step, and an evaluation / training step are provided. In the swallowing imaging step, if the swallowing material mixed with the contrast medium or coated on the surface is spherical, it becomes easy to quantify the movement of the bolus by image processing. The shooting may be a moving image or a still image as long as a continuous change in the swallowing process can be confirmed, and the type of the image does not matter. Furthermore, in the learning process and prediction process, for example, when LSTM is used as a prediction method for time-series data, long-term and short-term memory, which is a recurrent neural network architecture that learns long-term time dependence and short-term time dependence, is used. It is used to learn the movements of swallowing organs such as the tongue bone and the bolus based on the teacher signal and features. LSTM is a recurrent neural network architecture used in the field of deep learning, which solves problems that cannot be learned with long-term time dependence when training with conventional RNN, and long-term time dependence is also short-term time dependence. You can also learn sex. When new inputs and outputs arrive in the learning process, the new patterns are adapted, and the problems of input weight collisions and output weight collisions that occur in RNNs can be dealt with. For this reason, the subject first takes a biological signal at the sensor unit at the same time as at least one swallowing modeling test (VF test) or swallowing endoscopy (VE test), so that the subject can swallow. At least the characteristics related to the movement of one of the swallowing organs and the bolus are learned, and from the second time onward, at least the swallowing organs are obtained from only the characteristic amount of the biological signal newly detected after the learning process without VF examination. The movement of one of the bolus can be predicted. As a result, the X-ray fluoroscope required for VF examination is no longer required, and non-invasive and low-risk, swallowing function evaluation and training that easily predicts the movement of swallowing organs and bolus even at bedside and home medical care can be performed. It can be carried out. In addition, if swallowing is performed when the same amount and the same physical property value are swallowed in the same way, the learning data becomes appropriate data once, but in order to obtain more suitable learning data, the amount and physical property value are changed. Regarding swallowing at that time, the swallowing data under the condition may be added to the learning as learning data.

[2] Preferably, in the biosignal detection step, the suprahyoid muscle group biosignal is detected by the suprahyoid muscle group myoelectric sensor arranged in the suprahyoid muscle group portion, and the suprahyoid muscle group portion is detected. The placed subhyoid muscle group myoelectric sensor detects the biosignal of the subhyoid muscle group, and the placed laryngeal behavior sensor detects the laryngeal behavior signal.
In the feature amount extraction step, the feature amount is extracted from the suprahyoid muscle group biological signal, the infrahyoid muscle group biological signal, and the laryngeal behavior signal as the biological signal.

According to this configuration, in the biological signal detection step, the suprahyoid muscle group biological signal, the infrahyoid muscle group biological signal, and the laryngeal behavior signal are detected, so that at least one of the swallowing organs and the bolus is more accurate. Can predict the movement of.

[3] Preferably, in the learning step, the coordinate data of the swallowing organs and the bolus are used as the learning data, and the learning data does not include the same coordinate data of one original data in a predetermined cycle. It shifts like this and amplifies it to multiple.

According to such a configuration, in the learning process, one original data is shifted so as not to include the same coordinate data in a predetermined period and amplified into a plurality of pieces, so that the predicted value can be obtained in the first learning. It is possible to reduce the error of the measured value and predict the movement of at least one of the swallowing organs and the bolus with higher accuracy.

[4] Preferably, in the pretreatment step, the coordinate system for obtaining the coordinate data of the swallowing organs and the bolus and the origin thereof are determined, and the lower end of the anterior margin of the fifth cervical vertebra of the subject is set as the origin, and the subject's By setting a coordinate system with the upper end of the anterior edge of the third cervical vertebra as a point on one axis, the positions of the swallowing organs and the bolus on the sagittal plane or the front face of the subject are acquired in the coordinates of the coordinate system. do.

According to this configuration, in the pretreatment step, the coordinate system on the sagittal plane or the front face value of the subject is set, and the coordinates of the XY coordinate system at each position of the swallowing organs and the bolus are acquired. It is possible to easily understand the prediction of the movement of at least one of the swallowing organs and the bolus in each direction such as anterior-posterior and up-down.

[5] Preferably, the positions of the swallowing organs and the swallowing organs are acquired from the images obtained by photographing the movements of the bolus and the swallowing organs eaten by the subject and quantified as coordinates, and the swallowing organs and the swallowing organs and the swallowing organs are preferably obtained. A preprocessing unit that creates a teacher signal for the movement of the bolus,
A sensor unit that is placed on a predetermined skin surface of the subject and detects a biological signal during swallowing in synchronization with the acquisition of the image.
In addition to extracting features from the biometric signals, RNN (Recurrent Neural Network) and LSTM (Long Short-Term Memory), GRU (Gated Recurrent Unit), and LSTNet (Long- and Short-term Time-series) derived from the RNN. Using time-series data prediction methods including Network), AR (Autoregressive) models, ARMA (Autoregressive Moving Average), ARIMA (Autoregressive Integrated Moving Average), and SARIMA (Seasonal AutoRegressive Integrated Moving Average) models derived from the AR models. It is equipped with an analysis unit that learns and predicts at least the movements of one of the swallowing organs and the bolus based on the teacher signal and the feature quantity, and evaluates and trains the eating and swallowing function using the prediction results. There is.

According to this configuration, non-invasive, low-risk, time-series that can easily perform swallowing function evaluation / training that predicts the movement of at least one of the swallowing organs and the bolus at bedside or home medical care. It is possible to provide a swallowing function evaluation / training system using data prediction.

Non-invasive, low-risk, bedside and home medical care can easily perform swallowing function evaluation and training that predicts the movement of at least one of the swallowing organs and the bolus.

It is explanatory drawing which shows the suprahyoid muscle group and the infrahyoid muscle group. It is explanatory drawing which shows the movement of a hyoid bone. It is explanatory drawing which shows the mechanism of swallowing which consists of a voluntary movement and a swallowing reflex. It is explanatory drawing which shows the risk of aspiration. It is explanatory drawing which shows the information obtained by VF. It is a block diagram of the swallowing function evaluation / training method system using the time-series data prediction which concerns on this invention. It is explanatory drawing which shows the sensor part. It is explanatory drawing which shows the laryngeal behavior sensor and the jig for an electrode. It is explanatory drawing which shows the output voltage corresponding to the expansion / contraction ratio. It is explanatory drawing which shows the circuit structure. It is a schematic diagram of Σ-ΔAD conversion and the explanatory diagram which shows the parallelization and synchronization of each AD conversion module. It is explanatory drawing which shows the data processing / transfer circuit. It is explanatory drawing which shows the insulation structure. It is explanatory drawing which shows the microphone for synchronization and the portable multi-mixer. It is a flow chart which shows the swallowing function evaluation / training method using time-series data prediction. It is explanatory drawing which shows the state of a frame shift. It is explanatory drawing which shows the setting of the object in the X-ray image. It is explanatory drawing which shows the distance (X-axis and Y-axis) of the start point of a hyoid bone. It is explanatory drawing which shows the hyoid bone position (white circle) at the time of A to F on the VF image. It is explanatory drawing which shows the operation section determination by moving motion. It is explanatory drawing which shows the operation section determination by a biological signal. It is a basic diagram of RNN. It is a simplified diagram of the internal structure of the LSTM block. It is explanatory drawing which shows the internal structure of an LSTM block. It is explanatory drawing which shows the oblivion gate layer, the input gate layer, the cell update, and the output gate layer. It is explanatory drawing which shows the swallowing function evaluation / training system using an X-ray fluoroscope and time series data prediction. It is explanatory drawing which shows the arrangement of an electrode. It is explanatory drawing which shows the state which the sensor part was attached to the sensor part and the subject, and the sensor part was permeated. It is explanatory drawing which shows the quantification of the hyoid bone movement. It is explanatory drawing which shows the learning condition. It is explanatory drawing which shows the creation of the training data and the test data. It is explanatory drawing which shows the amplification of data. It is explanatory drawing which shows the relationship between data amplification and RMSE. It is explanatory drawing which shows the example of learning (corresponding to the condition 6 of FIG. 31). It is explanatory drawing which shows an example of the sEMG signal and the laryngeal movement of the first swallowing. It is explanatory drawing which shows the time series data of the sEMG signal, RMS, CC and the laryngeal movement and the movement of the hyoid bone of each muscle group. As an example, it is explanatory drawing which shows the result of learning A-prediction C. As an example, it is explanatory drawing which shows the trajectory of the hyoid bone in learning A-prediction C. It is explanatory drawing which shows the RMSE of the measured value and the predicted value on the X axis, and RMSE of the measured value and the predicted value on the Y axis. It is explanatory drawing which shows the correlation coefficient of the measured value and the predicted value in the X-axis direction, and the correlation coefficient of the measured value and the predicted value in the Y-axis direction. As an example, it is explanatory drawing which shows the result of learning A-prediction C. As an example, it is explanatory drawing which shows the locus of the tip of a bolus in learning A-prediction C.

As an embodiment of the present invention, the motion prediction of the hyoid bone will be described below as an example based on the attached figure. The drawings conceptually (schematically) show the schematic configuration of the swallowing function evaluation system.

First, the overall configuration of the eating and swallowing function evaluation system 10 according to the embodiment of the present invention will be described.
As shown in FIGS. 6 to 8, 26 and 27, the swallowing function evaluation system 10 has a sensor unit 20 that detects a biological signal during swallowing and a PC that amplifies the detected biological signal. A multifunctional myoelectric potential measuring device 30 that transmits, an analysis unit 40 that predicts the movement of at least one of the swallowing organs such as the hyoid bone and the bolus from biological signals, and uses it for evaluation and training of swallowing function. It includes a recording unit (not shown) for recording the evaluation / training results, a display unit 41 for displaying the evaluation / training results, and a battery (not shown) for supplying power to these.

Further, the eating and swallowing function evaluation system 10 includes an X-ray fluoroscope 50 that visualizes the movement of the bolus and the movements of various swallowing organs eaten by the subject 60 and obtains a feeding and swallowing contrast video, and a feeding and swallowing contrast video. It is provided with a preprocessing unit 51 that acquires the positions of the swallowing organs and the bolus from the above, quantifies at least the movement of one of the swallowing organs and the bolus as coordinates, and creates a teacher signal for each movement. The X-ray fluoroscope 50 is used only when the subject 60 first detects a biological signal by the sensor unit 20 at the same time as the VF examination and obtains learning data.

Next, the sensor unit 20 will be described.
The sensor unit 20 is a myoelectric sensor 21 for the suprahyoid muscle group, which is arranged in the suprahyoid muscle group portion and detects the biological signal of the suprahyoid muscle group due to the muscle activity of the suprahyoid muscle group, and the infrahyoid muscle group. A myoelectric sensor 22 for the infrahyoid muscles that is placed in the part and detects the biological signal of the infrahyoid muscles due to the muscle activity of the infrahyoid muscles, and a laryngeal behavior signal that is placed in the larynx and detects the larynx behavior signal due to the elevation of the larynx. It is equipped with a laryngeal behavior sensor 25. The sensor unit 20 is arranged on a predetermined skin surface of the subject 60 and detects a biological signal at the time of swallowing in synchronization with the acquisition of a swallowing contrast moving image.

As the myoelectric sensor 21 for the suprahyoid muscle group, an array-shaped electrode in which multi-channel electrodes 21a are aligned is used. As the myoelectric sensor 22 for the infrahyoid muscle group, an array-shaped electrode in which multi-channel electrodes 22a are aligned is used.

The multi-channel electrodes 21a and 22a are used by being connected to the multi-functional myoelectric potential measuring device 30. The suprahyoid muscle group myoelectric sensor 21 is shaped so as not to interfere with the back of the head and to measure the stylohyoid muscle portion existing in the back of the mandibular floor. The myoelectric sensor 22 for the infrahyoid muscle group has a shape that allows measurement without interfering with the movement of the adam's apple.

The thickness of the substrate itself is 0.3 mm, and the entire surface is covered with silicon to protect the substrate, and the surface electromyography (hereinafter referred to as sEMG signal) of the muscle is extracted via the silver electrode embedded in the silicon. do.

The silver electrodes had a diameter of 2 mm and a height of 2.5 mm, and electrodes 21a for the suprahyoid muscle group were embedded at intervals of 8 mm in length and 11.5 mm in width, and 22 electrodes were arranged so as to cover the entire mandibular floor. Electrodes 22a for the infrahyoid muscle group were implanted at intervals of 8 mm in length and 8 mm in width, and 22 electrodes were placed so as to cover the anterior surface of the neck. In addition, the GND electrode 23a and the reference electrode 23b of the bipolar electrode were placed in the left and right ear lobes, and the RLD electrode 24 was placed in the 7th cervical spinous process. At the time of measurement, multi-channel electrodes 21a and 22a coated with paste (Elefix, Nihon Kohden) on the electrode portion are attached to the subject in order to suppress contact resistance. The obtained signal is sent to the multifunctional myoelectric potential measuring device 30. In the present invention, the frequency band is 20 to 4000 Hz and the gain is 125 times.

Further, in the present invention, when the hyoid bone is raised, the larynx is also raised following the raising of the hyoid bone. Therefore, in order to record the change in the position of the Adam's apple, the laryngeal behavior sensor (stretchability) shown in FIG. A strain sensor) 25 was used. The laryngeal behavior sensor C-STRETCH (registered trademark) (F51FS01, Bando Chemical Industries, Ltd.) was used in the present invention. This sensor is a dielectric capacitance type strain sensor composed of an elastomer film and a protective film. By inputting a power supply voltage, it outputs an analog voltage according to the elongation of the sensor. The sensor telescopic portion has a length of 50 mm and a width of 5 mm. The expansion / contraction range of the sensor expansion / contraction portion is 0 to 100%, and the output voltage corresponding to the expansion / contraction displacement is the value shown in FIG.

Further, when the 22ch flexible electrode 21a and the 22ch flexible electrode 22a are attached, after taping, the jig 27 (hat and band) for the myoelectric sensor for the suprahyoid muscle group shown in FIG. 8 (b). And fixed with the jig 28 (band) for the myoelectric sensor for the infrahyoid muscle group shown in FIG. 8 (c).

Next, the multifunctional myoelectric potential measuring device 30 will be described.
The multifunctional myoelectric potential measuring device 30 is a measuring device for monitoring biological activity, which is designed on the premise that a plurality of different sensors are used at the same time. It is possible to sample up to 64 channels of sensors at the same time. It is connected to a PC for capturing measurement data via a USB2.0 (High Speed) interface. It is also possible to control the device from any application software. Operated by DC12 (V) AC adapter or external battery input power supply. As shown below, the circuit configuration of the multifunctional myoelectric potential measuring device is divided into four parts: a signal conditioning unit, an AD conversion unit, a data transfer unit, and an insulating unit.

As shown in FIG. 10, the signal conditioning unit can input up to two multi-channel electrodes, four general-purpose myoelectric potential sensors, and 16 arbitrary analog sensors. First, the differential amplifier circuit removes in-phase noise between the signal obtained from the reference electrode attached to the ear canal and the signal obtained from each electrode of the multi-channel electrode, and amplifies only the difference in signal components. Also called unipolar induction measurement.

Further, a low frequency band signal of 1 (Hz) or less is detected and removed from the obtained differential signal by the DC servo circuit. Next, the signal amplification circuit PGA (Programmable Gain Amplifier) amplifies the signal to either 125 times or 1000 times. Unnecessary high frequency noise is removed by a 3-pole anti-aliasing filter circuit. This also has the effect of preventing band wrapping during AD conversion. Finally, it is input to a high-speed amplifier for driving AD conversion, and an output signal is obtained.

In addition, in order to additionally remove the low frequency signal, it is also possible to perform a first-order low-cut filter processing by a digital filter. The cutoff frequency is one of disable, 0.01, 0.1, 1.0, 10.0 and 20.0 (Hz).

The signal processing circuit of the general-purpose myoelectric potential sensor has almost the same configuration as that of the multi-channel electrode, except that the signal obtained from two arbitrary electrodes attached to the body surface is differentially amplified. Also called bipolar induction measurement.

The signal processing circuit of a general-purpose analog sensor is designed to be able to input an analog signal of up to ± 15 (V) so that various sensors can be connected arbitrarily. When a signal with a large amplitude is input, the gain is adjusted by PGA in order to adjust it to the measurement range (± 2.5 (V)) of the multifunctional myoelectric potential measuring device. The PGA value is one of disable, 1/4, 1/2, and 1x. The output signal is obtained from a high speed amplifier for driving the AD conversion.

As shown in FIG. 11, the AD conversion function built in the multifunctional myoelectric potential measuring device 30 is a Σ-Δ conversion method, capable of simultaneous sampling of all channels with a resolution of 16 (bit) and a maximum of 10 (kHz). be. A schematic diagram of the Σ-Δ AD conversion is shown. By oversampling the sampling frequency fs (Hz) × n and Σ-Δ modulation of the analog signal Vin, the frequency spectrum of unnecessary noise is transferred to the high frequency band outside the band, and this is removed by the digital filter. do. Finally, downrate to fs (Hz) to obtain a digitized output signal. The SN ratio can be taken higher than the widely used successive approximation AD conversion, and the antialiasing filter can be simplified.

The digital filter of the multifunctional myoelectric potential measuring device 30 has a flat amplitude and linear phase characteristics (effective band is 1/2 of the sampling frequency). The sampling frequency is selected from 1k, 1.25k, 2k, 2.5k, 4k, 5k, 8k, and 10k (Hz).

Further, since the (64) Σ-Δ AD conversion modules corresponding to each channel are driven by the same clock source wired with the same length, each performs the AD conversion operation in synchronization.

As shown in FIG. 12, in the data transfer unit, the measurement data digitized by AD conversion is taken into the PC via the USB 2.0 (High Speed) interface. These processes are realized by the firmware written in the DSP (Digital Signal Processor). The measurement data transferred from each AD conversion module for each sampling frequency is transferred to the memory in the DSP by DMA (Direct Memory Access). The DSP performs additional signal processing such as a digital filter, and stores the measurement data in a FIFO (First In First Out) memory composed of SDRAM. When the USB transmission buffer becomes empty, the target measurement data is sequentially read from the FIFO memory and transmitted to the PC (USB host). By inserting the FIFO memory between the data processing and the USB transfer processing in this way, all the measurement data can be transferred to the PC without causing any omission.

As shown in FIG. 13, since the multifunctional myoelectric potential measuring device 30 is a measuring device for monitoring biological activity in the insulated portion, it is necessary to consider safety as well. The analog part (the above-mentioned signal conditioning part and AD conversion part) where the electrode and the living body come into contact, and the digital part (the above-mentioned data transfer part) that enables connection to the power supply and PC are electrically insulated. And said. The drive power of the analog unit is generated from the 12 (V) input by the isolated power supply circuit. Data communication between the digital unit and the analog unit is performed via a digital isolator.

Next, the X-ray fluoroscope 50 will be described.
As shown in FIG. 26, the VF inspection device used in the present invention is an X-ray fluoroscope 50 (SHIMADZU Corp, Safire II ZS-100), and the voltage output state at the time of inspection is 79 kVp and the current is 250 mA.

Next, the synchronization microphone 52 and the portable multi-mixer 53 will be described.
As shown in FIG. 14, the synchronization microphone is a synchronization microphone for the purpose of synchronizing the moving image obtained by the X-ray fluoroscope 50 and the sEMG signal obtained by the multifunctional myoelectric potential measuring device 30 at the time of inspection. 52 was used. A stereo microphone (AT9941, audio-technica (registered trademark)) is used for the main body of the synchronization microphone 52 shown in FIG. 14 (a), and the portable multi-mixer 53 (AT-PMX5P, shown in FIG. 14 (b)) is used. Synchronization was achieved by connecting audio-technica (registered trademark) to the main body of the synchronization microphone 52, the X-ray fluoroscope 50, and the multifunctional myoelectric potential measuring device 30.

Next, the analysis unit 40 will be described.
The analysis unit 40 (see FIGS. 6 and 26) is a recurrent neural network architecture that extracts features from biological signals and learns long-term time dependence and short-term time dependence, long / short-term memory. (LSTM) is used to learn and predict the movements of the hyoid bone and other swallowing organs and bolus based on teacher signals and features (details will be described later).

Next, the method for evaluating the swallowing function according to the embodiment of the present invention will be described.
As shown in FIG. 15, the eating and swallowing function evaluation methods include a swallowing imaging step (VF moving image step), a pretreatment step, a biological signal detection step (signal measurement step), a feature amount extraction step, a learning step, and a learning step. It has a prediction process (learning / prediction process) and an evaluation process of prediction results (operation prediction process).

In the swallowing imaging step (VF moving image step), a swallowing contrast moving image is obtained by a swallowing contrast examination in which the movement of the bolus and the movement of the swallowing-related organs eaten by the subject can be observed under fluoroscopy. In the pretreatment step, the positions of the movements of the hyoid bone and other swallowing organs and the bolus are obtained from the swallowing contrast video, and each movement is quantified as coordinates, and the teacher signal of the movements of the swallowing organs and the bolus is obtained. To create.

In the biological signal detection step (signal measurement step), the biological signal at the time of swallowing is detected by a sensor unit arranged on a predetermined skin surface of the subject in synchronization with the contrast moving image process. In the feature amount extraction step, the feature amount is extracted from the biological signal in the analysis unit.

In the learning process, swallowing including the tongue bone based on teacher signals and features using LSTM (long / short-term memory), which is a recurrent neural network architecture that learns long-term time dependence and short-term time dependence. A model (prediction model) that can predict the movements of swallowing organs and bolus from the features by learning the movements of various organs and bolus is generated. In the prediction process, the hyoid bone and other parts of the biological signal of the subject newly detected after the learning process or not used in the learning process by the motion prediction model of the subject generated in the learning process are used. Predict the movement of swallowing organs and hyoid bones. In addition, an evaluation / training process for evaluating / training the eating / swallowing function using the result of the prediction process is provided.

Further, in the biosignal detection step, the suprahyoid muscle group biosignal is detected by the suprahyoid muscle group myoelectric sensor placed in the suprahyoid muscle group part, and the infrahyoid muscle group part is placed in the infrahyoid muscle group part. The infrahyoid muscle group biological signal is detected by the muscle group myoelectric sensor, and the laryngeal behavior signal is detected by the laryngeal behavior sensor placed in the laryngeal part. Feature quantities are extracted from biological signals, infrahyoid muscle group biological signals, and laryngeal behavior signals.

Further, in the learning process, the coordinate data of the movements of the swallowing organs such as the hyoid bone and the bolus are used as the learning data, and the training data is not included in the same coordinate data in a predetermined cycle with one original data. It is shifted in this way and amplified to a plurality (see FIG. 32). Further, when amplifying the data, each point at each point (for example, data 1, data 2 and data 3 in FIG. 32) in which one original data is shifted so as not to include the same coordinate data in a predetermined period. An average value including (moving average) may be used.

Further, in the pretreatment step, an XY coordinate system is set in which the lower end of the anterior margin of the 5th cervical vertebra of the subject is the origin and the upper end of the anterior margin of the 3rd cervical vertebra of the subject is a point on the Y axis in the lateral view of the subject. The coordinates of the XY coordinate system are acquired with the lower end of the hyoid bone as the position of the hyoid bone.

Next, a hyoid bone motion prediction algorithm based on time-series data in the eating and swallowing function evaluation method will be described. A schematic diagram of the algorithm for predicting the movement of the hyoid bone based on the time series data is as shown in FIG.

Next, the feature section extraction section of the signal measurement will be described.
The inventors removed noise by applying a bandpass filter (250-700 Hz) to the suprahyoid muscle group 22ch and the infrahyoid muscle group 22ch obtained by signal measurement. Then, characteristic signal components (features) related to movement are extracted in each channel of the suprahyoid muscle group and the infrahyoid muscle group. In this study, features were extracted and created by shifting frames for 256 samples in length in a cycle of 16 samples for each sEMG signal of each channel of the suprahyoid muscle group and the infrahyoid muscle group. .. The following are used for this feature quantity.

RMS (Root Mean Square) is expressed by Equation 1, and features related to the amplitude of the EMG signal can be obtained.

CC (Cepstrum coefficient) is expressed by the formula 2. It is a feature quantity extracted from the frequency domain, and has a feature that the envelope shape of the power spectrum and the fine structure can be separated. When the order is low, the characteristic of the envelope shape appears, and when the order is high, the feature of the fine structure appears.

Here, n represents the total number of samples and sEMG represents the sEMG signal.

As shown in FIG. 16, the past n samples of EMG are used for the calculation of RMS. At this time, a frameshift method is used in which n samples are cut out as one frame and calculated, and the cutout range is shifted at regular intervals.

Next, the preprocessing of the VF inspection video will be described.
As shown in FIG. 17, the moving image obtained by the VF test was captured at a sampling rate of 30 fps using DIPP-Motion V (Detect Co., Ltd.), a video analysis software, and the movement of the hyoid bone was quantified. In the coordinate system, the upper end of the leading edge of the third cervical spine was P1, the lower end of the leading edge of the fifth cervical spine was P2, the lower end of the hyoid bone was P3, and the straight line passing through P1 and P2 was the Y axis. A straight line perpendicular to the Y axis and passing through P2 was set as the X axis. In the scale setting on the screen, the diameter of one of the 22 sterling silver rods of the multi-channel electrode was 2 mm.

As shown in FIG. 18, in the analysis of the movement of the lower end of the hyoid bone, the moving distance (mm) of the X-axis and the Y-axis during swallowing is calculated with the lower end of the hyoid bone in a resting state as the origin of the coordinates. After that, moving average was performed and smoothing was performed.

As shown in FIG. 19, from the image results, A is the start of movement of the hyoid bone with the start of voluntary swallowing, B is the start of hyoid bone elevation movement, C is the rapid start of hyoid bone elevation with the start of swallowing reflex, and D is the tongue. Reaching the foremost direction of the bone, E was determined as the rapid descent of the hyoid bone, F was determined as the resting position of the hyoid bone after the end of swallowing, and the section from A to F was defined as the section predicting the movement of the hyoid bone.

As shown in FIG. 20, regarding the determination of the motion section, the start point is the movement start A of the tongue tip accompanying the start of voluntary swallowing (see FIG. 19) from the movement of the hyoid bone which is the teacher signal, and the end point is after the end of swallowing. The hyoid bone resting position F was set.

Then, as shown in FIG. 21, the sEMG signal of the suprahyoid muscle group, the sEMG signal of the infrahyoid muscle group, and the laryngeal movement by the laryngeal behavior sensor (stretching strain sensor) are combined with the movement of the hyoid bone. , The operation section was decided.

Next, the learner will be described.
Here, LSTM, which is suitable for time-series data prediction, was used to predict the movement of the hyoid bone from the sEMG signal. LSTM (Long short-term memory) is a recurrent neural network (RNN) architecture used in the field of deep learning, and it takes a long time to train with a conventional RNN. It is a method that can solve long-term time dependence and short-term time dependence by solving problems that cannot be learned by dependence.

RNN is one of the deep learning that extends the neural network, and is a method with excellent performance in the field of time series data. Nowadays, it is often used in the fields of machine translation and speech recognition. As shown in FIG. 22, the network structure is such that the values obtained in the intermediate layer are input to the intermediate layer again in order to handle the variable length data in the neural network. The middle layer h _t looks at the input x _t and outputs the value h _t . The loop allowed information to be passed from one step to the next in the network. However, the longer the prediction, the more difficult it is to relate and learn in the case of RNNs, if the information related to the predicted value is located first.

As shown in FIG. 23, the LSTM has a CEC (Constant Error Carousel, also called a storage cell, a cell), an input gate, an output gate, and an oblivion gate, which are different from the RNN. CEC is a unit for storing past data and is also called a storage cell or cell. By introducing this, it becomes possible to detect long-period regularity. In the input gate and output gate, when a new input or output arrives in the learning process, it is possible to adapt to the new pattern and deal with the problems of input weight collision and output weight collision that occurred in RNN. .. In the case of a model without a forgetting gate, even if an input with a large change comes, the influence of that input will be relatively small, and only the same result as before will be output. By introducing a forgetting gate to deal with this problem, it is possible to update the cell state at once when the input pattern changes significantly.

As shown in FIG. 24, the internal details of the LSTM block are as shown in FIG. 24 (a). As shown in FIG. 25 (b), each line carries the entire vector from the output of one node to the input of another node. Small circles represent one-point operations such as vector addition, and rectangular boxes are layers of neural networks to be trained. A merging line means a concatenation, and a diverging line means that the content is copied and the copy goes to another location.

FIG. 25 shows a forgetting gate, and as shown in FIG. 25 (a), in the LSTM block, the information to be discarded in the first step is determined. This determination is made by a sigmoid layer called the "forgetting gate layer". Look at the input h _t-1 and x _t and output the number between 0 and 1 for each number in the cell state C _t-1 . 1 stands for "completely maintain" and 0 stands for "completely remove". The equation at that time is shown in Equation 3, and the equation of σ, which is a gate activation function, is shown in Equation 4.

Where W represents the weight of the input and b represents the bias.

(B) of FIG. 25 shows an input gate, and the next step is to determine a new state to be saved in the cell state. There are two parts to this. First, the sigmoid layer called the "input gate layer" determines which value to update. The tanh layer then creates a vector C _t of new candidate sites to be added to the cell state. Then combine these two to update the state in the next step. The formulas at that time are expressed in formulas 5 and 6. Where tanh represents a hyperbolic tangent function.

(C) of FIG. 25 shows the cell update, and updates from the old cell state C _t-1 to the new cell state C _t . Multiply the old cell state by f _t and forget what was determined to be forgotten earlier. Then add it _{x vector C t .} This is a new candidate value scaled at the rate determined to update each state value. The formula at that time is expressed in Equation 7.

(D) of FIG. 25 shows an output gate, and finally, it is necessary to determine what to output. This output is based on the cell state. First, the sigmoid layer is executed. This layer determines which part of the cell state to output and outputs only the determined part, so tanh is applied to the cell state (to compress the value between -1 and 1). Multiply that by the output of the sigmoid layer. The formulas at that time are expressed in formulas 8 and 9.

Next, the motion prediction of the hyoid bone by the sEMG signal will be described.
As a test condition, the subject was a female in her 70s who was suspected of having oral function, visited the Iwate Medical University Hospital, and agreed to perform a VF test. This inspection was carried out within the scope of the normal inspection with the approval of the Iwate Medical University School of Dentistry Ethics Review Board (No. 01304) and the Iwate University Research Ethics Review Board (No. 201905).

As for the measurement method and measurement operation, as shown in FIG. 26, the movement of the hyoid bone in the chair sitting position was photographed from the side of the neck using an X-ray fluoroscope (SafireII ZS-100, Shimadzu Corporation) at the time of the VF examination. The test food was 3 ml of a 98 w / w% barium sulfate solution having a thickness of 1%, and the tester injected it into the lower part of the tongue of the subject with a syringe, and then swallowed according to the instructions of the tester. The number of inspections was 3 trials, and the shooting speed was 30 fps. Further, as shown in FIGS. 27 and 28, at the same time as the VF examination, a 22-channel flexible electrode for the suprahyoid muscle group was placed in the lower jaw, a 22-channel flexible electrode for the infrahyoid muscle group was placed in the neck, and an ear electrode was placed in the ear canal. I put it on. The suprahyoid muscle group was attached at a position where the distance from the electrodes 1 and 2 in FIG. 7 (c) to the chin was between 25 mm and 30 mm and the electrodes did not touch the jawbone. The infrahyoid muscle group was attached so that the electrodes 5 and 6 in (d) of FIG. 7 were located at the most protruding part in front of the thyroid cartilage (Adam's apple).

For the measurement of laryngeal movement, a laryngeal behavior sensor (stretch strain sensor) was attached to the thyroid cartilage part and connected to the same multifunctional myoelectric potential measuring device as myoelectric measurement. The amplification factor of the sEMG signal was 125 times, and the sampling frequency of the sEMG signal and the laryngeal behavior sensor was 2000 Hz. Then, in order to synchronize the VF test with the myoelectric and laryngeal movement measurement system, a synchronization microphone was connected to the multifunctional myoelectric potential measuring device and the X-ray fluoroscopy device, and trigger input was performed by sound. FIG. 29 shows the process of quantifying the movement of the hyoid bone. According to the VF video, the movement of the hyoid bone in the X-axis direction (anterior-posterior direction) and the movement of the hyoid bone in the Y-axis direction (vertical direction). The movement is graphed. The main purpose of this measurement is VF examination, and the measurement of myoelectricity is accompanied.

Regarding the prediction of hyoid bone movement using LSTM, the learning condition as an analysis condition affects the prediction accuracy by adding the laryngeal movement and what happens when the muscle group used for input increases in the learning input value. In order to verify whether or not the above is given, verification was performed with 6 patterns as shown in FIG.

The training / prediction data set as an analysis condition was created as A, B, and C because the number of inspections was three. Then, as shown in FIG. 31, six types of learning and test data were created. The data was amplified in each of the training data for the purpose of improving the prediction accuracy. As shown in FIG. 32, in the present invention, points were scored in one cycle of 30 samples so as not to include the same points with respect to the number of samples of the original data, and the data was amplified up to 30 pieces. Since each data of the data amplified to 30 is 1/30 of the number of samples of the original data, the primary data is interpolated to match the number of samples of the original data, and the original data is used. According to the number of samples.

Regarding the effectiveness of data amplification, FIG. 33 shows a comparison with the case where the data is amplified up to 30 in the present invention but not amplified. FIG. 33 shows the relationship between data amplification and RMSE, and it can be seen from the error verification by RMSE that the error between the predicted value and the measured value is reduced by data amplification, and the effectiveness of data amplification can be seen.

FIG. 34 shows an example of learning, and when the input values of learning are the suprahyoid muscle group, the infrahyoid muscle group, and the laryngeal movement, a total of 265 dimensions is the number of input dimensions. The output value is two-dimensional, that is, the movement of the hyoid bone in the anterior-posterior direction and the movement in the vertical direction. In addition, when making predictions using test data, smoothing processing is performed on the prediction results.

Next, the evaluation index will be described.
RMSE (Root Mean Square Error) and Pearson's product moment correlation coefficient were used to verify the difference between the measured and predicted values obtained by the VF video. The result of RMSE and correlation coefficient shall be the average value of 6 kinds of learning results. RMSE is expressed by the formula 10 and is the most common performance index of the regression model, and it can be said that the smaller the error, the better the accuracy.

Pearson's product-moment correlation coefficient (hereinafter referred to as the correlation coefficient) is expressed by Equation 11, and it can be said that the larger the value, the higher the similarity of the waveforms and the higher the prediction accuracy.

Here, y _obs represents the measured value of the output, and y _pred represents the predicted value of the output.

Next, the result will be described.
Regarding the collected data, FIG. 35 shows the sEMG signal and laryngeal movement for one of the three swallows obtained in the present invention.

In addition, FIG. 36 shows time-series data of sEMG signals, RMS, CC and laryngeal movement and hyoid bone movement of each muscle group. The qr used here is the quefrency of the cepstrum coefficient (CC).

Regarding the prediction results in the X-axis direction (anterior-posterior direction) and the Y-axis direction (vertical direction), the measured values in each direction of the movement of the tongue bone are measured in the X-axis direction and the Y-axis direction of the tongue bone as shown in FIG. And one of the six predicted values is as shown in FIG. 37.

The locus of the movement of the hyoid bone in the sagittal plane (XY coordinate plane) corresponding to the result in FIG. 37 is as shown in FIG. 38. In addition, in FIG. 38, it is shown that the measured value and the predicted value are included.

Next, the accuracy of the motion prediction of the hyoid bone in the combination of muscle groups will be described.
FIG. 39 shows the results of RMSE, which are the results in the X-axis direction and the Y-axis direction in the combination of muscle groups.

FIG. 40 shows the results of the correlation coefficient, which are the results in the X-axis direction and the Y-axis direction in the combination of muscle groups.

From these results, it is considered that the suprahyoid muscles including the geniohyoid muscle act strongly on the muscles that move the hyoid bone in the X-axis direction. In addition, it is considered that the muscles that move the hyoid bone in the Y-axis direction are not moved only by each muscle group, but are moved by the coordinated movement of both muscle groups. Since the larynx is pulled up by the infrahyoid muscles in a follow-up manner when the hyoid bone moves upward by swallowing, it is considered that the accuracy is further improved by adding the information of the laryngeal movement by the elastic strain sensor.

In the prediction results of the sagittal plane of the hyoid bone, there were many prediction results that draw a triangular trajectory similar to the measured value. As a result, it is considered that a correct prediction could be made from the movement of the hyoid bone during swallowing.

Next, another example is shown in which the movement of the bolus rather than the movement of the hyoid bone is estimated. The teacher signal is the position change of the tip of the bolus (98w / w% barium sulfate solution 3ml with 1% thickening) taken by VF, and the learning conditions such as learning data including the feature amount are the motion estimation of the hyoid bone. It was the same as the case of. As shown in FIGS. 41 and 42, the measured value and the estimated value (learning A-prediction C) of the XY coordinates (origin is the lower end of the anterior margin of the fifth cervical vertebra) when the tip of the bolus passes through the esophageal entrance are almost the same. It can be confirmed that the movement of the bolus can be predicted with high accuracy.

The actions and effects of the above-mentioned method for evaluating the swallowing function and the system for evaluating the swallowing function will be described.
For example, other than the VF test, there is an echo test using an ultrasonic diagnostic device that can evaluate the swallowing function and the movement of the hyoid bone. However, in previous studies, adherent porridge is easy to detect with echo images, but it is difficult to observe the momentary dynamics from swallowing reflex to aspiration when swallowing, although there is fluidity such as water and jelly, and echography is performed. Then, it is difficult to identify aspiration. In addition, the observed image differs depending on how the probe is applied, making highly reproducible observation difficult. In this regard, according to the present invention, not only the hyoid bone but also the movements of at least one of the swallowing organs and the bolus, which can be observed by VF or VE, such as the timing of closing the larynx and the opening and closing of the esophageal entrance, can be predicted in the same manner. , Can be expected as an evaluation method without risks such as radiation exposure. In the present invention, if only one swallowing data can be acquired, the RMSE is 1.03 in the X-axis direction, the correlation coefficient is 0.64, the RMSE is 1.20 in the Y-axis direction, and the correlation coefficient is 0.96. It was confirmed that swallowing movement can be predicted under the same conditions.

According to the embodiment of the present invention, it includes a contrast moving image process (VF moving image process), a pretreatment process, a biological signal detection process synchronized with the VF moving image process, a feature amount extraction process, a learning process, and a prediction process. In the learning process, a teacher uses LSTM (long / short-term memory), which is a recurrent neural network architecture that learns long-term time dependence and short-term time dependence by determining the operation interval according to the feature quantity and the teacher signal. Learn the movement of the tongue bone based on signals and features. LSTM solves problems that cannot be learned with long-term time dependence when training with conventional RNNs, and can learn both long-term time dependence and short-term time dependence. When new inputs and outputs arrive in the learning process, the new patterns are adapted, and the problems of input weight collisions and output weight collisions that occur in RNNs can be dealt with. For this reason, the subject learns the characteristics of the subject's hyoid bone movement during swallowing by taking a biological signal at the sensor unit at the same time as the VF test only once at the beginning, and the VF test from the second time onward (prediction process). The movement of the hyoid bone can be predicted without it. As a result, the location of the X-ray fluoroscope is not required, and it is possible to easily evaluate the eating and swallowing function that predicts the movement of the hyoid bone even in bedside or home medical care, which is non-invasive and has low risk. In addition, if swallowing is performed when the same amount and the same physical property value are swallowed in the same way, the learning data becomes appropriate data once, but in order to obtain more suitable learning data, the amount and physical property value are changed. Regarding swallowing at that time, the swallowing data under that condition may be added to the learning as learning data.

Furthermore, if the subject first learns the movement of the subject's hyoid bone at least once at a hospital equipped with an X-ray fluoroscope, the movement of the hyoid bone can be predicted from the next time onward without any VF test. Can perform swallowing function evaluation and training. In addition, if swallowing is performed when the same amount and the same physical property value are swallowed in the same way, the learning data becomes appropriate data once, but in order to obtain more suitable learning data, the amount and physical property value are changed. It is possible to more preferably estimate the swallowing at that time by adding the swallowing data under the condition to the learning and using it as the learning data.

Further, in the biological signal detection step, the suprahyoid muscle group biological signal, the infrahyoid muscle group biological signal, and the laryngeal behavior signal are detected, so that the movement of the hyoid bone can be predicted with higher accuracy.

Further, in the learning process and the prediction process, one original data is shifted so as not to include the same coordinate data in a predetermined period and amplified into a plurality of pieces, so that the predicted value and the actual measurement are measured in the first learning. It is possible to reduce the error of the value and predict the movement of the hyoid bone with higher accuracy.

Furthermore, in the pretreatment step, the XY coordinate system in the lateral view of the subject is set, and the coordinates of the XY coordinate system are acquired with the lower end of the hyoid bone as the position of the hyoid bone. It is possible to make the prediction of movement easy to understand.

In the embodiment, the laryngeal behavior sensor is an elastic strain sensor, but the movement of the larynx is not limited to this, and the movement of the larynx may be a pressure sensor, an acceleration sensor, a non-contact sensor, or the like. Further, in the embodiment, the suprahyoid muscle group myoelectric sensor and the infrahyoid muscle group myoelectric sensor are used as array-shaped electrodes, but even if they are not arranged in an array at equal intervals, a plurality of (at least 2 channels) are used. It suffices to have the above-mentioned electrodes. A swallowing sound may be added. Further, in the above description and drawings, a portion meaning swallowing also has a portion described as swallowing, as appropriate.

Further, in the embodiment, LSTM (Long Short-Term Memory) derived from RNN (Recurrent Neural Network) was used as a prediction method of time series data, but the present invention is not limited to this, and RNN and GRU derived from RNN ( Gated Recurrent Unit), LSTNet (Long- and Short-term Time-series Network), AR (Autoregressive) model and ARMA (Autoregressive Moving Average), ARIMA (Autoregressive Integrated Moving Average), SARIMA (SARIMA) derived from the AR model. A prediction method for time series data such as a Seasonal AutoRegressive Integrated Moving Average) model may be used.

Further, in the embodiment, the swallowing imaging process is defined as a swallowing modeling test (VF test) that can be observed under fluoroscopy, but is not limited to this, and the swallowing endoscopy (VE test) using an endoscope is used. It may be observed, and further, the movement of the hyoid bone may be observed by echo and expressed in coordinates. The image taken in the swallowing photographing step is not limited to a moving image, and may be a still image as long as continuous changes are known, and the still image may be a frame-by-frame image.

In addition, in the prediction process, the movements of swallowing organs and bolus were predicted from biological signals during swallowing, but the prediction is not limited to this, and during training (direct training using food or no food is used). It may be used for predicting the movement of swallowing organs and eclipse from biological signals during indirect training (during so-called biofeedback training). Training for swallowing function such as direct training using food, which is a training method for swallowing function, evaluation of movement of swallowing organs in indirect training without food (basic training), and use for biofeedback training. Suitable for swallowing function training techniques.

That is, the present invention is not limited to the examples as long as the actions and effects of the present invention are exhibited.

The present invention detects a biological signal during swallowing, extracts a feature amount from the detected biological signal, predicts at least the movement of one of the swallowing organs and the bolus, and evaluates the swallowing function. Suitable for swallowing function evaluation technology. In addition, training on swallowing functions such as direct training using food, which is a training method for swallowing function, evaluation of movements of swallowing organs in indirect training without food (basic training), and use for biofeedback training. Suitable for swallowing function training techniques.

10 ... Eating and swallowing function evaluation system, 20 ... Sensor unit, 21 ... Suprahyoid muscle group myoelectric sensor, 21a ... Electrode, 22 ... Infrahyoid muscle group myoelectric sensor, 22a ... Electrode, 25 ... Laryngeal behavior Sensor (stretchable strain sensor), 30 ... multifunctional myoelectric potential measuring device, 40 ... analysis unit, 50 ... X-ray fluoroscope, 51 ... preprocessing unit, 60 ... subject.

Claims (5)

The swallowing imaging process, which captures the movement of the bolus and the movements of the swallowing organs eaten by the subject,
A pretreatment step of acquiring the positions of the swallowing organs and the bolus from the moving image of the swallowing imaging step and quantifying them as coordinates to create a teacher signal for the movement of the swallowing organs and the bolus.
A biological signal detection step of detecting a biological signal during swallowing by a sensor unit arranged on a predetermined skin surface of the subject in synchronization with the swallowing imaging step.
The feature amount extraction step of extracting the feature amount from the biological signal in the analysis unit,
The movements of the swallowing organs and the bolus are learned based on the teacher signal and the feature amount using the time-series data prediction method, and at least one of the swallowing organs and the bolus is moved from the feature amount. And the learning process to generate a model that can predict
Using the prediction model generated in the learning step, a prediction step of predicting the movement of at least one of the swallowing organs and the bolus from the feature amount,
Evaluation / training process for evaluating / training eating and swallowing function using the results of the prediction process, and
A swallowing function evaluation / training method using time-series data prediction, which is characterized by being equipped with. A swallowing function evaluation / training method using the time-series data prediction of claim 1.
In the biosignal detection step, the suprahyoid muscle group biosignal is detected by the suprahyoid muscle group myoelectric sensor placed in the suprahyoid muscle group portion, and the infrahyoid muscle placed in the suprahyoid muscle group portion. The suprahyoid muscle group biological signal is detected by the group myoelectric sensor, and the laryngeal behavior signal is detected by the laryngeal behavior sensor placed in the laryngeal part.
The feature amount extraction step is characterized in that the feature amount is extracted from the suprahyoid muscle group biological signal, the infrahyoid muscle group biological signal, and the laryngeal behavior signal as the biological signal. Swallowing function evaluation / training method using data prediction. A swallowing function evaluation / training method using the time-series data prediction according to claim 1 or 2.
In the learning step, the coordinate data of the swallowing organs and the bolus are used as the learning data, and one original data is shifted in a predetermined cycle so as not to include the same coordinate data, and a plurality of the learning data are used. A swallowing function evaluation / training method using time-series data prediction, which is characterized by being amplified to. A swallowing function evaluation / training method using the time-series data prediction according to any one of claims 1 to 3.
In the pretreatment step, a coordinate system for obtaining coordinate data of the swallowing organs and the bolus and its origin are determined, the lower end of the anterior margin of the fifth cervical vertebra of the subject is set as the origin, and the upper end of the anterior margin of the third cervical vertebra of the subject. By setting a coordinate system with Swallowing function evaluation / training method using time-series data prediction. The positions of the swallowing organs and the bolus are obtained from the images obtained by photographing the movements of the bolus and the swallowing organs eaten by the subject and quantified as coordinates, and the movements of the swallowing organs and the bolus are obtained. The pre-processing unit that creates the teacher signal and
A sensor unit that is placed on a predetermined skin surface of the subject and detects a biological signal during swallowing in synchronization with the acquisition of the image.
In addition to extracting the feature amount from the biometric signal, the movement of at least one of the swallowing organs and the bolus is learned and predicted based on the teacher signal and the feature amount by using the prediction method of time series data. A swallowing function evaluation / training system using time-series data prediction, which is characterized by having an analysis unit that evaluates / trains the swallowing function using this prediction result.

Images