WO2022227280A1 - 基于智能眼镜的灾害救援检伤分类及辅助诊断方法 - Google Patents

基于智能眼镜的灾害救援检伤分类及辅助诊断方法 Download PDF

Info

Publication number
WO2022227280A1
WO2022227280A1 PCT/CN2021/104948 CN2021104948W WO2022227280A1 WO 2022227280 A1 WO2022227280 A1 WO 2022227280A1 CN 2021104948 W CN2021104948 W CN 2021104948W WO 2022227280 A1 WO2022227280 A1 WO 2022227280A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
triage
auxiliary diagnosis
smart glasses
disaster rescue
Prior art date
Application number
PCT/CN2021/104948
Other languages
English (en)
French (fr)
Inventor
陈力
潘子杰
崔翔
王莉荔
刘红燕
陈骅
邓昭阳
杨华
王亚南
张爱舷
殷鹏
Original Assignee
中国人民解放军总医院第一医学中心
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 中国人民解放军总医院第一医学中心 filed Critical 中国人民解放军总医院第一医学中心
Publication of WO2022227280A1 publication Critical patent/WO2022227280A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G02B2027/0178—Eyeglass type

Definitions

  • the invention relates to the technical field of data classification, and more particularly, to a disaster rescue triage and auxiliary diagnosis method based on smart glasses.
  • the existing rescue medical system of "search and rescue-on-site rough inspection-transportation-hospital intensive inspection-treatment” has a prominent contradiction between the limited medical resources, detection capabilities and data at the disaster site and the need for accurate mass inspection (fast and not good), and the outstanding contradiction between the completeness of medical resources, detection capabilities and data in the rear hospital and the rapidity of mass triage at the disaster site (good but not fast).
  • the present invention researches the integration of artificial intelligence, big data technology and smart glasses technology to realize blood loss at disaster scene Efficient unification of timeliness and accuracy in sexual shock triage.
  • the traditional classification method adopts the manual diagnosis method and relies too much on the past experience of medical personnel, which not only wastes limited manpower, but also has shortcomings such as long time consumption, poor accuracy, and lack of quantitative standards, which make the triage classification unable to achieve the expected effect. It is particularly important to implement modern scientific, efficient and practical injury reduction classification methods. With the continuous development of artificial intelligence technology (AI) and augmented reality technology (AR), more and more new technologies are applied in the medical field. As a new thing that integrates the two technologies, smart glasses have been found to have great application prospects in triage and classification, and have achieved a certain degree of success.
  • AI artificial intelligence technology
  • AR augmented reality technology
  • Intelligent triage can not only classify and identify the injuries of the wounded quickly, accurately and efficiently, improve the timeliness and accuracy of triage, but also automate the traditional complex and professional triage technology, and non-professionals can also Complete the sorting, reduce the consumption of medical personnel, and improve the efficiency of rescue personnel.
  • the present invention provides a disaster rescue triage and auxiliary diagnosis method based on smart glasses.
  • the artificial intelligence auxiliary diagnosis system can not only overcome the artificial diagnosis results caused by fatigue caused by high-intensity work in a short period of time. Deviation, labor saving, fast and accurate injury classification can be achieved, and the new intelligent portable device, as the carrier of the auxiliary diagnosis system, can greatly promote the advancement of diagnosis and treatment. Therefore, the research and development of inspection and classification auxiliary diagnosis systems and equipment based on big data and artificial intelligence is an effective way to achieve "fast” and "accurate" inspection and classification.
  • the disaster rescue triage and auxiliary diagnosis method based on smart glasses includes the following steps:
  • the smart wearable device obtains visual, auditory and/or vital sign collection information
  • S5 Interact the first-generation blood loss-severe disease database obtained by S3 with the collection information obtained by S4 to generate the second-generation blood loss-severe disease database;
  • data preprocessing includes data cleaning, sample balance and standardization
  • Missing data processing includes deletion processing and filling processing. Delete features or samples with more missing data; fill in samples or features with few missing data; delete abnormal data;
  • the sample balance is used to modify the sample data set by using the resampling method, or adjust the weight of each category of data to correct the preference of the classifier in the case of sample imbalance;
  • Standardization Standardize or normalize numerical data in a cohort, scaling the data to a standard normal distribution.
  • the requirements for the casualty to obtain data include at least any of the following: the casualty is admitted to the hospital due to trauma, the casualty is aged ⁇ 18 years old, the casualty shock index is ⁇ 1.0, the average arterial pressure of the casualty is ⁇ 70 mmHg, and the casualty is in shock index. ⁇ 1.0 and mean arterial pressure ⁇ 70mmHg with blood transfusion records within 5 hours.
  • the data obtained by the wounded include: the whole process of vital signs, blood routine, blood biochemistry, coagulation function, blood gas analysis and/or urine routine during the treatment;
  • Feature engineering includes filtering, packing and embedding to select important features; principal component analysis, independent component analysis and linear discriminant analysis are used to form fewer new features from the original input attributes.
  • the filling processing method in the missing data processing includes the mean filling number algorithm, the median filling algorithm and the nearest neighbor filling algorithm.
  • the abnormal data processing method includes a scatter plot method and a box diagram method.
  • the data preprocessed in S2 is used to predict the outcome variables of traumatic hemorrhagic shock through machine learning algorithms, and the performance of the classifier model is evaluated, and the results obtained by various algorithms under different grouping index sets are obtained respectively. Calculate the accuracy, recall, precision and F value, and compare the results to obtain the key indicator model when the prediction result is optimal, and generate the first-generation blood loss-severe disease database.
  • the data preprocessed by S2 is independently repeated using the cellular genetic algorithm, and the identification ability is determined according to the number of times the index is retained in the screening, and a new key index set is formed; the application is based on precision
  • the AdaBoost algorithm is used to obtain the weight of each indicator when the prediction effect is optimal; the obtained indicators are screened, and the abnormal indicator data is deleted; the selected key indicators generate different key indicator data sets according to the number of reservations. , obtain classification models and key indicators through data grouping and machine learning algorithms.
  • the processing of auditory information includes recognizing the voice of the wounded, and extracting and identifying its category classification through voice intensity and intonation information.
  • the specific methods include preprocessing, feature extraction and classification model:
  • Pre-processing includes pre-emphasis, windowing and framing, and endpoint detection; pre-emphasis is used to pass the speech signal through a first-order high-pass digital filter to remove the radiation of the tongue and further improve the high-frequency resolution of the speech; Short-term stability assumption of speech signal, using Hamming window or rectangular window to divide speech into frames; endpoint detection is used to remove the silent part of speech signal, detect effective speech segments, and improve computational efficiency;
  • Feature extraction includes prosody feature extraction, spectral feature extraction and voice quality feature extraction
  • Classification models include Gaussian Mixture Models, Support Vector Machines, Recurrent Neural Networks, and Hidden Markov Models.
  • the specific method for extracting the features of the facial image from the image information obtained from the visual collection information includes the training of the Inception series network, the Resnet network or the Inception-Resnet-V2 network; when using Inception-Resnet-V2
  • the network training process includes the following steps:
  • p represents the model output after the activation function, that is, the probability that the predicted sample belongs to class 1, and the value is between 0 and 1;
  • y represents the label of the predicted sample, and the value is 0 or 1;
  • a parameter ⁇ is added to the cross entropy loss to balance the positive and negative samples, namely:
  • machine learning algorithms include K-nearest neighbor algorithm, logistic regression, naive Bayes, decision tree, support vector machine, Adaboost algorithm, random forest, gradient boosting tree and neural network.
  • the present invention has the following beneficial effects:
  • the artificial intelligence auxiliary diagnosis system can not only overcome the deviation of the diagnosis results caused by the fatigue caused by the labor of high-intensity work in a short period of time, save labor, and realize fast and accurate inspection and classification.
  • the carrier can greatly promote the advancement of diagnosis and treatment. Therefore, the research and development of inspection and classification auxiliary diagnosis systems and equipment based on big data and artificial intelligence is an effective way to achieve "fast” and "accurate” inspection and classification.
  • FIG. 1 is a schematic structural diagram of a smart glasses-based disaster rescue triage and auxiliary diagnosis method according to an embodiment of the present invention
  • FIG. 2 is a 10-fold cross-comparison diagram of an embodiment of the present invention.
  • the technical route to be adopted in the present invention is as follows: as shown in Fig. 1, in the modeling and key index acquisition part of blood loss-severe disease prediction, first collect data on traumatic hemorrhagic shock victims from massive trauma data through data retrieval technology, After cleaning and standardizing the obtained data, the key indicators of traumatic hemorrhagic shock were extracted by pre-screening method, and the hemorrhagic-severe disease database 1.0 was constructed using the screened key indicators.
  • Machine learning algorithms are applied to analyze and model these time series data, and a first-level blood loss-severe-severe prediction AI model is established, so that it can use clinical data to predict and warn of traumatic hemorrhagic shock.
  • FIG. 1 Design a smart wearable device for triage and classification, as shown in Figure 1.
  • the key technology part of smart glasses development is to collect relevant clinical indicator data through smart wearable technology and various sensors, including visual, auditory, vital signs and other indicators that must be collected and can be collected. index. Carry out research and development on the four main modules of data collection, data transmission, data analysis and data presentation involved in smart wearable devices, and study the key technologies of smart wearable device information collection, so as to accurately obtain the key visual, auditory, and vital signs of traumatic hemorrhagic shock. signs.
  • the prototype design and clinical data collection of the triage and classification glasses principle prototype are optimized and the verification part of the function of predicting hemorrhagic shock is based on the key indicators of vision, hearing and vital signs and the sensor function of smart wearable devices.
  • Demonstration on the basis of blood loss-severe disease database 1.0, integrate the clinical index data collected by smart wearable devices, and construct blood loss-severe disease database 2.0.
  • Blood Loss-Severe Disease Database 2.0 mainly uses machine learning or deep learning algorithm to model, analyze and verify the collected voice data, non-image data and image data through feature extraction/selection, and research and design corresponding multi-modal data
  • the fusion method can effectively extract the complete information of different modalities through different data fusion methods, so as to generate more abundant feature expressions to improve the prediction performance of the classification model, and finally establish a blood loss-severe prediction AI that can predict and warn traumatic hemorrhagic shock.
  • Model 2 the detailed scheme of the key method is as follows:
  • the blood loss-severe disease database 1.0 uses machine learning algorithms to analyze and model these time series data, so that it can predict and warn of traumatic hemorrhagic shock.
  • the individual information, biochemical indicators and other non-image data of the samples in the blood loss-severe disease data 1.0 and 2.0 need further processing, including: data cleaning, sample balance and standardization.
  • SI Shock Index
  • MBP Mean arterial pressure
  • Exclusion criteria (1) The above conditions were not met at the same time, or the situation occurred within 4 hours from the time point of hemorrhagic shock in the hospital (this time period is used for the prediction and early warning study of traumatic hemorrhagic shock in the later period).
  • Contents of data collection Extract the vital signs, blood routine, blood biochemistry, coagulation function, blood gas analysis, urine routine and other specific indicators of the wounded during their entire stay in the hospital.
  • Data cleaning includes missing data and abnormal data processing. There are two ways to deal with missing data: deletion or filling. For features or samples with more missing data, it is to be deleted directly; for samples or features with less missing data, fill in. Data filling methods include mean filling, median filling, and nearest neighbor filling algorithms. Identify outliers through data visualization methods, such as drawing scatter plots or box plots, and delete samples containing outliers.
  • Sample balance Observe the proportion of each category. If there is a sample imbalance, it is proposed to use the resampling method to modify the sample data set to ensure the sample balance, or adjust the weight of each category of data to correct the classifier's preference for the majority class.
  • Standardization Since the classification prediction model is sensitive to the feature dimension, in order to eliminate the influence of the dimension, it is necessary to standardize or normalize the numerical data in the queue, and scale the data to a standard normal distribution.
  • Blood loss-severe disease database construction predict the outcome variables of traumatic hemorrhagic shock through machine learning algorithms, evaluate the performance of the classifier model, and obtain the accuracy, recall, and precision rates calculated by various algorithms under different grouping index sets. , F value, and compare the results to obtain the key index model when the prediction results are optimal, and establish it as the blood loss-severe disease database 1.0.
  • the clinical index data collected by smart wearable devices was integrated to construct blood loss-severe disease database 2.0.
  • the selected key indicators have different ability to identify injuries, that is, the number of times they are screened and retained is different.
  • the key indicators are reorganized and grouped, and the time series data composed of indicators with a retention number of 6 or more are set as the key indicator data set 1, which will be retained
  • the time series data composed of indicators with a frequency greater than or equal to 7 times is the key indicator data set 2
  • the time series data composed of indicators with a retention frequency greater than or equal to 8 times is the key indicator data set 3.
  • the best classification models and key predictors can be obtained by grouping data and using machine learning classification algorithms.
  • Speech is a mixed carrier of various information, which can convey a large amount of information, including content information, speaker information and emotional information. Through the changes of voice and intonation, not only the emotional state of the speaker can be obtained, but also the severity of the injured in casualty events.
  • the intelligent wearable device for triage and classification designed and developed by the present invention will issue voice commands to the wounded according to the language evaluation module in the CRAMS scoring method and the scoring scale method, grade the injury through the wounded's verbal response, and collect the wounded's voice data to Speech Recognition for Blood Loss-Severe Prediction Models.
  • the speech recognition based on the blood loss-severe prediction model mainly extracts the sound intensity and intonation information contained in the speech and recognizes its classification, including three parts: preprocessing, feature extraction and classification model.
  • the preprocessing mainly includes three parts: pre-emphasis, windowing and framing, and endpoint detection.
  • Pre-emphasis is to pass the speech signal through a first-order high-pass digital filter to remove the radiation of the mouth and tongue and further improve the high-frequency resolution of the speech; windowing and framing are based on the inertia of the vocal organ, using the short-term stability assumption of the speech signal, using The Hamming window or the rectangular window divides the speech into frames.
  • the adjacent frames are usually partially overlapped.
  • the frame length is 20ms, and the frame shift is 10ms; endpoint detection is a method that can effectively remove speech signals. The silent part is detected, and the effective speech segment is detected, thereby improving the computational efficiency.
  • Prosody feature is one of the most important features extracted based on human phonetics knowledge in the field, including pitch frequency, zero-crossing rate, short-term energy and formant, etc.
  • the changes of these prosodic features constitute different emotions in speech, which can effectively characterize the changes of intonation and pronunciation intensity.
  • the spectral feature starts from the structure of the human ear and the sound processing mechanism, and uses the triangular Mel filter bank to simulate the characteristics of the human ear basilar membrane with different resolutions for different frequency signals.
  • the sound quality characteristics refer to the characteristics of human speech spectrum and timbre in different emotional states, and the harmonic-to-noise ratio is a commonly used one.
  • the goal of speech recognition is to classify it into different categories based on different features, which is a typical classification problem in machine learning.
  • the widely used model methods are: Gaussian mixture model, support vector machine, recurrent neural network and hidden Markov model.
  • the present invention intends to train classification models based on support vector machines and neural networks respectively, and uses the collected speech data to assist blood loss-severe prediction.
  • the non-image data in the present invention mainly refers to specific indicators such as vital signs, blood routine, blood biochemistry, coagulation function, blood gas analysis, and urine routine of the wounded during the hospitalization period collected by the critical care database, and the processing mainly includes feature engineering and machine learning.
  • the classification model has two parts.
  • Feature engineering is a way of transforming datasets into multi-column feature (attribute) data that can characterize samples.
  • Feature engineering is very important in machine learning. Its purpose is to screen out better features. Better features can be trained with simple models and can get better results.
  • Feature engineering generally includes feature extraction and feature selection, both of which have the same effect, and both aim to reduce the number of attributes (or features) in the feature data set, but the methods and methods used by the two are different.
  • Feature extraction is mainly through the relationship between attributes, such as combining different attributes to obtain new attributes, which changes the original feature space; feature selection is to select a subset from the original feature data set, which is an inclusive relationship. Change the original feature space.
  • the database constructed by the present invention contains a large amount of non-image data, and each sample has many specific clinical indicators, so feature selection can be used, such as filter method (Filter), wrapping method (Wrapper), embedded method (Embedded) to select important features,
  • feature extraction methods such as Principal Component Analysis (PCA), Independent Component Analysis (ICA), and Linear Discriminant Analysis (LDA) can be used to form fewer new features from the original input attributes.
  • PCA Principal Component Analysis
  • ICA Independent Component Analysis
  • LDA Linear Discriminant Analysis
  • Each sample in the database contains a large number of clinical data feature dimensions, and too many features may cause slow computation, resulting in low prediction and classification performance.
  • feature engineering that is to use feature extraction or feature selection to process non-image data, redundant features and features with low contribution can be found, these features can be eliminated, and those features with high representation are left.
  • a high-performance predictive classification model can be trained.
  • the intuitive clinical manifestations of hemorrhagic shock are pale skin, peripheral venous infill, and altered mental status.
  • the smart wearable device for inspection and classification in the present invention can capture facial images or videos of the wounded through the camera integrated on the glasses, so as to collect image information such as the complexion, expression, and demeanor of the wounded.
  • image information such as the complexion, expression, and demeanor of the wounded.
  • image feature extraction methods based on deep learning have shown great advantages compared with manual or traditional feature extraction methods.
  • the present invention intends to perform feature extraction on the facial image collected by the smart wearable device through a deep neural network method.
  • Inception series networks combine convolution kernels of different sizes in parallel to obtain sparse or non-sparse features on the same layer, which increases the "width" of the network and the adaptability of the network to the image scale; while Resnet introduces residual connections, It effectively solves the problems of gradient disappearance and gradient explosion, and can ensure good performance while training deeper networks.
  • Inception-Resnet-V2 is realized by adding Resnet's residual connection on the basis of Inception-V4 network, which makes the network converge faster, and the classification accuracy is also improved, which further improves the network performance. Therefore, the present invention intends to select Inception-Resnet-V2 as the basic image feature extraction network.
  • the Focal Loss function is a loss function improved on the basis of the standard cross entropy loss. By reducing the weight of easy-to-learn samples, the model is more focused on training. Hard to learn samples.
  • the standard binary cross-entropy loss function is as follows:
  • p represents the model output after the activation function, that is, the probability that the predicted sample belongs to class 1, and the value is between 0 and 1.
  • y represents the label of the predicted sample, and the value is 0 or 1.
  • CE in formula (1) is the standard cross-entropy loss, which can also be expressed by CE standard .
  • CE in formula (2) balances the cross-entropy loss of positive and negative samples, and can also be represented by CE positive and negative .
  • CE in formula (3) means that both positive and negative samples are balanced, and the cross-entropy loss of difficult and easy samples is balanced, and it can also be expressed by CE difficulty .
  • Machine learning is a discipline that studies computing methods from a large amount of data and uses experience to improve system performance. Machine learning uses statistical knowledge, designs algorithms based on existing data, and trains to generate certain models. When new data comes , use the model to make judgments.
  • Machine learning can be divided into supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, etc.
  • Supervised learning is a sample dataset with manual annotation, including the attribute features and category labels of the samples. At the same time, it is taught to the computer to learn the inherent relationship between attribute features and category labels. The focus here is to use supervised learning methods for modeling and classification. Commonly used classification algorithms include K-nearest neighbor algorithm, logistic regression, naive Bayes, decision tree, support vector machine, Adaboost algorithm, random forest, gradient boosting tree and neural network algorithms.
  • the training set was tested and compared with different classification algorithms, and the best model was obtained by repeated experiment verification, and the precision rate, recall rate and F1 value were calculated. Combined with the verification of medical experts, the best machine learning model was selected.
  • the feature weights are sorted again, and the important features of severe classification are screened out.
  • a knowledge map and knowledge base can be established to make these features interpretable for the classification of severe disease.
  • Deep residual learning is performed by skipping one or more convolutional layers by introducing a so-called Identity Shortcut Connection.
  • identity shortcut connections between convolutional residual blocks the deep neural network can learn the residuals mapped by the bottom layer instead of just learning the direct mapping from the previous hidden layer, so that it can be more It can solve the degradation problem caused by network deepening and the potential problem of gradient dispersion or gradient explosion, so that the model has better classification performance.
  • Multimodal fusion refers to combining the information of multiple modalities to classify or regress targets, which can often achieve better performance and effects than single-modality methods.
  • the data that the present invention can use for classification prediction are multimodal, including speech data, non-image data and image data.
  • Multimodal data fusion methods include: EarlyFusion/Data Level Fusion, Intermediate Fusion and Late Fusion/Decision Level Fusion.
  • Front-end fusion is to fuse each modal data into a feature vector, which is then used for model prediction; mid-end fusion converts different modal data into high-dimensional feature expressions, and then fuses them in the middle layer of the model; back-end fusion is It trains prediction models separately based on different modal data, and then fuses the model outputs.
  • Front-end fusion of multi-modal data often cannot fully utilize the complementarity between multiple modal data, and the raw data of front-end fusion usually contains a lot of redundant information. Therefore, front-end fusion methods usually need to be combined with feature extraction methods to remove redundant information.
  • Back-end fusion is to fuse the output decisions of classifiers trained on different modal data separately.
  • the advantage of this is that the errors of the fusion model come from different classifiers, and the errors from different classifiers are often uncorrelated and independent of each other, and will not cause further accumulation of errors.
  • One of the advantages of the intermediate fusion method is that the location of fusion can be selected flexibly, and the flexible fusion method can significantly improve the prediction performance of the algorithm.
  • the present invention intends to use back-end and mid-end fusion methods to fuse voice data, non-image data and image data, and compare their effects.

Landscapes

  • Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Biomedical Technology (AREA)
  • General Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Epidemiology (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • Primary Health Care (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Pathology (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Optics & Photonics (AREA)
  • Evolutionary Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Abstract

一种基于智能眼镜的灾害救援检伤分类及辅助诊断方法，涉及数据的分类技术领域，实现"快速"、"精准"检伤分类的效果，具体方案为：包括以下步骤：S1：获取数据；S2：数据预处理；S3：提取创伤失血性休克关键指标，生成初代失血-重症数据库；S4：智能穿戴设备获取视觉、听觉和/或生命体征的采集信息；S5：将S3获得的初代失血-重症数据库与S4获得的采集信息交互，生成二代失血-重症数据库。基于智能眼镜的灾害救援检伤分类及辅助诊断方法克服人工因短时间内高强度工作而带来的疲劳导致的诊断结果偏差，节省劳力，能够实现快速、精准的检伤分类，而且新型智能便携式设备作为辅助诊断系统的载体，能够极大推进诊疗前移。

Description

基于智能眼镜的灾害救援检伤分类及辅助诊断方法
本申请要求于2021年04月30日提交中国专利局、申请号为202110484108.2、发明名称为“基于智能眼镜的灾害救援检伤分类及辅助诊断方法”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本发明涉及数据的分类技术领域，更具体地说，它涉及基于智能眼镜的灾害救援检伤分类及辅助诊断方法。
背景技术
恐怖袭击、局部战争、自然灾害、火灾和重大交通事故等给全世界的急救医生带来了挑战。这些大规模伤亡事件，导致大量伤员不断涌现，造成伤员数量多、伤情复杂等情况，往往超过当地救治能力。在这种情况下，如何运用有限的医务人员、急救药品、仪器设备和运输工具，使更多的伤员得到及时有效的救治显得格外重要，因此检伤分类成为大规模伤亡事件救治过程中最为关键的环节。快速、准确的检伤分类可以将有限的资源最优化，使危重症患者优先得到救治，从而提高伤病员救治的成功率，尽可能挽救更多患者。然而现有的“搜救-现场粗检-转运-医院精检-治疗”救援医疗体系，存在灾难现场医疗资源、检测能力和数据的有限性与群体性检伤的精准性需求的突出矛盾(快而不好)，和后方医院院内医疗资源、检测能力和数据的完备性与灾难现场群体性检伤的快速性需求的突出矛盾(好而不快)。为了从根本上解决现有灾难现场搜救检伤分类中“快而不好”和“好而不快”的矛盾，本发明研究将人工智能与大数据技术和智能眼镜技术相融合，实现灾难现场失血性休克检伤时效性和准确性的高效统一。
传统的分类方法采用人工诊断方式，过度依赖医务人员的既往经验，不仅浪费有限的人力，同时还有耗时长、准确性差、缺乏量化标准等缺点，使检伤分类无法达到预期的效果，因此发展和施行现代化科学、高效、实用的减伤分类方法显得尤为重要。随着人工智能技术(AI)和增强现实 技术(AR)的不断发展，越来越多的新技术被应用于医疗领域。智能眼镜作为融合两种技术的新生事物，被发现在检伤分类方面有巨大应用前景，并取得了一定程度的成功。智能化的分检不仅可以快速、准确、高效地对伤员伤情进行分类辨识，提高检伤分类的时效性和准确性，还可以将传统复杂专业的检伤分类技术自动化，非专业人员也可以完成分检，减少医务人员的消耗，提高救援人员的使用效率。
发明内容
近年来，基于大数据与人工智能的辅助诊断技术、新型智能便携式与可穿戴设备、无人机系统等科技的迅速发展为以上难题的解决带来了新的思路。
为解决上述技术问题，本发明提供一种基于智能眼镜的灾害救援检伤分类及辅助诊断方法，人工智能辅助诊断系统不仅可以克服人工因短时间内高强度工作而带来的疲劳导致的诊断结果偏差，节省劳力，能够实现快速、精准的检伤分类，而且新型智能便携式设备作为辅助诊断系统的载体，能够极大推进诊疗前移。因此，研发基于大数据和人工智能的检伤分类辅助诊断系统和装备，是实现“快速”、“精准”检伤分类的效果途径。
本发明的上述技术目的是通过以下技术方案得以实现的：
基于智能眼镜的灾害救援检伤分类及辅助诊断方法，包括以下步骤：
S1：获取数据；
S2：数据预处理；
S3：提取创伤失血性休克关键指标，生成初代失血-重症数据库；
S4：智能穿戴设备获取视觉、听觉和/或生命体征的采集信息；
S5：将S3获得的初代失血-重症数据库与S4获得的采集信息交互，生成二代失血-重症数据库；
S2过程中，数据预处理包括数据清洗、样本均衡和标准化处理；
数据清洗包括缺失数据处理和异常数据处理，缺失数据处理包括删除处理和填充处理，对于缺失数据多的特征或样本删除；对于缺失数据少的样本或特征进行填充；对异常数据进行删除；
样本均衡用于在样本失衡情况下，采用重采样方法修改样本数据集，或调整各类别数据的权重修正分类器的偏好；
标准化处理：对队列中的数值数据进行标准化或归一化处理，将数据缩放为标准正态分布。
作为一种优选方案，S1过程中，获取数据的伤员要求至少包括以下任意一种：伤员因创伤入院，伤员年龄≥18岁，伤员休克指数≥1.0，伤员平均动脉压≤70mmHg，伤员在休克指数≥1.0且平均动脉压≤70mmHg后5小时内有输血记录。
作为一种优选方案，S1过程中，伤员获取的数据包括：治疗期间的全程生命体征、血常规、血生化、凝血功能、血气分析和/或尿液常规；
对数据处理的方法包括特征工程和机器学习分类模型；
特征工程包括过滤法、包装法和嵌入法选取重要特征；采用主成分分析、独立成分分析和线性判别分析从原始输入属性中形成较少的新特征。
作为一种优选方案，缺失数据处理中的充填处理方法包括平均数填充数算法、中位数填充算法和近邻填充算法。
作为一种优选方案，异常数据处理方法包括散点绘制图示法和箱体图示法。
作为一种优选方案，S3过程中，将S2预处理后的数据通过机器学习算法对创伤失血性休克结局变量进行预测，对分类器模型性能进行评价，分别获得不同分组指标集下各种算法所计算出的准确率、召回率、精准率和F值，并对结果进行比较，得到预测结果最优时的关键指标模型，生成初代失血-重症数据库。
作为一种优选方案，S3过程中，经S2预处理得到的数据使用元胞遗传算法进行独立重复试验，根据指标在筛选中被保留次数判定其识别能力，形成新的关键指标集；应用基于精度的AdaBoost算法，获得各指标在使 预测效果达到最优时所占权重；对获得的指标进行筛选，将异常指标数据删除；被筛选出的关键指标根据保留次数的不同生成不同的关键指标数据集，通过数据分组和机器学习算法得到分类模型和关键指标。
作为一种优选方案，S5过程中，听觉信息的处理包括将伤员语音进行识别，通过声音强弱及语调信息提取出来并识别出其类别分类，具体方法包括预处理、特征提取及分类模型：
预处理包括预加重、加窗分帧和端点检测；预加重用于将语音信号通过一个一阶高通数字滤波器，去除口舌辐射，进一步提高语音的高频分辨率；加窗分帧用于通过语音信号的短时平稳假设，使用汉明窗或矩形窗将语音划分成帧；端点检测用于去除语音信号的静音部分，检测出有效的语音片段，提高计算效率；
特征提取包括韵律学特征提取、谱特征提取和音质特征提取；
分类模型包括高斯混合模型、支持向量机、循环神经网络和隐马尔可夫模型。
作为一种优选方案，S5过程中，视觉采集信息获取的图像信息对面部图像进行特征提取的具体方法包括Inception系列网络、Resnet网络或Inception-Resnet-V2网络进行训练；在使用Inception-Resnet-V2网络进行训练过程中包括以下步骤：
标准的二分类交叉熵损失函数如下：
Figure PCTCN2021104948-appb-000001
其中：p表示经过激活函数的模型输出，即预测样本属于1类的概率，数值在0-1之间；y表示预测样本的标签，数值为0或1；
交叉熵损失的基础上加一个参数α，以平衡正负样本，即：
Figure PCTCN2021104948-appb-000002
再引入新的参数γ，公式如下：
Figure PCTCN2021104948-appb-000003
作为一种优选方案，机器学习算法包括K近邻算法、逻辑回归、朴素贝叶斯、决策树、支持向量机、Adaboost算法、随机森林、梯度提升树和神经网络。
综上所述，本发明具有以下有益效果：
人工智能辅助诊断系统不仅可以克服人工因短时间内高强度工作而带来的疲劳导致的诊断结果偏差，节省劳力，能够实现快速、精准的检伤分类，而且新型智能便携式设备作为辅助诊断系统的载体，能够极大推进诊疗前移。因此，研发基于大数据和人工智能的检伤分类辅助诊断系统和装备，是实现“快速”、“精准”检伤分类的效果途径。
说明书附图
图1是本发明实施例的基于智能眼镜的灾害救援检伤分类及辅助诊断方法的结构示意图；
图2是本发明实施例的10折交叉对比图。
具体实施方式
本说明书及权利要求并不以名称的差异来作为区分组件的方式，而是以组件在功能上的差异来作为区分的准则。如在通篇说明书及权利要求当中所提及的“包括”为一开放式用语，故应解释成“包括但不限定于”。“大致”是指在可接受的误差范围内，本领域技术人员能够在一定误差范围内解决所述技术问题，基本达到所述技术效果。
本说明书及权利要求的上下左右等方位名词，是结合附图以便于进一步说明，使得本申请更加方便理解，并不对本申请做出限定，在不同的场景中，上下、左右、里外均是相对而言。
以下结合附图对本发明作进一步详细说明。
参见图1，本发明拟采取的技术路线如下：如图1所示的失血-重症 预测的建模与关键指标获取部分，首先通过数据检索技术从海量创伤数据中收集创伤失血性休克伤员数据，将所获得的数据进行数据清洗和标准化处理后，通过预筛选方法提取创伤失血性休克关键指标，使用筛选后的关键指标构建失血-重症数据库1.0。
应用机器学习算法对这些时序数据进行分析及模型验证，建立第一级失血-重症预测AI模型，从而使其能利用临床数据对创伤失血性休克进行预测预警。
设计检伤分类智能穿戴设备，如图1所示的智能眼镜研发的关键技术部分，通过智能穿戴技术及各种传感器采集相关临床指标数据，包括视觉、听觉、生命体征等必须采集指标及可采集指标。对智能穿戴设备所涉及到的数据采集、数据传输、数据分析及数据呈现四个主要模块进行研究与开发，研究智能穿戴设备信息采集关键技术，以准确获取创伤失血性休克关键视觉、听觉、生命体征。如图1所示的检伤分类眼镜原理样机设计与临床数据采集以及检伤分类眼镜原理样机优化与预测失血性休克功能的验证部分，基于视觉、听觉、生命体征关键指标与智能穿戴设备传感器功能论证，在失血-重症数据库1.0的基础上整合智能穿戴设备所采集到的临床指标数据，构建失血-重症数据库2.0。失血-重症数据库2.0主要是通过特征提取/选择，使用机器学习或深度学习算法对采集到的语音数据、非图像数据及图像数据进行建模、分析与验证，并且研究设计相应的多模态数据融合方法，通过不同的数据融合方式有效地提取不同模态的完整信息，从而产生更为丰富的特征表达来提高分类模型的预测性能，最终建立可以预测预警创伤失血性休克的失血-重症预测AI模型2。其中，关键方法的详细方案如下所述：
(1)失血-重症数据库建立
通过数据检索技术从海量创伤数据中收集创伤失血性休克伤员数据，将所获得的数据进行数据清洗和标准化处理后，通过预筛选方法提取创伤失血性休克关键预测指标，使用筛选后的关键指标构建失血-重症数据库1.0，应用机器学习算法对这些时序数据进行分析及模型验证，使其能对创伤失血性休克进行预测预警。此外，失血-重症数据1.0和2.0中样本的 个体信息、生化指标等非图像数据还需进一步处理，具体包括：数据清洗、样本均衡及标准化处理。
纳入标准：(1)因创伤入院且年龄≥18岁；(2)休克指数(Shock Index，SI)≥1.0，即相同时间心率(次/分)/收缩压(mmHg)≥1；(3)平均动脉压(MeanBloodPressure，MBP)≤70mmHg；(4)在伤员生命体征同时满足SI≥1.0和MBP≤70mmHg后5小时内有输血记录。
排除标准：(1)未同时满足以上条件，或在院时间距离失血性休克时间点不足4小时即出现该情况(该时间段用于后期进行创伤失血性休克预测预警研究)。
数据收集内容：提取伤员在院期间全程生命体征、血常规、血生化、凝血功能、血气分析、尿液常规等具体指标，如为空值则记录为‘NA’。
数据清洗：数据清洗包括缺失数据和异常数据处理。缺失数据处理有两种方法：删除或填充，对于缺失数据较多的特征或样本拟直接删除；对于缺失数据较少的样本或特征进行填充。数据填充方法有平均数填充、中位数填充、近邻填充算法等。通过数据可视化的方法对异常值进行识别，如绘制散点图或箱体图，删除含有异常值的样本。
样本均衡：观察各类别的比例，若出现样本失衡，拟采用重采样方法修改样本数据集，保证样本均衡，或通过调整各类别数据的权重来修正分类器对多数类的偏好。
标准化处理：由于分类预测模型对特征量纲敏感，为消除量纲的影响，需对队列中的数值数据进行标准化或归一化处理，将数据缩放为标准正态分布。
失血-重症数据库构建：通过机器学习算法对创伤失血性休克结局变量进行预测，对分类器模型性能进行评价，分别获得不同分组指标集下各种算法所计算出的准确率、召回率、精确率、F值，并对结果进行比较，得到预测结果最优时的关键指标模型，并建立为失血-重症数据库1.0。基于视觉、听觉、生命体征关键指标与智能穿戴设备传感器功能论证，在失血-重症数据库1.0的基础上整合智能穿戴设备所采集到的临床指标数据，构建失血-重症数据库2.0。
(2)关键指标预筛选
从海量创伤数据中收集创伤失血性休克伤员数据，将所获得的数据进行数据清洗和标准化处理后，使用元胞遗传算法进行独立重复实验，根据指标在筛选中被保留次数判定其识别能力，进而形成新的关键指标集；应用基于精度的AdaBoost算法，获得各项指标在使预测效果达到最优时所占权重。从临床角度对指标筛选结果合理性进行解释说明，对部分与临床固有知识可能相悖的指标进行删除，得到与创伤失血性休克相关性较强的关键指标数据库。
被筛选出关键指标对伤情识别能力不同，即被筛选保留次数不同，对关键指标进行重组及分组，将保留次数大于等于6次的指标组成的时间序列数据为关键指标数据集①，将保留次数大于等于7次的指标组成的时间序列数据为关键指标数据集②，将保留次数大于等于8次的指标组成的时间序列数据为关键指标数据集③。通过数据分组和使用机器学习分类算法可以得到最好的分类模型及关键预测指标。
(3)语音数据处理及识别
语音是多种信息的混合载体，可以传递大量的信息，包括内容信息、说话人信息和情感信息等。通过语音和语调的变化，不仅可以获知说话人的情感状态，在伤亡事件中还可以了解伤者的严重程度。本发明所设计研发的检伤分类智能穿戴设备将依据CRAMS评分法及评分量表法中的语言评估模块对伤员发出语音指令，通过伤员的言语反应对伤情进行分级，并且采集伤员语音数据以用于失血-重症预测模型的语音识别。
基于失血-重症预测模型的语音识别主要是将蕴含在语音中的声音强弱及语调信息提取出来并识别出其类别分类，包括预处理、特征提取及分类模型三部分。预处理主要包括预加重、加窗分帧和端点检测3个部分。预加重是将语音信号通过一个一阶高通数字滤波器，去除口舌辐射，进一步提高语音的高频分辨率；加窗分帧是从发声器官的惯性出发，利用语音信号的短时平稳假设，使用汉明窗或者矩形窗将语音划分成帧，同时为了保证帧间平滑，通常使得相邻帧之间部分重叠，一般取帧长为20ms，帧移10ms；端点检测是一种能够有效去除语音信号的静音部分，检测出有 效的语音片段，从而提高计算效率的方法。
特征提取是语音识别问题的重难点，好的特征能够在有效区分不同类别的同时，对类间差异具有较好的鲁棒性。目前常用的特征主要有韵律学特征、谱特征和音质特征等。韵律学特征是领域内基于人类语音学知识提取的最为主要的特征之一，主要包括基音频率、过零率、短时能量和共振峰等。这些韵律特征的变化构成了语音中不同的情感，能够有效的表征语调的变化和发音强度。谱特征从人耳的构造和声音处理机制出发，利用三角梅尔滤波器组来模拟人耳基底膜对不同频率信号分辨率不同的特性。音质特征是指人在不同情感状态下语谱和音色方面的特征，谐波噪声比是目前常用的一种。
语音识别的目标是依据不同特征将其划分为不同类别，属于机器学习中典型的分类问题。目前被广泛使用到的模式方法有：高斯混合模型、支持向量机、循环神经网络和隐马尔可夫模型等。本发明拟分别训练基于支持向量机和神经网络的分类模型，利用采集到的语音数据辅助失血-重症预测。
(4)非图像数据处理
本发明中的非图像数据主要为重症数据库收集到的伤员在院期间全程生命体征、血常规、血生化、凝血功能、血气分析、尿液常规等具体指标，其处理主要包括特征工程及机器学习分类模型两部分。
特征工程是将数据集转成多列特征(属性)数据，能够表征样本的一种方式。特征工程在机器学习中非常重要，它的目的是筛选发现出更好的特征，更好的特征可以用简单的模型训练，可以得到更好的结果。特征工程一般包含特征提取和特征选择，这两者要达到的效果是一样的，均以减少特征数据集中的属性(或者称为特征)的数目为目的，但两者所采用的方式方法不同。
特征提取主要是通过属性间的关系，如组合不同的属性得到新的属性，这样就改变了原来的特征空间；特征选择是从原始特征数据集中选择出子集，是一种包含的关系，没有更改原始的特征空间。本发明构建的数据库包含海量非图像数据，且每例样本的临床具体指标众多，因此既可以采用 特征选择，如过滤法(Filter)、包装法(Wrapper)、嵌入法(Embedded)选取重要特征，又可以采用主成分分析(PCA)、独立成分分析(ICA)、线性判别分析(LDA)等特征提取的方法从原始输入属性中形成较少的新特征。数据库中每个样本所包含的临床数据特征维度非常多，过多的特征可能会造成计算缓慢，导致预测分类性能较低。通过特征工程，即使用特征提取或特征选择对非图像数据进行处理，可以找出冗余特征及贡献度低的特征，把这些特征剔除掉，留下那些具有高表征性的特征，在选择合适的机器学习算法后，可以训练出性能高的预测分类模型。
(5)图像数据特征提取
失血性休克直观临床表现为皮肤苍白、外周静脉不充盈和神志改变。通过分析图像中皮肤和头部静脉的纹理信息，可以判断伤者失血的重症程度，眼睛神态特征则可以在一定程度上反映伤者的精神状态。本发明中用于检伤分类的智能穿戴设备可以通过集成在眼镜上的摄像头拍摄伤员面部图像或视频，从而采集到伤员的面色、表情、神态等图像信息。此时，图像特征提取能力的好坏会直接影响后续分类模型的预测效果。近年来，基于深度学习的图像特征提取方法与人工或传统特征提取方法相比展现出了巨大的优势。为此，本发明拟通过深度神经网络方法对智能穿戴设备采集到的面部图像进行特征提取。
Inception系列网络是将不同尺寸的卷积核并行合并，在同一层上获得稀疏或非稀疏的特征，增加了网络的“宽度”以及网络对图像尺度的适应性；而Resnet通过引入残差连接，有效地解决了梯度消失和梯度爆炸问题，在训练更深网络的同时，又能保证良好的性能。Inception-Resnet-V2就是在Inception-V4网络基础上加入了Resnet的残差连接来实现的，从而使得网络收敛速度更快，分类精度也有所提升，进一步改善了网络性能。因此，本发明拟选用Inception-Resnet-V2作为基础的图像特征提取网络。
在实际训练过程中发现：样本数据集中存在少量的困难样本(Hard Sample)，即难学习的样本，其产生的损失(Loss)较大；而大多数样本都是易学习的样本，其损失(Loss)较小，进而造成难易样本数量不平衡的问题(区别于正负样本不平衡)。为此本发明拟采用Focal Loss函数解 决这一问题，Focal Loss函数是在标准交叉熵损失基础上改进得到的一种损失函数，其通过减少易学习样本的权重，使得模型在训练时更专注于难学习样本。标准的二分类交叉熵损失函数如下：
Figure PCTCN2021104948-appb-000004
p表示经过激活函数的模型输出，即预测样本属于1类的概率，数值在0-1之间。y表示预测样本的标签，数值为0或1。可见传统的交叉熵损失对于正样本而言，输出概率越大损失越小；对于负样本而言，输出概率越小则损失越小。为了解决正负样本不平衡的问题，通常会在交叉熵损失的基础上加一个参数α，即：
Figure PCTCN2021104948-appb-000005
尽管α平衡了正负样本，但对难易样本的不平衡并不起作用。易分类样本的Loss很低，但是由于数量极不平衡，易分类样本最终主导了总的Loss，而忽略了难分类样本。为解决这一问题，Foacl Loss在此基础上，引入的新的参数γ，公式如下：
Figure PCTCN2021104948-appb-000006
γ>0，以正类样本为例，当γ＝0时，对于易分样本，假设其预测概率达到了95％，则(1-0.95) 2＝0.0025，那么易分样本的Loss就会被衰减得很小；而对于难分样本，假设其预测概率为50％，则(1-0.5) 2＝0.25，即难分样本的Loss仅衰减了0.25倍，相对于易分样本而言Loss增大了很多。对于负类样本同理。因此，Foacl Loss更关注于这些难以区分的样本，削弱了简单样本的影响，一定程度上解决了难易样本不平衡问题。
公式(1)中的CE为标准的交叉熵损失，也可以用CE 标准表示。公式(2)中的CE为平衡了正负样本的交叉熵损失，也可以用CE 正负表示。公式(3)中的CE为即平衡了正负样本，又平衡了难易样本的交叉熵损失，也可以用CE 难易表示。
(6)机器学习分类模型
机器学习是从大量的数据中研究计算方法、利用经验来改善系统性能的一门学科，机器学习用到了统计学的知识、基于已有数据设计算法训练产生一定的模型，当有新的数据来临时，利用模型给出判断。
机器学习可以划分为监督学习、无监督学习、半监督学习、增强学习等。
监督学习是具有人工标注的样本数据集，包括样本的属性特征和类别标签两部分，同时给计算机学习，以学习属性特征和类别标签内在的关系。这里重点采用监督学习方法进行建模分类。常用的分类算法有K近邻算法、逻辑回归、朴素贝叶斯、决策树、支持向量机、Adaboost算法、随机森林、梯度提升树和神经网络等算法。
针对建立的重症数据库1.0和2.0中的非图像数据，利用前面标准化的数据特征进行建模分类，拟将数据集10折交叉验证取平均值，如图2所示。
将训练集分别使用不同的分类算法进行测试对比分析，反复实验验证得出最佳的模型，计算出精确率、召回率和F1值，结合医学专家验证，筛选出最佳机器学习模型。
针对训练得到的最佳机器学习模型，将特征权重再次进行排序，筛选出重症分类重要特征，结合对应的医学知识，可以建立知识图谱和知识库，为这些特征表征重症分类做出可解释性。
(7)深度学习分类模型
理论上，更深的网络其表达能力应该越强。事实上，深度卷积神经网络达到一定深度后，再一味地增加层数并不能进一步带来性能的提高。当网络达到适当深度时，随着层数增多和网络继续加深，收敛反而变得更慢，网络的性能趋于饱和甚至急速下降，分类的准确率也变得更差。此外，研究显示，网络加深所带来的退化现象并非由过拟合导致，网络深度的提升不能通过层与层的简单堆叠来实现。ResNet原是在VGG的基础上被提出来的一种用于图像分类的强有力的模型，所解决的就是在网络加深过程中所出现的退化现象及潜在的梯度弥散或梯度爆炸问题，核心思想是通过引 入所谓的恒等快捷连接(Identity Shortcut Connection)跳过一个或多个卷积层进行深度残差学习。此外，通过显式地增加卷积残差块之间的恒等快捷连接，该深度神经网络可以学习到底层所映射的残差，而非仅仅学习来自上一隐层的直接映射，从而可以更好地解决网络加深所带来的退化问题及潜在的梯度弥散或梯度爆炸问题，使模型具有更好的分类性能。
(8)多模态数据融合
“模态”定义为特定物理媒介上信息的表示及交换方式。多模态融合是指联合多个模态的信息，进行目标的分类或者回归，其往往能获得比单模态方法更好的性能和效果。本发明可用于分类预测的数据是多模态的，包括语音数据、非图像数据和图像数据。多模态数据融合方法包括：前端融合(EarlyFusion)/数据水平融合(Data Level Fusion)、中间融合(Intermediate Fusion)和后端融合(Late Fusion)/决策水平融合(Decision Level Fusion)。前端融合是将各个模态数据先融合成一个特征向量，然后用于模型预测；中端融合将不同模态数据先转化为高维特征表达，再在模型的中间层进行融合；后端融合则是基于不同模态数据分别训练预测模型，然后将模型输出融合。多模态数据前端融合往往无法充分利用多个模态数据间的互补性，且前端融合的原始数据通常包含大量冗余信息。因此，前端融合方法通常需要与特征提取方法相结合以剔除冗余信息。后端融合则是将不同模态数据分别训练好的分类器输出决策进行融合。这样做的好处是：融合模型的错误来自不同的分类器，而来自不同分类器的错误往往互不相关、互不影响，不会造成错误的进一步累加。中间融合方法的一大优势是可以灵活的选择融合的位置，灵活的融合方式可以明显提高算法的预测性能。本发明拟分别尝试采用后端和中端融合方法融合语音数据、非图像数据和图像数据，并比较其效果。
本具体实施例仅仅是对本发明的解释，其并不是对本发明的限制，本领域技术人员在阅读完本说明书后可以根据需要对本实施例做出没有创造性贡献的修改，但只要在本发明的权利要求范围内都受到专利法的保护。

Claims (10)

  1. 基于智能眼镜的灾害救援检伤分类及辅助诊断方法，其特征在于，包括以下步骤：
    S1：获取数据；
    S2：数据预处理；
    S3：提取创伤失血性休克关键指标，生成初代失血-重症数据库；
    S4：智能穿戴设备获取视觉、听觉和/或生命体征的采集信息；
    S5：将S3获得的初代失血-重症数据库与S4获得的采集信息交互，生成二代失血-重症数据库；
    S2过程中，数据预处理包括数据清洗、样本均衡和标准化处理；
    数据清洗包括缺失数据处理和异常数据处理，缺失数据处理包括删除处理和填充处理，对于缺失数据多的特征或样本删除；对于缺失数据少的样本或特征进行填充；对异常数据进行删除；
    样本均衡用于在样本失衡情况下，采用重采样方法修改样本数据集，或调整各类别数据的权重修正分类器的偏好；
    标准化处理：对队列中的数值数据进行标准化或归一化处理，将数据缩放为标准正态分布。
  2. 根据权利要求1所述的基于智能眼镜的灾害救援检伤分类及辅助诊断方法，其特征在于，所述S1过程中，获取数据的伤员要求至少包括以下任意一种：伤员因创伤入院，伤员年龄≥18岁，伤员休克指数≥1.0，伤员平均动脉压≤70mmHg，伤员在休克指数≥1.0且平均动脉压≤70mmHg后5小时内有输血记录。
  3. 根据权利要求2所述的基于智能眼镜的灾害救援检伤分类及辅助诊断方法，其特征在于，所述S1过程中，伤员获取的数据包括：治疗期间的全程生命体征、血常规、血生化、凝血功能、血气分析和/或尿液常规；
    对数据处理的方法包括特征工程和机器学习分类模型；
    特征工程包括过滤法、包装法和嵌入法选取重要特征；采用主成分分析、独立成分分析和线性判别分析从原始输入属性中形成较少的新特征。
  4. 根据权利要求3所述的基于智能眼镜的灾害救援检伤分类及辅助诊断方法，其特征在于，所述缺失数据处理中的充填处理方法包括平均数填充数算法、中位数填充算法和近邻填充算法。
  5. 根据权利要求4所述的基于智能眼镜的灾害救援检伤分类及辅助诊断方法，其特征在于，所述异常数据处理方法包括散点绘制图示法和箱体图示法。
  6. 根据权利要求5所述的基于智能眼镜的灾害救援检伤分类及辅助诊断方法，其特征在于，所述S3过程中，将S2预处理后的数据通过机器学习算法对创伤失血性休克结局变量进行预测，对分类器模型性能进行评价，分别获得不同分组指标集下各种算法所计算出的准确率、召回率、精准率和F值，并对结果进行比较，得到预测结果最优时的关键指标模型，生成初代失血-重症数据库。
  7. 根据权利要求6所述的基于智能眼镜的灾害救援检伤分类及辅助诊断方法，其特征在于，所述S3过程中，经S2预处理得到的数据使用元胞遗传算法进行独立重复试验，根据指标在筛选中被保留次数判定其识别能力，形成新的关键指标集；应用基于精度的AdaBoost算法，获得各指标在使预测效果达到最优时所占权重；对获得的指标进行筛选，将异常指标数据删除；被筛选出的关键指标根据保留次数的不同生成不同的关键指标数据集，通过数据分组和机器学习算法得到分类模型和关键指标。
  8. 根据权利要求7所述的基于智能眼镜的灾害救援检伤分类及辅助诊断方法，其特征在于，所述S5过程中，听觉信息的处理包括将伤员语音进行识别，通过声音强弱及语调信息提取出来并识别出其类别分类，具体方法包括预处理、特征提取及分类模型：
    预处理包括预加重、加窗分帧和端点检测；预加重用于将语音信号通过一个一阶高通数字滤波器，去除口舌辐射，进一步提高语音的高频分辨率；加窗分帧用于通过语音信号的短时平稳假设，使用汉明窗或矩形窗将语音划分成帧；端点检测用于去除语音信号的静音部分，检测出有效的语音片段，提高计算效率；
    特征提取包括韵律学特征提取、谱特征提取和音质特征提取；
    分类模型包括高斯混合模型、支持向量机、循环神经网络和隐马尔可夫模型。
  9. 根据权利要求8所述的基于智能眼镜的灾害救援检伤分类及辅助诊断方法，其特征在于，所述S5过程中，视觉采集信息获取的图像信息对面部图像进行特征提取的具体方法包括使用Inception系列网络、Resnet网络或Inception-Resnet-V2网络进行训练；在使用Inception-Resnet-V2网络进行训练过程中包括以下步骤：
    标准的二分类交叉熵损失函数如下：
    Figure PCTCN2021104948-appb-100001
    其中：p表示经过激活函数的模型输出，即预测样本属于1类的概率，数值在0-1之间；y表示预测样本的标签，数值为0或1；
    交叉熵损失的基础上加一个参数α，以平衡正负样本，即：
    Figure PCTCN2021104948-appb-100002
    再引入新的参数γ，公式如下：
    Figure PCTCN2021104948-appb-100003
  10. 根据权利要求9所述的基于智能眼镜的灾害救援检伤分类及辅助诊断方法，其特征在于，所述机器学习算法包括K近邻算法、逻辑回归、朴素贝叶斯、决策树、支持向量机、Adaboost算法、随机森林、梯度提升树和神经网络。
PCT/CN2021/104948 2021-04-30 2021-07-07 基于智能眼镜的灾害救援检伤分类及辅助诊断方法 WO2022227280A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110484108.2 2021-04-30
CN202110484108.2A CN113257406A (zh) 2021-04-30 2021-04-30 基于智能眼镜的灾害救援检伤分类及辅助诊断方法

Publications (1)

Publication Number Publication Date
WO2022227280A1 true WO2022227280A1 (zh) 2022-11-03

Family

ID=77223435

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/104948 WO2022227280A1 (zh) 2021-04-30 2021-07-07 基于智能眼镜的灾害救援检伤分类及辅助诊断方法

Country Status (2)

Country Link
CN (1) CN113257406A (zh)
WO (1) WO2022227280A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117494072A (zh) * 2023-12-29 2024-02-02 深圳永泰数能科技有限公司 一种基于数据融合的换电柜运行状态监测方法及系统

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114169374B (zh) * 2021-12-10 2024-02-20 湖南工商大学 一种斜拉桥斜拉索损伤识别方法及电子设备
CN115458148B (zh) * 2022-08-30 2023-06-16 中国人民解放军总医院第三医学中心 一种用于检伤分类方法的智能选择方法及智能选择装置
CN115444378A (zh) * 2022-11-10 2022-12-09 成都成电金盘健康数据技术有限公司 一种大型群体伤在线检伤系统
CN116309515A (zh) * 2023-03-31 2023-06-23 广东省人民医院 一种肺部亚厘米结节侵袭性预测模型的构建方法及该诊断模型与诊断器

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110195686A1 (en) * 2010-02-05 2011-08-11 Instant Care, Inc. Personal Emergency Response System with Alternative Voice Line Capability
CN106874663A (zh) * 2017-01-26 2017-06-20 中电科软件信息服务有限公司 心脑血管疾病风险预测方法及系统
CN109411083A (zh) * 2018-11-26 2019-03-01 中国人民解放军第二军医大学第二附属医院 检伤分类系统及方法
CN110051324A (zh) * 2019-03-14 2019-07-26 深圳大学 一种急性呼吸窘迫综合征死亡率预测方法及系统
CN110289061A (zh) * 2019-06-27 2019-09-27 黎檀实 一种创伤失血性休克伤情的时间序列预测方法
CN110680290A (zh) * 2019-09-12 2020-01-14 军事科学院系统工程研究院卫勤保障技术研究所 一种基于ar眼镜的检伤分类装置

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102982074A (zh) * 2012-10-29 2013-03-20 郑静晨 检伤分类处理方法和装置
CN103077324A (zh) * 2013-01-29 2013-05-01 郑静晨 医疗数据处理方法和医疗数据处理装置
EP3468457A4 (en) * 2016-06-13 2020-02-19 Flashback Technologies, Inc. FAST DETECTION OF BLEEDING AFTER INJURY
US11227507B2 (en) * 2018-10-11 2022-01-18 International Business Machines Corporation Wearable technology employed in injury detection, prevention and skills training

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110195686A1 (en) * 2010-02-05 2011-08-11 Instant Care, Inc. Personal Emergency Response System with Alternative Voice Line Capability
CN106874663A (zh) * 2017-01-26 2017-06-20 中电科软件信息服务有限公司 心脑血管疾病风险预测方法及系统
CN109411083A (zh) * 2018-11-26 2019-03-01 中国人民解放军第二军医大学第二附属医院 检伤分类系统及方法
CN110051324A (zh) * 2019-03-14 2019-07-26 深圳大学 一种急性呼吸窘迫综合征死亡率预测方法及系统
CN110289061A (zh) * 2019-06-27 2019-09-27 黎檀实 一种创伤失血性休克伤情的时间序列预测方法
CN110680290A (zh) * 2019-09-12 2020-01-14 军事科学院系统工程研究院卫勤保障技术研究所 一种基于ar眼镜的检伤分类装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117494072A (zh) * 2023-12-29 2024-02-02 深圳永泰数能科技有限公司 一种基于数据融合的换电柜运行状态监测方法及系统
CN117494072B (zh) * 2023-12-29 2024-04-19 深圳永泰数能科技有限公司 一种基于数据融合的换电柜运行状态监测方法及系统

Also Published As

Publication number Publication date
CN113257406A (zh) 2021-08-13

Similar Documents

Publication Publication Date Title
WO2022227280A1 (zh) 基于智能眼镜的灾害救援检伤分类及辅助诊断方法
Hammami et al. Voice pathologies classification and detection using EMD-DWT analysis based on higher order statistic features
Muhammad et al. Convergence of artificial intelligence and internet of things in smart healthcare: a case study of voice pathology detection
Chen et al. Automatic detection of Alzheimer’s disease using spontaneous speech only
JP2023538287A (ja) 呼吸器症候群を検出するためのアンサンブル機械学習モデル
CN109003677B (zh) 病历数据结构化分析处理方法
Zhao et al. Multi-head attention-based long short-term memory for depression detection from speech
Ye et al. A hybrid model for pathological voice recognition of post-stroke dysarthria by using 1DCNN and double-LSTM networks
Wang et al. A domain transfer based data augmentation method for automated respiratory classification
CN116616770A (zh) 基于语音语义分析的多模态抑郁症筛查评测方法及其系统
Lu et al. Speech depression recognition based on attentional residual network
CN117153393A (zh) 一种基于多模态融合的心血管疾病风险预测方法
Zhou et al. Tamfn: Time-aware attention multimodal fusion network for depression detection
Fang et al. A dual-stream deep neural network integrated with adaptive boosting for sleep staging
CN112466284B (zh) 一种口罩语音鉴别方法
CN117064333B (zh) 一种针对阻塞性睡眠呼吸暂停低通气综合症的初筛装置
CN108766462A (zh) 一种基于梅尔频谱一阶导数的语音信号特征学习方法
CN116844080A (zh) 疲劳程度多模态融合检测方法、电子设备及存储介质
Zhu et al. Emotion Recognition of College Students Based on Audio and Video Image.
Gheorghe et al. Using deep neural networks for detecting depression from speech
CN114881668A (zh) 一种基于多模态的欺骗检测方法
Meng et al. A lightweight CNN and Transformer hybrid model for mental retardation screening among children from spontaneous speech
Huang et al. Indexing Biosignal for integrated health social networks
Dutta et al. A Fine-Tuned CatBoost-Based Speech Disorder Detection Model
Coelho et al. Identification of Voice Disorders: A Comparative Study of Machine Learning Algorithms

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21938747

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE