CN113077874B - Intelligent auxiliary diagnosis and treatment system and method for rehabilitation of spine diseases based on infrared thermal images - Google Patents

Intelligent auxiliary diagnosis and treatment system and method for rehabilitation of spine diseases based on infrared thermal images Download PDF

Info

Publication number
CN113077874B
CN113077874B CN202110293245.8A CN202110293245A CN113077874B CN 113077874 B CN113077874 B CN 113077874B CN 202110293245 A CN202110293245 A CN 202110293245A CN 113077874 B CN113077874 B CN 113077874B
Authority
CN
China
Prior art keywords
treatment
patient
subsystem
module
diagnosis
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202110293245.8A
Other languages
Chinese (zh)
Other versions
CN113077874A (en
Inventor
金心宇
沈雪
刘磊
金昀程
田鹏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang University ZJU
Original Assignee
Zhejiang University ZJU
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang University ZJU filed Critical Zhejiang University ZJU
Priority to CN202110293245.8A priority Critical patent/CN113077874B/en
Publication of CN113077874A publication Critical patent/CN113077874A/en
Application granted granted Critical
Publication of CN113077874B publication Critical patent/CN113077874B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/0033Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/0033Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room
    • A61B5/004Features or image-related aspects of imaging apparatus classified in A61B5/00, e.g. for MRI, optical tomography or impedance tomography apparatus; arrangements of imaging apparatus in a room adapted for image acquisition of a particular organ or body part
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A61B5/0062Arrangements for scanning
    • A61B5/0064Body surface scanning
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/0059Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence
    • A61B5/0075Measuring for diagnostic purposes; Identification of persons using light, e.g. diagnosis by transillumination, diascopy, fluorescence by spectroscopy, i.e. measuring spectra, e.g. Raman spectroscopy, infrared absorption spectroscopy
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00ICT specially adapted for the handling or processing of medical images
    • G16H30/20ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/30ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02ATECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Public Health (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Animal Behavior & Ethology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Veterinary Medicine (AREA)
  • Surgery (AREA)
  • Epidemiology (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Primary Health Care (AREA)
  • Evolutionary Computation (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Databases & Information Systems (AREA)
  • Computational Linguistics (AREA)
  • Software Systems (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Abstract

The invention discloses an infrared thermal image-based intelligent auxiliary diagnosis and treatment system for rehabilitation of spinal diseases, which comprises a medical record management subsystem, an intelligent analysis subsystem, a treatment evaluation subsystem and a control center subsystem; the medical record management subsystem comprises a record uploading module, a database module and a search engine module; the medical record management subsystem, the intelligent analysis subsystem and the treatment evaluation subsystem are respectively connected with the control center subsystem. The invention also provides a using method of the intelligent auxiliary diagnosis and treatment system for rehabilitation of the vertebra diseases based on the infrared thermal image. According to the invention, the infrared thermal image of the vertebra part is analyzed by adopting a deep learning method, the rehabilitation condition of the vertebra disease reflected by the infrared thermal image of the vertebra part can be classified and evaluated, and the evaluation can assist doctors in diagnosis and treatment and provide objective treatment effect evaluation reports for patients.

Description

Intelligent auxiliary diagnosis and treatment system and method for rehabilitation of spine diseases based on infrared thermal images
Technical Field
The invention relates to the field of image recognition and intelligent auxiliary diagnosis, in particular to an infrared thermal image-based intelligent auxiliary diagnosis and treatment system and method for rehabilitation of spinal diseases.
Background
The spine diseases refer to diseases related to spine lesions, 97% of middle-aged and elderly people in China have spine diseases according to related investigation, and in recent years, the tendency of younger is shown, and in people under 40 years old, more than 40% of human vertebrates have various diseases, and the incidence rate of the children scoliosis is more than 25%. Spinal diseases have become a public health problem seriously threatening the health of people in China, and rehabilitation process evaluation of patients with spinal diseases has a great influence on treatment effects.
Research shows that the characteristics of the infrared thermal image of the vertebra part can reflect the health condition of the vertebra, the infrared thermal image diagnosis of the vertebra part needs an experienced pathologist to carefully observe the image and make a judgment, the rehabilitation of the vertebra disease is a long-term process, and the infrared thermal image of different rehabilitation treatment stages of a patient is difficult to give objective and dynamic assessment to the long-term procedural treatment effect in a manual judgment mode; in addition, the existing diagnosis and treatment system contains long-term data of various diseases of patients, does not collect the diseases of the spine, is inconvenient in inquiring historical data and infrared thermal imaging images, and does not have the function of intelligent auxiliary diagnosis and treatment.
Accordingly, improvements in the art are needed.
Disclosure of Invention
The invention aims to provide an infrared thermal image-based intelligent auxiliary diagnosis and treatment system and an infrared thermal image-based intelligent auxiliary diagnosis and treatment method for rehabilitation of a spinal disease, which are used for realizing intelligent auxiliary diagnosis and classified assessment of rehabilitation conditions of the spinal disease.
In order to solve the technical problems, the invention provides an infrared thermal imaging-based intelligent auxiliary diagnosis and treatment system for rehabilitation of spinal diseases, which comprises the following components: the medical record management subsystem, the intelligent analysis subsystem, the treatment evaluation subsystem and the control center subsystem;
the medical record management subsystem comprises a record uploading module, a database module and a search engine module;
the intelligent analysis subsystem comprises a target detection module and a classification module; the target detection module comprises an SSD network, an attention mechanism module and a detection module, and an infrared thermogram of a patient obtains a cervical vertebra disease classification result and a lumbar vertebra disease classification result of the patient through the intelligent analysis subsystem;
the treatment evaluation subsystem comprises a single treatment rehabilitation evaluation module and a plurality of treatment process rehabilitation evaluation modules;
the medical record management subsystem, the intelligent analysis subsystem and the treatment evaluation subsystem are respectively connected with the control center subsystem.
As an improvement of the intelligent auxiliary diagnosis and treatment system for rehabilitation of the spine diseases based on infrared thermal imaging, the invention comprises the following steps:
the back of each convolution layer Conv4_3, conv7, conv8_2, conv9_2, conv10_2 and Conv11_2 of the SSD network is respectively connected with an attention mechanism model, then each attention mechanism model is connected with a detection module, and the detection module comprises a non-maximum value suppression method; feature map F E R output by each convolution layer of SSD network C×H×W Respectively inputting attention mechanism models corresponding to all convolution layers of an SSD network to obtain feature images F ', and then inputting all the feature images F ' into a detection module to obtain feature images S ' marked with cervical vertebra areas and lumbar vertebra areas of a patient;
the classification module is an improved AlexNet network obtained by replacing a convolution layer with a convolution kernel size of 5 multiplied by 5 in the AlexNet network by using the acceptance module, and the characteristic diagram S' marked with the cervical vertebra region and the lumbar vertebra region of the patient is subjected to the classification module to obtain the cervical vertebra disease classification result and the lumbar vertebra disease classification result of the patient.
The invention also provides a using method of the intelligent auxiliary diagnosis and treatment system for rehabilitation of the spine diseases based on the infrared thermal image, which comprises the following steps:
step S01, diagnosis and treatment information management
Step S101, recording and updating diagnosis and treatment information
Recording diagnosis and treatment information comprising patient information, infrared thermal image images before and after treatment of the patient's spine diseases, treatment time, treatment scheme, medication scheme and diagnosis description information through a recording and uploading module, and uploading and uniformly storing the information into a database module; all diagnosis and treatment information in the database module form an electronic medical record database, and the infrared thermal image of the patient forms an infrared thermal image set S i
The medical record management subsystem processes the diagnosis and treatment information, and establishes, adds or updates single or multiple treatment records of the patient in the database module;
step S102, diagnosis and treatment information searching
The user inputs keywords including 'patient name', 'patient visit card number', 'primary doctor name' or 'primary doctor number' to the search engine module through the control center subsystem;
the search engine module searches and matches each piece of electronic medical record data in the database module according to the input keywords and presents search results on the upper computer;
s02, treatment evaluation:
step S201, evaluation of Single treatment
Acquiring diagnosis and treatment information of a patient through the diagnosis and treatment information search of the step S102;
the single-treatment rehabilitation evaluation module selects a single-treatment record from diagnosis and treatment information of a patient through a control center subsystem, and gathers an infrared thermal image S in the single-treatment record i Inputting into an intelligent analysis subsystem to obtain a cervical vertebra disease classification result and a lumbar vertebra disease classification result of a patient, and then evaluating a single treatment effect:respectively converting the cervical vertebra disease classification result and the lumbar vertebra disease classification result of the patient into 0-4 five digital grades to obtain and present the result of single treatment effect evaluation of the patient on an upper computer;
step S202, multiple treatment evaluation
Acquiring diagnosis and treatment information of a patient through the diagnosis and treatment information search of the step S102;
the multi-treatment rehabilitation evaluation unit selects a plurality of treatment records from the diagnosis and treatment information of the patient through the control center subsystem, and sets an infrared thermal image S in each treatment record of the patient i Inputting into an intelligent analysis subsystem to obtain cervical vertebra disease classification results and lumbar vertebra disease classification results of patients after treatment for each time, and then evaluating the treatment effects for multiple times: and converting the cervical vertebra disease classification result and the lumbar vertebra disease classification result of the patient after each treatment into five digital grades of 0-4, generating a multi-treatment evaluation line graph, and obtaining and presenting the multi-treatment effect evaluation result of the patient on an upper computer.
The beneficial effects of the invention are mainly as follows:
1. according to the invention, the infrared thermal image of the vertebra part is analyzed by adopting a deep learning method, the rehabilitation condition of the vertebra disease reflected by the infrared thermal image of the vertebra part can be classified and evaluated, and the evaluation can assist doctors in diagnosis and treatment and provide objective treatment effect evaluation reports for patients;
2. the invention fully utilizes the characteristics of non-invasiveness, painless, noninvasive and wide applicable crowd of infrared thermal image examination, combines with advanced image processing technology and analysis mode to be suitable for realizing automatic auxiliary diagnosis of the rehabilitation treatment process of the vertebra diseases, thereby rapidly and effectively providing reference diagnosis opinion for doctors, providing objective treatment effect evaluation report for patients and improving the working efficiency of doctors.
Drawings
The following describes the embodiments of the present invention in further detail with reference to the accompanying drawings.
FIG. 1 is a schematic block diagram of an infrared thermal imaging-based intelligent auxiliary diagnosis and treatment system for rehabilitation of spinal diseases;
FIG. 2 is a schematic flow diagram of the intelligent analysis subsystem of FIG. 1;
FIG. 3 is a flow diagram of the treatment evaluation subsystem of FIG. 1;
FIG. 4 is a schematic diagram of a network model of the object detection module in FIG. 1;
FIG. 5 is a schematic diagram of a network model of the classification module of FIG. 1;
fig. 6 is a line graph of multiple treatment assessments.
Detailed Description
The invention will be further described with reference to the following specific examples, but the scope of the invention is not limited thereto:
the embodiment 1, an infrared thermal image-based intelligent auxiliary diagnosis and treatment system for rehabilitation of the spine diseases, as shown in fig. 1, comprises a medical record management subsystem, an intelligent analysis subsystem, a treatment evaluation subsystem and a control center subsystem, wherein a relevant algorithm model is designed according to the characteristics of an infrared thermal image through reasonable database design and system architecture design, and the infrared thermal image-based intelligent auxiliary diagnosis and treatment system for rehabilitation of the spine diseases is built in an upper computer, so that a system for providing tracking, comparison and evaluation functions of rehabilitation treatment process for the spine diseases patients through unified management of patient medical records is realized, and doctors are helped to provide objective rehabilitation evaluation;
the medical record management subsystem comprises a record uploading module, a database module and a search engine module; the record uploading module is used for recording electronic medical record data, wherein the electronic medical record data comprises patient information (including patient name, patient visit card number and the like), main doctor information (including main doctor name, main doctor number and the like), infrared thermography before and after treatment of spine (cervical vertebra and lumbar vertebra) diseases, treatment time, treatment scheme, medication scheme and diagnosis description information, and uploading and uniformly storing single or multiple treatment records (electronic medical record data) of a patient to the database module; the database module is used for storing all electronic medical record data, all diagnosis and treatment information in the database module form an electronic medical record database, and the infrared thermal image of the patient forms an infrared thermal image set S i (i is a patientA patient medical record number); the search engine module can search and match the input relevant information of the patient in the 'patient name' field, the 'patient visit card number' field, the 'mainly used doctor name' field and the 'mainly used doctor number' field of each piece of electronic medical record data in the database module;
the intelligent analysis subsystem comprises a target detection module and a classification module, wherein the target detection module is used for extracting the cervical vertebra part and the lumbar vertebra part in the infrared thermal image of the patient, and the classification module is used for classifying and evaluating the rehabilitation condition of the vertebra diseases;
the target detection module comprises a SSD (Single Shot MultiBox Detector) network, an attention mechanism module and a detection module, as shown in fig. 4, one attention mechanism model is respectively connected behind each convolution layer conv4_3, conv7, conv8_2, conv9_2, conv10_2 and conv11_2 of the SSD network, and then each attention mechanism model is connected with the detection module; the resolution of the input pictures of the SSD network is generally 300×300 or 512×512, the original infrared thermogram resolution of the patient is 256×324, the original infrared thermogram resolution is scaled to 300×300, and then the image is input to the SSD network, the SSD network detects the cervical vertebra area and the lumbar vertebra area by utilizing a multi-scale convolution detection structure, and the characteristic pictures F epsilon R output by each convolution layer (namely Conv4_3, conv7, conv8_2, conv9_2, conv10_2 and Conv11_2) of the SSD network C×H×W Respectively inputting the attention mechanism models corresponding to all convolution layers of the SSD network to obtain a feature map F ', and then inputting the feature map F' output by all the attention mechanism models into a detection module;
the detection module comprises a Non-maximum suppression (NMS) method, the obtained characteristic diagram F ' after passing through an SSD network and an attention mechanism module is provided with a plurality of target boundary frames, the boundary frames with higher overlapping degree are removed through the Non-maximum suppression in the detection module, and the characteristic diagram S ' marked with the cervical vertebra region and the lumbar vertebra region of the patient is obtained, namely the target detection module outputs the characteristic diagram S ' marked with the cervical vertebra region and the lumbar vertebra region of the patient;
SSD network directly detects after extracting features in several convolution layers, obtains the offset of a priori frame to determine the final edgeThe position of the bounding box is faster in operation speed, in addition, the back infrared thermal image is smaller in cervical vertebra area and larger in lumbar vertebra area, a small target and a large target are required to be detected simultaneously, the multi-scale convolution detection structure of the SSD can well meet the requirement of the point, although the SSD network model can extract multi-scale characteristics, the information extracted by each convolution layer has useful infrared image information and some useless background information, and the attention introducing module can further process convolution results to effectively eliminate interference information; in the attention mechanism model, the feature map F ε R C×H×W First pass through channel attention mapping matrix M c ∈R C×1×1 (wherein M c ∈R C×1×1 The calculation method of (1) is as follows: for the input feature map F ε R C×H×W Respectively carrying out maximum pooling treatment and average pooling treatment, then inputting the results of the two pooling treatments into the same multi-layer perceptron, adding the respective outputs, and obtaining a mapping matrix M of the spatial attention after Sigmoid activation function treatment c ∈R C ×1×1 ) The method comprises the steps of carrying out a first treatment on the surface of the By the formulaObtaining a feature map F ', carrying out maximum pooling treatment and average pooling treatment on the feature map F', carrying out convolution operation on the results of the maximum pooling treatment and the average pooling treatment to extract features, and obtaining a spatial attention mapping matrix M after Sigmiod activation function treatment s ∈R 1×H×W Then the feature map F' is passed through a space attention mapping matrix M s ∈R 1×H×W By the formula->Obtaining a characteristic diagram F';
the classification module is based on an improved AlexNet network built by the AlexNet network, the input of the classification module is a characteristic diagram S' marked with the cervical vertebra region and the lumbar vertebra region of a patient, and the output is a cervical vertebra disease classification result and a lumbar vertebra disease classification result (the disease type with the largest probability is used as the classification result), wherein the cervical vertebra disease classification result is sequentially classified into 5 types according to the increment of the severity: normal (j_a), fatigue (j_b), cervical muscle strain or inflammatory change (j_c), cervical overload (j_d), cervical degenerative disease (j_e), lumbar spine disease classification results are sequentially classified into 5 categories according to increasing severity: normal (y_a), fatigue (y_b), myofascitis (y_c), lumbar muscle strain (y_d), lumbar degenerative changes (y_e). For example, for a patient, the cervical vertebra disease classification result is normal (0.02), fatigue (0.02), cervical muscle strain or inflammatory change (0.03), cervical vertebra overload (0.09), cervical vertebra degeneration (0.84), the lumbar vertebra disease classification result is normal (0), fatigue (0.02), myofascitis (0.03), lumbar muscle strain (0.86), lumbar degeneration (0.09), the classification module outputs the cervical vertebra disease classification result of the patient as cervical vertebra degeneration (j_e), the lumbar vertebra disease classification result as lumbar muscle strain (y_d); besides global information, local overheating in infrared thermal imaging is also an important basis for judging disease conditions, so that the network also has the capability of extracting characteristics in multiple scales, an AlexNet network is further improved based on an acceptance module, the AlexNet network sequentially comprises 1 convolution layer with the size of 11 x 11 convolution cores, 1 convolution layer with the size of 5*5 convolution layers, 3 convolution layers with the size of 3*3 convolution layers and 3 full connection layers, and the acceptance module is utilized to replace the convolution layer with the size of 5 x 5 convolution cores in the AlexNet network so as to form a classification module, as shown in fig. 5; the acceptance module comprises a plurality of convolution kernels with different sizes, wherein convolution has two-point effect, namely, the combination of all characteristic information on a certain pixel point is realized, the nonlinearity of the characteristic is improved under the condition that the receptive field is kept unchanged, and the dimension reduction of any scale is carried out according to the number of the convolution kernels passing through a high-dimensional characteristic diagram of a previous layer, so that the operand is reduced; the convolution kernels with different sizes are used for extracting features with different scales, and when the features of the upper layer are input into the acceptance module, the network can automatically select and output information with what scale so as to achieve the effect of extracting the features with multiple scales;
training the intelligent analysis subsystem, configuring an upper computer used for training as a central Linux 7.5.1804 operating system, selecting a CPU with E5-2667 v4@3.20GHz and 8 cores, and 2 GPUs with model number Tesla P4; the training data set is a human body infrared thermal image data set collected in a trimethyl hospital, the resolution of an infrared thermal image is 256 multiplied by 324, the total number of infrared thermal image patterns of 5400 cases of 300 patients is 5400, the 5400 cases of infrared thermal image patterns are all marked by a professional doctor according to the actual condition setting of the patients, the classification of cervical vertebra diseases (normal (J_A), fatigue (J_B), cervical muscle strain or inflammatory change (J_C), cervical vertebra overload (J_D) and cervical vertebra degenerative disease (J_E)) and the classification of lumbar vertebra diseases (normal (Y_A), fatigue (Y_B), myofascitis (Y_C), lumbar muscle strain (Y_D) and lumbar vertebra degenerative disease (Y_E)), and then the marked infrared thermal image patterns are used for data set with the following weight ratio of 7:3 into a training set and a testing set, training an intelligent analysis subsystem by using the training set, setting the batch_size of an AlexNet network to be 32, setting the learning rate to be 5 multiplied by 10 < -5 >, and setting the iteration number to be 40000; setting the batch_size of the SSD network to be 32, setting the learning rate to be 0.0001, setting the iteration number to be 6000, and finally obtaining a trained intelligent analysis subsystem; and then inputting the test set into a trained intelligent analysis subsystem for verification, comparing the cervical vertebra disease classification result and the lumbar vertebra disease classification result of the patient output by the intelligent analysis subsystem with labels set by a professional doctor, and counting that the ratio of the number of correctly classified samples to the total number of samples is 94.69%, thereby meeting the requirement of actual use.
The treatment evaluation subsystem comprises a single treatment recovery evaluation module and a multi-treatment process recovery evaluation module, as shown in fig. 3, the single treatment recovery evaluation module queries the medical record management subsystem through the control center subsystem to obtain a single treatment record, then inputs an infrared thermal image in the single treatment record into the intelligent analysis subsystem to obtain a cervical vertebra disease classification result and a lumbar vertebra disease classification result of a patient, and then evaluates the single treatment effect, so that a single treatment effect evaluation result of the patient is obtained, namely the cervical vertebra disease classification result and the lumbar vertebra disease classification result of the patient are respectively converted into 0-4 five digital grades, and the single treatment effect evaluation results are respectively:
1) Cervical spondylosis: normal (j_a) =0, fatigue (j_b) =1, cervical muscle strain or inflammatory change (j_c) =2, cervical overload (j_d) =3, cervical degenerative disease (j_e) =4;
2) Lumbar disease state: normal (y_a) =0, fatigue (y_b) =1, myofascitis (y_c) =2, lumbar muscle strain (y_d) =3, lumbar degenerative disease (y_e) =4;
the multi-treatment process rehabilitation evaluation module inquires the medical record management subsystem through the control center subsystem to obtain multi-treatment process records, then inputs an infrared thermal image in the multi-treatment process records into the intelligent analysis subsystem to obtain cervical vertebra disease classification results and lumbar vertebra disease classification results of patients after each treatment, and then evaluates the multi-treatment effects of the patients to obtain multi-treatment effect evaluation results of the patients, namely, the cervical vertebra disease classification results and the lumbar vertebra disease classification results of the patients are converted into 0-4 digital grades (consistent with the definition of the five digital grades of the single treatment effect evaluation results), then the digital grade of each patient is converted into a multi-treatment evaluation line graph, the horizontal coordinate is the number of times of treatment of the patients, and the vertical coordinate is the evaluation effect digital grade, so that the evaluation effect of the patients for multiple treatments is displayed clearly.
The control center subsystem is a convergence point of the medical record management subsystem, the intelligent analysis subsystem and the treatment evaluation subsystem, and the medical record management subsystem, the intelligent analysis subsystem and the treatment evaluation subsystem are respectively connected with the control center subsystem.
The application method of the invention is as follows:
step S01, diagnosis and treatment information management
Step S101, recording and updating diagnosis and treatment information
Recording diagnosis and treatment information comprising patient information, infrared thermal image images before and after treatment of the patient's spine diseases, treatment time, treatment scheme, medication scheme and diagnosis description information through a recording and uploading module, and uploading and uniformly storing the information into a database module; all diagnosis and treatment information in the database module form an electronic medical record database, and the infrared thermal image of the patient forms an infrared thermal image set S i
The medical record management subsystem processes the diagnosis and treatment information, and establishes, adds or updates single or multiple treatment records of the patient in the database module;
step S102, diagnosis and treatment information searching
The user inputs keywords including 'patient name', 'patient visit card number', 'primary doctor name' or 'primary doctor number' to the search engine module through the control center subsystem;
the search engine module searches and matches each piece of electronic medical record data in the database module according to the input keywords and presents search results on the upper computer;
s02, treatment evaluation:
step S201, evaluation of Single treatment
Acquiring diagnosis and treatment information of a patient through the diagnosis and treatment information search of the step S102;
the single-treatment rehabilitation evaluation module selects a single-treatment record from diagnosis and treatment information of a patient through a control center subsystem, and gathers an infrared thermal image S in the single-treatment record i Inputting the data into an intelligent analysis subsystem to obtain a cervical vertebra disease classification result and a lumbar vertebra disease classification result of a patient, and then evaluating the single treatment effect to obtain and present a single treatment effect evaluation result of the patient on an upper computer;
step S202, multiple treatment evaluation
Acquiring diagnosis and treatment information of a patient through the diagnosis and treatment information search of the step S102;
the multi-treatment rehabilitation evaluation unit selects a plurality of treatment records from the diagnosis and treatment information of the patient through the control center subsystem, and sets an infrared thermal image S in each treatment record of the patient i Inputting the results into an intelligent analysis subsystem to obtain cervical vertebra disease classification results and lumbar vertebra disease classification results of patients after treatment, then evaluating the treatment effects for multiple times to obtain and presenting the results of the evaluation of the treatment effects for multiple times of the patients on an upper computer.
Experiment 1:
taking the training set and the testing set used for training in the embodiment 1 as the data set of the experiment 1, respectively taking the testing set as the input of the intelligent analysis subsystem and other three network models in the invention, and carrying out a comparison classification experiment; in the classification problem, for one of the classes, the class is called Positive, the class is not called Negative, true Positive (TP) represents the Positive classification of Positive, true Negative (TN) represents the Negative classification of Negative, false Positive (FP) represents the Positive classification of Negative errors, false Negative (FN) represents the Negative classification of Positive errors, for example for a 5-classification problem, the confusion matrix for class D is as follows:
accuracy (Accuracy) refers to the ratio of the number of correctly classified samples to the total number of samples,
precision refers to the ratio of the actual positive samples among all samples predicted to be positive.
Recall (Recall), also known as true positive, refers to the ratio of positive samples that are correctly predicted out of all true positive samples,
f1 is an index for comprehensively considering the accuracy and recall,
calculation of accuracy in this experiment: directly calculating the ratio of all the correctly classified samples to the total number of samples;
calculation of average accuracy: first according toCalculating the precision of each category, and then calculating the average value of the precision and the precision as the average precision;
calculation of average recall: first according toCalculating the recall rate of each category, and then calculating the average value of the recall rates as the average recall rate;
calculation of the average F1 value: first according toThe F1 value for each class is calculated and then their average value is calculated as the average F1 value.
Finally, the statistical comparison results are shown in the following table:
under the same experimental data and the same test method, the intelligent analysis subsystem has the highest test accuracy.
Experiment 2:
1. single treatment effect evaluation
Visual Analog Scoring (VAS), commonly used for clinical pain assessment, (patients scored from 0 to 10 according to their pain level, 0-2 for painless comfort, 3-4 for mild pain, 5-6 for pain evident, 7-8 for pain severe, and 9-10 for severe pain) the VAS Score of the patient was divided into 5 categories of numerical scale at intervals of 2: 0-2 is class 0, 3-4 is class 1, 5-6 is class 2, 7-8 is class 3, 9-10 is class 4, corresponding to 5 number classes of results of a single treatment assessment, respectively.
Selecting 300 patients, recording 900 times of treatment, scoring the patients according to actual experiences of the patients after each treatment, performing single treatment evaluation on infrared thermal image images shot after each treatment of the 300 patients to obtain digital grades corresponding to and converted from cervical vertebra disease classification results and lumbar vertebra disease classification results of the patients, performing comparison statistics with the VAS scores of the patients, and if the type of single treatment effect evaluation is consistent with the VAS score type of the patients, considering the patients as correct, otherwise, considering the patients as incorrect, and dividing the correct number of evaluation by the total number of the evaluation to obtain the accuracy: the total number of participation evaluations was 900, with the correct number of evaluations 819 and the accuracy 91%.
2. Multiple treatment effect assessment
The cervical vertebra disease classification result and the lumbar vertebra disease classification result of the patient are converted into digital grades, the digital grade of the patient is converted into a multi-treatment evaluation line graph, the abscissa is the number of times of treatment of the patient, the ordinate is the evaluation effect digital grade and the VAS score, and the trend of the evaluation effect digital grade line of the patient is consistent with the trend of the VAS score after the multi-treatment as shown in figure 6; experimental results demonstrate the effectiveness of the treatment evaluation subsystem.
Finally, it should also be noted that the above list is merely a few specific embodiments of the present invention. Obviously, the invention is not limited to the above embodiments, but many variations are possible. All modifications directly derived or suggested to one skilled in the art from the present disclosure should be considered as being within the scope of the present invention.

Claims (2)

1. The intelligent auxiliary diagnosis and treatment system for the rehabilitation of the vertebra diseases based on the infrared thermal images is characterized by comprising a medical record management subsystem, an intelligent analysis subsystem, a treatment evaluation subsystem and a control center subsystem;
the medical record management subsystem comprises a record uploading module, a database module and a search engine module;
the intelligent analysis subsystem comprises a target detection module and a classification module; the target detection module comprises an SSD network, an attention mechanism module and a detection module, and an infrared thermogram of a patient obtains a cervical vertebra disease classification result and a lumbar vertebra disease classification result of the patient through the intelligent analysis subsystem;
the treatment evaluation subsystem comprises a single treatment rehabilitation evaluation module and a plurality of treatment process rehabilitation evaluation modules;
the medical record management subsystem, the intelligent analysis subsystem and the treatment evaluation subsystem are respectively connected with the control center subsystem;
the back of each convolution layer Conv4_3, conv7, conv8_2, conv9_2, conv10_2 and Conv11_2 of the SSD network is respectively connected with an attention mechanism model, then each attention mechanism model is connected with a detection module, and the detection module comprises a non-maximum value suppression method; feature map F E R output by each convolution layer of SSD network C×H×W Respectively inputting attention mechanism models corresponding to all convolution layers of an SSD network to obtain feature images F ', and then inputting all the feature images F ' into a detection module to obtain feature images S ' marked with cervical vertebra areas and lumbar vertebra areas of a patient; in the attention mechanism model, the feature map F ε R C×H×W First pass through channel attention mapping matrix M c ∈R C×1×1 The method comprises the steps of carrying out a first treatment on the surface of the By the formulaObtaining a feature map F ', carrying out maximum pooling treatment and average pooling treatment on the feature map F by the feature map F', carrying out convolution operation on the results of the maximum pooling treatment and the average pooling treatment to extract features, and obtaining a spatial attention mapping matrix M after Sigmoid activation function treatment s ∈R 1×H×W Then the feature map F' is passed through a space attention mapping matrix M s ∈R 1×H×W By the formulaObtaining a characteristic diagram F';
the classification module is an improved AlexNet network obtained by replacing a convolution layer with a convolution kernel size of 5 multiplied by 5 in the AlexNet network by using the acceptance module, and the characteristic diagram S' marked with the cervical vertebra region and the lumbar vertebra region of the patient is subjected to the classification module to obtain the cervical vertebra disease classification result and the lumbar vertebra disease classification result of the patient.
2. The method for using the intelligent auxiliary diagnosis and treatment system for rehabilitation of the spinal diseases based on infrared thermal imaging as claimed in claim 1, which is characterized by comprising the following steps:
step S01, diagnosis and treatment information management
Step S101, recording and updating diagnosis and treatment information
Recording diagnosis and treatment information comprising patient information, infrared thermal image images before and after treatment of the patient's spine diseases, treatment time, treatment scheme, medication scheme and diagnosis description information through a recording and uploading module, and uploading and uniformly storing the information into a database module; all diagnosis and treatment information in the database module form an electronic medical record database, and the infrared thermal image of the patient forms an infrared thermal image set S i
The medical record management subsystem processes the diagnosis and treatment information, and establishes, adds or updates single or multiple treatment records of the patient in the database module;
step S102, diagnosis and treatment information searching
The user inputs keywords including 'patient name', 'patient visit card number', 'primary doctor name' or 'primary doctor number' to the search engine module through the control center subsystem;
the search engine module searches and matches each piece of electronic medical record data in the database module according to the input keywords and presents search results on the upper computer;
s02, treatment evaluation:
step S201, evaluation of Single treatment
Acquiring diagnosis and treatment information of a patient through the diagnosis and treatment information search of the step S102;
the single-treatment rehabilitation evaluation module selects a single-treatment record from diagnosis and treatment information of a patient through a control center subsystem, and gathers an infrared thermal image S in the single-treatment record i Inputting into an intelligent analysis subsystem to obtain a cervical vertebra disease classification result and a lumbar vertebra disease classification result of a patient, and then evaluating a single treatment effect: respectively converting the cervical vertebra disease classification result and the lumbar vertebra disease classification result of the patient into 0-4 five digital grades to obtain and present the result of single treatment effect evaluation of the patient on an upper computer;
step S202, multiple treatment evaluation
Acquiring diagnosis and treatment information of a patient through the diagnosis and treatment information search of the step S102;
the multi-treatment rehabilitation evaluation unit selects a plurality of treatment records from the diagnosis and treatment information of the patient through the control center subsystem, and sets an infrared thermal image S in each treatment record of the patient i Inputting into an intelligent analysis subsystem to obtain cervical vertebra disease classification results and lumbar vertebra disease classification results of patients after treatment for each time, and then evaluating the treatment effects for multiple times: and converting the cervical vertebra disease classification result and the lumbar vertebra disease classification result of the patient after each treatment into five digital grades of 0-4, generating a multi-treatment evaluation line graph, and obtaining and presenting the multi-treatment effect evaluation result of the patient on an upper computer.
CN202110293245.8A 2021-03-19 2021-03-19 Intelligent auxiliary diagnosis and treatment system and method for rehabilitation of spine diseases based on infrared thermal images Active CN113077874B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110293245.8A CN113077874B (en) 2021-03-19 2021-03-19 Intelligent auxiliary diagnosis and treatment system and method for rehabilitation of spine diseases based on infrared thermal images

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110293245.8A CN113077874B (en) 2021-03-19 2021-03-19 Intelligent auxiliary diagnosis and treatment system and method for rehabilitation of spine diseases based on infrared thermal images

Publications (2)

Publication Number Publication Date
CN113077874A CN113077874A (en) 2021-07-06
CN113077874B true CN113077874B (en) 2023-11-28

Family

ID=76612910

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110293245.8A Active CN113077874B (en) 2021-03-19 2021-03-19 Intelligent auxiliary diagnosis and treatment system and method for rehabilitation of spine diseases based on infrared thermal images

Country Status (1)

Country Link
CN (1) CN113077874B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103020446A (en) * 2012-12-10 2013-04-03 吴志宏 Data processing system for information of chronicle spinal degenerative diseases and deformity
WO2016101351A1 (en) * 2014-12-26 2016-06-30 深圳市前海安测信息技术有限公司 Network hospital based general practitioner auxiliary diagnosis and treatment system and method
CN106897546A (en) * 2017-01-13 2017-06-27 深圳市前海安测信息技术有限公司 Medical information aids in doctor's diagnosis and therapy system and method
KR20190059440A (en) * 2017-11-23 2019-05-31 주식회사 아이엠투지 System and method for diagnostic support through automatic search of similar patient
CN111986177A (en) * 2020-08-20 2020-11-24 太原科技大学 Chest rib fracture detection method based on attention convolution neural network
CN112488234A (en) * 2020-12-10 2021-03-12 武汉大学 End-to-end histopathology image classification method based on attention pooling

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180350459A1 (en) * 2017-06-05 2018-12-06 University Of Florida Research Foundation, Inc. Methods and apparatuses for implementing a semantically and visually interpretable medical diagnosis network
US20200397313A1 (en) * 2017-10-06 2020-12-24 Mayo Foundation For Medical Education And Research Ecg-based cardiac ejection-fraction screening

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103020446A (en) * 2012-12-10 2013-04-03 吴志宏 Data processing system for information of chronicle spinal degenerative diseases and deformity
WO2016101351A1 (en) * 2014-12-26 2016-06-30 深圳市前海安测信息技术有限公司 Network hospital based general practitioner auxiliary diagnosis and treatment system and method
CN106897546A (en) * 2017-01-13 2017-06-27 深圳市前海安测信息技术有限公司 Medical information aids in doctor's diagnosis and therapy system and method
KR20190059440A (en) * 2017-11-23 2019-05-31 주식회사 아이엠투지 System and method for diagnostic support through automatic search of similar patient
CN111986177A (en) * 2020-08-20 2020-11-24 太原科技大学 Chest rib fracture detection method based on attention convolution neural network
CN112488234A (en) * 2020-12-10 2021-03-12 武汉大学 End-to-end histopathology image classification method based on attention pooling

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
"基于红外热像的颈椎病变辅助诊疗系统研究与应用";李鑫;《中国优秀硕士学位论文全文数据库 医药卫生科技辑》;全文 *
医学影像疾病诊断的残差神经网络优化算法研究进展;周涛;霍兵强;陆惠玲;师宏斌;;中国图象图形学报(10);全文 *

Also Published As

Publication number Publication date
CN113077874A (en) 2021-07-06

Similar Documents

Publication Publication Date Title
CN101911077B (en) For the method and apparatus of hierarchical search
Hsu et al. SPIRS: a Web-based image retrieval system for large biomedical databases
US20040193036A1 (en) System and method for performing probabilistic classification and decision support using multidimensional medical image databases
US20090082637A1 (en) Multi-modality fusion classifier with integrated non-imaging factors
WO2010005656A2 (en) Brain condition assessment
CN115131642B (en) Multi-modal medical data fusion system based on multi-view subspace clustering
US11978208B2 (en) Trained model, learning method, learning program, medical information acquisition device, medical information acquisition method, and medical information acquisition program
Mansourvar et al. Automated web based system for bone age assessment using histogram technique
CN113782184A (en) Cerebral apoplexy auxiliary evaluation system based on facial key point and feature pre-learning
CN112862749A (en) Automatic identification method for bone age image after digital processing
Modi et al. A comprehensive analysis of artificial intelligence techniques for the prediction and prognosis of lifestyle diseases
Miao et al. Neural pattern recognition model for breast cancer diagnosis
Ji et al. Detection of cervical spondylotic myelopathy based on gait analysis and deterministic learning
Cheng et al. Report of clinical bone age assessment using deep learning for an Asian population in Taiwan
CN113077874B (en) Intelligent auxiliary diagnosis and treatment system and method for rehabilitation of spine diseases based on infrared thermal images
Jayachandran et al. Multi-dimensional cascades neural network models for the segmentation of retinal vessels in colour fundus images
Paunksnis et al. The use of information technologies for diagnosis in ophthalmology
Jabbar et al. Deep learning based classification of wrist cracks from X-ray imaging
Chang et al. Using machine learning algorithms in medication for cardiac arrest early warning system construction and forecasting
Jai-Andaloussi et al. Content Based Medical Image Retrieval based on BEMD: optimization of a similarity metric
Hsu et al. Predicting thorax diseases with NIH chest X-rays
Chen et al. Alzheimer's Disease Clinical Scores Prediction based on the Label Distribution Learning using Brain Structural MRI
Kaur et al. An Accurate Integrated System to detect Pulmonary and Extra Pulmonary Tuberculosis using Machine Learning Algorithms
Hsu et al. A comprehensive study of age-related macular degeneration detection
Korchiyne et al. New Approach Based on Multifractal Spectrum Features for Detection of Osteoporosis

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant