CN116994718A - VR technology-based mental disorder auxiliary treatment method - Google Patents
VR technology-based mental disorder auxiliary treatment method Download PDFInfo
- Publication number
- CN116994718A CN116994718A CN202311271544.7A CN202311271544A CN116994718A CN 116994718 A CN116994718 A CN 116994718A CN 202311271544 A CN202311271544 A CN 202311271544A CN 116994718 A CN116994718 A CN 116994718A
- Authority
- CN
- China
- Prior art keywords
- mental disorder
- facial image
- patient
- disorder
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 208000020016 psychiatric disease Diseases 0.000 title claims abstract description 289
- 238000011282 treatment Methods 0.000 title claims abstract description 106
- 238000000034 method Methods 0.000 title claims abstract description 77
- 238000005516 engineering process Methods 0.000 title claims abstract description 41
- 230000001815 facial effect Effects 0.000 claims abstract description 162
- 238000012549 training Methods 0.000 claims abstract description 102
- 238000013136 deep learning model Methods 0.000 claims abstract description 68
- 230000000694 effects Effects 0.000 claims abstract description 43
- 230000036387 respiratory rate Effects 0.000 claims abstract description 43
- 230000029058 respiratory gaseous exchange Effects 0.000 claims abstract description 41
- 208000019901 Anxiety disease Diseases 0.000 claims description 40
- 208000028017 Psychotic disease Diseases 0.000 claims description 40
- 238000003066 decision tree Methods 0.000 claims description 26
- 230000008451 emotion Effects 0.000 claims description 26
- 238000000605 extraction Methods 0.000 claims description 24
- 230000008569 process Effects 0.000 claims description 24
- 238000012795 verification Methods 0.000 claims description 23
- 238000011156 evaluation Methods 0.000 claims description 22
- 208000020401 Depressive disease Diseases 0.000 claims description 21
- 208000020925 Bipolar disease Diseases 0.000 claims description 15
- 238000010276 construction Methods 0.000 claims description 15
- 208000019022 Mood disease Diseases 0.000 claims description 14
- 208000017194 Affective disease Diseases 0.000 claims description 12
- 230000004913 activation Effects 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 12
- 238000013480 data collection Methods 0.000 claims description 12
- 238000011360 adjunctive therapy Methods 0.000 claims description 11
- 238000004364 calculation method Methods 0.000 claims description 8
- 238000005192 partition Methods 0.000 claims description 8
- 230000036651 mood Effects 0.000 claims description 7
- 210000004709 eyebrow Anatomy 0.000 claims description 6
- 230000006870 function Effects 0.000 claims description 5
- 206010016825 Flushing Diseases 0.000 claims description 4
- 230000002457 bidirectional effect Effects 0.000 claims description 4
- 238000005520 cutting process Methods 0.000 claims description 4
- 238000005259 measurement Methods 0.000 claims description 4
- 238000011176 pooling Methods 0.000 claims description 4
- 238000003860 storage Methods 0.000 claims description 3
- 231100000957 no side effect Toxicity 0.000 abstract description 2
- 230000036506 anxiety Effects 0.000 description 8
- 230000008921 facial expression Effects 0.000 description 8
- 238000012545 processing Methods 0.000 description 8
- 208000024891 symptom Diseases 0.000 description 7
- 238000004422 calculation algorithm Methods 0.000 description 6
- 238000001914 filtration Methods 0.000 description 6
- ORILYTVJVMAKLC-UHFFFAOYSA-N Adamantane Natural products C1C(C2)CC3CC1CC2C3 ORILYTVJVMAKLC-UHFFFAOYSA-N 0.000 description 4
- 230000002146 bilateral effect Effects 0.000 description 4
- 230000014509 gene expression Effects 0.000 description 4
- 230000007794 irritation Effects 0.000 description 4
- 238000005457 optimization Methods 0.000 description 4
- 230000002040 relaxant effect Effects 0.000 description 4
- 238000009098 adjuvant therapy Methods 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000006872 improvement Effects 0.000 description 3
- 238000010801 machine learning Methods 0.000 description 3
- 230000003340 mental effect Effects 0.000 description 3
- 206010054196 Affect lability Diseases 0.000 description 2
- 208000017164 Chronobiology disease Diseases 0.000 description 2
- 206010042209 Stress Diseases 0.000 description 2
- 230000003044 adaptive effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 2
- 230000008859 change Effects 0.000 description 2
- 230000009194 climbing Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000010485 coping Effects 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 229940079593 drug Drugs 0.000 description 2
- 239000003814 drug Substances 0.000 description 2
- 238000011010 flushing procedure Methods 0.000 description 2
- 238000009499 grossing Methods 0.000 description 2
- 208000024714 major depressive disease Diseases 0.000 description 2
- 230000037081 physical activity Effects 0.000 description 2
- 238000012552 review Methods 0.000 description 2
- 239000011435 rock Substances 0.000 description 2
- 230000035945 sensitivity Effects 0.000 description 2
- 230000003860 sleep quality Effects 0.000 description 2
- 238000006467 substitution reaction Methods 0.000 description 2
- 230000002889 sympathetic effect Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000005406 washing Methods 0.000 description 2
- 208000033618 Elevated mood Diseases 0.000 description 1
- 206010019233 Headaches Diseases 0.000 description 1
- 206010028813 Nausea Diseases 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000009225 cognitive behavioral therapy Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 208000035475 disorder Diseases 0.000 description 1
- 206010013663 drug dependence Diseases 0.000 description 1
- 238000002651 drug therapy Methods 0.000 description 1
- 231100000869 headache Toxicity 0.000 description 1
- 238000002372 labelling Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000008693 nausea Effects 0.000 description 1
- 238000011338 personalized therapy Methods 0.000 description 1
- 229940001470 psychoactive drug Drugs 0.000 description 1
- 238000001671 psychotherapy Methods 0.000 description 1
- 239000004089 psychotropic agent Substances 0.000 description 1
- 239000000126 substance Substances 0.000 description 1
- 208000011117 substance-related disease Diseases 0.000 description 1
- 238000002560 therapeutic procedure Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
- G16H20/70—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mental therapies, e.g. psychological therapy or autogenous training
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
- G06F3/04815—Interaction with a metaphor-based environment or interaction object displayed as three-dimensional, e.g. changing the user viewpoint with respect to the environment or object
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/0464—Convolutional networks [CNN, ConvNet]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Evolutionary Computation (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Psychology (AREA)
- Psychiatry (AREA)
- Epidemiology (AREA)
- Public Health (AREA)
- Primary Health Care (AREA)
- Hospice & Palliative Care (AREA)
- Developmental Disabilities (AREA)
- Medical Informatics (AREA)
- Social Psychology (AREA)
- Child & Adolescent Psychology (AREA)
- Human Computer Interaction (AREA)
- Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
Abstract
The invention relates to the technical field of medical treatment, and discloses a VR technology-based mental disorder auxiliary treatment method, which comprises the steps of collecting names, sexes, ages, facial images, heart rates, respiratory rates and skin reactance values of mental disorder patients; training a deep learning model according to the facial image of the mental disorder patient, and judging the facial image type of the mental disorder patient; determining a specific mental disorder type according to the heart rate, the breathing rate and the skin reactance value of the mental disorder patient and the facial image type; creating a virtual environment by utilizing VR technology according to specific mental disorder types, and simulating various real-world situations and trigger factors to provide personalized auxiliary treatment; after the treatment is completed, heart rate, respiratory rate and skin reactance values of the mental disorder patients are collected again to evaluate the treatment effect. The whole method has no side effect, can automatically judge specific mental disorder types, and can carry out personalized treatment on mental disorder patients.
Description
Technical Field
The invention relates to the technical field of medical treatment, in particular to an auxiliary treatment method for mental disorder based on VR technology.
Background
Psychotic disorders mainly include bipolar disorder, depressive disorder, anxiety disorder, and traditional methods of treatment of psychotic disorder mainly include drug therapy, psychotherapy, and cognitive behavioral therapy. However, these methods have some problems: the limited reality, the traditional treatment method often cannot provide real environment and situation, and the experience and participation of patients are limited; lack of personalization: the traditional method is difficult to carry out personalized treatment according to specific symptoms and demands of patients, so that the treatment effect is limited; the treatment process is limited: the particular circumstances and triggers that the patient needs to face can be difficult to replicate in reality, limiting the comprehensiveness and effectiveness of the treatment.
For example, chinese patent application publication No. CN108417271a discloses a method and system for recommending a psychotropic drug based on a subtype classification of psychotic disorder, which treats patients with psychotic disorder by means of a drug, however, the use of a drug to treat psychotic disorder is often accompanied by a series of side effects such as headache, nausea, drug addiction, which may seriously affect the quality of life of the psychotic disorder patients. As another example, chinese patent with publication number CN114270446a discloses a virtual reality treatment system, which uses virtual reality technology to treat patients with mental disorder, but cannot judge the specific type of mental disorder suffered by patients with mental disorder, so that personalized treatment cannot be performed on patients, resulting in limited treatment effect; there is a need for a method that automatically determines the specific type of mental disorder, and that is free of side effects, and that allows personalized treatment of patients with mental disorders.
Disclosure of Invention
The invention aims to provide an auxiliary treatment method for mental disorder based on VR technology, which aims to solve the existing problems in the background technology: the traditional auxiliary treatment method for the mental disorder has great side effect, can not automatically judge the specific type of the mental disorder, and can not carry out personalized treatment on the mental disorder patient;
in order to achieve the above purpose, the present invention provides the following technical solutions: an auxiliary treatment method for mental disorder based on VR technology comprises the following specific steps:
collecting the name, sex, age, facial image, heart rate, respiratory rate and skin reactance value of the patient suffering from mental disorder;
training a deep learning model according to the facial image of the mental disorder patient, and judging the facial image type of the mental disorder patient;
determining a specific type of mental disorder according to the heart rate, the breathing rate, the skin reactance value and the facial image type of the mental disorder patient;
creating a virtual environment by utilizing VR technology according to specific mental disorder types, and simulating various real-world situations and trigger factors to provide personalized auxiliary treatment;
after the treatment is completed, heart rate, respiratory rate and skin reactance values of the mental disorder patients are collected again to evaluate the treatment effect.
The invention is further improved in that the collection of the name, sex, age, facial image, heart rate, breathing rate and skin reactance value of the mental disorder patient is realized by a data collection module, and the data collection module comprises a basic information collection unit, a facial image arrangement unit, a facial image marking unit and a sensor unit;
the basic information collection unit collects the name, sex and age of the mental disorder patient and uploads the name, sex and age to the database of the remote server; a facial image collection unit captures a facial image of a mental disorder patient using a depth camera; the facial image arrangement unit is used for arranging the acquired facial images of the mental disorder patients, and the arrangement process comprises the steps of cutting, adjusting contrast and denoising the images of the mental disorder patients; after finishing, marking the facial image of each mental disorder patient by a facial image marking unit according to the image characteristics of the mental disorder patient, wherein the facial image marking unit marks the facial image of each mental disorder patient with facial flushing, anger, happiness, dullness, facial tightness, eyebrow tightness, and lips tightness; after the marking is finished, the facial image of the mental disorder patient is stored in a folder of a remote server.
A further development of the invention is that the sensor unit comprises a pulse sensor, a respiration sensor, a skin electrical activity sensor;
wherein, the pulse sensor measures heart rate of the mental disorder patient in real time; the respiration sensor measures the respiration rate of the mental disorder patient in real time; the skin electric activity sensor measures the skin reactance value of a mental disorder patient in real time; after the measurement is finished, the heart rate, the breathing rate and the skin reactance value of the mental disorder patient are respectively uploaded to a database of a remote server.
The invention further improves that a deep learning model is trained according to the facial image of the mental disorder patient, and the judgment of the facial image type of the mental disorder patient is realized by a model construction module, wherein the model construction module comprises a feature extraction unit;
wherein the feature extraction unit comprises a deep learning model; the feature extraction unit obtains from a folder of a remote serverFacial image of patients with mental disorder, will +.>Facial image of patients with mental disorder is divided into training set +.>And verification set->Two subsets, each subset comprising facial images of a plurality of patients with mental disorders; training set->The number of facial images of patients suffering from mental disorder is verification set +. >8 times the number of facial images of a psychotic disorder patient; the feature extraction unit uses the training set +.>Training a deep learning model according to the facial image of each mental disorder patient and the corresponding category of the facial image, and judging the facial image category of the mental disorder patient; after training is finished, giveVerification set->The facial image of the mental disorder patient is evaluated, the performance of the deep learning model is evaluated, and finally the deep learning model is obtained.
A further improvement of the present invention is that the deep learning model comprises a primary network and a secondary network;
the main network comprises four convolution blocks, wherein the four convolution blocks are respectively marked as a first convolution block, a second convolution block, a third convolution block and a fourth convolution block, each convolution block comprises a convolution layer, an activation layer, and the second convolution block and the fourth convolution block additionally comprise a pooling layer; the discarding layer, the first full-connection layer, the second full-connection layer and the softmax layer are sequentially connected to the back of the fourth convolution block; four convolution blocks in the secondary network and the primary network and a discarding layer are connected in parallel; the secondary network comprises two convolution blocks, a full connection layer and an activation layer, wherein the two convolution blocks are sequentially marked as a fifth convolution block and a sixth convolution block, the secondary network comprises a full connection layer which is marked as a third full connection layer, and the activation layer is connected behind the third full connection layer.
A further improvement of the present invention is that the training deep learning model comprises the steps of;
(1) Let the deep learning model beDuring the training process, each time from training set +.>Random extraction of->Facial image of patient with mental disorder is fed into deep learning model +.>Let->The facial images of the patients with mental disorder are in turn，/>The corresponding labels are->The loss function corresponding to each training process is +.>，/>The calculation formula of (2) is as follows:
in the above-mentioned description of the invention,indicate->Facial image of a person suffering from a mental disorder, wherein +.>Indicate->Class corresponding to facial image of patients with mental disorder, < +.>Indicate->Predictive category of facial image of patients with mental disorder, < +.>Representing hyper-parameters->Representation regularization->The weights of the first full connection layer and the second full connection layer are represented;
(2) From training setRandom extraction of +.>Facial image feed-in deep learning model for patients with mental disorderUntil the loss function->Stopping training when convergence is carried out;
(3) After training, in the verification setPerformance evaluation is performed on the deep learning model.
The invention is further improved in that the specific mental disorder type is determined by a mental disorder identification module according to the heart rate, the breathing rate, the skin reactance value and the facial image type of the mental disorder patient, and the mental disorder identification module comprises a decision tree unit;
Wherein the type of mental disorder includes bipolar disorder, depressive disorder, anxiety disorder; according to heart rate, respiratory rate, skin reactance value and facial image type of the mental disorder patient, the decision tree unit determines specific mental disorder type by constructing a decision tree model.
The invention further improves that the construction of the decision tree model comprises the following specific processes:
(a) CollectingPieces of data, each piece of data comprising 4 features of a patient suffering from a mental disorder, respectively a heart rate,Respiratory rate, skin reactance value, facial image category, and labeling each piece of data with specific mental disorder types, including 3, respectively bipolar disorder, depressive disorder, anxiety disorder;
(b) Will beThe strip data is divided into training sets->And verification set->Let training set->Middle->The data set for the individual specific psychotic disorder types is +.>For each feature +.>, wherein />Calculate each feature +.>The information gain ratio of (2) is calculated as follows:
；
in the above-mentioned method, the step of,is based on the characteristics->From training set->Subset of the middle partition, +.>Representation->Information entropy of->For training set->Information entropy of-> and />The calculation formulas are respectively as follows:
；
wherein ,representing training set +.>The amount of data contained in>Indicate->Data set of individual specific psychotic disorder types +.>The amount of data contained in the data;
；
representation->Middle->A data set corresponding to a specific type of mental disorder; />Is based on the characteristics->From training set->A subset of the middle partitions;
(c) For each featureCalculating the corresponding information gain rate, selecting the characteristic corresponding to the maximum information gain rate as a node, and adding the training set according to the characteristic corresponding to the maximum information gain rate>Dividing into subsets such that the data characteristic values in each subset are the same;
(d) Repeating steps (b) and (c) for each subset divided according to the feature corresponding to the maximum information gain rate until the subset cannot be divided into smaller subsets;
(e) Training the decision tree model in a verification setAnd performing performance evaluation, and finally completing the construction of the decision tree model.
A further improvement of the present invention is that, according to the specific type of mental disorder, a virtual environment is created by VR technology, simulating various real world situations and triggers to provide personalized adjuvant therapy by VR personalized therapy module; the VR personalized treatment module comprises eight units, namely an announce and education unit, an emotion unit, a sleep unit, a positive idea meditation unit, an artistic relaxation unit, a sports training unit, a role playing unit and an interpersonal conflict unit;
For all patients with mental disorder, the declaration unit is used for the treatment, and for patients with the bi-directional affective disorder, the role playing unit and the interpersonal conflict unit are used for the treatment; treatment with an mood unit and a motor training unit for patients suffering from depressive disorders; the patients suffering from anxiety disorder are treated with sleep unit, positive-concept meditation unit, artistic relaxation unit.
The invention is further improved in that after the treatment is finished, heart rate, respiratory rate and skin reactance values of the mental disorder patient are collected again to evaluate the treatment effect through the effect evaluation module;
the effect evaluation module acquires real-time heart rate, respiratory rate and skin reactance value of the mental disorder patient after the auxiliary treatment from the sensor unit, acquires heart rate, respiratory rate and skin reactance value of the mental disorder patient before the auxiliary treatment from a database of the remote server, analyzes front-back changes of the heart rate, respiratory rate and skin reactance value of the mental disorder patient, and evaluates treatment effect; after the treatment of the patient suffering from the mental disorder is finished, referring to the heart rate, the breathing rate and the skin reactance value of the normal standard, if the heart rate, the breathing rate and the skin reactance value of the patient suffering from the mental disorder are converged to the normal standard, the treatment is effective.
A mental disorder auxiliary treatment system based on VR technology comprises the following specific modules:
the data collection module is used for collecting the name, sex, age, facial image, heart rate, respiratory rate and skin reactance value of the patient suffering from the mental disorder;
the model construction module trains a deep learning model according to the facial images of the mental disorder patients and judges the facial image types of the mental disorder patients;
the mental disorder identification module is used for determining a specific mental disorder type according to the heart rate, the breathing rate, the skin reactance value and the facial image type of a mental disorder patient;
the VR personalized treatment module is used for creating a virtual environment by utilizing a VR technology according to specific mental disorder types, and simulating various real-world situations and trigger factors to provide personalized auxiliary treatment;
and the effect evaluation module is used for collecting heart rate, respiratory rate and skin reactance values of the mental disorder patient again after the treatment is finished so as to evaluate the treatment effect.
A computer-readable storage medium, having stored thereon a computer program which, when executed by a processor, implements a method for assisting in the treatment of mental disorders based on VR technology.
A controller comprising a memory for storing a computer program and a processor for implementing steps of a VR technology based psychotic disorder adjunctive therapy method when the computer program is executed.
Compared with the prior art, the invention has the beneficial effects that:
the invention provides a VR technology-based mental disorder auxiliary treatment method, which comprises the steps of collecting name, gender, age, facial images, heart rate, breathing rate and skin reactance value of a mental disorder patient, and judging the specific type of mental disorder of the mental disorder patient by utilizing a machine learning technology according to the facial images, heart rate, breathing rate and skin reactance value of the mental disorder patient; and providing individualized VR therapy services to the psychotic disorder patient according to the specific type of psychotic disorder; the whole method has no side effect, can automatically judge specific mental disorder types, can carry out personalized treatment on mental disorder patients, and has great value for improving the life quality of the mental disorder patients.
Drawings
FIG. 1 is a flow chart of a method for assisting in the treatment of mental disorders based on VR techniques;
FIG. 2 is a frame diagram of a deep learning model;
FIG. 3 is a frame diagram of a personalized treatment of a psychotic disorder;
fig. 4 is a frame diagram of a mental disorder adjuvant therapy system based on VR technology.
Detailed Description
It should be noted that the following detailed description is exemplary and is intended to provide further explanation of the invention. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.
It is noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of exemplary embodiments according to the present invention.
Embodiments of the invention and features of the embodiments may be combined with each other without conflict.
Example 1
The present embodiment provides an auxiliary treatment method for mental disorder based on VR technology, which has great value for improving the quality of life of mental disorder patients, as shown in fig. 1, and comprises the following specific steps:
name, gender, age, facial image, heart rate, respiration rate, skin reactance values were collected for patients with mental disorder. It should be noted that: the method comprises the steps that the step is realized through a data collection module, wherein the data collection module comprises a basic information collection unit, a facial image arrangement unit, a facial image marking unit and a sensor unit; the name, sex and age of the mental disorder patient are independently filled in by the mental disorder patient, and the name, sex and age of the mental disorder patient are collected by the basic information collecting unit and uploaded to a database of a remote server; a facial image collection unit captures a facial image of a mental disorder patient using a depth camera; captured images of the face of a psychotic disorder patient may be affected by noise, blurring, low contrast; these problems may result in loss of information in the image or reduce the usability of the image. Processing low quality images may negatively impact the accuracy of subsequent tasks, in addition to which the collected images may have different sizes and proportions, which may lead to difficulties in subsequent processing and analysis. Therefore, the face image arrangement unit is required to arrange the acquired face images of the mental disorder patients, the arrangement process comprises uniformly cutting the images of the mental disorder patients into 256 pixels by 256 pixels, adjusting contrast and denoising, and the specific denoising mode adopts bilateral filtering, and the bilateral filtering keeps edge information while smoothing the images, so that the face image arrangement method is suitable for removing noise and keeping the definition and detail of the face boundary. The method considers the difference between the space distance between pixels and the pixel value, so that the filtering result is more natural; after finishing, marking the facial image of each mental disorder patient by a facial image marking unit according to the image characteristics of the mental disorder patient, wherein the facial image marking unit marks the facial image of each mental disorder patient with facial flushing, anger, happiness, dullness, facial tightness, eyebrow tightness, and lips tightness; mental disorders mainly include bipolar disorder, depressive disorder, anxiety disorder; patients with different mental disorders, often have different manifestations in facial features, bipolar disorder: when a person suffers from a bi-directional affective disorder, during an elevated emotion, his face may appear as a flushing, in addition to which his facial expression changes rapidly and consistently, often switching between different facial expressions quickly in a short time, such as happy, angry or sad; for patients with depressive disorders, their facial expressions are reduced and the face often presents with tight or dull expressions; for patients suffering from anxiety disorders, their facial expressions often exhibit the characteristics of tightening the eyebrows, tightening the lips; after the marking is finished, the facial image of the mental disorder patient is stored in a folder of a remote server.
Patients with bipolar disorder, during elevated emotions, the heart rate of the patient may go beyond the normal range, the respiration rate may be increased, the skin may exhibit a lower reactance value, while during lowered emotions, the heart rate may generally fall below the normal range, the respiration rate may be slowed, and the skin may exhibit a higher reactance value.
The heart rate of a patient suffering from depression will be lower than normal, during major depression the heart rate may decrease further, the respiration rate will slow down significantly, and the patient's skin will show a higher impedance value, which is related to a lower mood and reduced physical activity of the patient.
The heart rate of a patient suffering from anxiety disorder may exceed the normal range, the heart rate may further accelerate during the anxiety episode, the skin reactance value of the anxiety disorder patient may show a lower value, which may be related to increased sympathetic activity in the patient's anxiety and stress state.
For the purpose of more accurate subsequent evaluation of the specific type of mental disorder suffered by the mental disorder patient, the heart rate, the respiratory rate and the skin reactance value of the mental disorder patient are respectively measured by the sensor unit; the sensor unit comprises a pulse sensor, a respiration sensor and a skin electric activity sensor; wherein, the pulse sensor measures heart rate of the mental disorder patient in real time; the respiration sensor measures the respiration rate of the mental disorder patient in real time; the skin electric activity sensor measures the skin reactance value of a mental disorder patient in real time; after the measurement is finished, heart rate, respiratory rate and skin reactance values of the mental disorder patient are respectively uploaded to a database of a remote server.
Training a deep learning model according to the facial image of the mental disorder patient, and judging the facial image type of the mental disorder patient. It should be noted that: the step is realized by a model building module, and the model building module comprises a feature extraction unit;
wherein the feature extraction unit comprises a deep learning model; the feature extraction unit obtains from a folder of a remote serverFacial image of patients with mental disorder, will +.>Facial image of patients with mental disorder is divided into training set +.>And verification set->Two subsets, each subset comprising facial images of a plurality of patients with mental disorders; training deviceTraining set->The number of facial images of patients suffering from mental disorder is verification set +.>8 times the number of facial images of a psychotic disorder patient; the feature extraction unit uses the training set +.>Training a deep learning model according to the facial image of each mental disorder patient and the corresponding category of the facial image, and judging the facial image category of the mental disorder patient; after training, give verification set +.>The facial image of the mental disorder patient is evaluated, the performance of the deep learning model is evaluated, and finally the deep learning model is obtained.
As shown in fig. 2, the deep learning model includes a primary network and a secondary network; the main network comprises four convolution blocks, wherein the four convolution blocks are respectively marked as a first convolution block, a second convolution block, a third convolution block and a fourth convolution block, each convolution block comprises a convolution layer, an activation layer, and the second convolution block and the fourth convolution block additionally comprise a pooling layer; the discarding layer, the first full-connection layer, the second full-connection layer and the softmax layer are sequentially connected to the back of the fourth convolution block; four convolution blocks in the secondary network and the primary network and a discarding layer are connected in parallel; the auxiliary network comprises two convolution blocks, a full connection layer and an activation layer, wherein the two convolution blocks are sequentially marked as a fifth convolution block and a sixth convolution block, the full connection layer of the auxiliary network is marked as a third full connection layer, and the activation layer is connected behind the third full connection layer; the main network and the auxiliary network can improve the grasp of the deep learning model on the detailed textures in the facial images of the patients with the mental disorder, can enable the model to be better suitable for the changes of the facial images of the patients with different mental disorders, and reduce the sensitivity to noise, so that the deep learning model is helpful for improving the capability of judging the facial image types of the patients with the mental disorder. After the deep learning model is built, training the deep learning model, wherein the training process comprises the following steps:
(1) Let the deep learning model beInitializing the deep learning model, wherein the initialized parameters meet Gaussian distribution, and the training process is performed from the training set every time>Random extraction of->Facial image of patient with mental disorder is fed into deep learning model +.>Let->The facial image of the patient suffering from mental disorder is +.>，The corresponding labels are->The loss function corresponding to each training process is +.>，/>The calculation formula of (2) is as follows:
in the above-mentioned description of the invention,indicate->Facial image of a person suffering from a mental disorder, wherein +.>Indicate->Class corresponding to facial image of patients with mental disorder, < +.>Indicate->Predictive category of facial image of patients with mental disorder, < +.>Representing hyper-parameters->Representation regularization->The weights of the first full connection layer and the second full connection layer are represented;
(2) From training setRandom extraction of +.>Facial image feed-in deep learning model for patients with mental disorderIn the training process, an Adam gradient descent optimization algorithm is used for training the deep learning model, the learning rate is 0.003, and the Adam algorithm is widely applied to the training process of the deep learning model. The method combines the ideas of a momentum method and an adaptive learning rate, and can adaptively adjust the learning rate of each parameter, thereby accelerating the convergence speed and improving the optimization The effect of the chemical, when the loss functionWhen convergence is carried out, the deep learning model stops training;
(3) After training, in the verification setPerformance evaluation is performed on the deep learning model.
And deploying the trained deep learning model on a remote server, wherein in an actual scene, the facial image collecting unit is used for collecting facial images of the mental disorder patients in real time, and the deep learning model is used for judging the types corresponding to the facial images of the mental disorder patients in real time.
The specific type of mental disorder is determined according to the heart rate, the breathing rate and the skin reactance value of the mental disorder patient and the facial image type. It should be noted that: the step is realized by a mental disorder recognition module, wherein the mental disorder recognition module comprises a decision tree unit;
the types of mental disorders mainly include bipolar disorder, depressive disorder, anxiety disorder; according to heart rate, respiratory rate, skin reactance value and facial image type of the mental disorder patient, the decision tree unit determines specific mental disorder type by constructing a decision tree model. The decision tree is a machine learning algorithm based on a tree structure and is used for solving the classification or regression problem; the facial image categories are gradually divided into a tree-shaped decision process by inputting heart rate, breathing rate and skin reactance values, so that the specific mental disorder type of a mental disorder patient is predicted; the construction of the decision tree model comprises the following specific processes:
(a) CollectingA plurality of data, each data comprising 4 characteristics of a patient suffering from a mental disorder, namely heart rate, respiratory rate, skin reactance value, facial image category, and each data being marked with a specific mental disorder type, comprising 3, namely bipolar disorder, depressive disorder, focal distance, respectivelyAnxiety disorder;
(b) Will beThe strip data is divided into training sets->And verification set->Let training set->Middle->The data set for the individual specific psychotic disorder types is +.>For each feature +.>, wherein />Calculate each feature +.>The information gain ratio of (2) is calculated as follows:
；
in the above-mentioned method, the step of,is based on the characteristics->From training set->Subset of the middle partition, +.>Representation->Information entropy of->For training set->Information entropy of-> and />The calculation formulas are respectively as follows:
；
wherein ,representing training set +.>The amount of data contained in>Indicate->Data set of individual specific psychotic disorder types +.>The amount of data contained in the data;
；
representation->Middle->A data set corresponding to a specific type of mental disorder; />Is based on the characteristics->From training set->A subset of the middle partitions;
(c) For each featureCalculating the corresponding information gain rate, selecting the characteristic corresponding to the maximum information gain rate as a node, and adding the training set according to the characteristic corresponding to the maximum information gain rate >Dividing into subsets such that the data characteristic values in each subset are the same;
(d) Repeating steps (b) and (c) for each subset divided according to the feature corresponding to the maximum information gain rate until the subset cannot be divided into smaller subsets;
(e) Training the decision tree model in a verification setAnd performing performance evaluation, and finally completing the construction of the decision tree model.
Depending on the particular type of mental disorder, VR techniques are utilized to create a virtual environment that simulates various real world scenarios and triggers to provide personalized adjuvant therapy. It should be noted that: the step is realized by a VR personalized treatment module which comprises eight units, namely an announce and education unit, an emotion unit, a sleep unit, a positive idea meditation unit, an artistic relaxation unit, a sports training unit, a role playing unit and an interpersonal conflict unit;
for all patients with mental disorder, the education unit is used for the start;
patients with bipolar affective disorders often switch quickly between different emotions, such as happy, angry or sad, in a short time, have problems of emotional instability, and are prone to conflict with others; treatment with a role playing unit and an interpersonal conflict unit for a patient suffering from a bipolar disorder;
Patients with depressive disorder lose interest in activities and things of previous interest and have symptoms of multiple negative emotions, low power and easy fatigue; treatment with an mood unit and a motor training unit for patients suffering from depressive disorders;
anxiety disorder patients frequently experience tension, anxiety or fear. These emotions may persist and be difficult to control or relax; treatment with a sleep unit, a positive-thought meditation unit, and an artistic relaxation unit for patients suffering from anxiety disorders;
the system comprises a ventilating and teaching unit, a video playing unit and a processing unit, wherein the ventilating and teaching unit enables a mental disorder patient to be in a safe room, and shows mental disorder related knowledge to the mental disorder patient in a video playing mode, wherein the related knowledge comprises mental disorder definition, performance, causes, treatment modes and coping skills;
a role playing unit for converting the role into cut-in so that the patient plays different identities in different environments; the patients with the bi-directional affective disorder learn to cope with and improve the dilemma of the patients, and help the patients with the bi-directional affective disorder to improve the problems of unstable emotion and easy irritation;
the interpersonal conflict unit is used for helping the patients with mental disorder to learn a correct communication mode and a method for processing the problems by simulating the interpersonal conflict scene generated in public places, so that the problems of unstable emotion and easy irritation of the patients with the bidirectional affective disorder are improved;
The emotion unit is used for guiding a patient to carry out emotion expression by introducing a daemon into a place of play in a safe room and guiding the patient to review the important emotion change of the patient in the week and timely dredge negative emotion by using a video playing mode and a fun flower mode;
the exercise training unit is used for helping depressive disorder patients to improve physical quality and improve low-power and fatigue-prone symptoms by simulating scenes such as rock climbing, car washing and the like;
the sleep unit is used for helping the anxiety disorder patient to know the definition of good sleep and find a sleep-entering mode suitable for the anxiety disorder patient, improving the sleep quality and improving the circadian rhythm disorder problem through an ancient magic book form;
the positive-concept meditation unit selects a visual angle open scene to enable anxiety disorder patients to be integrated into a remote natural environment, and the tension, anxiety and fear symptoms of the anxiety disorder patients are improved through a positive-concept meditation mode;
and the artistic relaxing unit is used for relaxing mind and body and relieving energy and stress symptoms of the anxiety disorder patients by passing through the graffiti artistic form.
After the treatment is completed, heart rate, respiratory rate and skin reactance values of the mental disorder patients are collected again to evaluate the treatment effect. It should be noted that: the step is realized through an effect evaluation module;
The effect evaluation module acquires real-time heart rate, respiratory rate and skin reactance value of the psychotic disorder patient after the auxiliary treatment from the sensor unit, acquires heart rate, respiratory rate and skin reactance value of the psychotic disorder patient before the auxiliary treatment from a database of the remote server, and analyzes the front-back changes of the heart rate, respiratory rate and skin reactance value of the psychotic disorder patient by referring to the standard of the normal heart rate, respiratory rate and skin reactance value, so as to evaluate the treatment effect; after the treatment of the patient suffering from the mental disorder is finished, the treatment is effective if the heart rate, the breathing rate and the skin reactance value are converged to normal standards.
Example two
The present embodiment is a second embodiment of the present invention, and a mental disorder auxiliary treatment system based on VR technology includes the following modules:
the data collection module is used for collecting the name, sex, age, facial image, heart rate, respiratory rate and skin reactance value of the patient suffering from the mental disorder; the data collection module comprises a basic information collection unit, a facial image arrangement unit, a facial image marking unit and a sensor unit; the name, sex and age of the mental disorder patient are independently filled in by the mental disorder patient, and the name, sex and age of the mental disorder patient are collected by the basic information collecting unit and uploaded to a database of a remote server; a facial image collection unit captures a facial image of a mental disorder patient using a depth camera; captured images of the face of a psychotic disorder patient may be affected by noise, blurring, low contrast; these problems may result in loss of information in the image or reduce the usability of the image. Processing low quality images may negatively impact the accuracy of subsequent tasks, in addition to which the collected images may have different sizes and proportions, which may lead to difficulties in subsequent processing and analysis. Therefore, the face image arrangement unit is required to arrange the acquired face images of the mental disorder patients, the arrangement process comprises uniformly cutting the images of the mental disorder patients into 256×256 pixels, adjusting contrast and denoising, and the specific denoising mode adopts bilateral filtering, and the bilateral filtering is suitable for removing noise and keeping the definition and detail of the face boundary while smoothing the images. The method considers the difference between the space distance between pixels and the pixel value, so that the filtering result is more natural; after finishing, marking the facial image of each mental disorder patient by a facial image marking unit according to the image characteristics of the mental disorder patient, wherein the facial image marking unit marks the facial image of each mental disorder patient with facial flushing, anger, happiness, dullness, facial tightness, eyebrow tightness, and lips tightness; mental disorders mainly include bipolar disorder, depressive disorder, anxiety disorder; patients with different mental disorders, often have different manifestations in facial features, bipolar disorder: when a person suffers from a bi-directional affective disorder, during an elevated emotion, his face may appear as a flushing, in addition to which his facial expression changes rapidly and consistently, often switching between different facial expressions quickly in a short time, such as happy, angry or sad; for patients with depressive disorders, their facial expressions are reduced and the face often presents with tight or dull expressions; for patients suffering from anxiety disorders, their facial expressions often exhibit the characteristics of tightening the eyebrows, tightening the lips; after the marking is finished, the facial image of the mental disorder patient is stored in a folder of a remote server.
Patients with bipolar disorder, during periods of elevated mood, the heart rate of the patient will be outside the normal range, the respiration rate will be increased, the skin will exhibit a lower reactance value, while during periods of lowered mood, the heart rate will generally be below the normal range, the respiration rate will be slowed down, the skin will exhibit a higher reactance value; the heart rate of a patient suffering from depression will be lower than normal, during major depression the heart rate may decrease further, the respiration rate will slow down significantly, and the patient's skin will show a higher impedance value, which is related to a lower mood and reduced physical activity of the patient. The heart rate of a patient suffering from anxiety disorder may exceed the normal range, the heart rate may further accelerate during the anxiety episode, the skin reactance value of the anxiety disorder patient may show a lower value, which may be related to increased sympathetic activity in the patient's anxiety and stress state; for the purpose of more accurate subsequent evaluation of the specific type of mental disorder suffered by the mental disorder patient, the heart rate, the respiratory rate and the skin reactance value of the mental disorder patient are respectively measured by the sensor unit; the sensor unit comprises a pulse sensor, a respiration sensor and a skin electric activity sensor; wherein, the pulse sensor measures heart rate of the mental disorder patient in real time; the respiration sensor measures the respiration rate of the mental disorder patient in real time; the skin electric activity sensor measures the skin reactance value of a mental disorder patient in real time; after the measurement is finished, heart rate, respiratory rate and skin reactance values of the mental disorder patient are respectively uploaded to a database of a remote server.
The model construction module trains a deep learning model according to the facial images of the mental disorder patients and judges the facial image types of the mental disorder patients; the model construction module comprises a feature extraction unit;
wherein the feature extraction unit comprises a deep learning model; the feature extraction unit obtains from a folder of a remote serverFacial image of patients with mental disorder, will +.>Facial image of patients with mental disorder is divided into training set +.>And verification set->Two subsets, each subset comprising facial images of a plurality of patients with mental disorders; training set->The number of facial images of patients suffering from mental disorder is verification set +.>8 times the number of facial images of a psychotic disorder patient; the feature extraction unit uses the training set +.>Training a deep learning model according to the facial image of each mental disorder patient and the corresponding category of the image, and judging the facial image category of the mental disorder patient; after training, give verification set +.>The facial image of the mental disorder patient is evaluated, the performance of the deep learning model is evaluated, and finally the deep learning model is obtained.
As shown in fig. 2, the deep learning model includes a primary network and a secondary network; the main network comprises four convolution blocks, wherein the four convolution blocks are respectively marked as a first convolution block, a second convolution block, a third convolution block and a fourth convolution block, each convolution block comprises a convolution layer, an activation layer, and the second convolution block and the fourth convolution block additionally comprise a pooling layer; the discarding layer, the first full-connection layer, the second full-connection layer and the softmax layer are sequentially connected to the back of the fourth convolution block; four convolution blocks in the secondary network and the primary network and a discarding layer are connected in parallel; the auxiliary network comprises two convolution blocks, a full connection layer and an activation layer, wherein the two convolution blocks are sequentially marked as a fifth convolution block and a sixth convolution block, the full connection layer of the auxiliary network is marked as a third full connection layer, and the activation layer is connected behind the third full connection layer; the main network and the auxiliary network can improve the grasp of the deep learning model on the detailed textures in the facial images of the patients with the mental disorder, can enable the model to be better suitable for the changes of the facial images of the patients with different mental disorders, and reduce the sensitivity to noise, so that the deep learning model is helpful for improving the capability of judging the facial image types of the patients with the mental disorder. After the deep learning model is built, training the deep learning model, wherein the training process comprises the following steps:
(1) Let the deep learning model beInitializing the deep learning model, wherein the initialized parameters meet Gaussian distribution, and the training process is performed from the training set every time>Random extraction of->Facial image of patient with mental disorder is fed into deep learning model +.>Let->The facial image of the patient suffering from mental disorder is +.>，The corresponding labels are->Each of thenThe loss function corresponding to the secondary training process is +.>，/>The calculation formula of (2) is as follows:
in the above-mentioned description of the invention,indicate->Facial image of a person suffering from a mental disorder, wherein +.>Indicate->Class corresponding to facial image of patients with mental disorder, < +.>Indicate->Predictive category of facial image of patients with mental disorder, < +.>Representing hyper-parameters->Representation regularization->The weights of the first full connection layer and the second full connection layer are represented;
(2) From training setRandom extraction of +.>Facial image feed-in deep learning model for patients with mental disorderIn the training process, an Adam gradient descent optimization algorithm is used for training the deep learning model, the learning rate is 0.003, and the Adam algorithm is widely applied to the training process of the deep learning model. The method combines the ideas of a momentum method and an adaptive learning rate, and can adaptively adjust the learning rate of each parameter, thereby accelerating the convergence speed and improving the optimization effect when the function is lost When convergence is carried out, the deep learning model stops training;
(3) After training, in the verification setPerformance evaluation is performed on the deep learning model.
And deploying the trained deep learning model on a remote server, wherein in an actual scene, the facial image collecting unit is used for collecting facial images of the mental disorder patients in real time, and the deep learning model is used for judging the types corresponding to the facial images of the mental disorder patients in real time.
The mental disorder identification module is used for determining specific mental disorder types according to heart rate, breathing rate and skin reactance value of mental disorder patients and facial image types; the mental disorder recognition module comprises a decision tree unit;
the types of mental disorders mainly include bipolar disorder, depressive disorder, anxiety disorder; according to heart rate, respiratory rate, skin reactance value and facial image type of the mental disorder patient, the decision tree unit determines specific mental disorder type by constructing a decision tree model. The decision tree is a machine learning algorithm based on a tree structure and is used for solving the classification or regression problem; the facial image categories are gradually divided into a tree-shaped decision process by inputting heart rate, breathing rate and skin reactance values, so that the specific mental disorder type of a mental disorder patient is predicted; the construction of the decision tree model comprises the following specific processes:
(a) CollectingA data set, each data set comprising 4 features of a patient suffering from a mental disorder, namely heart rate, respiratory rate, skin reactance value and facial image category, and marking each data set with a specific mental disorder type, wherein the specific mental disorder type comprises 3 data, namely bidirectional affective disorder, depressive disorder and anxiety disorder;
(b) Will beThe strip data is divided into training sets->And verification set->Let training set->Middle->The data set for the individual specific psychotic disorder types is +.>For each feature +.>, wherein />Calculate each feature +.>The information gain ratio of (2) is calculated as follows:
；
in the above-mentioned method, the step of,is based on the characteristics->From training set->Subset of the middle partition, +.>Representation->Information entropy of->For training set->Information entropy of-> and />The calculation formulas are respectively as follows:
；
wherein ,representing training set +.>The amount of data contained in>Indicate->Data set of individual specific psychotic disorder types +.>The amount of data contained in the data;
；
representation->Middle->A data set corresponding to a specific type of mental disorder; />Is based on the characteristics->From training set->A subset of the middle partitions;
(c) For each featureCalculating the corresponding information gain rate, selecting the characteristic corresponding to the maximum information gain rate as a node, and adding the training set according to the characteristic corresponding to the maximum information gain rate >Dividing into subsets such that the data characteristic values in each subset are the same;
(d) Repeating steps (b) and (c) for each subset divided according to the feature corresponding to the maximum information gain rate until the subset cannot be divided into smaller subsets;
(e) Decision tree model to be trainedAt the verification setAnd performing performance evaluation, and finally completing the construction of the decision tree model.
The VR personalized treatment module is used for creating a virtual environment by utilizing a VR technology according to specific mental disorder types, and simulating various real-world situations and trigger factors to provide personalized auxiliary treatment; the VR personalized treatment module comprises eight units, namely an announce and education unit, an emotion unit, a sleep unit, a positive idea meditation unit, an artistic relaxation unit, a sports training unit, a role playing unit and an interpersonal conflict unit;
for all patients with mental disorder, the education unit is used for the start;
patients with bipolar affective disorders often switch quickly between different emotions, such as happy, angry or sad, in a short time, have problems of emotional instability, and are prone to conflict with others; treatment with a role playing unit and an interpersonal conflict unit for a patient suffering from a bipolar disorder;
Patients with depressive disorder lose interest in activities and things of previous interest and have symptoms of multiple negative emotions, low power and easy fatigue; treatment with an mood unit and a motor training unit for patients suffering from depressive disorders;
anxiety disorder patients frequently experience tension, anxiety or fear. These emotions may persist and be difficult to control or relax; treatment with a sleep unit, a positive-thought meditation unit, and an artistic relaxation unit for patients suffering from anxiety disorders;
the system comprises a ventilating and teaching unit, a video playing unit and a processing unit, wherein the ventilating and teaching unit enables a mental disorder patient to be in a safe room, and shows mental disorder related knowledge to the mental disorder patient in a video playing mode, wherein the related knowledge comprises mental disorder definition, performance, causes, treatment modes and coping skills;
a role playing unit for converting the role into cut-in so that the patient plays different identities in different environments; the patients with the bi-directional affective disorder learn to cope with and improve the dilemma of the patients, and help the patients with the bi-directional affective disorder to improve the problems of unstable emotion and easy irritation;
the interpersonal conflict unit is used for helping the patients with mental disorder to learn a correct communication mode and a method for processing the problems by simulating the interpersonal conflict scene generated in public places, so that the problems of unstable emotion and easy irritation of the patients with the bidirectional affective disorder are improved;
The emotion unit is used for guiding a patient to carry out emotion expression by introducing a daemon into a place of play in a safe room and guiding the patient to review the important emotion change of the patient in the week and timely dredge negative emotion by using a video playing mode and a fun flower mode;
the exercise training unit is used for helping depressive disorder patients to improve physical quality and improve low-power and fatigue-prone symptoms by simulating scenes such as rock climbing, car washing and the like;
the sleep unit is used for helping the anxiety disorder patient to know the definition of good sleep and find a sleep-entering mode suitable for the anxiety disorder patient, improving the sleep quality and improving the circadian rhythm disorder problem through an ancient magic book form;
the positive-concept meditation unit selects a visual angle open scene to enable anxiety disorder patients to be integrated into a remote natural environment, and the tension, anxiety and fear symptoms of the anxiety disorder patients are improved through a positive-concept meditation mode;
and the artistic relaxing unit is used for relaxing mind and body and relieving energy and stress symptoms of the anxiety disorder patients by passing through the graffiti artistic form.
The effect evaluation module is used for collecting heart rate, respiratory rate and skin reactance values of the mental disorder patient again after treatment is finished so as to evaluate the treatment effect; the effect evaluation module acquires real-time heart rate, respiratory rate and skin reactance value of the psychotic disorder patient after the auxiliary treatment from the sensor unit, acquires heart rate, respiratory rate and skin reactance value of the psychotic disorder patient before the auxiliary treatment from a database of the remote server, and analyzes the front-back changes of the heart rate, respiratory rate and skin reactance value of the psychotic disorder patient by referring to the standard of the normal heart rate, respiratory rate and skin reactance value, so as to evaluate the treatment effect; after the treatment of the patient suffering from the mental disorder is finished, the treatment is effective if the heart rate, the breathing rate and the skin reactance value are converged to normal standards.
Example III
A computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, enables a method for assisted treatment of mental disorders based on VR technology. It should be noted that:
the computer program of the mental disorder auxiliary treatment method based on VR technology is realized by C++, python, wherein, a data collection module, a VR personalized treatment module and an effect evaluation module are realized by the same STM32 embedded microcontroller, wireless WIFI is arranged on the STM32, facial images, heart rate, respiratory rate and skin reactance values of mental disorder patients are uploaded to a remote server through the wireless WIFI, and driving programs required by the data collection module, the VR personalized treatment module and the effect evaluation module are written in C++ language; the model building module and the mental disorder recognition module are realized by Python language and controlled by Intel Core i7, and the STM32 embedded microcontroller and the Intel Core i7 comprise a memory and a processor, wherein the memory is used for storing a computer program; the processor is used for executing the computer program, so that the Intel CORE i7 and STM32 embedded microcontroller execute to realize the mental disorder auxiliary treatment system based on VR technology.
Finally, it should be noted that: the above examples are only specific embodiments of the present invention, and are not intended to limit the scope of the present invention, but it should be understood by those skilled in the art that the present invention is not limited thereto, and that the present invention is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention, and are intended to be included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.
Claims (13)
1. An auxiliary treatment method for mental disorder based on VR technology is characterized in that: the method comprises the following specific steps:
collecting the name, sex, age, facial image, heart rate, respiratory rate and skin reactance value of the patient suffering from mental disorder;
training a deep learning model according to the facial image of the mental disorder patient, and judging the facial image type of the mental disorder patient;
Determining a specific type of mental disorder according to the heart rate, the breathing rate, the skin reactance value and the facial image type of the mental disorder patient;
creating a virtual environment by utilizing VR technology according to specific mental disorder types, and simulating various real-world situations and trigger factors to provide personalized auxiliary treatment;
after the treatment is completed, heart rate, respiratory rate and skin reactance values of the mental disorder patients are collected again to evaluate the treatment effect.
2. The VR technology-based psychotic disorder adjunctive therapy method of claim 1, wherein: the method comprises the steps that the name, the sex, the age, the facial image, the heart rate, the breathing rate and the skin reactance value of a mental disorder patient are collected through a data collection module, and the data collection module comprises a basic information collection unit, a facial image arrangement unit, a facial image marking unit and a sensor unit;
the basic information collection unit collects the name, sex and age of the mental disorder patient and uploads the name, sex and age to the database of the remote server; a facial image collection unit captures a facial image of a mental disorder patient using a depth camera; the facial image arrangement unit is used for arranging the acquired facial images of the mental disorder patients, and the arrangement process comprises the steps of cutting, adjusting contrast and denoising the images of the mental disorder patients; after finishing, marking the facial image of each mental disorder patient by a facial image marking unit according to the image characteristics of the mental disorder patient, wherein the facial image marking unit marks the facial image of each mental disorder patient with facial flushing, anger, happiness, dullness, facial tightness, eyebrow tightness, and lips tightness; after the marking is finished, the facial image of the mental disorder patient is stored in a folder of a remote server.
3. The VR technology-based psychotic disorder adjunctive therapy method of claim 2, wherein: the sensor unit comprises a pulse sensor, a respiration sensor and a skin electric activity sensor;
wherein, the pulse sensor measures heart rate of the mental disorder patient in real time; the respiration sensor measures the respiration rate of the mental disorder patient in real time; the skin electric activity sensor measures the skin reactance value of a mental disorder patient in real time; after the measurement is finished, the heart rate, the breathing rate and the skin reactance value of the mental disorder patient are respectively uploaded to a database of a remote server.
4. A method for the assisted treatment of mental disorders based on VR technology as set forth in claim 3, wherein: training a deep learning model according to facial images of a mental disorder patient, and judging facial image types of the mental disorder patient through a model construction module, wherein the model construction module comprises a feature extraction unit;
wherein the feature extraction unit comprises a deep learning model; the feature extraction unit obtains from a folder of a remote serverFacial image of patients with mental disorder, will +.>Facial image of patients with mental disorder is divided into training set +. >And verification set->Two subsets, each subset comprising facial images of a plurality of patients with mental disorders;training set->The number of facial images of patients suffering from mental disorder is verification set +.>8 times the number of facial images of a psychotic disorder patient; the feature extraction unit uses the training set +.>Training a deep learning model according to the facial image of each mental disorder patient and the corresponding category of the facial image, and judging the facial image category of the mental disorder patient; after training, give verification set +.>The facial image of the mental disorder patient is evaluated, the performance of the deep learning model is evaluated, and finally the deep learning model is obtained.
5. The VR technology-based psychotic disorder adjunctive therapy method of claim 4, wherein: the deep learning model comprises a main network and a secondary network;
the main network comprises four convolution blocks, wherein the four convolution blocks are respectively marked as a first convolution block, a second convolution block, a third convolution block and a fourth convolution block, each convolution block comprises a convolution layer, an activation layer, and the second convolution block and the fourth convolution block additionally comprise a pooling layer; the discarding layer, the first full-connection layer, the second full-connection layer and the softmax layer are sequentially connected to the back of the fourth convolution block; four convolution blocks in the secondary network and the primary network and a discarding layer are connected in parallel; the secondary network comprises two convolution blocks, a full connection layer and an activation layer, wherein the two convolution blocks are sequentially marked as a fifth convolution block and a sixth convolution block, the secondary network comprises a full connection layer which is marked as a third full connection layer, and the activation layer is connected behind the third full connection layer.
6. The VR technology-based psychotic disorder adjunctive therapy method of claim 5, wherein: the training deep learning model comprises the following steps;
(1) Let the deep learning model beDuring the training process, each time from training set +.>Random extraction of->Facial image of patient with mental disorder is fed into deep learning model +.>Let->The facial images of the patients with mental disorder are in turn，/>The corresponding labels are->The loss function corresponding to each training process is +.>，/>The calculation formula of (2) is as follows:
；
in the above-mentioned description of the invention,indicate->Facial image of a person suffering from a mental disorder, wherein +.>Indicate->Class corresponding to facial image of patients with mental disorder, < +.>Indicate->Predictive category of facial image of patients with mental disorder, < +.>Representing hyper-parameters->Representation regularization->The weights of the first full connection layer and the second full connection layer are represented;
(2) From training setRandom extraction of +.>Facial image of patient with mental disorder is fed into deep learning model +.>Until the loss function->Stopping training when convergence is carried out;
(3) After training, in the verification setPerformance evaluation is performed on the deep learning model.
7. The VR technology-based psychotic disorder adjunctive therapy method of claim 6, wherein: determining specific mental disorder type according to heart rate, respiratory rate, skin reactance value and facial image type of mental disorder patient, wherein the mental disorder identification module comprises a decision tree unit;
Wherein the type of mental disorder includes bipolar disorder, depressive disorder, anxiety disorder; according to heart rate, respiratory rate, skin reactance value and facial image type of the mental disorder patient, the decision tree unit determines specific mental disorder type by constructing a decision tree model.
8. The VR technology-based psychotic disorder adjunctive therapy method of claim 7, wherein the constructing the decision tree model comprises the following specific steps:
(a) CollectingA data set, each data set comprising 4 features of a patient suffering from a mental disorder, namely heart rate, respiratory rate, skin reactance value and facial image category, and marking each data set with a specific mental disorder type, wherein the specific mental disorder type comprises 3 data, namely bidirectional affective disorder, depressive disorder and anxiety disorder;
(b) Will beThe strip data is divided into training sets->And verification set->Let training set->Middle->The data set for the individual specific psychotic disorder types is +.>For each feature +.>, wherein />Calculate each feature +.>The information gain ratio of (2) is calculated as follows:
；
in the above-mentioned method, the step of,is based on the characteristics->From training set->Subset of the middle partition, +.>Representation->Is used for the information entropy of (a), For training set->Information entropy of-> and />The calculation formulas are respectively as follows:
；
wherein ,representing training set +.>The amount of data contained in>Indicate->Data set of individual specific psychotic disorder types +.>The amount of data contained in the data;
；
representation->Middle->A data set corresponding to a specific type of mental disorder; />Is based on the characteristics->From training set->A subset of the middle partitions;
(c) For each featureCalculating the corresponding information gain rate, selecting the characteristic corresponding to the maximum information gain rate as a node, and adding the training set according to the characteristic corresponding to the maximum information gain rate>Dividing into subsets such that the data characteristic values in each subset are the same;
(d) Repeating steps (b) and (c) for each subset divided according to the feature corresponding to the maximum information gain rate until the subset cannot be divided into smaller subsets;
(e) Training the decision tree model in a verification setAnd performing performance evaluation, and finally completing the construction of the decision tree model.
9. The VR technology-based psychotic disorder adjunctive therapy method of claim 8, wherein: creating a virtual environment by utilizing VR technology according to the specific mental disorder type, and simulating various real-world situations and trigger factors to provide personalized auxiliary treatment through a VR personalized treatment module; the VR personalized treatment module comprises eight units, namely an announce and education unit, an emotion unit, a sleep unit, a positive idea meditation unit, an artistic relaxation unit, a sports training unit, a role playing unit and an interpersonal conflict unit;
For all patients with mental disorder, the declaration unit is used for the treatment, and for patients with the bi-directional affective disorder, the role playing unit and the interpersonal conflict unit are used for the treatment; treatment with an mood unit and a motor training unit for patients suffering from depressive disorders; the patients suffering from anxiety disorder are treated with sleep unit, positive-concept meditation unit, artistic relaxation unit.
10. The VR technology-based psychotic disorder adjunctive therapy method of claim 9, wherein: after the treatment is finished, heart rate, respiratory rate and skin reactance value of the mental disorder patient are collected again to evaluate the treatment effect, and the treatment effect is achieved through an effect evaluation module;
the effect evaluation module acquires real-time heart rate, respiratory rate and skin reactance value of the mental disorder patient after the auxiliary treatment from the sensor unit, acquires heart rate, respiratory rate and skin reactance value of the mental disorder patient before the auxiliary treatment from a database of the remote server, analyzes front-back changes of the heart rate, respiratory rate and skin reactance value of the mental disorder patient, and evaluates treatment effect; after the treatment of the patient suffering from the mental disorder is finished, referring to the heart rate, the breathing rate and the skin reactance value of the normal standard, if the heart rate, the breathing rate and the skin reactance value of the patient suffering from the mental disorder are converged to the normal standard, the treatment is effective.
11. A VR technology-based psychotic disorder adjunctive therapy system, implemented on the basis of a VR technology-based psychotic disorder adjunctive therapy method according to any of claims 1-10, characterized in that: comprising
The data collection module is used for collecting the name, sex, age, facial image, heart rate, respiratory rate and skin reactance value of the patient suffering from the mental disorder;
the model construction module trains a deep learning model according to the facial images of the mental disorder patients and judges the facial image types of the mental disorder patients;
the mental disorder identification module is used for determining a specific mental disorder type according to the heart rate, the breathing rate, the skin reactance value and the facial image type of a mental disorder patient;
the VR personalized treatment module is used for creating a virtual environment by utilizing a VR technology according to specific mental disorder types, and simulating various real-world situations and trigger factors to provide personalized auxiliary treatment;
and the effect evaluation module is used for collecting heart rate, respiratory rate and skin reactance values of the mental disorder patient again after the treatment is finished so as to evaluate the treatment effect.
12. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements a method for assisting in the treatment of mental disorders based on VR technology as set forth in any one of claims 1-10.
13. A controller comprising a memory and a processor, the memory for storing a computer program,
the processor being configured to implement the steps of a method for assisting in the treatment of mental disorders based on VR technology as set forth in any one of claims 1 to 10 when executing the computer program.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311271544.7A CN116994718B (en) | 2023-09-28 | 2023-09-28 | VR technology-based mental disorder auxiliary treatment method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311271544.7A CN116994718B (en) | 2023-09-28 | 2023-09-28 | VR technology-based mental disorder auxiliary treatment method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116994718A true CN116994718A (en) | 2023-11-03 |
| CN116994718B CN116994718B (en) | 2023-12-01 |
Family
ID=88521750
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311271544.7A Active CN116994718B (en) | 2023-09-28 | 2023-09-28 | VR technology-based mental disorder auxiliary treatment method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116994718B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117438048A (en) * | 2023-12-20 | 2024-01-23 | 深圳市龙岗区第三人民医院 | Method and system for assessing psychological disorder of psychiatric patient |
| CN117637118A (en) * | 2024-01-27 | 2024-03-01 | 南京元域绿洲科技有限公司 | Anxiety disorder virtual reality training system |
Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050216243A1 (en) * | 2004-03-02 | 2005-09-29 | Simon Graham | Computer-simulated virtual reality environments for evaluation of neurobehavioral performance |
| CA2833398A1 (en) * | 2013-11-20 | 2015-05-20 | El-Mar Inc. | Method of identifying an individual with a disorder or efficacy of a treatment of a disorder |
| CN107320836A (en) * | 2017-05-02 | 2017-11-07 | 华南理工大学 | Depression auxiliary treatment virtual scene system and its implementation based on VR technologies |
| CN109215804A (en) * | 2018-10-09 | 2019-01-15 | 华南理工大学 | Mental disorder assistant diagnosis system based on virtual reality technology and physio-parameter detection |
| WO2019161050A1 (en) * | 2018-02-18 | 2019-08-22 | Akili Interactive Labs, Inc. | Cognitive platform including computerized elements coupled with a therapy for mood disorder |
| CN113611395A (en) * | 2021-08-09 | 2021-11-05 | 江苏嘉纳宝医疗科技有限公司 | Mental and psychological illness user auxiliary training method based on virtual reality technology |
| WO2022087116A1 (en) * | 2020-10-22 | 2022-04-28 | Tibi Health, Inc. | Systems and methods for mental health assessment |
| WO2023012818A1 (en) * | 2021-07-31 | 2023-02-09 | Sparcolife Digital Healthcare Technologies Private Limited | A non-invasive multimodal screening and assessment system for human health monitoring and a method thereof |
| CN116386845A (en) * | 2023-06-06 | 2023-07-04 | 北京回龙观医院(北京心理危机研究与干预中心) | Schizophrenia diagnosis system based on convolutional neural network and facial dynamic video |
| WO2023139587A1 (en) * | 2022-01-24 | 2023-07-27 | Nihaytech Ltd. | Therapeutical virtual reality application |
| CN116665847A (en) * | 2023-08-02 | 2023-08-29 | 南京元域绿洲科技有限公司 | VR-based mood adjustment training system surrounding mental disorder |
-
2023
- 2023-09-28 CN CN202311271544.7A patent/CN116994718B/en active Active
Patent Citations (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050216243A1 (en) * | 2004-03-02 | 2005-09-29 | Simon Graham | Computer-simulated virtual reality environments for evaluation of neurobehavioral performance |
| CA2833398A1 (en) * | 2013-11-20 | 2015-05-20 | El-Mar Inc. | Method of identifying an individual with a disorder or efficacy of a treatment of a disorder |
| CN107320836A (en) * | 2017-05-02 | 2017-11-07 | 华南理工大学 | Depression auxiliary treatment virtual scene system and its implementation based on VR technologies |
| WO2019161050A1 (en) * | 2018-02-18 | 2019-08-22 | Akili Interactive Labs, Inc. | Cognitive platform including computerized elements coupled with a therapy for mood disorder |
| CN109215804A (en) * | 2018-10-09 | 2019-01-15 | 华南理工大学 | Mental disorder assistant diagnosis system based on virtual reality technology and physio-parameter detection |
| WO2022087116A1 (en) * | 2020-10-22 | 2022-04-28 | Tibi Health, Inc. | Systems and methods for mental health assessment |
| WO2023012818A1 (en) * | 2021-07-31 | 2023-02-09 | Sparcolife Digital Healthcare Technologies Private Limited | A non-invasive multimodal screening and assessment system for human health monitoring and a method thereof |
| CN113611395A (en) * | 2021-08-09 | 2021-11-05 | 江苏嘉纳宝医疗科技有限公司 | Mental and psychological illness user auxiliary training method based on virtual reality technology |
| WO2023139587A1 (en) * | 2022-01-24 | 2023-07-27 | Nihaytech Ltd. | Therapeutical virtual reality application |
| CN116386845A (en) * | 2023-06-06 | 2023-07-04 | 北京回龙观医院(北京心理危机研究与干预中心) | Schizophrenia diagnosis system based on convolutional neural network and facial dynamic video |
| CN116665847A (en) * | 2023-08-02 | 2023-08-29 | 南京元域绿洲科技有限公司 | VR-based mood adjustment training system surrounding mental disorder |
Non-Patent Citations (9)
| Title |
|---|
| BRYANT NIEDERRITER; ALICE RONG; FAISAL AQLAN; HUI YANG: "Sensor-Based Virtual Reality for Clinical Decision Support in the Assessment of Mental Disorders", 2020 IEEE CONFERENCE ON GAMES (COG), pages 666 - 669 * |
| IMOGEN H. BELL，JENNIFER NICHOLAS: "Virtual reality as a clinical tool in mental health research and practice", DIALOGUES CLIN NEUROSCI, vol. 22, no. 02, pages 169 * |
| SHAYAN HASSANTABAR, JOE ZHANG, HONGXU YIN, NIRAJ K. JHA: "MHDeep: Mental Health Disorder Detection System Based on Wearable Sensors and Artificial Neural Networks", ACM TRANSACTIONS ON EMBEDDED COMPUTING SYSTEMS, vol. 21, no. 06, pages 1 - 22, XP058930034, DOI: 10.1145/3527170 * |
| YIXUE HAO; ZAIYANG TANG; BANDER ALZAHRANI: "An End-to-End Human Abnormal Behavior Recognition Framework for Crowds With Mentally Disordered Individuals", IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, vol. 26, no. 08, pages 3618 * |
| 剧友飞;徐忠平: "应用虚拟现实技术治疗创伤后应激障碍的研究", 考试周刊, no. 45, pages 239 - 240 * |
| 朱佳倩;袁勇贵;: "虚拟现实技术在精神障碍中的应用进展", 国际精神病学杂志, no. 04, pages 666 - 669 * |
| 李丹阳;张晶晶: "精神障碍研究中机器学习的应用——基于EEG的特征提取", 科学技术创新, no. 08, pages 44 - 46 * |
| 林弋琪;王希;彭凯平;倪士光;: "虚拟现实技术与自闭症谱系障碍治疗:科技新希望", 心理科学进展, no. 03, pages 518 - 526 * |
| 韩佳丽; 丰雷: "人工智能在抑郁症领域中应用的研究进展", 北京医学, vol. 42, no. 04, pages 317 - 319 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117438048A (en) * | 2023-12-20 | 2024-01-23 | 深圳市龙岗区第三人民医院 | Method and system for assessing psychological disorder of psychiatric patient |
| CN117438048B (en) * | 2023-12-20 | 2024-02-23 | 深圳市龙岗区第三人民医院 | Method and system for assessing psychological disorder of psychiatric patient |
| CN117637118A (en) * | 2024-01-27 | 2024-03-01 | 南京元域绿洲科技有限公司 | Anxiety disorder virtual reality training system |
| CN117637118B (en) * | 2024-01-27 | 2024-03-29 | 南京元域绿洲科技有限公司 | Anxiety disorder virtual reality training system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116994718B (en) | 2023-12-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN116994718B (en) | VR technology-based mental disorder auxiliary treatment method | |
| CN106851216B (en) | A kind of classroom behavior monitoring system and method based on face and speech recognition | |
| CN110070944A (en) | Training system is assessed based on virtual environment and the social function of virtual role | |
| Tognetti et al. | Modeling enjoyment preference from physiological responses in a car racing game | |
| CN110334626A (en) | A kind of on-line study system based on affective state | |
| CN108921284A (en) | Interpersonal interactive body language automatic generation method and system based on deep learning | |
| CN102354349A (en) | Human-machine interaction multi-mode early intervention system for improving social interaction capacity of autistic children | |
| CN109773807B (en) | Motion control method and robot | |
| CN109447001A (en) | A kind of dynamic Emotion identification method | |
| Takacs | Special education and rehabilitation: teaching and healing with interactive graphics | |
| CN108615439A (en) | Method, apparatus, equipment and medium for formulating ability training scheme | |
| CN104268921A (en) | 3D face expression control method and system | |
| CN106372729A (en) | Depth learning method and device for mental analysis | |
| CN107544675A (en) | Brain control formula virtual reality method | |
| CN113593671B (en) | Automatic adjustment method and device of virtual rehabilitation game based on Leap Motion gesture recognition | |
| Liu et al. | Multimodal face aging framework via learning disentangled representation | |
| CN109902824A (en) | It is a kind of to generate confrontation network method with self adaptive control learning improvement | |
| CN113357773A (en) | Method and device for air conditioner control and air conditioner | |
| Neff et al. | Animation of natural virtual characters | |
| Li et al. | Automatic generation of 3D natural anime-like non-player characters with machine learning | |
| Streck et al. | Neomento-Towards building a universal solution for virtual reality exposure psychotherapy | |
| CN106940592B (en) | A kind of brain control visual human avoidance obstacle method | |
| Jalali et al. | Model based optimization of the cardiopulmonary resuscitation (CPR) procedure | |
| Vallance | Independently Supporting Learners in VR with an AI-enabled Non-Player Character (NPC) | |
| CN106156842A (en) | A kind of integration provides the method for optimally controlling of neuron models |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |