CN117711611A - MDT remote consultation system and method based on scene fusion and mr - Google Patents
MDT remote consultation system and method based on scene fusion and mr Download PDFInfo
- Publication number
- CN117711611A CN117711611A CN202410163784.3A CN202410163784A CN117711611A CN 117711611 A CN117711611 A CN 117711611A CN 202410163784 A CN202410163784 A CN 202410163784A CN 117711611 A CN117711611 A CN 117711611A
- Authority
- CN
- China
- Prior art keywords
- information
- data
- patient
- dimensional
- medical history
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 230000004927 fusion Effects 0.000 title claims abstract description 38
- 238000000034 method Methods 0.000 title claims abstract description 32
- 238000003745 diagnosis Methods 0.000 claims abstract description 17
- 238000005516 engineering process Methods 0.000 claims abstract description 12
- 230000002452 interceptive effect Effects 0.000 claims description 12
- 201000010099 disease Diseases 0.000 claims description 8
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 claims description 8
- 230000005540 biological transmission Effects 0.000 claims description 7
- 210000003484 anatomy Anatomy 0.000 claims description 6
- 210000003205 muscle Anatomy 0.000 claims description 6
- 210000000056 organ Anatomy 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000004458 analytical method Methods 0.000 claims description 4
- 230000003902 lesion Effects 0.000 claims description 4
- QVGXLLKOCUKJST-UHFFFAOYSA-N atomic oxygen Chemical compound [O] QVGXLLKOCUKJST-UHFFFAOYSA-N 0.000 claims description 3
- 239000008280 blood Substances 0.000 claims description 3
- 210000004369 blood Anatomy 0.000 claims description 3
- 230000036772 blood pressure Effects 0.000 claims description 3
- 230000036760 body temperature Effects 0.000 claims description 3
- 238000004364 calculation method Methods 0.000 claims description 3
- 238000010276 construction Methods 0.000 claims description 3
- 238000013135 deep learning Methods 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 238000009499 grossing Methods 0.000 claims description 3
- TVMXDCGIABBOFY-UHFFFAOYSA-N octane Chemical compound CCCCCCCC TVMXDCGIABBOFY-UHFFFAOYSA-N 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 229910052760 oxygen Inorganic materials 0.000 claims description 3
- 239000001301 oxygen Substances 0.000 claims description 3
- 230000036387 respiratory rate Effects 0.000 claims description 3
- 238000000547 structure data Methods 0.000 claims description 3
- 208000028399 Critical Illness Diseases 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 2
- 238000010295 mobile communication Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000000265 homogenisation Methods 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000000877 morphologic effect Effects 0.000 description 1
- 230000001737 promoting effect Effects 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Abstract
The invention belongs to the technical field of remote consultation, and provides an MDT remote consultation system and method based on scene fusion and mr, comprising the following steps: the method comprises the steps of collecting data information of a patient, calculating error degree between three-dimensional data and medical history data information of the patient, judging whether the three-dimensional model is subjected to refined reconstruction, obtaining the three-dimensional data of the patient again, fusing with technologies such as virtual reality and the like, constructing a remote consultation environment based on scene fusion, transmitting the data information of the patient and the information such as diagnosis results of doctors to a remote consultation center, and transmitting the final consultation results to a system terminal of a hospital, wherein the hospital treats the patient according to the consultation results.
Description
Technical Field
The invention belongs to the technical field of remote consultation, and particularly relates to an MDT remote consultation system and method based on scene fusion and mr.
Background
Through remote consultation, medical resources in different areas can be integrated and shared, and the utilization efficiency of the medical resources is improved, so that the remote consultation has important significance for relieving medical resource shortage and improving medical service quality, can break regional limitation, enables doctors in all areas to mutually communicate and learn, promotes popularization and popularization of medical technology, and has important significance for improving medical level and promoting medical progress.
One chinese patent application publication No. CN113284597a discloses a critical and critical MDT system and method based on scene fusion and MR, comprising: the system comprises a lead hospital system, a plurality of primary hospital systems and a plurality of emergency ambulance systems, wherein the lead hospital system is respectively connected with the primary hospital system and the emergency ambulance systems through networks; the medical holographic scene fusion terminal, the management end equipment, the three-dimensional holographic environment server and the application server are organically connected to form the emergency critical illness MDT collaborative treatment system capable of fully utilizing the holographic scene fusion and mixed reality technology, so that the series of problems existing in other existing schemes or products are solved, accurate, visual, complete and unambiguous natural interaction can be provided for multi-party MDT collaborative treatment based on multi-party fusion and unified medical holographic scenes, the homogenization of the emergency medical level and medical quality of a primary hospital and the emergency medical treatment of the primary hospital is realized, and the timeliness and effectiveness of emergency critical illness treatment are effectively improved.
In the prior art, the technical problems of remote consultation and the cooperative treatment of the critical and critical MDT based on scene fusion and MR are solved, but the two methods are not effectively combined, the remote consultation can be realized at the same time of the cooperative treatment of the critical and critical MDT based on scene fusion and MR, and in the prior art, the three-dimensional model is not finely reconstructed according to the deviation degree between the three-dimensional data of the human body of a patient and the actual data, so that the diagnosis and the treatment of the patient are more accurate.
Therefore, the invention provides an MDT remote consultation system and method based on scene fusion and mr.
Disclosure of Invention
In order to overcome the deficiencies of the prior art, at least one technical problem presented in the background art is solved.
The technical scheme adopted for solving the technical problems is as follows: an MDT remote consultation method based on scene fusion and mr comprises the following steps:
step one: collecting three-dimensional data of a patient through a three-dimensional model;
step two: collecting physiological data and medical history information of a patient;
step three: comparing and analyzing the three-dimensional data of the acquired patient with the medical history information to obtain an error degree representation value between the three-dimensional data and the medical history information, and judging whether the three-dimensional model needs to be reconstructed finely;
step four: processing the three-dimensional model which needs to be finely reconstructed, judging whether the finely reconstructed model accords with the standard, and repeatedly finely reconstructing the finely reconstructed model which does not accord with the standard;
step five: based on a three-dimensional model which does not need to be reconstructed or a three-dimensional model which meets the standard after the fine reconstruction, three-dimensional data of a patient is obtained again, and is fused with technologies such as virtual reality and the like, so as to construct a remote consultation environment based on scene fusion;
step six: transmitting three-dimensional data, physiological data, medical history information, diagnosis results of doctors and other relevant information of the patients to a remote consultation center;
step seven: and the multi-disciplinary doctor carries out academic exchange discussion on the illness state of the patient through the interactive learning module at the remote consultation center, and transmits the final consultation result to the system terminal of the hospital.
The invention further adopts the technical scheme that: the three-dimensional data includes: morphology data, structure data, and appearance data;
the physiological data includes: body temperature, heart rate, blood pressure, respiratory rate, blood oxygen saturation of the patient;
the medical history information includes: the information of the height, weight, three-dimensional size, length, width and thickness of each part of the body, the skeletal structure of the human body, the distribution of muscles, the position of organs, the skin color, texture and hair of the human body, etc. are recorded in the past.
The invention further adopts the technical scheme that: the morphology data includes: information such as height, weight, three-dimensional size, anatomical structure, focus position and the like of the patient, and information such as length, width, thickness and the like of each part of the body;
the structural data includes: information such as skeletal structure, muscle distribution, organ position and the like of a human body;
the appearance data includes: skin color, texture, hair, etc. of the human body.
The invention further adopts the technical scheme that: comparing and analyzing the three-dimensional data of the acquired patient with the medical history information to obtain an error degree representation value between the three-dimensional data and the medical history information;
marking the obtained information quantity error rate as ai and the obtained information numerical error rate as di;
the obtained information quantity error rate ai and the information numerical error rate di are quantized, and the values are substituted into a formula:
and obtaining an error degree representation value ei between the three-dimensional data and the medical history data information, wherein alpha is a preset natural constant, and alpha is more than 0.
The invention further adopts the technical scheme that: the process of obtaining the information quantity error rate ai is:
comparing each item of information contained in the three-dimensional data with each item of information of a patient recorded in the medical history information, counting the number of difference information existing after comparison, and carrying out ratio processing on the number of the difference information and the total number of the compared information to obtain an information number error rate ai.
The invention further adopts the technical scheme that: the process of obtaining the information numerical error rate di comprises:
marking the difference information in an X-Y axis coordinate system, marking the name of the difference information on an X axis, and correspondingly marking the specific numerical value of the difference information and the specific numerical value of the standard information above the X axis;
performing linear connection between specific numerical value marking points of all the difference information to form a difference information folding line, performing linear connection on specific numerical value marks of all the standard information to form a standard information folding line, calculating the area enclosed between the difference information folding line and the standard information folding line, and marking the area as bi;
selecting a standard information folding line or a difference information folding line, respectively making vertical lines to the X axis at two ends of the folding line, measuring the horizontal length of the folding line through the X axis, and marking the horizontal length as ci;
the area bi enclosed by the obtained difference information folding lines and the standard information folding lines and the horizontal length ci of the folding lines are quantized, and the values are substituted into a formula:
an information value error rate di is obtained, which, among other things,s 1 、s 2 is a preset proportion parameters 1 Ands 2 are all greater than 0.
The invention further adopts the technical scheme that: judging whether the three-dimensional model needs to be reconstructed in a refined mode, and presetting a threshold value of an error degree representation value between the three-dimensional data and medical history data information as fi;
if the error degree representation value ei between the three-dimensional data and the medical history data information is smaller than fi, the three-dimensional model is not required to be reconstructed in a refined mode;
if the error degree representation value ei between the three-dimensional data and the medical history data information is more than or equal to fi, the three-dimensional model is required to be reconstructed in a refined mode.
The invention further adopts the technical scheme that: the three-dimensional model to be finely reconstructed is processed, and the processing steps comprise:
s201, data preprocessing: preprocessing the acquired three-dimensional data, namely denoising the data by adopting an Octane renderer, smoothing and registering the data by utilizing a PSTPS registration method, and the like;
s202, feature extraction: extracting characteristics related to the disease state of the patient from the preprocessed three-dimensional data, wherein the characteristics related to the disease state of the patient comprise: anatomical structure, lesion location, etc.;
s203, model reconstruction: according to the extracted features, carrying out fine reconstruction on a three-dimensional model of a patient by utilizing a three-dimensional reconstruction algorithm or a deep learning algorithm;
s204, model optimization: and optimizing the reconstructed model, including adjusting the proportion, shape, texture and the like of the model.
The invention further adopts the technical scheme that: judging whether the model after the fine reconstruction accords with the standard, acquiring three-dimensional data of a patient again based on the model after the fine reconstruction, calculating the error degree between the three-dimensional data and the medical history data information again, judging whether the model after the fine reconstruction needs to be subjected to fine reconstruction again, if not, indicating that the model after the fine reconstruction accords with the standard, if so, indicating that the model after the fine reconstruction does not accord with the standard, and then carrying out fine reconstruction again until the model accords with the standard.
An MDT remote consultation system based on scene fusion and mr, comprising:
the acquisition module is used for acquiring three-dimensional data of a patient according to the three-dimensional model, and is also used for acquiring physiological data and medical history information of the patient;
the comparison module is used for comparing and analyzing the acquired three-dimensional data of the patient with the medical history data information;
the calculation module is used for calculating the error degree between the three-dimensional data and the medical history data information according to the comparison analysis result of the comparison module;
the judging module is used for judging whether the three-dimensional model is subjected to fine reconstruction according to the error degree between the three-dimensional data and the medical history data information;
the reconstruction module is used for carrying out fine reconstruction processing on the three-dimensional model according to the judging result of the judging module;
the construction module is used for a remote consultation environment based on scene fusion;
the transmission module is used for transmitting three-dimensional data, physiological data, medical history information, diagnosis results of doctors and other relevant information of the patients to the remote consultation center;
the interactive learning module is used for interactive learning among multiple disciplines;
the system terminal is used for receiving the final consultation result;
the remote consultation center is used for receiving the three-dimensional data, the physiological data, the medical history information and the diagnosis results of doctors of the patient transmitted by the transmission module.
The beneficial effects of the invention are as follows:
1. according to the MDT remote consultation system and method based on scene fusion and mr, three-dimensional data of a patient are acquired through the three-dimensional model, the acquired three-dimensional data of the patient and medical history information are compared and analyzed, the error degree between the three-dimensional data and the medical history information is obtained, whether the three-dimensional model is subjected to fine reconstruction or not is judged, the three-dimensional data of the patient is obtained again based on the three-dimensional model which does not need to be reconstructed or the three-dimensional model which accords with the standard after fine reconstruction, and the three-dimensional data are fused with technologies such as virtual reality, a scene fusion remote consultation environment is constructed, the three-dimensional data, physiological data, the medical history information of the patient, the diagnosis result of a doctor and other related information of the patient are transmitted to a remote consultation center, a multidisciplinary doctor carries out academic exchange discussion on the patient at the remote consultation center, and a final consultation result is transmitted to a system terminal of a hospital, and the hospital carries out treatment on the patient according to the consultation result.
2. According to the MDT remote consultation system and method based on scene fusion and mr, whether the three-dimensional model is subjected to fine reconstruction is judged according to the error degree between the acquired three-dimensional data and medical history data information of the patient, then fine reconstruction processing is carried out on the three-dimensional model which needs fine reconstruction, whether the model after fine reconstruction meets the standard is judged, repeated fine reconstruction is carried out on the model after fine reconstruction which does not meet the standard, accuracy of information such as patient condition data is improved, and diagnosis of the patient condition can be more accurate and treatment of the patient is more targeted.
Drawings
The invention is further described below with reference to the accompanying drawings.
FIG. 1 is a flow chart of a method according to a first embodiment of the invention;
FIG. 2 is a flowchart of a method for determining whether a three-dimensional model needs fine reconstruction in accordance with an embodiment of the present invention;
fig. 3 is a system block diagram of a second embodiment of the present invention.
Detailed Description
The invention is further described in connection with the following detailed description in order to make the technical means, the creation characteristics, the achievement of the purpose and the effect of the invention easy to understand.
Example 1
As shown in fig. 1, an MDT remote consultation method based on scene fusion and mr according to an embodiment of the present invention includes:
step one: scanning a patient by using mr equipment to generate a three-dimensional model of the patient, and acquiring three-dimensional data of the patient through the three-dimensional model;
specifically, the three-dimensional data includes: morphology data, structure data, and appearance data;
more specifically, the morphological data in the three-dimensional data includes: information such as height, weight, three-dimensional size, anatomical structure, focus position and the like of the patient, and information such as length, width, thickness and the like of each part of the body;
the structural data includes: information such as skeletal structure, muscle distribution, organ position and the like of a human body;
the appearance data includes: information such as skin color, texture, hair and the like of a human body;
step two: collecting physiological data and medical history information of a patient;
wherein the physiological data comprises: body temperature, heart rate, blood pressure, respiratory rate, blood oxygen saturation of the patient;
the medical history information includes: the information of the height, weight, three-dimensional size, length, width and thickness of each part of the body, the skeletal structure, muscle distribution, organ position, skin color, texture and hair of the human body, etc. of the patient recorded in the past;
step three: comparing and analyzing the three-dimensional data of the acquired patient with the medical history information to obtain an error degree representation value between the three-dimensional data and the medical history information, and judging whether the three-dimensional model needs to be reconstructed finely or not based on the error degree representation value between the three-dimensional data and the medical history information;
the specific comparison and analysis steps comprise:
s101, comparing each item of information contained in the three-dimensional data with each item of information of a patient recorded in medical history information, counting the number of difference information existing after the comparison, carrying out ratio processing on the number of difference information and the total number of compared information to obtain an information number error rate, and marking the information number error rate as ai, for example, correspondingly comparing the height, weight, three-dimensional size and length, width and thickness of each part of a body of the patient contained in the three-dimensional data with the height, weight, three-dimensional size and length, width and thickness of each part of the body of the patient recorded in the medical history information, taking each item of information in the medical history information as standard information, and marking the specific value of the information in the three-dimensional data as difference information if the specific value of the information in the three-dimensional data is different from the specific value of the standard information in the medical history information;
it should be noted that, the difference information is only selected from the information contained in the three-dimensional data, and the total amount of the comparison information is the information amount commonly compared in the three-dimensional data and the medical history information;
s102, marking the difference information counted in the step S101 in an X-Y axis coordinate system, marking the name of the difference information on an X axis, and correspondingly marking the specific numerical value of the difference information and the specific numerical value of the standard information above the X axis;
it should be noted that, the difference information is the same as the standard information in name but different in specific numerical value;
performing linear connection between specific numerical value marking points of all the difference information to form a difference information folding line, performing linear connection on specific numerical value marks of all the standard information to form a standard information folding line, calculating the area enclosed between the difference information folding line and the standard information folding line, and marking the area as bi;
selecting a standard information folding line or a difference information folding line, respectively making vertical lines to the X axis at two ends of the folding line, measuring the horizontal length of the folding line through the X axis, and marking the horizontal length as ci;
in the X-Y axis, unit distances between the mark points of the information names are the same;
the area bi enclosed by the obtained difference information folding lines and the standard information folding lines and the horizontal length ci of the folding lines are quantized, and the values are substituted into a formula:
an information value error rate di is obtained, which, among other things,s 1 、s 2 is a preset proportion parameters 1 Ands 2 are all greater than 0;
s103, performing quantization processing on the information quantity error rate ai and the information value error rate di obtained in step S101 and step S102, and substituting the values into a formula:
obtaining an error degree representation value ei between three-dimensional data and medical history data information, wherein alpha is a preset natural constant, and alpha is more than 0;
as shown in fig. 2, a determination is made as to whether the three-dimensional model needs to be reconstructed in detail, and the specific determination process includes:
presetting a threshold value of an error degree representation value between three-dimensional data and medical history data information as fi;
if the error degree representation value ei between the three-dimensional data and the medical history data information is smaller than fi, the three-dimensional model is not required to be reconstructed in a refined mode;
if the error degree representation value ei between the three-dimensional data and the medical history data information is more than or equal to fi, the three-dimensional model is required to be reconstructed in a refined mode;
step four: carrying out fine reconstruction processing on a three-dimensional model which needs fine reconstruction, judging whether the fine reconstructed model accords with a standard or not, and carrying out repeated fine reconstruction on the fine reconstructed model which does not accord with the standard, wherein the fine reconstruction processing steps comprise:
s201, data preprocessing: preprocessing the acquired three-dimensional data, namely denoising the data by adopting an Octane renderer, smoothing and registering the data by utilizing a PSTPS registration method, and the like, so that the accuracy and the stability of the data are improved;
s202, feature extraction: extracting characteristics related to the disease state of the patient from the preprocessed three-dimensional data, wherein the characteristics related to the disease state of the patient comprise: anatomical structure, lesion location, etc.;
s203, model reconstruction: according to the extracted features, carrying out fine reconstruction on a three-dimensional model of a patient by utilizing a three-dimensional reconstruction algorithm or a deep learning algorithm;
s204, model optimization: optimizing the reconstructed model, including adjusting the proportion, shape, texture and the like of the model to make the model more vivid and accurate;
judging whether the model after the fine reconstruction accords with the standard, acquiring three-dimensional data of a patient based on the model after the fine reconstruction again, calculating the error degree between the three-dimensional data and the medical history data information again, judging whether the model after the fine reconstruction needs to be subjected to fine reconstruction again, if not, indicating that the model after the fine reconstruction accords with the standard, if so, indicating that the model after the fine reconstruction does not accord with the standard, and carrying out fine reconstruction again until the model accords with the standard;
step five: the three-dimensional data of a patient is obtained again based on a three-dimensional model which does not need to be reconstructed or a three-dimensional model which accords with the standard after being reconstructed in a refined way, and is fused with a Virtual Reality (VR) technology and an Augmented Reality (AR) technology to construct a remote consultation environment based on scene fusion;
it should be noted that in this environment, a doctor can view and analyze a three-dimensional model of a patient through an MR apparatus, and can simultaneously make in-depth knowledge and diagnosis of the model through VR/AR technology;
step six: transmitting three-dimensional data, physiological data, medical history information, diagnosis results of doctors and other relevant information of the patients to a remote consultation center;
it should be noted that, the data transmission may be implemented through the internet and a mobile communication network;
step seven: and the multi-disciplinary doctor carries out academic exchange discussion on the illness state of the patient through the interactive learning module at the remote consultation center, and transmits a final consultation result to a system terminal of the hospital, and the hospital treats the patient according to the consultation result.
It should be noted that, the interactive learning module may implement interactive learning among multiple doctors through video conference or online conference using mobile communication device;
the invention firstly scans a patient by MR equipment to generate a three-dimensional model of the patient, acquires three-dimensional data of the patient through the three-dimensional model, acquires physiological data and medical history information of the patient, compares and analyzes the acquired three-dimensional data and the medical history information of the patient to obtain the error degree between the three-dimensional data and the medical history information, judges whether to carry out fine reconstruction on the three-dimensional model based on the error degree between the acquired three-dimensional data and the medical history information of the patient, carries out fine reconstruction processing on the three-dimensional model which needs fine reconstruction, judges whether the model which meets the standard after fine reconstruction, repeatedly carries out fine reconstruction on the model which does not meet the standard, and obtains the three-dimensional data of the patient again based on the three-dimensional model which does not need reconstruction or the three-dimensional model which meets the standard after fine reconstruction, and fusing with virtual reality VR technology and augmented reality AR technology to construct a scene fusion-based remote consultation environment in which a doctor can view and analyze a three-dimensional model of a patient through MR equipment, and simultaneously can deeply understand and diagnose the model through VR/AR technology, finally, three-dimensional data, physiological data, medical history information of the patient, diagnosis results of the doctor and other relevant information of the doctor are transmitted to a remote consultation center, a multidisciplinary doctor performs academic communication discussion on the condition of the patient at the remote consultation center through an interactive learning module, and transmits the final consultation result to a system terminal of a hospital, the hospital treats the patient according to the consultation result, the invention realizes scene fusion and MDT remote consultation of the patient, on one hand, the virtual technology and the multidisciplinary consultation are fused, and doctors can diagnose and treat in the virtual environment, so that the actual contact and operation of patients are reduced, the diagnosis efficiency is improved, the dependence on medical equipment and sites can be reduced by the virtual technology, meanwhile, the time and cost of repeated consultation and consultation of the patients can be reduced by the multidisciplinary consultation, the medical cost is comprehensively reduced as a whole, and on the other hand, the accuracy of information such as patient disease data is improved through reconstruction of a three-dimensional model of the patients, and the disease diagnosis of the patients is more accurate and the treatment of the patients is more targeted.
Example 2
As shown in fig. 3, an MDT remote consultation system based on scene fusion and mr according to an embodiment of the present invention includes:
the acquisition module is used for acquiring three-dimensional data of a patient according to the three-dimensional model, and is also used for acquiring physiological data and medical history information of the patient;
the comparison module is used for comparing and analyzing the acquired three-dimensional data of the patient with the medical history data information;
the calculation module is used for calculating the error degree between the three-dimensional data and the medical history data information according to the comparison analysis result of the comparison module;
the judging module is used for judging whether the three-dimensional model is subjected to fine reconstruction according to the error degree between the three-dimensional data and the medical history data information;
the reconstruction module is used for carrying out fine reconstruction processing on the three-dimensional model according to the judging result of the judging module;
the construction module is used for a remote consultation environment based on scene fusion;
the transmission module is used for transmitting three-dimensional data, physiological data, medical history information, diagnosis results of doctors and other relevant information of the patients to the remote consultation center;
the interactive learning module is used for interactive learning among multiple disciplines;
the system terminal is used for receiving the final consultation result;
the remote consultation center is used for receiving the three-dimensional data, the physiological data, the medical history information and the diagnosis results of doctors of the patient transmitted by the transmission module.
The foregoing has shown and described the basic principles, principal features and advantages of the invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, and that the above embodiments and descriptions are merely illustrative of the principles of the present invention, and various changes and modifications may be made without departing from the spirit and scope of the invention, which is defined in the appended claims. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (10)
1. An MDT remote consultation method based on scene fusion and mr is characterized by comprising the following steps of: comprising the following steps:
step one: collecting three-dimensional data of a patient through a three-dimensional model;
step two: collecting physiological data and medical history information of a patient;
step three: comparing and analyzing the three-dimensional data of the acquired patient with the medical history information to obtain an error degree representation value between the three-dimensional data and the medical history information, and judging whether the three-dimensional model needs to be reconstructed finely;
step four: processing the three-dimensional model which needs to be finely reconstructed, judging whether the finely reconstructed model accords with the standard, and repeatedly finely reconstructing the finely reconstructed model which does not accord with the standard;
step five: based on a three-dimensional model which does not need to be reconstructed or a three-dimensional model which meets the standard after the fine reconstruction, three-dimensional data of a patient is obtained again, and is fused with a virtual reality technology to construct a remote consultation environment based on scene fusion;
step six: transmitting three-dimensional data, physiological data, medical history information, diagnosis results of doctors and other relevant information of the patients to a remote consultation center;
step seven: and the multi-disciplinary doctor carries out academic exchange discussion on the illness state of the patient through the interactive learning module at the remote consultation center, and transmits the final consultation result to the system terminal of the hospital.
2. The MDT remote consultation method based on scene fusion and mr according to claim 1, wherein: the three-dimensional data includes: morphology data, structure data, and appearance data;
the physiological data includes: body temperature, heart rate, blood pressure, respiratory rate, blood oxygen saturation of the patient;
the medical history information includes: the height, weight, three-dimensional size and thickness of each part of the body, skeleton structure, muscle distribution, organ position, skin color, texture and hair information of the human body are recorded in the past.
3. The MDT remote consultation method based on scene fusion and mr according to claim 2, wherein: the morphology data includes: patient height, weight, three-dimensional size, anatomy, lesion location information, length, width, and thickness information of various parts of the body;
the structural data includes: skeletal structure, muscle distribution, and organ position information of the human body;
the appearance data includes: skin color, texture, hair information of the human body.
4. The MDT remote consultation method based on scene fusion and mr according to claim 1, wherein: comparing and analyzing the three-dimensional data of the acquired patient with the medical history information to obtain an error degree representation value between the three-dimensional data and the medical history information;
marking the obtained information quantity error rate as ai and the obtained information numerical error rate as di;
the obtained information quantity error rate ai and the information numerical error rate di are quantized, and the values are substituted into a formula:
；
and obtaining an error degree representation value ei between the three-dimensional data and the medical history data information, wherein alpha is a preset natural constant, and alpha is more than 0.
5. The MDT remote consultation method based on scene fusion and mr according to claim 4, wherein: the process of obtaining the information quantity error rate ai is as follows:
comparing each item of information contained in the three-dimensional data with each item of information of a patient recorded in the medical history information, counting the number of difference information existing after comparison, and carrying out ratio processing on the number of the difference information and the total number of the compared information to obtain an information number error rate ai.
6. The MDT remote consultation method based on scene fusion and mr according to claim 4, wherein: the process of obtaining the information numerical error rate di comprises the following steps:
marking the difference information in an X-Y axis coordinate system, marking the name of the difference information on an X axis, and correspondingly marking the specific numerical value of the difference information and the specific numerical value of the standard information above the X axis;
performing linear connection between specific numerical value marking points of all the difference information to form a difference information folding line, performing linear connection on specific numerical value marks of all the standard information to form a standard information folding line, calculating the area enclosed between the difference information folding line and the standard information folding line, and marking the area as bi;
selecting a standard information folding line or a difference information folding line, respectively making vertical lines to the X axis at two ends of the folding line, measuring the horizontal length of the folding line through the X axis, and marking the horizontal length as ci;
the area bi enclosed by the obtained difference information folding lines and the standard information folding lines and the horizontal length ci of the folding lines are quantized, and the values are substituted into a formula:
；
an information value error rate di is obtained, which, among other things,s 1 、s 2 is a preset proportion parameters 1 Ands 2 are all greater than 0.
7. The MDT remote consultation method based on scene fusion and mr according to claim 1, wherein: judging whether the three-dimensional model needs to be reconstructed in a refined mode, and presetting a threshold value of an error degree representation value between three-dimensional data and medical history data information as fi;
if the error degree representation value ei between the three-dimensional data and the medical history data information is smaller than fi, the three-dimensional model is not required to be reconstructed in a refined mode;
if the error degree representation value ei between the three-dimensional data and the medical history data information is more than or equal to fi, the three-dimensional model is required to be reconstructed in a refined mode.
8. The MDT remote consultation method based on scene fusion and mr according to claim 1, wherein: the three-dimensional model to be finely reconstructed is processed, and the processing steps comprise:
s201, data preprocessing: preprocessing the acquired three-dimensional data, namely denoising the data by adopting an Octane renderer, and smoothing and registering the data by utilizing a PSTPS registration method;
s202, feature extraction: extracting characteristics related to the disease state of the patient from the preprocessed three-dimensional data, wherein the characteristics related to the disease state of the patient comprise: anatomical structure, lesion location;
s203, model reconstruction: according to the extracted features, carrying out fine reconstruction on a three-dimensional model of a patient by utilizing a three-dimensional reconstruction algorithm or a deep learning algorithm;
s204, model optimization: and optimizing the reconstructed model, including adjusting the proportion, shape and texture of the model.
9. The MDT remote consultation method based on scene fusion and mr according to claim 1, wherein: judging whether the model after the fine reconstruction accords with the standard, acquiring three-dimensional data of a patient again based on the model after the fine reconstruction, calculating the error degree between the three-dimensional data and the information of the medical history data again, judging whether the model after the fine reconstruction needs to be subjected to fine reconstruction again, if not, indicating that the model after the fine reconstruction accords with the standard, if so, indicating that the model after the fine reconstruction does not accord with the standard, and then carrying out fine reconstruction again until the model accords with the standard.
10. An MDT remote consultation system based on scene fusion and mr, which implements an MDT remote consultation method based on scene fusion and mr according to any one of claims 1 to 9, and is characterized in that: the system comprises:
the acquisition module is used for acquiring three-dimensional data of a patient according to the three-dimensional model, and is also used for acquiring physiological data and medical history information of the patient;
the comparison module is used for comparing and analyzing the acquired three-dimensional data of the patient with the medical history data information;
the calculation module is used for calculating the error degree between the three-dimensional data and the medical history data information according to the comparison analysis result of the comparison module;
the judging module is used for judging whether the three-dimensional model is subjected to fine reconstruction according to the error degree between the three-dimensional data and the medical history data information;
the reconstruction module is used for carrying out refined reconstruction processing on the three-dimensional model according to the judging result of the judging module;
the construction module is used for a remote consultation environment based on scene fusion;
the transmission module is used for transmitting three-dimensional data, physiological data, medical history information, diagnosis results of doctors and other relevant information of the patients to the remote consultation center;
the interactive learning module is used for interactive learning among multiple disciplines;
the system terminal is used for receiving a final consultation result;
the remote consultation center is used for receiving the three-dimensional data, the physiological data, the medical history information and the diagnosis results of doctors of the patient transmitted by the transmission module.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410163784.3A CN117711611B (en) | 2024-02-05 | 2024-02-05 | MDT remote consultation system and method based on scene fusion and mr |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410163784.3A CN117711611B (en) | 2024-02-05 | 2024-02-05 | MDT remote consultation system and method based on scene fusion and mr |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117711611A true CN117711611A (en) | 2024-03-15 |
| CN117711611B CN117711611B (en) | 2024-04-19 |
Family
ID=90153821
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410163784.3A Active CN117711611B (en) | 2024-02-05 | 2024-02-05 | MDT remote consultation system and method based on scene fusion and mr |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117711611B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117711610A (en) * | 2024-02-05 | 2024-03-15 | 四川省医学科学院·四川省人民医院 | Remote consultation vehicle system of networked MDT multidisciplinary |
Citations (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100280350A1 (en) * | 2009-05-02 | 2010-11-04 | Xinyu Zhang | Chinese medicine tele-diagnostics and triage system |
| CN103961178A (en) * | 2013-01-30 | 2014-08-06 | 深圳先进技术研究院 | Remote surgical planning and navigating system |
| CN108742519A (en) * | 2018-04-02 | 2018-11-06 | 上海中医药大学附属岳阳中西医结合医院 | Machine vision three-dimensional reconstruction technique skin ulcer surface of a wound intelligent auxiliary diagnosis system |
| KR20190092699A (en) * | 2018-01-31 | 2019-08-08 | 주식회사 지르코리아 | Simulation system and method for dental patient consultation |
| CN110706825A (en) * | 2019-10-24 | 2020-01-17 | 陈�峰 | Orthopedic medical platform system and method based on three-dimensional modeling and 3D printing |
| CN111163099A (en) * | 2019-12-31 | 2020-05-15 | 重庆亚德科技股份有限公司 | Medical remote consultation system based on 5G |
| CN111603136A (en) * | 2020-05-11 | 2020-09-01 | 马文足医疗科技(北京)有限公司 | Remote consultation system for foot diagnosis and treatment |
| WO2020259712A1 (en) * | 2019-06-27 | 2020-12-30 | 埃客(上海)医用设备有限公司 | Smart orthopaedic external fixation system based on cloud platform |
| CN112365587A (en) * | 2020-12-01 | 2021-02-12 | 成都中创五联科技有限公司 | System and method for multi-mode three-dimensional modeling of tomographic image suitable for auxiliary diagnosis and treatment |
| CN112906205A (en) * | 2021-01-29 | 2021-06-04 | 广东工业大学 | Virtual learning method for total hip replacement surgery |
| KR20210100248A (en) * | 2020-02-05 | 2021-08-17 | 오스템임플란트 주식회사 | Method for establishing diagnosis and treatment plan and 3D dental image providing device therefor |
| CN113284597A (en) * | 2021-07-23 | 2021-08-20 | 杭州乾瑾科技有限公司 | Critical MDT system and method based on scene fusion and MR |
| CN113808762A (en) * | 2021-09-30 | 2021-12-17 | 贾婉艳 | Internet hospital remote consultation system |
| WO2022021309A1 (en) * | 2020-07-31 | 2022-02-03 | 深圳市大疆创新科技有限公司 | Method and apparatus for establishing model, electronic device, and computer readable storage medium |
| CN114708197A (en) * | 2022-03-15 | 2022-07-05 | 北京泽桥医疗科技股份有限公司 | Human body three-dimensional animation grouping label display method based on mobile terminal |
| WO2022165112A1 (en) * | 2021-01-29 | 2022-08-04 | Covidien Lp | Systems and methods for c-arm fluoroscope camera pose refinement with secondary movement compensation |
| US20220240886A1 (en) * | 2021-01-29 | 2022-08-04 | Covidien Lp | Systems and methods for c-arm fluoroscope camera pose refinement with secondary movement compensation |
| CN115527645A (en) * | 2022-09-29 | 2022-12-27 | 浙江大学医学院附属第一医院 | Clinical simulation virtual patient system for teaching and examination |
| CN116313028A (en) * | 2022-09-06 | 2023-06-23 | 上海联影智能医疗科技有限公司 | Medical assistance device, method, and computer-readable storage medium |
| WO2023240235A1 (en) * | 2022-06-10 | 2023-12-14 | Icahn School Of Medicine At Mount Sinai | Use of three-dimensional models of specimens and defects to enhance communication and documentation of frozen and permanent sections in cancer surgery |
-
2024
- 2024-02-05 CN CN202410163784.3A patent/CN117711611B/en active Active
Patent Citations (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20100280350A1 (en) * | 2009-05-02 | 2010-11-04 | Xinyu Zhang | Chinese medicine tele-diagnostics and triage system |
| CN103961178A (en) * | 2013-01-30 | 2014-08-06 | 深圳先进技术研究院 | Remote surgical planning and navigating system |
| KR20190092699A (en) * | 2018-01-31 | 2019-08-08 | 주식회사 지르코리아 | Simulation system and method for dental patient consultation |
| CN108742519A (en) * | 2018-04-02 | 2018-11-06 | 上海中医药大学附属岳阳中西医结合医院 | Machine vision three-dimensional reconstruction technique skin ulcer surface of a wound intelligent auxiliary diagnosis system |
| WO2020259712A1 (en) * | 2019-06-27 | 2020-12-30 | 埃客(上海)医用设备有限公司 | Smart orthopaedic external fixation system based on cloud platform |
| CN110706825A (en) * | 2019-10-24 | 2020-01-17 | 陈�峰 | Orthopedic medical platform system and method based on three-dimensional modeling and 3D printing |
| CN111163099A (en) * | 2019-12-31 | 2020-05-15 | 重庆亚德科技股份有限公司 | Medical remote consultation system based on 5G |
| KR20210100248A (en) * | 2020-02-05 | 2021-08-17 | 오스템임플란트 주식회사 | Method for establishing diagnosis and treatment plan and 3D dental image providing device therefor |
| CN111603136A (en) * | 2020-05-11 | 2020-09-01 | 马文足医疗科技(北京)有限公司 | Remote consultation system for foot diagnosis and treatment |
| WO2022021309A1 (en) * | 2020-07-31 | 2022-02-03 | 深圳市大疆创新科技有限公司 | Method and apparatus for establishing model, electronic device, and computer readable storage medium |
| CN112365587A (en) * | 2020-12-01 | 2021-02-12 | 成都中创五联科技有限公司 | System and method for multi-mode three-dimensional modeling of tomographic image suitable for auxiliary diagnosis and treatment |
| CN112906205A (en) * | 2021-01-29 | 2021-06-04 | 广东工业大学 | Virtual learning method for total hip replacement surgery |
| WO2022165112A1 (en) * | 2021-01-29 | 2022-08-04 | Covidien Lp | Systems and methods for c-arm fluoroscope camera pose refinement with secondary movement compensation |
| US20220240886A1 (en) * | 2021-01-29 | 2022-08-04 | Covidien Lp | Systems and methods for c-arm fluoroscope camera pose refinement with secondary movement compensation |
| CN113284597A (en) * | 2021-07-23 | 2021-08-20 | 杭州乾瑾科技有限公司 | Critical MDT system and method based on scene fusion and MR |
| CN113808762A (en) * | 2021-09-30 | 2021-12-17 | 贾婉艳 | Internet hospital remote consultation system |
| CN114708197A (en) * | 2022-03-15 | 2022-07-05 | 北京泽桥医疗科技股份有限公司 | Human body three-dimensional animation grouping label display method based on mobile terminal |
| WO2023240235A1 (en) * | 2022-06-10 | 2023-12-14 | Icahn School Of Medicine At Mount Sinai | Use of three-dimensional models of specimens and defects to enhance communication and documentation of frozen and permanent sections in cancer surgery |
| CN116313028A (en) * | 2022-09-06 | 2023-06-23 | 上海联影智能医疗科技有限公司 | Medical assistance device, method, and computer-readable storage medium |
| CN115527645A (en) * | 2022-09-29 | 2022-12-27 | 浙江大学医学院附属第一医院 | Clinical simulation virtual patient system for teaching and examination |
Non-Patent Citations (4)
| Title |
|---|
| 姜霞;戎月;李丽;王晓娜;薛红元;: "超声造影与增强MR、增强CT诊断MDT肝脏肿瘤108例的价值比较", 疑难病杂志, no. 08, 18 August 2020 (2020-08-18), pages 819 - 822 * |
| 李小军;朱潇;杏福宝;高雄;张雷;: "三维重建及3D打印在微创肺外科中的应用", 中华全科医学, no. 07, 1 July 2020 (2020-07-01), pages 1190 - 1194 * |
| 胡文歆: "结合情境分析和用户研究理论的医疗AR眼镜产品设计研究与实践", 《中国优秀硕士学位全文全文数据库-工程科技I辑》, no. 08, 15 August 2021 (2021-08-15), pages 024 - 564 * |
| 邱憬;孙健;叶铭;熊耀阳;顾晓宇;张富强;: "颌面缺损结构光三维测量与螺旋CT扫描图像的配准", 中国组织工程研究与临床康复, no. 39, 24 September 2010 (2010-09-24), pages 7319 - 7322 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117711610A (en) * | 2024-02-05 | 2024-03-15 | 四川省医学科学院·四川省人民医院 | Remote consultation vehicle system of networked MDT multidisciplinary |
| CN117711610B (en) * | 2024-02-05 | 2024-04-26 | 四川省医学科学院·四川省人民医院 | Remote consultation vehicle system of networked MDT multidisciplinary |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117711611B (en) | 2024-04-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN117711611B (en) | MDT remote consultation system and method based on scene fusion and mr | |
| CN111127441B (en) | Multi-modal brain image depression recognition method and system based on graph node embedding | |
| CN104424385B (en) | A kind of evaluation method and device of medical image | |
| CN107247971B (en) | Intelligent analysis method and system for ultrasonic thyroid nodule risk index | |
| CN110491480A (en) | A kind of medical image processing method, device, electromedical equipment and storage medium | |
| Tian et al. | DCPR-GAN: dental crown prosthesis restoration using two-stage generative adversarial networks | |
| CN110490851A (en) | Galactophore image dividing method, apparatus and system based on artificial intelligence | |
| Li et al. | Medical image coloring based on gabor filtering for internet of medical things | |
| CN112086197A (en) | Mammary nodule detection method and system based on ultrasonic medicine | |
| CN110021445A (en) | A kind of medical system based on VR model | |
| CN116110597B (en) | Digital twinning-based intelligent analysis method and device for patient disease categories | |
| CN108710890A (en) | Preoperative grading of cerebral glioma method, apparatus, equipment and storage medium | |
| CN112669925A (en) | Report template for CT (computed tomography) reexamination of new coronary pneumonia and forming method | |
| CN115064254A (en) | Remote rapid diagnosis system based on medical image | |
| Tian et al. | Efficient tooth gingival margin line reconstruction via adversarial learning | |
| CN117237351B (en) | Ultrasonic image analysis method and related device | |
| CN110473176A (en) | Image processing method and device, method for processing fundus images, electronic equipment | |
| CN109948569A (en) | A kind of three-dimensional hybrid expression recognition method using particle filter frame | |
| CN107887018A (en) | Preoperative evaluation system and preoperative evaluation method | |
| CN117224082A (en) | Dynamic brain function network learning method based on graph learning optimization algorithm | |
| Megalooikonomou et al. | Medical data fusion for telemedicine | |
| CN108596900A (en) | Thyroid-related Ophthalmopathy medical image data processing unit, method, computer readable storage medium and terminal device | |
| CN108577966A (en) | A kind of Intelligent orthopaedic surgery systems | |
| CN106570310A (en) | Medical clinic information management system, and management method thereof | |
| CN113706644A (en) | Image processing method, image processing apparatus, and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |