CN113947959A

CN113947959A - Remote teaching system and live broadcast problem screening system based on MR technology

Info

Publication number: CN113947959A
Application number: CN202111236672.9A
Authority: CN
Inventors: 刘丽萍; 刘欣; 唐雨
Original assignee: Beijing Tiantan Hospital
Current assignee: Beijing Tiantan Hospital
Priority date: 2021-10-23
Filing date: 2021-10-23
Publication date: 2022-01-18

Abstract

The invention relates to a remote teaching system and a live broadcast problem screening system based on an MR (magnetic resonance) technology. The system comprises: the teaching terminal, the learning terminal and the control system; the teaching end comprises an MR device used for shooting and synthesizing a video picture; the control system includes: the video stream layer is used for receiving a video picture pushed by the MR equipment; the MR3D information body enhancement display information layer is used for displaying a virtual MR3D information body; an enhanced display information generating module, configured to add the MR3D information body enhanced display information layer on the video stream layer to form a synthesized video picture, where the MR3D information body enhanced display information layer covers the video stream layer; and the video source station is used for receiving the video pictures transmitted by the MR live broadcast control processing system and playing videos outwards. The system of the application has important clinical application value.

Description

Remote teaching system and live broadcast problem screening system based on MR technology

Technical Field

The invention relates to the field of intelligent systems, in particular to a remote teaching system and a live broadcast problem screening system based on an MR (magnetic resonance) technology.

Background

The currently clinically adopted teaching mode can be divided into two types, namely an offline training system and an online training system. These two modes each have advantages and disadvantages. The offline teaching training can lead the doctors in the subordinate hospitals to directly go to the wards in the superior hospitals, observe and study the diagnosis thought of the doctors in the superior hospitals, and lead the effect of the teaching training to be the best through the face-to-face communication. The defects are that the number of trained doctors and the training times are limited, and the repeatability is not high. In recent years, an online training mode has also been gradually adopted. The method has the advantages that the trained doctors do not need to go to the site in person, and can visit and study the ward online and listen to the disease analysis of the doctors in the superior hospitals. And can repeatedly watch the study. The on-line real-time communication with the superior doctor cannot be realized, and simultaneously, the lack of reality of on-line teaching ward-round results in low participation and greatly reduces the efficiency of teaching. The method has the advantages that a high-efficiency teaching system with multiple audiences and strong sense of participation capable of being repeatedly learned is established, the method has a positive guiding effect on improving the clinical skill and thinking mode of doctors, and especially, the standard and systematic teaching system is established in the fields of critical nervous system diseases with complex disease conditions, large difficulty in treatment decision, strong unpredictability and obvious individuation trend and has great significance on improving the clinical capability of doctors and improving the good prognosis of patients.

The MR (Mixed Reality) technology is a further development of the virtual Reality technology, and the technology is used for enhancing the Reality sense of the user experience by presenting virtual scene information in a real scene and building an interactive feedback information loop among the real world, the virtual world and the user. Through the front camera on the MR equipment, the first visual angle camera shooting acquisition content of the MR equipment wearer is synthesized with the space virtual 3D information body and then pushed to the live broadcast source site. When the MR equipment carries out video live broadcast through the front-facing camera, the video content shot and collected by the camera and the MR are superposed in a virtual 3D information body of a real space to be synthesized in a video stream. Due to the problems of the resolution of a camera of the device and the visual angle of a wearer, discomfort of a viewer of live video can be caused, for example, the information such as 3D rendering characters and tables has jagged and unsmooth line outlines and unclear display due to low resolution; the information display is incomplete and not durable due to the movement of the visual angle of the wearer; the anti-shake function of the equipment is insufficient, and shaking causes discomfort such as dizziness.

In addition, in the live broadcast teaching of tens of people and even hundreds of people simultaneously, a great number of people can ask questions or perform small-talk interaction, the teaching personnel cannot effectively answer the questions of people in time, and especially when the repeated questions are many or the small-talk interaction content is many, the teaching personnel receive too much useless information, so that the efficiency of answering the questions is not high.

Disclosure of Invention

An object of the present application is to provide a remote teaching system based on MR technology, comprising: the teaching terminal, the learning terminal and the control system; the teaching end comprises MR equipment, the learning end comprises a video source station, and the control system comprises an MR live broadcast control processing system;

the teaching end includes:

the MR device is used for shooting and synthesizing a video picture, and the video picture comprises the synthesis of a real scene object and a virtual MR3D information body;

the MR live broadcast control processing system comprises:

the video stream layer is used for receiving a video picture pushed by the MR equipment;

the MR3D information body enhancement display information layer is used for displaying a virtual MR3D information body;

an enhanced display information generating module, configured to add the MR3D information body enhanced display information layer on the video stream layer to form a synthesized video picture, where the MR3D information body enhanced display information layer covers the video stream layer;

and the video source station is used for receiving the video pictures transmitted by the MR live broadcast control processing system and playing videos outwards.

Further, the MR equipment contains an MR live broadcast APP for displaying and interacting the virtual MR3D information body to a teaching person;

optionally, the MR live APP includes an information body display control module, where the information body display control module is configured to control whether a virtual MR3D information body displays and displays content;

optionally, the content displayed in the virtual MR3D information body includes static content or dynamic content;

optionally, the display content type in the virtual MR3D information body includes a plane graph, a video, or a 3D model;

optionally, when the MR device wearer performs live video, the information display control module is used to control display content in the MR3D information body, and operate the virtual MR3D information body, where the operation includes, optionally, calling, displaying, moving, or zooming.

Further, the information body display control module adopts an automatic trigger mode or a manual trigger mode for control;

optionally, in the automatic trigger mode, when the MR device performs operations such as opening, displaying, zooming, moving and the like on the virtual MR3D information body, information content of the virtual MR3D information body is automatically and synchronously displayed in the MR3D information body enhanced display information layer; the manual trigger mode is that a teaching person triggers the information content of a virtual MR3D information body in the MR equipment to be displayed in an MR3D information body enhanced display information layer by operating a button in the MR live broadcast APP.

Furthermore, the learning end further comprises a live audience question input module, the control system further comprises a question processing module, the live audience question input module is used for acquiring a question of a user and submitting the question to the question processing module, and the question processing module classifies and summarizes the question to obtain a question list and pushes the question list to the teaching end.

Further, the question processing module classifies and summarizes the question, including:

NLP module: the system comprises a query question processing unit, a query processing unit and a query processing unit, wherein the query question processing unit is used for performing NLP (NLP) on a query question, the NLP comprises text abstract generation and keyword extraction, and an abstract text and a keyword weight list of the query question are output;

the analysis module is used for carrying out problem classification analysis on the abstract text and the keyword weight list of the question to obtain a clustering family;

and the generating module is used for selecting a question from each cluster family according to the result of the analyzing module and generating a question list.

Further, the analysis module also comprises relevance analysis, the relevance analysis of the live content subject is carried out on the abstract text of the question and the keyword weight list, and the problems lower than the low threshold value of the relevance of the live content subject are filtered;

optionally, the association analysis may be performed in parallel with the classification analysis or before or after the classification analysis, and the analysis module further includes a heat analysis for evaluating the importance of the problem as a ranking basis.

The system further comprises a sorting module for sorting the generated question list, wherein the sorting is carried out based on the number of the cluster family questions or the association degree or the heat degree.

Furthermore, the MR device of the teaching end also comprises a live broadcast question display module which is used for displaying a question list;

optionally, the MR live broadcast APP further comprises a live broadcast question prompt module for prompting a teaching staff to ask a question.

Furthermore, an information retrieval module in the control system is connected with an in-hospital information system and used for retrieving clinical information of a patient and outputting the clinical information to an MR (magnetic resonance) live broadcast APP to form display content in an MR3D (magnetic resonance) information body;

optionally, the information retrieval module in the control system is connected with the medical equipment system and used for retrieving dynamic monitoring data of a patient and outputting the dynamic monitoring data to the MR live APP to form display content in the MR3D information body.

An object of this application is to provide a live problem screening system, includes:

the live broadcast audience question input module is used for acquiring a question of a user;

the question processing module is used for classifying and summarizing the question to obtain a question list, and comprises:

the generating module is used for selecting a question from each cluster family according to the result of the analyzing module and generating a question list;

optionally, the analysis module further includes relevance analysis, which performs live broadcast content topic relevance analysis on the abstract text of the question and the keyword weight list, and filters out questions lower than a low threshold of the live broadcast content topic relevance;

optionally, the association analysis may be performed in parallel with the classification analysis or before or after the classification analysis, and the analysis module further includes a heat analysis for evaluating the importance of the problem as a sorting basis;

optionally, the system further comprises a sorting module, configured to sort the generated question list, where the sorting is performed based on the number of the cluster family questions or the association degree or the heat degree.

The application has the advantages that:

1. according to the method, the enhanced display information generating module of the control system is used for covering the MR3D information body enhanced display information layer on the video stream layer, so that the problems that an existing viewer is uncomfortable or the display of a virtual MR3D information body is incomplete and unclear and the like are solved;

2. according to the method and the system, through a questioning problem processing module of the control system, an artificial intelligence method is adopted to perform text abstract generation, keyword extraction, classification, clustering, filtering and association degree analysis on the questioning problem, and the efficiency of answering the problem by a teaching person is improved.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without creative efforts.

FIG. 1 is a block diagram of a remote instruction system based on MR technology according to an embodiment of the present invention;

FIG. 2 is a video flow diagram of a remote instruction system based on MR technology according to an embodiment of the present invention;

FIG. 3 is a flow diagram of a live problem provided by an embodiment of the invention;

FIG. 4 is a block diagram of a live broadcast question screening system provided by an embodiment of the present invention;

FIG. 5 is a block diagram of MR3D information volume retrieval provided by an embodiment of the present invention;

wherein, 1, teaching end; 2. a control system; 3. a learning end; 4. a hospital information system; 5. a medical device system; 12. MR live broadcast APP; 13. a real scene object; 14. an MR3D infosome; 15. an information body display control module; 21. an MR live broadcast control system; 22. a video stream layer; 23. the MR3D information body enhances the display information layer; 24. MR3D information body on the layer; 25. an enhanced display information generation module; 31. a video source station; 32. a live audience question input module; 26. a questioning question processing module; 16. a live broadcast question display module; 17. a live broadcast question prompting module; 27. and an information calling module.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention.

In some of the flows described in the description and claims of the invention and in the above figures, a number of operations are included that occur in a particular order, but it should be clearly understood that these operations may be performed out of order or in parallel as they occur herein. Additionally, the flows may include more or fewer operations, and the operations may be performed sequentially or in parallel.

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to some drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only some embodiments of the present invention, and not all embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Referring to fig. 1-5, there are block diagrams of a remote teaching system based on MR technology according to an embodiment of the present invention:

the method comprises the following steps: the teaching terminal 1, the learning terminal 3 and the control system 2; the teaching terminal 1 comprises MR equipment, the learning terminal 3 comprises a video source station 31, and the control system 2 comprises an MR live broadcast control processing system;

the teaching terminal 1 includes:

an MR device for capturing and composing a video picture comprising a composition of a real scene object 13 and a virtual MR3D informer 14;

the MR equipment contains an MR live broadcast APP12 used for displaying and interacting the virtual MR3D information body 14 to a teaching person;

the MR live broadcast APP12 includes an information body display control module 15, where the information body display control module 15 is used to control whether the virtual MR3D information body 14 displays and displays content;

the information body display control module 15 controls in an automatic trigger mode or a manual trigger mode: in the automatic trigger mode, when the virtual MR3D information body 14 is opened, displayed, zoomed, moved, and the like in the MR device, the information content of the virtual MR3D information body 14 is automatically and synchronously displayed in the MR3D information body enhanced display information layer 23; the manual trigger mode is that the instructor triggers the information content of the virtual MR3D information body 14 in the MR equipment to be displayed in the MR3D information body enhanced display information layer 23 by operating a button in the MR live APP 12.

The content shown in the virtual MR3D information volume 14 includes static content or dynamic content;

the type of content presented in the virtual MR3D body of information 14 includes flat graphics, video or 3D models.

When the MR device wearer performs live video, the display control module 15 is used to control the display content in the MR3D information body 14, and operate the virtual MR3D information body 14, optionally, the operation includes calling, displaying, moving, or zooming.

The MR live broadcast APP12 further comprises a live broadcast question display module 16 for displaying and generating a question and question list;

the MR live APP12 also comprises a live question prompt module 17 for reminding a teacher that a new question arrives.

The MR live broadcast control processing system comprises:

the video stream layer 22 is used for receiving a video picture pushed by the MR device;

the MR3D information body enhancement display information layer 23 is used for displaying the virtual MR3D information body 14;

an enhanced display information generating module 25, configured to add the MR3D information enhanced display information layer 23 to the video stream layer 22 to form a synthesized video picture, where the MR3D information enhanced display information layer 23 covers the video stream layer 22;

the virtual MR3D information body 14 displayed on the MR3D information body enhanced display information layer 23 is also referred to as an MR3D information body 24 on the layer, the contents of the MR3D information body 24 on the layer and the MR3D information body 14 in the MR device are consistent, and the display form of the contents is a two-dimensional plane.

A question processing module 26, wherein the live viewer question input module 32 is configured to obtain a question input by a user at the video source station 31 and submit the question to the question processing module 26, and the question processing module 26 classifies and summarizes the question to obtain a question list and pushes the question list to the MR live APP 12.

The question processing module 26 includes:

a preprocessing module: preprocessing question text, the preprocessing comprising: processing non-character numbers: recognizing non-character symbols (Chinese and English punctuation marks, blank symbols, separation symbols, icon symbols and the like) in the text, and performing conversion and filtering processing according to rule alignment; sensitive word processing: and identifying the sensitive words by using the sensitive word bank and a third-party sensitive word interface, and carrying out desensitization or shielding treatment to obtain a converted text and transmitting the converted text to the next stage or shielding and blocking the subsequent transmission of the content.

NLP module: performing NLP (NLP) on the question text, wherein the NLP comprises text abstract generation and keyword extraction, and outputting the abstract text and a keyword weight list of the question; the NLP module is composed of two parallel modules: the text abstract generating module and the keyword extracting module.

The text abstract generating module is used as an independent functional module and can be completed by various technologies, including algorithms such as TextRank, RNN, CNN and the like, and the output of the text abstract generating module is a generated abstract text; and after processing by the keyword extraction module, generating a list of keywords and extreme weights.

And the analysis module is used for carrying out problem classification analysis on the abstract text and the keyword weight list of the question to remove repeated or approximate problems.

The classification analysis is used for classifying and de-weighting the question questions. This can be achieved in a number of ways. Text clustering is performed on the question questions, for example, through clustering analysis in machine learning; and the classification category of the question can be set in advance according to the live content before the live broadcast starts. Such as: classification can be divided into two levels, with one level of classification being based on categories, such as: disease classification, course topic academic questions, subject field academic questions, course teaching questions, greeting, personal chat, and the like.

The secondary classification (label) is set according to the platform domain or live content under the primary classification, for example:

TABLE 1

First order classification	Second class classification (Label)
		Disease classification	Craniocerebral injury, atherosclerosis
Subject matter academic problems of course	Cerebral infarction, cerebral hemorrhage
		Problem in course teaching	Lecture arrangement, registration and examination
Greeting	Hello, morning, hi

For the secondary classification (label), a theme association degree attribute is set in advance for each prefabricated content, and the labeling value range of the association attribute is 0-1, for example: cerebral infarction (0.9), cerebral hemorrhage (0.74).

The analysis module is used for carrying out question classification analysis, association degree analysis and heat degree analysis or any one or combination of the above analysis on the abstract text of the question and the keyword weight list;

and the relevancy analysis is used for filtering out the questioning question text with low relevancy with the live content theme.

The relevancy analysis may be implemented in a number of ways. For example, the abstract text and the keyword extraction result and a pre-prepared second-level classification label are subjected to relevance judgment. The method can be carried out by adopting a machine learning mode or a mode of artificial training and machine learning.

The heat analysis is used for evaluating the importance of the problem as a sequencing basis.

The heat analysis can be achieved in a number of ways. For example, the heat value is calculated according to the specific weight of the problem in all effective problems; or how many members of the family are clustered according to questions, etc.

And the generating module is used for selecting one question from each classified family according to the result of the analyzing module and generating a question list.

And further eliminating and filtering the problems in the question list, and directly eliminating the problems which are matched with the keyword blacklist and the classification blacklist and have the classification relevance lower than a threshold value and are not displayed in the question list.

And sequencing the generated question list, sequencing the sequencing based on the results of the cluster family question number, the association degree analysis or the heat degree analysis in the classification analysis to obtain a sequenced question list, and pushing the sequenced question list to the MR live broadcast APP 12.

Further, a popularity ranking list of the question is obtained based on the popularity analysis result, the questions entering the popularity ranking list are aggregated according to the second-level classification contact ratio, and a text abstract with more details is regenerated to serve as question subject contents.

The information retrieval module 27 in the control system 2 is connected with the hospital information system 4, and is used for retrieving clinical information of a patient for processing, outputting the processed clinical information to the MR live APP12, and becoming display content in the MR3D information body 14. The clinical information includes baseline information, past history information, baseline laboratory indicators, baseline image information, intraoperative conditions, and other information. In one embodiment, taking clinical information of patients in the field of neurological diseases as an example, the clinical information includes: baseline information (B) age, gender, baseline NIHSS score, baseline blood pressure, time to onset; the previous history information (H) comprises hypertension, coronary heart disease, atrial fibrillation and previous stroke, and the previous medication history (antiplatelet therapy and anticoagulant therapy); baseline laboratory indices (L) include admission blood glucose, white blood cell count, neutrophil count, PLT count, LDL); the baseline image information (I) comprises infarct volume, Mismatch volume, ASPECT score, infarct position, responsible blood vessel, occlusion degree, early signs, collateral circulation, leukoencephalopathy degree, microhemorrhage condition and the like; the intraoperative conditions (E) include intravenous thrombolysis treatment, anesthesia mode, intravascular treatment mode (stent removal, aspiration thrombolysis, stent formation, etc.), number of thrombolysis removal, intraoperative anticoagulation, antiplatelet application, postoperative TICI grading, residual stenosis, postoperative NIHSS score, time of surgery, etc.

The information retrieval module 27 in the control system 2 is connected with the medical equipment system 5, and is used for retrieving dynamic monitoring data of a patient and outputting the dynamic monitoring data to the MR live APP12 to become display contents in the MR3D information body 14. In contemplated embodiments, the dynamic monitoring data includes ecg (e), resp (r), nibp (nb), abp (ab), SPO2(s), pulse (p), eeg (eeg), tcd (tcd), and the like.

The learning terminal 3 includes a video source station 31, which is used to receive the video pictures transmitted by the MR live broadcast control processing system and play the video to the outside.

The learning terminal 3 comprises a mobile phone or a computer, and the video source station 31 can be logged in by the mobile phone or the computer.

And the live audience question input module 32 is used for acquiring the question questions input by the user at the video source station 31 and submitting the questions to the question processing module 26.

In one embodiment, the teaching person wears the MR device for distance teaching, and the front camera of the MR device captures the real scene object 13 and synthesizes a video picture, which includes the synthesis of the real scene object 13 and the virtual MR3D information body 14. The information content displayed by the virtual MR3D information body 14 can be in the form of various 3D rendered objects, including 2D graphics information, image information, 3D dynamic and static models, and the display content includes static content (such as detection report) or dynamic content (such as electrocardiographic monitoring). MR equipment contains live APP12 of MR for show and interact to the teaching personnel virtual MR3D informer 14, including informer display control module 15 in the live APP12 of MR for whether show and show content of control virtual MR3D informer 14. The teaching staff can also perform operations including calling, displaying, moving or zooming on the virtual MR3D information body 14, the teaching staff controls the virtual MR3D information body 14 through the information body display control module 15, the information body display control module 15 performs control in an automatic trigger mode or a manual trigger mode, the automatic trigger mode is that when the virtual MR3D information body 14 is opened, displayed, zoomed, moved and the like in the MR equipment, the information content of the virtual MR3D information body 14 is automatically and synchronously displayed in the MR3D information body enhanced display information layer 23; the manual trigger mode is that the instructor triggers the information content of the virtual MR3D information body 14 in the MR equipment to be displayed in the MR3D information body enhanced display information layer 23 by operating a button in the MR live APP 12.

In one embodiment, the content of the virtual MR3D information 14 includes baseline information, prior history information, baseline laboratory indices, baseline imaging information, clinical information such as intraoperative conditions, including basic information, demographic characteristics, pre-hospital first aid (, prior history, family history, prior medication, admission subjects, admission diagnoses, baseline scoring, auxiliary examinations during hospital stay, treatments during hospital stay, final diagnoses, time, extent, cause of patient's onset, and follow-up information, imaging information, and the like.

In an embodiment, when a teaching person calls, displays, moves, or scales a virtual MR3D information body 14, an enhanced display information generation module 25 in the MR live broadcast control processing system adds the MR3D information body enhanced display information layer 23 to the video stream layer 22, the content of the virtual MR3D information body 14 is displayed on the MR3D information body enhanced display information layer 23, that is, the MR3D information body 24 on the layer and the content of the MR3D information body 14 in the MR device are identical, the MR3D information body enhanced display information layer 23 and the MR3D information body 24 on the layer are covered on the video stream layer 22 to form a synthesized video picture, and the synthesized video is pushed to the video source station 31 to play a video to the outside.

In one embodiment, a student at the learning terminal 3 asks teaching questions in live broadcast through the live broadcast audience question input module 32, the live broadcast audience question input module 32 submits questions of the student to the question processing module 26, and the question processing module 26 classifies and summarizes the questions to obtain a question list and pushes the question list to the teaching terminal 1.

In one embodiment, the control system 2 receives a question from the live audience question input module 32, and pre-processes the text of the question by a pre-processing module of the question processing module 26, the pre-processing including: processing non-character numbers: recognizing non-character symbols (Chinese and English punctuation marks, blank symbols, separation symbols, icon symbols and the like) in the text, and performing conversion and filtering processing according to rule alignment; sensitive word processing: and identifying the sensitive words by using the sensitive word bank and a third-party sensitive word interface, and carrying out desensitization or shielding treatment to obtain a converted text and transmitting the converted text to the next stage or shielding and blocking the subsequent transmission of the content. After preprocessing, performing NLP on the text through an NLP module, wherein the NLP comprises text abstract generation and keyword extraction, and outputting abstract texts and a keyword weight list of a question; the NLP module is composed of two parallel modules: the text abstract generating module and the keyword extracting module. Further, the abstract text and the keyword weight list of the question are subjected to problem classification analysis through an analysis module, the classification analysis can be cluster analysis in machine learning, the abstract text and the keywords of the question are subjected to text clustering to obtain N (N is a natural number) cluster families, one question is selected from each cluster family as a representative according to the result of the classification analysis to generate a question list, and a large number of repeated problems existing in live broadcast are deleted from the question list. The analysis module can also comprise relevance analysis or heat analysis, for example, the relevance analysis is taken as an example, the abstract text, the keyword extraction result and a second-level classification label prefabricated by the system are subjected to relevance judgment, problems are eliminated and filtered, and the problems which are matched with the keyword blacklist and the classification blacklist and have the classification relevance lower than a threshold value are directly eliminated.

In one embodiment, the relevancy analysis can filter a large number of greeting sentences existing in the live broadcast or questions irrelevant to the subject content before or after the classification analysis, so as to further narrow the question-asking question list and provide a very good auxiliary function for the teaching staff to efficiently answer the questions.

In one embodiment, the results of the group member number, the relevance analysis or the heat analysis in the classification analysis of the generated question list are sorted to obtain a sorted question list, the sorted question list is pushed to the MR live broadcast APP12, the live broadcast question prompt module 17 in the MR live broadcast APP12 prompts the teaching staff of questions to be answered, the teaching staff clicks the live broadcast question display module 16, and the live broadcast question display module 16 displays the results to generate the question list.

In one embodiment, a doctor wears the MR device for distance teaching, all the doctor who wears the MR device sees and hears (real world images + virtual reality images + virtual data + sounds), and the sounds collected by the microphone are uploaded to a remote live broadcast server for processing, and other doctors can take the same first person perspective as the doctor to keep their own experience.

It is clear to those skilled in the art that, for convenience and brevity of description, the specific working processes of the above described systems, apparatuses and units may refer to the corresponding processes in the foregoing embodiments, and are not described herein again.

In the several embodiments provided in the present application, it should be understood that the disclosed system may be implemented in other ways. For example, the above-described device embodiments are merely illustrative, and for example, the division between the teaching end and the learning end is only one logical division, and there may be other divisions in actual implementation, and in addition, the shown or discussed mutual coupling or direct coupling or communication connection may be through some interfaces, indirect coupling or communication connection of devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable storage medium, and the storage medium may include: read Only Memory (ROM), Random Access Memory (RAM), magnetic or optical disks, and the like.

It will be understood by those skilled in the art that all or part of the steps in the method for implementing the above embodiments may be implemented by hardware that is instructed to implement by a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a magnetic disk or an optical disk, etc.

While the invention has been described in detail with reference to specific embodiments thereof, it will be apparent to one skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention.

Claims

1. A remote instruction system based on MR technology, comprising: the teaching terminal, the learning terminal and the control system; the teaching end comprises MR equipment, the learning end comprises a video source station, and the control system comprises an MR live broadcast control processing system;

the teaching end includes:

the MR live broadcast control processing system comprises:

2. The remote instruction system based on MR technology of claim 1, wherein the MR device contains a MR live APP for displaying and interacting the virtual MR3D information body to the instructor;

optionally, the content displayed in the virtual MR3D information body includes static content or dynamic content; optionally, the display content type in the virtual MR3D information body includes a plane graph, a video, or a 3D model;

3. The remote teaching system based on MR technology of claim 2 wherein the infomiation body display control module is controlled in an automatic triggering mode or a manual triggering mode;

4. The remote teaching system based on MR technology as claimed in claim 1, wherein the learning terminal further includes a live audience question input module, the control system further includes a question processing module, the live audience question input module is used to obtain the user's question and submit it to the question processing module, the question processing module classifies and summarizes the question, obtains a question list, and pushes it to the teaching terminal.

5. The remote teaching system based on MR technology as claimed in claim 4, wherein said question processing module classifies and summarizes question questions including:

6. The remote education system based on MR technology of claim 5 wherein the analysis module further includes relevance analysis for analyzing the relevance of the subject matter of the live content to the abstract text of the question and the weighted list of keywords, filtering out questions lower than the low threshold of relevance of the subject matter of the live content; optionally, the association analysis may be performed in parallel with or before or after the classification analysis;

optionally, the analysis module further includes a heat analysis for evaluating the importance of the problem as a ranking basis.

7. The remote teaching system based on MR technology of claim 7 further comprising a ranking module for ranking the generated question list, said ranking being based on the number of cluster family questions or the degree of association or the degree of heat.

8. The remote teaching system based on MR technology as claimed in any one of claims 4-7, wherein the MR device of the teaching end further comprises a live question display module for displaying a question list;

9. The remote teaching system based on MR technology as claimed in claim 1, wherein the information retrieving module in the control system is connected with the information system in the hospital for retrieving the clinical information of the patient and outputting the clinical information to the MR live APP as the display content in the MR3D information body;

10. A live question screening system, comprising: