CN111613280B - H.I.P.S multi-touch teaching interaction system for medical treatment - Google Patents

H.I.P.S multi-touch teaching interaction system for medical treatment Download PDF

Info

Publication number
CN111613280B
CN111613280B CN202010290090.8A CN202010290090A CN111613280B CN 111613280 B CN111613280 B CN 111613280B CN 202010290090 A CN202010290090 A CN 202010290090A CN 111613280 B CN111613280 B CN 111613280B
Authority
CN
China
Prior art keywords
information
medical
module
data
page
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010290090.8A
Other languages
Chinese (zh)
Other versions
CN111613280A (en
Inventor
孙尧伟
赵亮
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Jiuxiang Digital Technology Co ltd
Original Assignee
Shanghai Jiuxiang Digital Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Jiuxiang Digital Technology Co ltd filed Critical Shanghai Jiuxiang Digital Technology Co ltd
Priority to CN202010290090.8A priority Critical patent/CN111613280B/en
Publication of CN111613280A publication Critical patent/CN111613280A/en
Application granted granted Critical
Publication of CN111613280B publication Critical patent/CN111613280B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H10/00ICT specially adapted for the handling or processing of patient-related medical or healthcare data
    • G16H10/20ICT specially adapted for the handling or processing of patient-related medical or healthcare data for electronic clinical trials or questionnaires
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating 3D models or images for computer graphics
    • G06T19/006Mixed reality
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10Character recognition
    • G06V30/14Image acquisition
    • G06V30/142Image acquisition using hand-held instruments; Constructional details of the instruments
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00Electrically-operated educational appliances
    • G09B5/08Electrically-operated educational appliances providing for individual presentation of information to a plurality of student stations
    • G09B5/14Electrically-operated educational appliances providing for individual presentation of information to a plurality of student stations with provision for individual teacher-student communication
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B7/00Electrically-operated teaching apparatus or devices working with questions and answers
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09BEDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B9/00Simulators for teaching or training purposes
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H70/00ICT specially adapted for the handling or processing of medical references
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02ATECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
    • Y02A90/00Technologies having an indirect contribution to adaptation to climate change
    • Y02A90/10Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Educational Technology (AREA)
  • Educational Administration (AREA)
  • Business, Economics & Management (AREA)
  • Primary Health Care (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Epidemiology (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Software Systems (AREA)
  • Human Computer Interaction (AREA)
  • Medical Treatment And Welfare Office Work (AREA)

Abstract

The invention discloses an H.I.P.S. multi-touch-control teaching interaction system for medical treatment, which comprises a background terminal, an equipment terminal and an application program system, wherein the application program system comprises an SLAM (SLAM) implementation positioning module, an information interaction module, a monitoring module, a first database module and a testing module, and the background terminal comprises a second database module; the SLAM implementation positioning module comprises a laser SLAM and a depth camera-based SLAM. The invention can present the operation treatment to the patient through the digital model with interactivity, reduce the fear and rejection psychology of the patient, and the communication disorder between the doctor and the patient, and increase the coordination degree between the doctor and the patient; by the same means, medical students and medical staff can master medical knowledge more quickly, and learning curves are reduced; the preoperative simulation can be carried out on the elements such as the operation process, the risk and the like, so that unnecessary medical accidents are reduced; the invention can also promote the consciousness of the public self-learning correct medical common sense and improve the effects of self-nursing and rehabilitation training.

Description

H.I.P.S multi-touch teaching interaction system for medical treatment
Technical Field
The invention relates to the technical field of medical treatment, in particular to an H.I.P.S. multi-touch teaching interaction system for medical treatment.
Background
H.i.p.s system, holographic Interactive Propaganda System: holographic interactive propaganda system. Is a Augmented Reality (AR)/Virtual Reality (VR)/Mixed Reality (MR) integrated module. The H.I.P.S. can not only create a holographic visual effect, but also generate a strong ductile interaction relationship with a user. And a data analysis library is established by means of the same platform AR scanning identification assessment, touch control recording, path recording and the like, and the degree of knowledge and understanding depth of the watched audience to the content are accurately recorded.
The characteristics include: a multi-dimensional modular operation mode; a friendly application interface; a simple interesting data model; continuously maintaining rapid data update; and the multiple systems are compatible.
The current similar medical knowledge courseware is ppt or video and is planar; and the conventional similar technical teaching courseware is propagated based on a platform or hardware running environment. The invention can be compatible and use more equipment types at the same time: a cell phone or tablet computer or a common pc computer or wearable equipment (smart glasses, vr helmets) and the like.
In use, the conventional approach suffers from the following drawbacks:
1. the existing medical training content software technology on the market is scattered, and misdiagnosis and unnecessary medical accidents caused by individual understanding differences are easy to occur during use;
2. the patient's awareness and awareness rate are generally low in efficiency, misleading opinions and divergences are easily generated due to invalid communication in the treatment process, and the patient and family members thereof have low coordination degree to medical staff;
3. the public has poor knowledge on medical treatment, is easy to cause misoperation in self-treatment in families, and has lower safety of self-treatment.
Therefore, an H.I.P.S. multi-touch teaching interaction system for medical treatment is designed to solve the problems.
Disclosure of Invention
The invention aims to solve the defects in the prior art, and provides an H.I.P.S. multi-touch teaching interaction system for medical treatment.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
the H.I.P.S. multi-touch teaching interaction system for medical treatment comprises a background terminal, an equipment terminal and an application program system, wherein the application program system comprises an SLAM implementation positioning module, an information interaction module, a monitoring module, a first database module and a testing module, and the background terminal comprises a second database module; wherein,,
the SLAM implementation positioning module comprises a laser SLAM and a depth camera-based SLAM, wherein the laser SLAM and the depth camera-based SLAM are used for directly acquiring point cloud data in an environment, generating the point cloud data and then transmitting the point cloud data to the first database module and the second database module;
the information interaction module comprises an information display unit and an audio dialogue unit, and is used for carrying out information interaction between users, wherein the information interaction mode between users comprises one or more of text input transmission, voice input transmission, real-time picture transmission of a camera unit and picture input transmission;
the monitoring module comprises a shooting scanning unit and an information acquisition unit, wherein the shooting scanning unit is used for scanning and extracting image information and transmitting stored information to a first database module of the application program system through signals; the first database module is used for identifying and analyzing the information in the operation room identified by scanning, transmitting analysis data information to the application program system and carrying out information interaction on users; the information acquisition unit is used for browsing the picture display information of the application program system by a user, browsing single data time to acquire information, transmitting the data to the first database module and the second database module for analysis, transmitting analysis data information to the application program system, and carrying out information interaction on the user;
the first database module comprises a data information collecting unit, a data storage unit, a data analysis unit and a data information feedback unit, and is used for receiving information transmitted by the monitoring module and the information interaction module, storing the information and analyzing the data, transmitting the information to the application program system and carrying out information interaction on a user;
the testing module comprises a questionnaire unit, a questionnaire information collecting unit, a questionnaire result generating unit and a testing information duplicate checking unit, and is used for enabling a user to know information deeply and confirm the information;
the application method of the H.I.P.S. multi-touch teaching interaction system for medical treatment comprises the following steps:
s1, starting: the method comprises the steps that a user opens an application program through a device terminal to perform use operation, wherein the operation mode comprises one or more of gesture interaction, natural semantics and voice recognition interaction, and in the beginning, a positioning module and a wireless network are used for performing surrounding type scanning rapid analysis on a specific environment of a hospital where the user is located through SLAM, and an application program system is connected with background data of the hospital;
s2, browsing page information: after information login and mode selection are performed in the step S1, a page is displayed, wherein the page comprises picture information, text information, video information, audio information, three-dimensional model information, animation information and special effect information; the page information browsing content comprises one or more of current medical technology content, current symptom content, symptom treatment technology content, human symptom organ information content, medical mechanical equipment type identification content and operation real-time guidance content;
s3, content interaction: in the page browsing in the step S2, a user performs operation control on a page through page information to acquire interactive information, wherein the interactive information acquisition mode comprises one or more of page information browsing inquiry, window consultation and video voice consultation; according to different equipment terminals, the method for operating the control page comprises clicking a screen, gesture control, voice command or handle operation, wherein when clicking is performed, the timing unit collects information according to clicked data and sends the information to the application program system, and the data is not presented to a user;
s4, data statistics: after the page browsing in the step S4 is finished, the device terminal directly sends the signal to the background terminal, the second database module of the background terminal performs a save record, and if the device terminal has no signal, the device terminal temporarily saves the signal in the first database module of the application program system.
Preferably, in the beginning process of the step S1, a usage description and a service agreement are performed on the user, wherein the service agreement has a definite description on the data content monitored by the monitoring module; the equipment terminals used by the user comprise one or more of mobile phones, tablet personal computers, intelligent wearable equipment, intelligent glasses, touch screen computers and televisions.
Preferably, the camera scanning unit comprises a common camera, a depth camera or a laser camera, and the common camera comprises a monocular camera or a binocular RGB camera.
Preferably, the manner of operating the control page in step S3 includes: zoom in, zoom out, move, rotate, edit.
Preferably, in the page browsing process of step S2, the page may be automatically adjusted according to the movement of the user.
Preferably, the application system supports support use of ar+vr+mr.
Preferably, the display mode of the information display unit comprises one or more of two-dimensional display, three-dimensional display and four-dimensional display.
Compared with the prior art, the invention has the beneficial effects that:
1. the method has the advantages that the traditional scattered medical training contents can be unified, the standard is formulated, the precious experience of a higher-order doctor is quantified by means of the technology, the memory coding rate of trained personnel is improved, deep understanding is achieved, the learning curve is quickly mastered, the treatment standard is effectively shortened, the operation standard is unified, and misdiagnosis and unnecessary medical accidents caused by individual understanding differences are reduced;
2. the panic degree of patients and families on the conditions can be reduced, the awareness and the awareness rate of the patients are increased, misleading opinions and divergences caused by invalid communication are reduced, the trust feeling of masses on medical staff and hospitals is built from the fundamental treatment understanding, the contradiction between doctors and patients is reduced, the patients and the families thereof can also learn a lot of daily necessary medical knowledge in the software use learning process, and the coordination degree of the patients and the families thereof on medical staff can be increased to a certain extent;
3. the method can improve the self-learning and self-management capacity of the masses, improve the consciousness of correct basic medical treatment means of the mass self-learning, reduce wrong methods of belief and blind health preservation, and re-establish the authoritative image of medical science.
Detailed Description
The invention is further illustrated below in connection with specific embodiments.
The H.I.P.S. multi-touch teaching interaction system for medical treatment comprises a background terminal, an equipment terminal and an application program system, wherein the application program system comprises an SLAM implementation positioning module, an information interaction module, a monitoring module, a first database module and a testing module, and the background terminal comprises a second database module; wherein,,
the SLAM implementation positioning module comprises a laser SLAM and a depth camera-based SLAM, wherein the laser SLAM and the depth camera-based SLAM are used for directly acquiring point cloud data in an environment, generating the point cloud data and then transmitting the point cloud data to the first database module and the second database module;
the information interaction module comprises an information display unit and an audio dialogue unit, and is used for carrying out information interaction between users, wherein the information interaction mode of the users comprises one or more of text input transmission, voice input transmission, real-time picture transmission of a camera unit and picture input transmission;
the monitoring module comprises a camera scanning unit and an information acquisition unit, and is used for scanning and extracting image information, storing the information and transmitting the information to a first database module of the application program system through signals; the first database module is used for identifying and analyzing the information in the operation room identified by scanning, transmitting analysis data information to the application program system and carrying out information interaction on users; the information acquisition unit can be used for browsing the picture display information of the application program system by a user, browsing single data time to acquire information, transmitting data to the first database module and the second database module for analysis, transmitting analysis data information to the application program system, and carrying out information interaction on the user; the habit of the user and the interested points can be captured and combed, and the points are sent to the background for data analysis, so that preparation is made for accurate pushing;
the first database module comprises a data information collecting unit, a data storage unit, a data analysis unit and a data information feedback unit, and is used for receiving information transmitted by the monitoring module and the information interaction module, storing the information and analyzing the data, transmitting the information to the application program system and carrying out information interaction on a user;
the testing module comprises a questionnaire unit, a questionnaire information collecting unit, a questionnaire result generating unit and a testing information duplicate checking unit, and is used for enabling a user to know information deeply and confirm the information.
Example 1
The user terminal: hospital (doctor)
The application method of the H.I.P.S. multi-touch teaching interaction system for medical treatment comprises the following application steps:
s1, starting: the user opens an application program through the equipment terminal to perform use operation, wherein the operation mode comprises one or more of gesture interaction, natural semantics and voice recognition interaction; at the beginning, the positioning module and the wireless network can be realized through SLAM to carry out surrounding type scanning rapid analysis on the specific environment of the hospital where the user is located, and an application program system is connected with background data of the hospital;
s2, browsing page information: after information login and mode selection are performed in the step S1, a page is displayed, wherein the page comprises picture information, text information, video information, audio information, three-dimensional model information, animation information and special effect information; the page information browsing content comprises one or more of current medical technology content, current symptom content, symptom treatment technology content, human symptom organ information content, medical mechanical equipment type identification content and operation real-time guidance content;
the page information browsing content comprises current medical technology content, current symptom content, symptom treatment technology content, human symptom organ information content, medical mechanical equipment type identification content, operation real-time guidance and the like;
specifically, the surgical real-time guidance is presented through an application technology of MR mixed reality, including:
preoperative: evaluation prediction and operation scheme establishment: the method combines the graph acquisition and related data of the real case of the medical imaging equipment to give out proper graph image processing, is convenient for an operator to observe the lesion position of the organ at multiple angles, gives out a more accurate and effective operation scheme, and can predict the possible risk of the operation by the reference points in the graph image processing; meanwhile, the monitoring module can scan and identify information in an operating room, store the information and transmit the information to the first database module of the application program system through signals, the first database module can rapidly identify and analyze the information in the operating room which is scanned and identified, timely and accurately analyze the type of the operation and the required operation equipment tool according to the condition in the operating room, automatically recommend all equipment, consumables, medicines and the like required by the operation by combining the real vital sign data of a patient after determining the type of the operation, wherein the equipment comprises specification types, quantity and the like, rapidly know the type of the operation which can be performed in the environment and the position, state and availability of the equipment in a hospital, and is simultaneously connected with a background database of the hospital, can rapidly know the medical insurance condition, inventory, state, department and other relevant information and qualification conditions of consumable medicines required in the operation, and transmit the information to the application program system to a doctor user in time, and reduce risks existing in the operation;
in the operation: guiding and navigating in operation; through the three-dimensional reconstructed image of pathological organ, the size measurement of many specific tissue positions, intelligent demonstration suggestion doctor adopts the apparatus size to give the route that the surgical instrument goes on, direction and degree of depth, through wear-type mixed reality equipment, the virtual influence of H.I.P.S. system can overlap with actual operation position, makes things convenient for the operator to know the condition of operation passageway. (e.g., creation of orthopedic spinal channels, access of a burn artery or femoral artery surgical instrument during interventional procedures);
s3, content interaction: in the page browsing in the step S2, a user can operate and control a page through page information to acquire interactive information, wherein the interactive information acquisition mode comprises one or more of page information browsing inquiry, window consultation and video voice consultation; preferably, according to different equipment terminals, the method for operating the control page comprises clicking a screen, gesture control, voice command or handle operation, wherein when clicking is performed, the timing unit collects information according to clicked data and sends the information to the application program system, and the data is not presented to a user;
wherein, the content interaction comprises training of doctors/medical skills/nursing theories and skills, training modes and content:
1. guiding the daily workflow of a doctor;
2. visual teaching of medical theory knowledge, wherein the teaching mode is AR+VR+MR (space positioning+three-dimensional/audio-video and other various media forms);
3. multidimensional teaching of standardized operation flow (two-dimensional to three-dimensional of medical images, multi-angle learning, intraoperative difficulty triggered with teaching content flow and corresponding mode, and real-time remote expert guidance intervention);
4. a brand new operation technology/operation type recommendation platform (a multi-angle analysis technology demonstration platform) immersive practical operation training;
simulation operation training ware: the physical hardware simulator and the sensing equipment thereof are matched, and specific equipment and environments are rendered, so that the immersive training experience in the scene is achieved;
5. physician daily workflow guidance: the latest national policies and laws and regulations aim at the subjects corresponding to users to precisely push and publicize (such as sensing and controlling requirements, quality control checking methods and the like).
The method is characterized by integrating the previously scattered medical training contents, formulating standards and quantifying the precious experience of a higher-order doctor by technical means. The memory coding rate of trained personnel is improved, deep understanding is achieved, the training personnel can master quickly, and the learning curve is effectively shortened. The treatment standard is reached, the operation standard is unified, and misdiagnosis and unnecessary medical accidents caused by individual understanding differences are reduced.
Specifically, the content interaction further includes:
surgical guidance (application of MR mixed reality):
preoperative: evaluating predictions/surgical planning/surgical simulation/selection of instruments;
the image acquisition and related data of the real case of the medical imaging equipment are combined to give out proper image processing of the image, so that an operator can observe the lesion position of the organ at multiple angles conveniently, and a more accurate and effective operation scheme is given out. And can anticipate the risk that surgery may exist by referencing points in the graphical image processing. Selection of the critical materials for the procedure (e.g., how large a balloon stent is to be placed, what size guide wire, etc.);
in the operation: intraoperative guidance/navigation
Through the three-dimensional reconstructed image of the lesion organ, the size measurement of a plurality of specific tissue parts is intelligently displayed, the size of the instrument adopted by a doctor is recommended, and the path, the direction and the depth of the surgical instrument are given. Through the head-mounted mixed reality equipment, the virtual image of the HIPS system can be overlapped with the actual operation position, so that an operator can know the condition of an operation channel conveniently. ( Examples: bifurcation lesions in cardiac coronary intervention operation, the path of access of radial artery or femoral artery surgical instruments, and establishment of channels. )
Marketing, use and management of medical devices/consumables/medicines
1. Virtual display and popularization of medical equipment/consumables/medicines: the paper color page propaganda mode is replaced, a prototype is replaced on the exhibition, the communication cost of the sales of manufacturers is reduced, and unnecessary prototypes are reduced to provide the exhibition. Medical staff can know the information of the virtual product through HIPS software at any time, at any time and any place, can observe and know the product more deeply at multiple angles, can communicate on line with manufacturer professionals through remote video streaming media, and sales personnel's popularization can reach the standardization.
2. Operation/use procedure steps/maintenance/troubleshooting of medical equipment (for subjects: physicians, technicians, medical maintenance personnel, caregivers): medical staff can identify specific points on purchased products by utilizing a scanning identification function in software, know using steps and notes of the products, and can be in butt joint with the existing equipment management internet of things of a hospital;
3. preparation of equipment consumable materials during operation on the table (point cloud identification technology):
the circumferential scan of a specific hospital environment quickly knows the types of surgery that can be performed in that environment, and by combining the real vital sign data of the patient after the surgery types are determined, all equipment/consumables/medicines and the like required by the surgery are automatically recommended, wherein the equipment/consumables/medicines comprise specification models, quantity and the like. The system can be in butt joint with the Internet of things managed by the existing equipment in the hospital to check whether the position, the state and the availability of the equipment in the hospital are available. The system can be connected with the database of the existing consumable medicine purchasing warehouse of the hospital to know the medical insurance condition, the stock quantity and the state of the required consumable medicine, and belongs to departments and other related information and qualification conditions;
drug instruction/management: drug information, molecular structure, method and metering for correct use, contraindications in operation, coping methods, etc.;
establishing a medical consumable/medicine intelligent secondary library: the self-building library can be realized, and the hospital related management database can be docked;
conclusion: evidence-based medical demonstration prior to purchase of medical devices/consumables/medicines in a hospital or department, multi-party comparison, demonstration of whether purchase and purchase quantity are required. The accurate asset management efficiency can be improved.
S4, data statistics: after finishing the page browsing in the step S4, the equipment terminal directly sends the page browsing to the background terminal under the condition of signals, a second database module of the background terminal saves records, and if the equipment terminal has no signals, the page browsing is temporarily saved in a first database module of an application program system for saving;
through the above content, the medical training content scattered in the past can be unified, standards are formulated, precious experiences of Gao Jiesheng are quantified by means of the technology, the memory coding rate of trained personnel is improved, deep understanding is achieved, the learning curve is quickly mastered, the treatment standard is achieved, the operation standard is unified, and misdiagnosis and unnecessary medical accidents caused by individual understanding differences are reduced.
Example two
The user: patient and masses
The application method of the H.I.P.S. multi-touch teaching interaction system for medical treatment comprises the following application steps:
s1, starting: the method comprises the steps that a user opens an application program through a device terminal to perform use operation, wherein the operation mode comprises one or more of gesture interaction, natural semantics and voice recognition interaction, and in the beginning, a positioning module and a wireless network can be used for performing surrounding type scanning rapid analysis on a specific environment of a hospital where the user is located through SLAM, an application program system is connected with background data of the hospital, and information and illness state of the user are timely transmitted to a background database of the hospital;
s2, browsing page information: after information login and mode selection are performed in the step S1, a page is displayed, wherein the page comprises picture information, text information, video information, audio information, three-dimensional model information, animation information and special effect information; the page information browsing content comprises one or more of current medical technology content, current symptom content, symptom treatment technology content, human symptom organ information content, medical mechanical equipment type identification content and operation real-time guidance content; the education of family members of the patient is facilitated through a first database module and a test module of the application program system;
s3, content interaction: in the page browsing in the step S2, a user can operate and control a page through page information to acquire interactive information, wherein the interactive information acquisition mode comprises one or more of page information browsing inquiry, window consultation and video voice consultation; preferably, according to different equipment terminals, the method for operating the control page comprises clicking a screen, gesture control, voice command or handle operation, wherein when clicking is performed, the timing unit collects information according to clicked data and sends the information to the application program system, and the data is not presented to a user;
s4, data statistics: after finishing the page browsing in the step S4, the equipment terminal directly sends the page browsing to the background terminal under the condition of signals, a second database module of the background terminal saves records, and if the equipment terminal has no signals, the page browsing is temporarily saved in a first database module of an application program system for saving;
wherein, the main interactive contents include:
preoperative education: the method can replace doctors to explain the whole operation flow (three-dimensional graph, UI interface) of patients and families thereof in a game interactive guiding mode, and the conditions and risks possibly occurring in the operation and various expense notices generated in the operation. After the explanation is finished, the patient and the family members thereof are checked in the form of questions (question bank), and finally, the electronic signature and the electronic fingerprint of the patient or the family members thereof confirm that the operation is agreed, and the report is generated by combining the audio and video data collected by the front-mounted camera of the equipment, and is automatically uploaded to a second database of the back-stage system in the hospital; meanwhile, the data can be repeatedly checked through the test information duplicate checking unit in the later period.
The method aims at reducing fear of patients and family members of the patients to the treatment scheme which is unknown, and effectively improving the awareness and awareness rate of the patients to the treatment scheme. b. Medical disputes are reduced, accurate management efficiency of hospitals is improved, responsibility and authority are separated, and data can be described. c. Can help patients and families better understand medical technology limitations, and can further understand risks possibly existing in treatment. The desire for medical services is controlled to be within a rational acceptance range.
Postoperative guidance and care guidance: the medical nursing care system can replace doctors to patients and families thereof, and can explain the notes (three-dimensional graphics, UI interfaces) of postoperative nursing in a game interaction guiding mode, hospital related regulations, meal management, later follow-up visit, self-nursing management after discharge and the like. After the explanation is finished, the patient and the family members thereof are checked in the form of questions (question bank), and finally, the electronic signature and the electronic fingerprint of the patient or the family members thereof are confirmed to be effectively informed, or the application for discharge is carried out;
through the information interaction between the equipment terminal and the patient, the patient or the family members of the patient can clearly and definitely know the illness state of the patient, the treatment scheme of the illness state and the later-period nursing information; the purpose is to make the patient improve the consciousness of self-care and management, and prevent and reduce secondary hospitalization caused by human;
rehabilitation guidance: the method comprises the steps of recovering cardiac function, recovering orthopaedics, recovering nerves and the like, a patient looks at a person and combines the virtual graphic image special for an H.I.P.S. system with a real scene, and the basic requirements of recovery treatment are calculated by utilizing the development of a camera and the matching degree of the action position guided by the recovery action and software to form scores; the data can be automatically uploaded to a recovery data platform of a responsible community hospital, a family doctor and a review hospital, and can be uploaded to a national resident health data file for archiving;
the page browsable information comprises promotion of medical science popularization common medical knowledge: 1. popular medical science popularization: the authoritative person can make the video through the software content (case: jia center) to form vivid high-quality medical science popularization video; 2. pre-hospital emergency/emergency self-rescue: the correct use of emergency drugs, CPR artificial cardiopulmonary resuscitation, use of ADE, etc.; 3. chronic disease management (post discharge/out of hospital): inertia disease management and diet structure management, and culture of self-care consciousness; 4. and one key is reported: first-aid channels can be connected in line, for example: chest pain center/stroke center/heart failure center/atrial fibrillation center, etc.
Through the adoption of the method, the self-learning and self-management capacity of the masses can be improved, the awareness of correct basic medical treatment means of the mass self-learning is improved, wrong methods of belief and blind health maintenance are reduced, and the authoritative image of medical science is built again.
Meanwhile, the awareness and awareness rate of the patient are increased, misleading opinions and divergences caused by invalid communication are reduced, and the panic degree of the patient is reduced. From the fundamental treatment understanding, the trust sense of the public on medical staff and hospitals is established, and the contradiction and conflict between doctors and patients are reduced. The patient and the family members thereof can learn a lot of daily necessary medical common knowledge in the software use learning process, and the coordination degree of the patient and the family members thereof to medical staff can be increased to a certain extent.
In the foregoing embodiment, the techniques used may be:
AR technology: PTC vuforia provides local tile identification tracking, cloud identification tracking and planar identification;
MR technique: using Microsoft's hollens, magic leap
And (3) voice recognition: using science-big news fly, tengxue hundred degree speech recognition, nuance, microsoft ice, search dog speech recognition, etc
Gesture interaction: using leapmotion, hololens, magic leap
SLAM technology:
hollens, magic leaps, kinect based on depth or laser cameras
Based on monocular, binocular: the apple device uses an arkit and the android device uses arcoreVR device from google: immersive vr device such as htc device and oculus
Intelligent glasses: intelligent glasses equipment such as hollens, magic leap, epson, vuzix, google glass and the like
Communication with the hospital equipment internet of things: typically by bluetooth or wifi communication
Connect to a hospital or self-built database: typically, the database is mysql, sqlserver, oracle, etc., the cloud host is a virtual host or a physical host, and the system is windows or linux, typically by API interface or websocket.
The above technical solution is only a preferred embodiment of the present invention, but the technology used in the present invention is not limited thereto;
it should be noted that the number of the substrates,
in the using process of the hospital:
(1) Hospitals can address the needs, can propose custom-made needs related to own hospital features, and require relevant clinical laboratory medical staff to use when communicating with patients and their families. The doctor orally explains the propaganda treatment scheme, the operation scheme, the risk notification and the patient nursing beard. Instead, the patient learns the necessary information through software learning independently, and medical staff assist the explanation from side to side.
(2) The hospital installs the software on intelligent equipment (such as a touch screen television and the like) arranged in the ward, and the patient and family members thereof can learn the software content by themselves (such as orthopedics rehabilitation training, COPD slow pulmonary resistance rehabilitation respiratory operations, hemodialysis patient nursing beard and the like).
(3) The medical science and education uses: through software content, standardized training is carried out on training doctors, the previously scattered medical training content sets are unified, standards are formulated, and precious experience of advanced doctors is quantified by means of technology.
Advantages are:
(1) The efficiency of patient management in hospitals can be improved, and the direct and effective data statistics of patients can be obtained;
(2) Improving the communication efficiency of doctors and patients and improving the medical service level; improving the awareness rate and signature rate of emergency patients;
(3) The experience of patients is improved, the authority image of medical institutions is improved, and the advancement of medical technology is improved;
(4) The memory coding rate of trained medical staff is improved, deep understanding is achieved, the trained medical staff can master quickly, and the learning curve is effectively shortened. The treatment standard is reached, the operation standard is unified, and misdiagnosis and unnecessary medical accidents caused by individual understanding differences are reduced.
In use, the medical staff:
(1) The main doctor can explain the operation to be performed on the patient before operation through software, quickly check the patient through the software so as to learn the understanding degree of the patient, record the understanding degree and upload the understanding degree to a database for summarization;
(2) Nurses can conduct preoperative and postoperative nursing propaganda and education to patients through software, help the patients to quickly master nursing knowledge and importance of the nursing knowledge, and quickly check through the software to obtain explanation effectiveness data, record the explanation effectiveness data and upload the explanation effectiveness data to a database for summarization;
(3) Through the special holographic presentation mode of software, medical staff can combine real people with the multidimensional content rendered by the software, shoot videos which can be used for social media and public media transmission, and carry out medical propaganda education on the public.
Advantages are:
(1) The method of doctor-patient communication is standardized;
(2) The correct knowledge of the patient and the family members thereof on the diseases can be increased, the knowledge of medical conditions, diagnosis, diseases or disabilities can be improved, and the fear psychology of the patient can be reduced;
(3) Can enhance the understanding of methods and means for various aspects of the medical condition and allow the patient to correctly recognize possible limitations in the medical technology;
(4) The system can obtain higher compliance, namely, quick and effective communication and patient education, improve the power of patient compliance and increase the trust degree of the patient on medical staff;
(5) Can help doctors and patients to generate mutual cooperative actions, improve the coordination degree and induce the health management consciousness of patients;
(6) Increasing the success of informed consent-patients more easily think that medical staff and institutions have provided the information they need;
(7) Can help medical institutions and medical staff to perform more effective risk management, and reduce the malpractice risk of the medical staff.
Patients and their family members:
(1) The patient and family members thereof can be used in medical institutions;
(2) The patient and family members thereof can download the disease knowledge aiming at own diseases or interested diseases on a download platform provided by a medical institution and repeatedly use the disease knowledge through own intelligent mobile terminal equipment (mobile phones, tablet computers and the like);
(3) The general knowledge of the medical science can be watched through the video on the public health system platform, and the learning is continued by downloading the software, so that the general knowledge of the medical science and the basic medical skill of the public can be improved; the software can be used to improve the correct processing capacity of diseases and the capacity of self-help and helping others.
Advantages are:
(1) The software can increase more effective doctor-patient communication, and the patient obtains correct medical education through the use of the software, so that the power of the patient to follow national medical services is improved;
(2) Can help patients and families better understand medical technology limitations, and can further understand risks possibly existing in treatment. Controlling the desire for medical services to a rationally accepted range;
(3) The medical nursing bed can help patients and family members thereof to understand the illness state more quickly, understand the necessity and rationality of medical services provided by hospitals and medical staff, and enhance the matching degree of the patients and the family members thereof;
(4) The patient can learn through software, and the treatment means of the hospital can be matched more effectively, so that the occurrence of complications and the probability of rescue delay are reduced, the medical basis is provided for self-management after treatment, invalid repeated medical treatment is reduced, and irrational dependence on the hospital is reduced;
(5) Can help the patient to more reasonably utilize medical service resources;
(6) The layout of the national medical policies, such as grading diagnosis and treatment, secondary referral and the like, can be better understood by the patient;
(7) The nursing bed can help the family members of the patients to provide better and better nursing outside the hospital for the patients and help the patients to recover early;
(8) The interest of the general knowledge of the public learning medicine can be increased, unnecessary medical risks are avoided, the self-management consciousness of the patient is enhanced, and the final outcome can be greatly improved.
The foregoing is only a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art, who is within the scope of the present invention, should make equivalent substitutions or modifications according to the technical scheme of the present invention and the inventive concept thereof, and should be covered by the scope of the present invention.

Claims (8)

1. The H.I.P.S. multi-touch advertising interaction system for medical treatment comprises a background terminal, an equipment terminal and an application program system, and is characterized in that the application program system comprises an SLAM implementation positioning module, an information interaction module, a monitoring module, a first database module and a testing module, and the background terminal comprises a second database module; wherein,,
the SLAM implementation positioning module comprises a laser SLAM and a depth camera-based SLAM, wherein the laser SLAM and the depth camera-based SLAM are used for directly acquiring point cloud data in an environment, generating the point cloud data and then transmitting the point cloud data to the first database module and the second database module;
the information interaction module comprises an information display unit and an audio dialogue unit, and is used for carrying out information interaction between users, wherein the information interaction mode between users comprises one or more of text input transmission, voice input transmission, real-time picture transmission of a camera unit and picture input transmission;
the monitoring module comprises a shooting scanning unit and an information acquisition unit, wherein the shooting scanning unit is used for scanning and extracting image information and transmitting stored information to a first database module of the application program system through signals; the first database module is used for identifying and analyzing the information in the operation room identified by scanning, transmitting analysis data information to the application program system and carrying out information interaction on users; the information acquisition unit is used for browsing the picture display information of the application program system by a user, browsing single data time to acquire information, transmitting the data to the first database module and the second database module for analysis, transmitting analysis data information to the application program system, and carrying out information interaction on the user;
the first database module comprises a data information collecting unit, a data storage unit, a data analysis unit and a data information feedback unit, and is used for receiving information transmitted by the monitoring module and the information interaction module, storing the information and analyzing the data, transmitting the information to the application program system and carrying out information interaction on a user;
the testing module comprises a questionnaire unit, a questionnaire information collecting unit, a questionnaire result generating unit and a testing information duplicate checking unit, and is used for enabling a user to know information deeply and confirm the information.
2. The h.i.p.s. multi-touch system for medical education of claim 1 wherein the camera scanning unit comprises a normal camera, a depth camera, or a laser camera, the normal camera comprising a monocular camera or a binocular RGB camera.
3. The h.i.p.s. multi-touch ventilating and teaching interaction system for medical treatment according to claim 1, wherein the application system supports supporting use of ar+vr+mr.
4. The h.i.p.s. multi-touch system for medical education of claim 1 wherein the information display unit is displayed in one or more of two-dimensional display, three-dimensional display, and four-dimensional display.
5. The h.i.p.s. multi-touch advertising interaction system for medical treatment according to any of claims 1-4, further providing a method of using the system, comprising the steps of:
s1, starting: the method comprises the steps that a user opens an application program through a device terminal to perform use operation, wherein the operation mode comprises one or more of gesture interaction, natural semantics and voice recognition interaction, and in the beginning, a positioning module and a wireless network are used for performing surrounding type scanning rapid analysis on a specific environment of a hospital where the user is located through SLAM, and an application program system is connected with background data of the hospital;
s2, browsing page information: after information login and mode selection are performed in the step S1, a page is displayed, wherein the page comprises picture information, text information, video information, audio information, three-dimensional model information, animation information and special effect information; the page information browsing content comprises one or more of current medical technology content, current symptom content, symptom treatment technology content, human symptom organ information content, medical mechanical equipment type identification content and operation real-time guidance content;
s3, content interaction: in the page browsing in the step S2, a user performs operation control on a page through page information to acquire interactive information, wherein the interactive information acquisition mode comprises one or more of page information browsing inquiry, window consultation and video voice consultation; according to different equipment terminals, the method for operating the control page comprises clicking a screen, gesture control, voice command or handle operation, wherein when clicking is performed, the timing unit collects information according to clicked data and sends the information to the application program system, and the data is not presented to a user;
s4, data statistics: after the page browsing in the step S2 is finished, the equipment terminal directly sends the page browsing to the background terminal under the condition of signals, a second database module of the background terminal stores the record, and if the equipment terminal does not have the signals, the page browsing is temporarily stored in a first database module of an application program system for storage.
6. The h.i.p.s. multi-touch system for medical treatment according to claim 5, wherein the user is presented with an introduction and a service agreement during the beginning of step S1, wherein the service agreement has a clear introduction of the data content monitored by the monitoring module; the equipment terminals used by the user comprise one or more of mobile phones, tablet personal computers, intelligent wearable equipment, intelligent glasses, touch screen computers and televisions.
7. The h.i.p.s. multi-touch teaching interaction system for medical treatment according to claim 5, wherein the manner of the operation control page in step S3 comprises: zoom in, zoom out, move, rotate, edit.
8. The h.i.p.s. multi-touch system for medical treatment according to claim 5, wherein the page is automatically adjustable according to the movement of the user during the page browsing in step S2.
CN202010290090.8A 2020-04-14 2020-04-14 H.I.P.S multi-touch teaching interaction system for medical treatment Active CN111613280B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010290090.8A CN111613280B (en) 2020-04-14 2020-04-14 H.I.P.S multi-touch teaching interaction system for medical treatment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010290090.8A CN111613280B (en) 2020-04-14 2020-04-14 H.I.P.S multi-touch teaching interaction system for medical treatment

Publications (2)

Publication Number Publication Date
CN111613280A CN111613280A (en) 2020-09-01
CN111613280B true CN111613280B (en) 2023-04-28

Family

ID=72199666

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010290090.8A Active CN111613280B (en) 2020-04-14 2020-04-14 H.I.P.S multi-touch teaching interaction system for medical treatment

Country Status (1)

Country Link
CN (1) CN111613280B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112562870B (en) * 2020-12-03 2024-05-28 厦门大学附属中山医院 Evaluation early warning system for enhancing doctor-patient mutual trust and reducing doctor-patient contradiction
CN112820419B (en) * 2021-01-27 2023-10-31 泰安市康福宝医疗科技有限公司 Preoperative ventilating and teaching system for hospitals

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106845120A (en) * 2017-01-19 2017-06-13 杭州古珀医疗科技有限公司 A kind of Telemedicine System and its operating method based on mixed reality technology
CN106981040A (en) * 2017-02-21 2017-07-25 中国人民解放军第三军医大学第附属医院 Health propaganda and education system
CN107480462A (en) * 2017-09-05 2017-12-15 上海亲睦医院管理有限公司 Wisdom clinic interactive system
CN108154927A (en) * 2017-12-21 2018-06-12 华子昂 A kind of intelligence traditional Chinese medical science robot software's framework
CN110021203A (en) * 2019-04-23 2019-07-16 南方医科大学口腔医院 A kind of Oral healthy education experiencing system, method and medical education device

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20200005949A1 (en) * 2018-02-20 2020-01-02 Pristine Surgical, Llc Engagement and Education of Patients for Endoscopic Surgery

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106845120A (en) * 2017-01-19 2017-06-13 杭州古珀医疗科技有限公司 A kind of Telemedicine System and its operating method based on mixed reality technology
CN106981040A (en) * 2017-02-21 2017-07-25 中国人民解放军第三军医大学第附属医院 Health propaganda and education system
CN107480462A (en) * 2017-09-05 2017-12-15 上海亲睦医院管理有限公司 Wisdom clinic interactive system
CN108154927A (en) * 2017-12-21 2018-06-12 华子昂 A kind of intelligence traditional Chinese medical science robot software's framework
CN110021203A (en) * 2019-04-23 2019-07-16 南方医科大学口腔医院 A kind of Oral healthy education experiencing system, method and medical education device

Also Published As

Publication number Publication date
CN111613280A (en) 2020-09-01

Similar Documents

Publication Publication Date Title
Rachmad MediVerse: Challenges And Development Of Digital Health Transformation Towards Metaverse in Medicine
Gorman et al. Simulation and virtual reality in surgical education: real or unreal?
CN107480462B (en) Intelligent clinical interaction system
US20030022141A1 (en) Interactive patient educational tool
Nepal et al. A framework for telehealth program evaluation
Ibrahim et al. Computer mediated reality technologies: A conceptual framework and survey of the state of the art in healthcare intervention systems
CN111613280B (en) H.I.P.S multi-touch teaching interaction system for medical treatment
CN104246857A (en) Medical procedure training system
Deutsch et al. Leveraging health care simulation technology for human factors research: closing the gap between lab and bedside
Joseph et al. Communication and teamwork during telemedicine-enabled stroke care in an ambulance
Zhou et al. [Retracted] VR/AR Technology in Human Anatomy Teaching and Operation Training
CN110111862A (en) Information platform
CN115917492A (en) Method and system for video collaboration
JP2016018224A (en) Service system for using health condition prediction diagnosis
Farahat et al. The Implication of Metaverse in the Traditional Medical Environment and Healthcare Sector: Applications and Challenges
Mohamed et al. The use of metaverse in the healthcare sector: Analysis and applications
CN111192655B (en) Online medical method and system, electronic equipment and computer storage medium
JP2016194681A (en) Physical assessment education service providing system and physical assessment education service providing method
US10296529B2 (en) Automated advanced resuscitation training generation system and method therefor
US20140315172A1 (en) Systems and methods for interactive scenario-based medical instruction
Pamplin et al. Prolonged, high-fidelity simulation for study of patient care in resource-limited medical contexts and for technology comparative effectiveness testing
Gupta Investigation of a holistic human-computer interaction (HCI) framework to support the design of extended reality (XR) based training simulators
Kanak et al. An IoT-based triangular methodology for plastic surgery simulation enriched with augmented and virtual reality
Shluzas et al. Mobile augmented reality for distributed healthcare
Welch et al. 3D medical collaboration technology to enhance emergency healthcare

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant