CN108154778A - Based on motion-captured and mixed reality ophthalmologic operation training system and method - Google Patents
Based on motion-captured and mixed reality ophthalmologic operation training system and method Download PDFInfo
- Publication number
- CN108154778A CN108154778A CN201711466016.1A CN201711466016A CN108154778A CN 108154778 A CN108154778 A CN 108154778A CN 201711466016 A CN201711466016 A CN 201711466016A CN 108154778 A CN108154778 A CN 108154778A
- Authority
- CN
- China
- Prior art keywords
- mixed reality
- laser pulse
- operation tool
- central processing
- processing unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012549 training Methods 0.000 title claims abstract description 94
- 238000000034 method Methods 0.000 title claims abstract description 42
- 238000012545 processing Methods 0.000 claims abstract description 57
- 210000005252 bulbus oculi Anatomy 0.000 claims abstract description 26
- 241001465754 Metazoa Species 0.000 claims abstract description 23
- 238000001356 surgical procedure Methods 0.000 claims abstract description 21
- 238000012937 correction Methods 0.000 claims abstract description 9
- 238000004891 communication Methods 0.000 claims description 43
- 210000001508 eye Anatomy 0.000 claims description 27
- 230000002093 peripheral effect Effects 0.000 claims description 3
- 230000000694 effects Effects 0.000 abstract description 5
- 238000005516 engineering process Methods 0.000 description 12
- 210000003128 head Anatomy 0.000 description 4
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 3
- 238000010586 diagram Methods 0.000 description 3
- 230000005611 electricity Effects 0.000 description 3
- 229910052710 silicon Inorganic materials 0.000 description 3
- 239000010703 silicon Substances 0.000 description 3
- 230000003190 augmentative effect Effects 0.000 description 2
- 239000011521 glass Substances 0.000 description 2
- 230000003902 lesion Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000001953 sensory effect Effects 0.000 description 2
- 208000002177 Cataract Diseases 0.000 description 1
- 241000905957 Channa melasoma Species 0.000 description 1
- 230000005483 Hooke's law Effects 0.000 description 1
- 210000001367 artery Anatomy 0.000 description 1
- 238000005452 bending Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 239000012141 concentrate Substances 0.000 description 1
- 210000004087 cornea Anatomy 0.000 description 1
- 238000007405 data analysis Methods 0.000 description 1
- 230000002950 deficient Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 201000010099 disease Diseases 0.000 description 1
- 208000037265 diseases, disorders, signs and symptoms Diseases 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 210000000744 eyelid Anatomy 0.000 description 1
- 230000008713 feedback mechanism Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000005457 optimization Methods 0.000 description 1
- 210000000056 organ Anatomy 0.000 description 1
- 229920000728 polyester Polymers 0.000 description 1
- 239000007787 solid Substances 0.000 description 1
- 238000010561 standard procedure Methods 0.000 description 1
- 238000002054 transplantation Methods 0.000 description 1
- 210000003462 vein Anatomy 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
- G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
- G09B23/00—Models for scientific, medical, or mathematical purposes, e.g. full-sized devices for demonstration purposes
- G09B23/28—Models for scientific, medical, or mathematical purposes, e.g. full-sized devices for demonstration purposes for medicine
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computational Mathematics (AREA)
- Mathematical Analysis (AREA)
- Medicinal Chemistry (AREA)
- General Health & Medical Sciences (AREA)
- Algebra (AREA)
- Health & Medical Sciences (AREA)
- Chemical & Material Sciences (AREA)
- Medical Informatics (AREA)
- Mathematical Optimization (AREA)
- Mathematical Physics (AREA)
- Pure & Applied Mathematics (AREA)
- Business, Economics & Management (AREA)
- Educational Administration (AREA)
- Educational Technology (AREA)
- Theoretical Computer Science (AREA)
- Processing Or Creating Images (AREA)
Abstract
The present invention provides a kind of based on motion-captured and mixed reality ophthalmologic operation training system and method.The front 3-D view for acquiring animal eyeball sample in surgical procedure in real time by 3D photographic microscopes based on motion-captured and mixed reality ophthalmologic operation training system, by the way that pulsed laser detector is set to position the position of the operation tool on operation tool, the position of the operation tool and the difference of normal place are compared by central processing unit, information is corrected to out position, tutorial message is provided according to procedure by central processing unit, and position correction information and tutorial message are added in the front 3-D view of the 3D photographic microscopes acquisition, it is shown by mixed reality display terminal, the sense of reality and telepresenc of ophthalmologic operation training can be promoted, improve the real-time instruction effect in surgery training, shorten the ophthalmologic operation training period, reduce ophthalmologic operation training cost, promote training quality.
Description
Technical field
The present invention relates to medical training technical fields more particularly to a kind of based on motion-captured and mixed reality ophthalmology hand
Art training system and method.
Background technology
Eyes are the very important organs of head part, extremely fragile since its is complicated, position on the head of people, because
This ophthalmologic operation carried out to eyes is one kind in the most difficult operation of hospital, doctor is needed to have in order to complete ophthalmologic operation
There is high technology.The shared oculist 28000 or so in the whole nation at present, but can carry out intraocular surgery less than 4000 people, energy
For the doctor of the highly difficult ophthalmologic operations such as enough complete independently corneal transplantations less than hundred people, oculist's resource is still very deficient
It is weary so that the treatment of ophthalmology disease receives very big limitation, and how more, quickly the doctor of culture grasp ophthalmologic operation becomes
The a difficult problem urgently to be solved.
Traditional ophthalmologic operation Training Methodology is mainly to concentrate instruction, demonstration teaching, substitute (eyeball as selected pig)
Rehearsal and on-the-spot guidance formula practical operation, during due to training student's viewing angle and doctor's work angle have differences, demonstration teaching people
Reasons, the even senior oculists such as number is limited, insufficient, the live practical operation risk of substitute rehearsal telepresenc is big grasp this hand
Art is required for the long period, and the cultivation cycle of young doctor is just more very long.The problem of virtual ophthalmologic operation training is equipment
With program complex and expensive, and simulating degree or relatively large distance with really performing the operation, the sense of reality is not strong.
Virtual reality (Virtual Reality, VR) technology is a kind of computer that can be created with the experiencing virtual world
Emulation technology, it creates a Virtual Space using computer, and user can be immersed to the Virtual Space completely by VR glasses
In, using Binocular Vision Principle, Virtual Space is 3D solids in glasses, on the spot in person so as to achieve the effect that.By virtual
Augmented reality (Augmented Reality, AR) technology that reality technology develops is to really by virtual information superposition
It in the world, is perceived by human sensory, so as to reach the sensory experience of exceeding reality, different from virtual reality technology, using increasing
Strong reality technology user can not only see the virtual information generated by computer, additionally it is possible to see the true letter in the existing world
Breath, telepresenc and the sense of reality are all stronger.Mixed reality technology (Mixed Reality, MR) combines AR technologies and VR technologies, it
For the image transmission for the real physical world that optical lens is taken to processor, processor carries out image procossing to the image,
And enhancing superposition is carried out, since processing speed is fast, shown in display equipment is immersed, by the figure for being superimposed and optimizing
Picture with the time delay of real physical world in a few tens of milliseconds, can be regarded as the real-time imaging of real physical world completely.
Invention content
It, can the purpose of the present invention is to provide a kind of based on motion-captured and mixed reality ophthalmologic operation training system
True reappearance ophthalmologic operation process, and real-time instruction is carried out to start-up, the sense of reality of ophthalmologic operation training is promoted with coming personally
The ophthalmologic operation training period is shortened in sense, reduces ophthalmologic operation training cost.
It is a kind of based on motion-captured and mixed reality ophthalmologic operation Training Methodology, energy the present invention also aims to provide
Enough true reappearance ophthalmologic operation processes, and real-time instruction is carried out to start-up, the sense of reality of ophthalmologic operation training is promoted with facing
Field sense, shortens the ophthalmologic operation training period, reduces ophthalmologic operation training cost.
To achieve the above object, the present invention provides a kind of based on motion-captured and mixed reality ophthalmologic operation training system
System, including：Emulation operating table, the animal eyeball sample on the emulation operating table, the corresponding animal eyeball sample are set
3D photographic microscopes, central processing unit, the corresponding animal eyeball sample with 3D photographic microscopes communication connection put
What is set and at least three laser pulse emission devices are connect with the central processing unit communication, connect with central processing unit communication is mixed
Close real display terminal and the operation tool being connect with the central processing unit communication；
The operation tool is equipped with laser pulse detector and equal with the laser pulse detector and central processing unit
Communicate the communication module of connection；
The 3D photographic microscopes for shooting the front 3-D view of animal eyeball sample in surgical procedure, and transmit
To central processing unit；
The laser pulse emission device, for generating and sending laser pulse signal；
The laser pulse detector, for detecting and receiving the laser pulse letter that each laser pulse emission device is sent out
Number, obtain location data；
The communication module, for the location data of operation tool to be transferred to central processing unit；
The central processing unit, for providing the corresponding guidance letter being pre-stored in central processing unit according to the process of operation
Breath, and according to the location data of the operation tool, obtains the location information of the operation tool, and by the position of operation tool
Information and preset standard location information are compared in real time, are corrected information to out position and are entangled tutorial message and position
Positive information is added on the front 3-D view of 3D photographic microscopes acquisition, generates mixed reality picture signal and is sent to mixing
Real display terminal；
The mixed reality display terminal, for carrying out 3D display, start-up according to the mixed reality picture signal
Ophthalmologic operation training is carried out according to the image that the mixed reality display terminal is shown.
The operation tool includes：Deformation class operation tool and non-deformation class operation tool are performed the operation in each deformation class
The deformation position of tool is additionally provided with posture and captures sensor, and the posture captures sensor and connected with communication module communication；
The posture captures sensor, for acquiring the deformation data of the deformation class operation tool；
The communication module is additionally operable to the deformation data of deformation class operation tool being transferred to central processing unit；
The central processing unit is additionally operable to the deformation data according to the deformation class operation tool, obtains the deformation class
The attitude information of operation tool, and the attitude information of deformation class operation tool and preset standard attitude information are compared in real time
It is right, it provides posture and corrects information, and posture correction information and tutorial message, position correction information are added to 3D camera shootings and shown together
On the front 3-D view of micro mirror acquisition, generate mixed reality picture signal and be sent to mixed reality display terminal.
The deformation class operation tool includes：Surgical glove and surgical scissors, the non-deformation class operation tool include：Hand
Art knife.
It is miniature elastic resistance that the posture, which captures sensor, and the posture captures sensor according to miniature elastic electricity
The deformation data of the resistance variations acquisition operation tool of resistance.
The communication module is wireless communication module.
The laser pulse letter that the quantity of the laser pulse emission device is three and three laser pulse emission devices are sent out
Number frequency it is different.
On the peripheral surface of camera lens that three laser pulse emission devices are uniformly distributed in the 3D photographic microscopes.
The mixed reality display terminal includes：Left eye display screen and right eye display screen, the mixed reality picture signal
Including：Left-eye image signal and eye image signal, the left eye display screen and right eye display screen are respectively used to display left eye figure
As signal and eye image signal, to realize 3D display.
The laser pulse detector is detected with to be more than the rate of 10 pulse/secs and receives each laser pulse emission device and sent out
The laser pulse signal gone out.
The present invention also provides a kind of based on motion-captured and mixed reality ophthalmologic operation Training Methodology, applied to above-mentioned
Based on motion-captured and mixed reality ophthalmologic operation training system, include the following steps：
Step S1,3D photographic microscopes shoot the front 3-D view of animal eyeball sample in surgical procedure, and are transmitted to
Central processing unit；
Step S2, laser pulse emission device generates and sends laser pulse signal, and the laser pulse detector detecting is simultaneously
The laser pulse signal that each laser pulse emission device is sent out is received, obtains location data；
The location data of operation tool is transferred to central processing unit by step S3, described communication module；
Step S4, central processing unit provides the corresponding guidance letter being pre-stored in central processing unit according to the process of operation
Breath, and according to the location data of the operation tool, obtains the location information of the operation tool, and by the position of operation tool
Information and preset standard location information are compared in real time, and information is corrected, and tutorial message and position are corrected to out position
Information is added on the front 3-D view of 3D photographic microscopes acquisition, generates mixed reality picture signal and is sent to mixing now
Real display terminal；
Step S5, mixed reality display terminal carries out 3D display, start-up's root according to the mixed reality display terminal
Ophthalmologic operation training is carried out according to the image that the mixed reality display terminal is shown.
Beneficial effects of the present invention：The present invention provides a kind of based on motion-captured and mixed reality ophthalmologic operation training
System, it is described that operation is acquired based on motion-captured and mixed reality ophthalmologic operation training system in real time by 3D photographic microscopes
The front 3-D view of animal eyeball sample in the process, by pulsed laser detector being set to position the hand on operation tool
The position of art tool is compared the position of the operation tool and the difference of normal place by central processing unit, is entangled to out position
Positive information provides tutorial message, and position is corrected information and tutorial message superposition by central processing unit according to procedure
In the front 3-D view acquired to the 3D photographic microscopes, shown by mixed reality display terminal, eye can be promoted
The sense of reality and telepresenc of section's surgery training improve the real-time instruction effect in surgery training, shorten the ophthalmologic operation training period,
Ophthalmologic operation training cost is reduced, promotes training quality.The present invention also provides a kind of based on motion-captured and mixed reality eye
Section's surgery training method, can true reappearance ophthalmologic operation process, and to start-up carry out real-time instruction, promoted ophthalmologic operation
The sense of reality and telepresenc of training shorten the ophthalmologic operation training period, reduce ophthalmologic operation training cost.
Description of the drawings
For further understanding of the features and technical contents of the present invention, it please refers to below in connection with the detailed of the present invention
Illustrate and attached drawing, however, the drawings only provide reference and explanation, is not intended to limit the present invention.
In attached drawing,
Fig. 1 is the module diagram based on motion-captured and mixed reality ophthalmologic operation training system of the present invention；
Fig. 2 is the structure diagram based on motion-captured and mixed reality ophthalmologic operation training system of the present invention；
Fig. 3 is the present invention based on operation tool and laser in motion-captured and mixed reality ophthalmologic operation training system
The structure diagram of impulse sender；
Fig. 4 is the flow chart based on motion-captured and mixed reality ophthalmologic operation Training Methodology of the present invention.
Specific embodiment
The technological means and its effect taken further to illustrate the present invention, below in conjunction with the preferred implementation of the present invention
Example and its attached drawing are described in detail.
It please refers to Fig.1 and Fig. 2, the present invention provides a kind of based on motion-captured and mixed reality ophthalmologic operation training system
System, including：Emulate operating table 1, the animal eyeball sample 2 on the emulation operating table 1, the corresponding animal eyeball sample
The 3D photographic microscopes 3 of 2 settings, central processing unit 4, the corresponding animal eyes with the 3D photographic microscopes 3 communication connection
At least three laser pulse emission devices 5 and central processing unit 4 that ball sample 2 sets and connected with the central processing unit 4 communication
Communicate the mixed reality display terminal 6 of connection and the operation tool 7 with the central processing unit 4 communication connection.
Wherein, the operation tool 7 includes：Deformation class operation tool and non-deformation class operation tool, in each non-shape
Become on class operation tool and be equipped with laser pulse detector 72 and equal with the laser pulse detector 72 and central processing unit 4
The communication module 73 of connection is communicated, laser pulse detector 72 is equipped on each deformation class operation tool, is swashed with described
Light pulse detector 72 and central processing unit 4 communicate the communication module 73 of connection and positioned at the deformation class operation tools
Deformation position and the posture capture sensor 71 connected with the communication module 73 communication.
For example, as shown in Figures 2 and 3, the non-deformation class operation tool can include：Scalpel 703, described
Scalpel 703 is equipped with laser pulse detector 72 and is communicated with the laser pulse detector 72 and central processing unit 4
The communication module 73 of connection.The deformation class operation tool can include：Surgical glove 701 and surgical scissors 702, in the hand
The back of the hand position of art gloves 701 is equipped with laser pulse detector 72 and the laser pulse detector 72 and central processing unit 4 is equal
The communication module 73 of connection is communicated, the position between described each articulations digitorum manus of surgical glove 701 and two fingers is equipped with one
A posture captures sensor 71, and multiple attitude transducers 71 on same finger are connected in series with；In the surgical scissors 702
Center be equipped with laser pulse detector 72, communicate connection with the laser pulse detector 72 and central processing unit 4
Communication module 73, the rendezvous position of two blades of the surgical scissors 702 set there are one posture capture sensor 71.
Specifically, the 3D photographic microscopes 3, for shooting the front graphics of animal eyeball sample 2 in surgical procedure
Picture, and it is transmitted to central processing unit 4；
The laser pulse emission device 5, for generating and sending laser pulse signal；The laser pulse detector 72,
For detecting and receiving the laser pulse signal that each laser pulse emission device 5 is sent out, location data is obtained；
The communication module 73, for the location data of operation tool 7 to be transferred to central processing unit 4；The centre
Device 4 is managed, for providing the corresponding tutorial message being pre-stored in central processing unit 4 according to the process of operation, and according to the hand
The location data of art tool 7, obtains the location information of the operation tool 7, and by the location information of operation tool 7 and preset
Standard location information is compared in real time, is corrected information to out position and is added to tutorial message and position correction information
On the front 3-D view that 3D photographic microscopes 3 acquire, generate mixed reality picture signal and be sent to mixed reality display eventually
End 6；
The mixed reality display terminal 6, for carrying out 3D display, start-up according to the mixed reality picture signal
Ophthalmologic operation training is carried out according to the image that the mixed reality display terminal 6 is shown.
Preferably, the communication module 73 is wireless communication module, and the present invention is not limited thereto, the communication module
73 under some necessary occasions or wire communication module.
Specifically, the quantity of the laser pulse emission device 5 is three and what three laser pulse emission devices 5 were sent out swashs
The frequency of light pulse signal is different, and during implementation, three laser pulse emission devices 5 are uniformly distributed in the 3D photographic microscopes
On the peripheral surface of 3 camera lens, two adjacent laser pulse emission devices 5 are spaced 120 °, during work, three laser pulse emissions
Device 5 continuously sends out pulse signal, when the laser pulse detector 72 being fixed on operation tool 7 detects three group pulse signals,
According to triangle polyester fibre algorithm, the spatial position coordinate of the operation tool where it is calculated, obtains the positioning number of the operation tool
According to.
Preferably, the laser pulse detector 72 is detected with to be more than the rate of 10 pulse/secs and receives each laser arteries and veins
The laser pulse signal that transmitter 5 is sent out is rushed, so as to the position of at least 10 times operation tools 7 of detection per second during surgery training
Confidence ceases, and locating speed and positioning accuracy are all very high.
In addition, when the operation tool 7 is deformation class operation tool, appearance can be also set on the deformation class operation tool
State captures sensor 71；The posture captures sensor 71, for acquiring the deformation data of the deformation class operation tool；It is described
Communication module 73 is additionally operable to the deformation data of deformation class operation tool being transferred to central processing unit 4；The central processing unit 4,
The deformation data according to the deformation class operation tool is additionally operable to, obtains the attitude information of the deformation class operation tool, and will
The attitude information of deformation class operation tool and preset standard attitude information are compared in real time, are provided posture and are corrected information, and
Posture is corrected into information and tutorial message, position correction information are added to the front graphics that 3D photographic microscopes 3 acquire together
As upper, generation mixed reality picture signal is simultaneously sent to mixed reality display terminal 6.
Specifically, it is miniature elastic resistance that the posture, which captures sensor 71, and the posture captures sensor 71 according to institute
State the deformation data of the resistance variations acquisition operation tool 7 of miniature elastic resistance, the miniature elastic resistance, when both ends stress
When, it can be stretched, but its volume V is remained unchanged, i.e. V=L*S, wherein L are miniature elastic resistance length, and S is miniature elastic electricity
Cross-sectional area is hindered, according to resistance formula R=ρ L/S and Hooke's law F=K* Δ L, wherein, R is the electricity of the miniature elastic resistance
Resistance, ρ are the resistivity of miniature elastic resistance, and F is the external force size being subject to of the miniature elastic resistance, and K is the miniature bullet
Property resistance coefficient of elasticity, Δ L be length variable quantity of the miniature elastic resistance before and after deformation, so as to according to miniature elastic
The change in resistance of resistance represents the stressing conditions for the deformation position for obtaining the deformation operation tool, so as to according to power
Size converses the deformation situation of deformation operation tool, for example, on surgical glove 701 finger bending degree or surgical scissors
The distance between 702 two blades etc. so as to accordingly be modeled for different deformation operation tools, obtain deformation hand
The deformation data of art tool.
Specifically, the emulation operating table 1 copies the operating table of true ophthalmologic operation to make, so that start-up
The experience when participating in the cintest really performed the operation is obtained,
Particularly, in order to further enhance the sense of reality of operation, human head model 9 is equipped on the emulation operating table 1,
During operation, animal eyeball sample 2 is put in the eye of the human head model 9, at the same with the operating table phase of true ophthalmologic operation
Together, which also is provided with foot-operated 8, the focal length that can adjust the shooting of 3D photographic microscopes 3 by foot-operated 8 and position etc.
Parameter.
Preferably, as shown in Fig. 2, the 3D photographic microscopes 3 are corresponded to positioned at the surface of the animal eyeball sample 2,
To acquire the front 3-D view consistent with start-up visual angle, the sense of reality of operation is promoted.
Specifically, storage unit is equipped in the central processing unit 4, the storage unit preserves the mark modeled in advance
Accurate ophthalmologic operation process, the tutorial message of each process, the standard position signal of each process and attitude information, a variety of ophthalmology
Lesion and a variety of operation accidents are believed by the position for obtaining laser pulse detector 72 and posture capture sensor 71
Breath is compared in real time with attitude information and the standard position signal and attitude information of each process, and real time correction is to start-up
Wrong surgical action in surgical procedure.
Wherein, the ophthalmologic operation process of the standard, the tutorial message of each process, each process standard procedures move
Make, a variety of ophthalmology lesions and it is a variety of operation accident with reference to a large amount of expert perform the operation case formulated, guarantee perform the operation into
The accuracy of journey and the diversity of training and practicability, by adding in what accident was handled in the ophthalmologic operation process of standard
Situation, start-up can obtain the experience for handling a variety of operation accidents, so as to reach better result of training.By swashing
Light pulse detector 72 and posture capture the application of sensor 71 so that start-up is able to know that the surgical procedure action of oneself
It is whether consistent with standard operation.
In addition, the voice module with the communication connection of central processing unit 4 can also be set, in training process in the system
In, tutorial message, position correction information and posture are reported for start-up by real-time voice module and correct information, so as to
To accelerate to train speed, training quality is promoted.
Specifically, the mixed reality display terminal 6 includes：Left eye display screen 61 and right eye display screen 62, the mixing
Real world images signal includes：Left-eye image signal and eye image signal, the left eye display screen 61 and right eye display screen 62 divide
Left-eye image signal and eye image signal Yong Yu not be shown, to realize 3D display.
Preferably, the left-eye image display screen 61 and eye image display screen 62 are liquid crystal on silicon (Liquid
Crystal on Silicon, LCOS) display screen, with using LCOS display screens are in bright gay color and clarity is high the characteristics of show height
The image of quality.Certainly, the left-eye image display screen 61 and eye image display screen 62 are not limited to LCOS display screens,
Such as silicon-based organic light-emitting diode (Si-OLED) display screen or micro- light emitting diode (Micro-LED) display screen may be used
Etc. other kinds of display screen.
Preferably, the animal eyeball sample 2 is pig eyeball, while surgery training authenticity is taken into account, reduces operation
The cost of training.
Specifically, this system can further include：With the remote guide mould of the central processing unit 4 communication connection
Block, the remote guide module, the image for mixed reality display terminal 6 to be shown are uploaded to high in the clouds via internet, with
The procedure of start-up can be obtained in real time beyond the clouds for the training instruction personnel of the profession such as ophthalmologist, oculist and ophthalmology professor,
Long-range real-time instruction is carried out to it so that training process it is more various with it is flexible, promote the efficiency of surgery training.
Preferably, the animal eyeball sample 2 is pig eyeball, while surgery training authenticity is taken into account, reduces operation
The cost of training, since the present invention uses true animal eyeball sample 2, compared to using including containing various force feedback mechanisms
Deng ARTIFICIAL EYELID wherein and artificial scalpel and the ophthalmologic operation training system of progress compare, not only greatly reduce and be
System cost, and the feel of closer actual operation so that the value higher of training, start-up is straight when being trained compared to tradition
It connects viewing or is performed the operation by the ancillary equipments direct viewing eyeball such as simple microscope, the present invention can ensure the sense of reality
Under the premise of, tutorial message is provided to start-up also as the process of operation and correct information, promote the efficiency and quality of training.
Specifically, the present invention can also set History Parser Module, and it is each to record start-up by History Parser Module
The process of secondary training, and big data analysis is carried out to record, the training data such as difficult point and result of training are obtained, to Optimization Training
Scheme accelerates training process, promotes training efficiency.
It is noted that it present invention can be suitably applied to all kinds of ophthalmology including cornea operation and cataract operation etc.
Operation, it is preferable that invention which are intended primarily for the trainings of the emergency situations in ophthalmologic operation process.
Referring to Fig. 4, the present invention also provides a kind of based on motion-captured and mixed reality ophthalmologic operation Training Methodology, it should
For above-mentioned based on motion-captured and mixed reality ophthalmologic operation training system, include the following steps：
Step S1,3D photographic microscopes 3 shoot the front 3-D view of animal eyeball sample 2 in surgical procedure, and transmit
To central processing unit 4；
Step S2, laser pulse emission device 5 generates and sends laser pulse signal, and the laser pulse detector 72 is detected
And the laser pulse signal that each laser pulse emission device 5 is sent out is received, obtain location data；
Step S3, the location data of operation tool 7 is transferred to central processing unit 4 by communication module 73；
Step S4, central processing unit 4 provides the corresponding guidance letter being pre-stored in central processing unit 4 according to the process of operation
Breath, and according to the location data of the operation tool 7, obtains the location information of the operation tool 7, and by operation tool 7
Location information and preset standard location information are compared in real time, out position are given to correct information, and by tutorial message and position
It corrects information to be added on the front 3-D view of the acquisition of 3D photographic microscopes 3, generate mixed reality picture signal and is sent to
Mixed reality display terminal 6；
Step S5, mixed reality display terminal 6 carries out 3D display, start-up's root according to the mixed reality picture signal
Ophthalmologic operation training is carried out according to the image of the mixed reality display terminal 6 display.
In conclusion the present invention provides a kind of based on motion-captured and mixed reality ophthalmologic operation training system, institute
It states and is acquired in surgical procedure in real time by 3D photographic microscopes based on motion-captured and mixed reality ophthalmologic operation training system
The front 3-D view of animal eyeball sample, by pulsed laser detector being set to position the operation tool on operation tool
Position, the position of the operation tool and the difference of normal place are compared by central processing unit, give out position correct information,
Tutorial message is provided according to procedure by central processing unit, and information is corrected into position and tutorial message is added to the 3D
It in the front 3-D view of photographic microscope acquisition, is shown by mixed reality display terminal, ophthalmologic operation training can be promoted
The sense of reality and telepresenc of instruction improve the real-time instruction effect in surgery training, shorten the ophthalmologic operation training period, reduce ophthalmology
Surgery training cost promotes training quality.The present invention also provides a kind of based on motion-captured and mixed reality ophthalmologic operation training
Method for training, can true reappearance ophthalmologic operation process, and to start-up carry out real-time instruction, promoted ophthalmologic operation training it is true
True feeling and telepresenc shorten the ophthalmologic operation training period, reduce ophthalmologic operation training cost
The above, for those of ordinary skill in the art, can according to the technique and scheme of the present invention and technology
Other various corresponding changes and deformation are made in design, and all these changes and deformation should all belong to the claims in the present invention
Protection domain.
Claims (10)
- It is 1. a kind of based on motion-captured and mixed reality ophthalmologic operation training system, which is characterized in that including：Emulate operating table (1), the 3D that the animal eyeball sample (2) on the emulation operating table (1), the corresponding animal eyeball sample (2) are set Photographic microscope (3), central processing unit (4), the corresponding animal eyeball mark with the 3D photographic microscopes (3) communication connection This (2) setting and at least three laser pulse emission devices (5) and central processing connected with the central processing unit (4) communication The mixed reality display terminal (6) of device (4) communication connection and the operation tool with the central processing unit (4) communication connection (7)；The operation tool (7) be equipped with laser pulse detector (72) and with the laser pulse detector (72) and center Processor (4) communicates the communication module (73) of connection；The 3D photographic microscopes (3), for shooting the front 3-D view of animal eyeball sample (2) in surgical procedure, and pass Transport to central processing unit (4)；The laser pulse emission device (5), for generating and sending laser pulse signal；The laser pulse detector (72), for detecting and receiving the laser pulse that each laser pulse emission device (5) sends out Signal obtains location data；The communication module (73), for the location data of operation tool (7) to be transferred to central processing unit (4)；The central processing unit (4), for providing the corresponding guidance being pre-stored in central processing unit (4) according to the process of operation Information, and according to the location data of the operation tool (7), obtain the location information of the operation tool (7), and the work that will perform the operation The location information and preset standard location information for having (7) are compared in real time, are corrected information to out position and are believed guidance Breath and position are corrected information and are added on the front 3-D view of 3D photographic microscopes (3) acquisition, generate mixed reality image letter Number and be sent to mixed reality display terminal (6)；The mixed reality display terminal (6), for carrying out 3D display, trainer according to the mixed reality display image signals Member carries out ophthalmologic operation training according to the image that the mixed reality display terminal (6) shows.
- 2. as described in claim 1 based on motion-captured and mixed reality ophthalmologic operation training system, which is characterized in that institute Operation tool (7) is stated to include：Deformation class operation tool and non-deformation class operation tool, in the shape of each deformation class operation tool Become position and be additionally provided with posture capture sensor (71), the posture captures sensor (71) and connects with the communication module (73) communication It connects；The posture captures sensor (71), for acquiring the deformation data of the deformation class operation tool；The communication module (73) is additionally operable to the deformation data of deformation class operation tool being transferred to central processing unit (4)；The central processing unit (4), is additionally operable to the deformation data according to the deformation class operation tool, obtains the deformation class hand The attitude information of art tool, and the attitude information of deformation class operation tool and preset standard attitude information are compared in real time It is right, it provides posture and corrects information, and posture correction information and tutorial message, position correction information are added to 3D camera shootings and shown together On the front 3-D view of micro mirror (3) acquisition, generate mixed reality picture signal and be sent to mixed reality display terminal (6).
- 3. as claimed in claim 2 based on motion-captured and mixed reality ophthalmologic operation training system, which is characterized in that institute Deformation class operation tool is stated to include：Surgical glove (701) and surgical scissors (702), the non-deformation class operation tool include：Hand Art knife (703).
- 4. as claimed in claim 2 based on motion-captured and mixed reality ophthalmologic operation training system, which is characterized in that institute It is miniature elastic resistance to state posture and capture sensor (71), and the posture captures sensor (71) according to the miniature elastic resistance Resistance variations acquisition operation tool (7) deformation data.
- 5. as described in claim 1 based on motion-captured and mixed reality ophthalmologic operation training system, which is characterized in that institute Communication module (73) is stated as wireless communication module.
- 6. as described in claim 1 based on motion-captured and mixed reality ophthalmologic operation training system, which is characterized in that institute The laser pulse signal that the quantity for stating laser pulse emission device (5) is three and three laser pulse emission devices (5) send out Frequency is different.
- 7. as claimed in claim 6 based on motion-captured and mixed reality ophthalmologic operation training system, which is characterized in that institute On the peripheral surface for stating the camera lens that three laser pulse emission devices (5) are uniformly distributed in the 3D photographic microscopes (3).
- 8. as described in claim 1 based on motion-captured and mixed reality ophthalmologic operation training system, which is characterized in that institute Mixed reality display terminal (6) is stated to include：Left eye display screen (61) and right eye display screen (62), the mixed reality picture signal Including：Left-eye image signal and eye image signal, the left eye display screen (61) and right eye display screen (62) are respectively used to show Show left-eye image signal and eye image signal, to realize 3D display.
- 9. as described in claim 1 based on motion-captured and mixed reality ophthalmologic operation training system, which is characterized in that institute It states laser pulse detector (72) and detects and receive each laser pulse emission device (5) to be more than the rate of 10 pulse/secs and send out Laser pulse signal.
- It is 10. a kind of based on motion-captured and mixed reality ophthalmologic operation Training Methodology, which is characterized in that will applied to such as right 1 to 9 any one of them is asked to include the following steps based on motion-captured and mixed reality ophthalmologic operation training system：Step S1,3D photographic microscopes (3) shoot the front 3-D view of animal eyeball sample (2) in surgical procedure, and transmit To central processing unit (4)；Step S2, laser pulse emission device (5) generates and sends laser pulse signal, laser pulse detector (72) detecting And the laser pulse signal that each laser pulse emission device (5) sends out is received, obtain location data；Step S3, the location data of operation tool (7) is transferred to central processing unit (4) by communication module (73)；Step S4, central processing unit (4) provides the corresponding guidance letter being pre-stored in central processing unit (4) according to the process of operation Breath, and according to the location data of the operation tool (7), obtains the location information of the operation tool (7), and by operation tool (7) location information and preset standard location information is compared in real time, and out position is given to correct information, and by tutorial message and Position is corrected information and is added on the front 3-D view of 3D photographic microscopes (3) acquisition, generates mixed reality picture signal simultaneously It is sent to mixed reality display terminal (6)；Step S5, mixed reality display terminal (6) according to the mixed reality picture signal carry out 3D display, start-up according to The image of mixed reality display terminal (6) display carries out ophthalmologic operation training.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711466016.1A CN108154778A (en) | 2017-12-28 | 2017-12-28 | Based on motion-captured and mixed reality ophthalmologic operation training system and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711466016.1A CN108154778A (en) | 2017-12-28 | 2017-12-28 | Based on motion-captured and mixed reality ophthalmologic operation training system and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108154778A true CN108154778A (en) | 2018-06-12 |
Family
ID=62463652
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711466016.1A Pending CN108154778A (en) | 2017-12-28 | 2017-12-28 | Based on motion-captured and mixed reality ophthalmologic operation training system and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108154778A (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108492657A (en) * | 2018-03-20 | 2018-09-04 | 天津工业大学 | A kind of mixed reality simulation system for being trained before temporal bone surgery |
| CN109326166A (en) * | 2018-12-05 | 2019-02-12 | 济南大学 | A kind of virtual microscopic material object external member and its application |
| CN110349661A (en) * | 2019-07-08 | 2019-10-18 | 艾炳蔚 | A kind of acupuncture remote diagnosis system |
| CN110379259A (en) * | 2019-08-07 | 2019-10-25 | 泉州师范学院 | Performer's Training Methodology based on motion capture |
| CN111616800A (en) * | 2020-06-09 | 2020-09-04 | 电子科技大学 | Ophthalmic surgery navigation system |
| CN112489541A (en) * | 2019-09-12 | 2021-03-12 | 吴文钧 | Multimedia auxiliary system for medical training |
| CN112991854A (en) * | 2021-02-05 | 2021-06-18 | 四川大学华西医院 | Ultrasonic teaching method, device and system and electronic equipment |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104020445A (en) * | 2014-06-10 | 2014-09-03 | 西安电子科技大学 | Indoor laser positioning system based on frequency labeling method |
| CN104866121A (en) * | 2014-02-21 | 2015-08-26 | 上海市七宝中学 | Drawing system and drawing method for three-dimensional model |
| CN105183166A (en) * | 2015-09-15 | 2015-12-23 | 北京国承万通信息科技有限公司 | Virtual reality system |
| CN105458483A (en) * | 2014-11-20 | 2016-04-06 | 南通大学 | Automatic correcting and ultrasonic impacting system of post-welding seam tracking robot |
| CN105788390A (en) * | 2016-04-29 | 2016-07-20 | 吉林医药学院 | Medical anatomy auxiliary teaching system based on augmented reality |
| CN106297472A (en) * | 2016-10-25 | 2017-01-04 | 深圳市科创数字显示技术有限公司 | The cornea intelligent operation training system that AR with VR combines |
| CN106873787A (en) * | 2017-04-10 | 2017-06-20 | 武汉大学 | A kind of gesture interaction system and method for virtual teach-in teaching |
| CN107340858A (en) * | 2017-06-14 | 2017-11-10 | 贵州电网有限责任公司 | One kind is based on Substation Design SCD automation equipment virtual reality devices |
| CN107402373A (en) * | 2016-05-19 | 2017-11-28 | 财团法人金属工业研究发展中心 | Domain type locating module |
-
2017
- 2017-12-28 CN CN201711466016.1A patent/CN108154778A/en active Pending
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104866121A (en) * | 2014-02-21 | 2015-08-26 | 上海市七宝中学 | Drawing system and drawing method for three-dimensional model |
| CN104020445A (en) * | 2014-06-10 | 2014-09-03 | 西安电子科技大学 | Indoor laser positioning system based on frequency labeling method |
| CN105458483A (en) * | 2014-11-20 | 2016-04-06 | 南通大学 | Automatic correcting and ultrasonic impacting system of post-welding seam tracking robot |
| CN105183166A (en) * | 2015-09-15 | 2015-12-23 | 北京国承万通信息科技有限公司 | Virtual reality system |
| CN105788390A (en) * | 2016-04-29 | 2016-07-20 | 吉林医药学院 | Medical anatomy auxiliary teaching system based on augmented reality |
| CN107402373A (en) * | 2016-05-19 | 2017-11-28 | 财团法人金属工业研究发展中心 | Domain type locating module |
| CN106297472A (en) * | 2016-10-25 | 2017-01-04 | 深圳市科创数字显示技术有限公司 | The cornea intelligent operation training system that AR with VR combines |
| CN106873787A (en) * | 2017-04-10 | 2017-06-20 | 武汉大学 | A kind of gesture interaction system and method for virtual teach-in teaching |
| CN107340858A (en) * | 2017-06-14 | 2017-11-10 | 贵州电网有限责任公司 | One kind is based on Substation Design SCD automation equipment virtual reality devices |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108492657A (en) * | 2018-03-20 | 2018-09-04 | 天津工业大学 | A kind of mixed reality simulation system for being trained before temporal bone surgery |
| CN109326166A (en) * | 2018-12-05 | 2019-02-12 | 济南大学 | A kind of virtual microscopic material object external member and its application |
| CN109326166B (en) * | 2018-12-05 | 2020-11-06 | 济南大学 | Virtual microscope object kit and application thereof |
| CN110349661A (en) * | 2019-07-08 | 2019-10-18 | 艾炳蔚 | A kind of acupuncture remote diagnosis system |
| CN110379259A (en) * | 2019-08-07 | 2019-10-25 | 泉州师范学院 | Performer's Training Methodology based on motion capture |
| CN110379259B (en) * | 2019-08-07 | 2024-01-26 | 泉州师范学院 | Actor training method based on motion capture |
| CN112489541A (en) * | 2019-09-12 | 2021-03-12 | 吴文钧 | Multimedia auxiliary system for medical training |
| CN111616800A (en) * | 2020-06-09 | 2020-09-04 | 电子科技大学 | Ophthalmic surgery navigation system |
| CN111616800B (en) * | 2020-06-09 | 2023-06-09 | 电子科技大学 | Ophthalmic surgery navigation system |
| CN112991854A (en) * | 2021-02-05 | 2021-06-18 | 四川大学华西医院 | Ultrasonic teaching method, device and system and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108154778A (en) | Based on motion-captured and mixed reality ophthalmologic operation training system and method | |
| CN106297472A (en) | The cornea intelligent operation training system that AR with VR combines | |
| US10965933B2 (en) | Medical devices, systems, and methods using eye gaze tracking for stereo viewer | |
| JP5887026B2 (en) | Head mounted system and method for computing and rendering a stream of digital images using the head mounted system | |
| US10231614B2 (en) | Systems and methods for using virtual reality, augmented reality, and/or a synthetic 3-dimensional information for the measurement of human ocular performance | |
| CN110390851A (en) | Augmented reality training system | |
| CN107067856B (en) | Medical simulation training system and method | |
| CN106297471A (en) | The removable cornea intelligent operation training system that AR and VR combines | |
| CN109288591A (en) | Surgical robot system | |
| CN109069103A (en) | ultrasound imaging probe positioning | |
| JP2018511359A (en) | Operating room and surgical site recognition | |
| US20150004581A1 (en) | Interactive physical therapy | |
| CN104094162A (en) | Wide field-of-view 3d stereo vision platform with dynamic control of immersive or heads-up display operation | |
| JP4129527B2 (en) | Virtual surgery simulation system | |
| Soto et al. | AR stereoscopic 3D human eye examination app | |
| CN206301504U (en) | The cornea intelligent operation training system that AR is combined with VR | |
| CN207264655U (en) | Based on motion-captured and mixed reality ophthalmologic operation training system | |
| Huang et al. | CatAR: a novel stereoscopic augmented reality cataract surgery training system with dexterous instruments tracking technology | |
| EP3905225A1 (en) | System and method for evaluating simulation-based medical training | |
| WO2020091385A1 (en) | Virtual reality device for medical training including microsurgery and injection | |
| CN206301505U (en) | The removable cornea intelligent operation training system that AR and VR are combined | |
| Wieben | Virtual and augmented reality in medicine | |
| JP2020134710A (en) | Surgical operation training device | |
| EP4052636A1 (en) | A system for treating visual neglect | |
| TWI819654B (en) | Systems for improving vision of a viewer's eye with impaired retina |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |