CN109419554A - A kind of carpal bone system of virtual operation and method based on unity3d - Google Patents
A kind of carpal bone system of virtual operation and method based on unity3d Download PDFInfo
- Publication number
- CN109419554A CN109419554A CN201710785586.0A CN201710785586A CN109419554A CN 109419554 A CN109419554 A CN 109419554A CN 201710785586 A CN201710785586 A CN 201710785586A CN 109419554 A CN109419554 A CN 109419554A
- Authority
- CN
- China
- Prior art keywords
- image
- carpal bone
- operating robot
- unity3d
- display methods
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/003—Navigation within 3D models or images
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/10—Computer-aided planning, simulation or modelling of surgical operations
- A61B2034/101—Computer-aided simulation of surgical operations
- A61B2034/102—Modelling of surgical devices, implants or prosthesis
Landscapes
- Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Software Systems (AREA)
- Surgery (AREA)
- Public Health (AREA)
- Radar, Positioning & Navigation (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Robotics (AREA)
- Veterinary Medicine (AREA)
- Biomedical Technology (AREA)
- Remote Sensing (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Computer Graphics (AREA)
- Computer Hardware Design (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses a kind of carpal bone system of virtual operation and method based on unity3d, it is main including medical image in the display methods of screen terminal, 3 d medical images in VR displaying method of terminal, pre-operative surgical simulation system.The display methods of medical image screen terminal mainly realizes that two dimensional image of the medical image in unity3d on the interface UI is shown and 3-D image is shown；3 d medical images are mainly realized in VR displaying method of terminal and are shown in VR terminal device to the threedimensional model after reconstruct；Pre-operative surgical simulation system mainly includes operation pathway planning, carpal bone operating robot intelligently punctures revolving speed control and according to the anti-arm angle for solving carpal bone operating robot in the path cooked up automatically, and the carpal bone operating robot model imported in advance by C# Script controlling on this basis.The present invention combines virtual reality technology with medical image, improves doctor to the perception precision of patient's lesions position, reduces the operation wound of patient.
Description
Technical field
The invention belongs to technical field of virtual reality, and in particular to a kind of carpal bone system of virtual operation based on unity3d
And method.
Background technique
Current carpal bone surgical procedure is as follows: a guide pin being penetrated skin first and is against on the nut bone of patient, and in hand
Fluoroscopy during operation determines guide pin direction, and guide pin is then pierced nut bone, and hollow nail is pierced nut bone along guide pin using surgical electric drill,
It finally extracts guide pin and sews up a wound.Current carpal bone operation wound is larger, longer so as to cause the recovery period of patient；Simultaneously
The rehabilitation efficacy that experience difference, the state difference of doctor also results in patient is different.
With the development of virtual reality technology, virtual reality device has obtained very big popularization and application, base in every field
The lesion threedimensional model of patient is shown in VR equipment, allows doctor that there can be one intuitively to understand the lesions position of patient.
The parsing of medical image files of the country based on unity3d is in the starting stage at present, or even does not have relative recording
And report；The country also belongs to the starting stage about the light projecting algorithm of unity3d.
Traditional light projecting algorithm, it is the same to the number of sampling points of 3 d data field on each light, seriously limit
The drafting speed of this method, drawing effect on the poor computer of performance cannot show well.Present invention improves over conventional bulks
Rendering algorithm, the length in the 3 d data field that different light are passed through is different, and the data point number sampled is different, can be very well
The computer suitable for common configuration, can be used on the computer of any windows operating system, can not have to be equipped with
VR equipment, it is extremely low using threshold.
Summary of the invention
The technical problem to be solved in the present invention is that being shown using a kind of VR equipment and several method realization to medical image
Show, medical image three-dimensional reconstruct and interactively cook up carpal bone operating robot execute end puncture path, intelligently count
Calculate the puncture revolving speed of carpal bone operating robot, and the operation to carpal bone operating robot in virtual world.
To solve the above problems, solution proposed by the invention is: a kind of carpal bone virtual hand based on unity3d
Art system and method, it is characterised in that: medical image is shown in the display methods of screen terminal, three-dimensionalreconstruction image in VR terminal
Method and preoperative carpal bone surgery simulation system.
The image includes display to the display methods of two-dimensional medical images and three in the display methods of screen terminal
Tie up the display methods of medical image.
Steps are as follows in the display methods of screen terminal for the two-dimensional medical images:
(1) medical image files are read, grey scale pixel value, pixel separation distance, interfloor distance, patient information are parsed；
(2) the original pixels gray value parsed is carried out adjusting window setting technique processing, original pixels gray value is adjusted to display
The digit that can be shown, and will be on the grey scale pixel value Output matrix to display screen after adjusting；
(3) pixel grey scale value matrix is filtered, improves doctor to the diagnosis accuracy of patient's lesion；
Steps are as follows in the display methods of screen terminal for the 3 d medical images:
(1) medical image files are read, grey scale pixel value, pixel separation distance, interfloor distance, patient information are parsed；
(2) basis thinks effect to be shown, design colours transfer function and transparency transfer function；
(3) according to above-mentioned designed color transmission function and transparency transfer function, grey scale pixel value is subjected to color division
And transparency divides；
(4) 3 d data field after division is passed in GPU；
(5) emit emergent ray from video camera, it, (should be away from pre-set distance since the starting point for entering 3 d data field
From different at a distance from traditional light projecting algorithm, length difference of traditional light projecting algorithm based on every ray is calculated
Distance is different, and the distance in the present invention on every ray is identical) data are sampled；
(6) according to the transparency that adds up whether be greater than 1 or ray whether pass through that 3 d data field terminates in advance ray and data are adopted
Sample.
(7) it adds up, and is transmitted in video card frame buffer to pixel value collected on same ray.
Steps are as follows in the display methods of VR terminal for the 3 d medical images:
(1) based on 3 d medical images in [0010] the display methods of screen terminal the step of, according to the left and right two of VR equipment
The position of a camera in space generates two images；
(2) two images are passed to respectively in VR or so display equipment, to constitute stereo display effect.
The pre-operative surgical simulation system mainly includes carpal bone operating robot threedimensional model, carpal bone operating robot mould
Type control module, interactive intra-procedural path planning module, carpal bone operating robot intelligently puncture rotational speed control module.
Operating robot threedimensional model described in [0012] is by solidworks with according to true operating robot
Size draw corresponding three-dimensional display model, be then introduced into 3dmax software, finally imported into unity3d three-dimensional scenic,
To constitute the carpal bone operating robot in system of virtual operation.
Operating robot model cootrol module described in [0012], which receives, comes from doctor interaction institute on xbox handle
The data of generation, and each joint of robot is calculated according to the pose of the execution end of the desired operating robot of doctor
Angle, and control joint of robot and reach the calculated joint angles of institute, to realize operating robot model cootrol module
Function.
Interactive intra-procedural path planning module described in [0012], first doctor are cured using virtual hollow needle in three-dimensional
Learn image on generated position and posture, cook up one or more of operation pathway automatically, it is ensured that operating robot it is each
Equipment and patient in component and environment have certain safe distance.
Carpal bone operating robot described in [0012] intelligently punctures rotational speed control module, first according to other equipment institutes
After detecting the true bone density of patient's carpal bone, by the data value and patient age by being inputted on the interface UI in unity3d, thus
Intelligently calculate carpal bone operating robot tip penetration revolving speed, it is ensured that the poor patient of sclerotin could be used that in this product.
Contemplated method compared with prior art, there is advantage through the invention:
(1) in light projecting algorithm, ray is different by the length of 3 d data field, and corresponding number of sampling points is different,
Therefore image taking speed is improved；
(2) virtual reality technology and operating robot are combined, substantially increase doctor to the diagnosis precision of patient's lesion,
Pre-operative surgical simulation system reduces operation wound；
(3) system evaluates the sclerotin quality degree of the patient to the age and bone density according to patient, and according to evaluating
Sclerotin quality degree intelligently calculate the carpal bone operating robot that is adapted to the patient and puncture revolving speed；
(4) system can use on the computer routinely configured, can not wear VR equipment, therefore the present invention is with stronger
Adaptability.
Detailed description of the invention
Fig. 1 is a kind of allomeric function figure of system of virtual operation and method based on unity3d of the present invention；
Fig. 2 is the display methods in medical image screen terminal provided in an embodiment of the present invention and VR device end；
Fig. 3 is improved light projecting algorithm process in this method；
Fig. 4 is preoperative system of virtual operation process provided in an embodiment of the present invention；
Specific embodiment
To keep the purpose of the present invention, technical solution more explicit, the present invention is made furtherly with reference to the accompanying drawing
It is bright；
Shown in Fig. 2, steps are as follows in the display methods of screen terminal for two-dimensional medical images of the present invention:
(1) C# script is write, medical image is read in a manner of binary stream；
It (2) is that (7fe0,0010) starts reads pixel data with tag, tag is that the data content of (0018,0050) is image sequence
Column interfloor distance, tag are that the data content of (0028,0030) is pixel separation distance etc., parse these data using script；
(3) decimal number is converted by the pixel data parsed, window position information, the window width information being then arranged according to window will
Pixel data is converted into the displayable pixel coverage of computer display screen, and it is as follows to jump window formula:
C in formula, w are window position and window width respectively；
(4) image after above-mentioned convert is filtered according to options menu；
(5) filtered image is passed in the image component in the UI of unity3d, is realized based on unity3d in display
Display methods of the terminal to two-dimensional medical images.
Shown in Fig. 2, steps are as follows in the display methods of screen terminal for 3 d medical images of the present invention:
(1) it according to the medical image of [0023] described reading, is transmitted respectively by color transfer function algorithm and transparency
Function divides image data；
(2) data after division are assigned to the texture3d of unity3d, so that structured 3 d data field is constituted, and from
It is passed in script in the shader of unity3d, namely data are passed in GPU from CPU；
(3) opposite vertexes coordinate is converted in the vertex shader of unity3d, and improved light is used in fragment shader
Projection algorithm is calculated to obtaining 3 d data field among the above, thus obtain the corresponding color value of each pixel on screen and
Transparency, and in incoming computer monitor terminal device, wherein the improved part of light projecting algorithm is as shown in Fig. 3, wherein
Equidistant sampling be being equidistant of being sampled in all rays, and traditional light projecting algorithm is the sampled distance under identical ray
Equally, different interradial sampled distances are different；
It is above-mentioned according to 3 d medical images described in [0024] based on the display methods of screen terminal shown in Fig. 2
The shader of unity3d carries out light projecting algorithm calculating according to the position of the left and right display of VR equipment respectively, obtained figure
Shape is passed in VR equipment respectively；
Operating robot threedimensional model described in [0012], be according to practical carpal bone operating robot actual look,
It with solidworks Software on Drawing, is then introduced into 3dmax software, generates the file format that unity3d can be imported, and import
In unity3d three-dimensional scenic, the relative position of carpal bone operating robot model is set according to the master-slave relationship of model, is realized to wrist
The model foundation of osseous surgery robot；
Interactive intra-procedural path planning module described in [0012], doctor pass through in the interface UI in unity3d
Virtual hollow needle is selected, the pose in threedimensional model space needed to set is placed on；Interactive intra-procedural path planning module according to
The pose of virtual hollow needle, using the distance of other equipment in each member distance patient of carpal bone operating robot and environment as target
Function establishes Optimized model, to cook up the path from the initial pose of carpal bone operating robot art pose in one's hands；Then in C#
It is realized by Kinematics analysis algorithm to the mechanical arm relative angle of carpal bone operating robot in script according to the pose on path
It calculates；
Carpal bone operating robot model cootrol module described in [0012] is inputted according to the xbox equipment that doctor uses
Signal resolution goes out the pose that desired virtual operation robot executes end, uses
Described in each arm angle for clearing out of Kinematics analysis algorithm, pass through C# script and change robot model's
Rotational angle realizes the control to carpal bone operating robot model；
It is close according to the patient bone of input that carpal bone operating robot described in [0012] intelligently punctures rotational speed control module
Degree according to and patient age information intelligent calculate carpal bone operating robot and puncture revolving speed, calculation formula is as follows:
In formula, k is intensity factor, w0For initial rotation speed, bmFor bone density, y is patient age, and w is after intelligence computation
Puncture rotation speed；
Further include other features in the embodiment of the present invention:
(1) switching to X-Y scheme is realized by mouse roller and directionkeys, mouse roller and keyboard is obtained in C# script
Direction key events, corresponding receptance function remove picture to be shown needed for the table；
(2) by mouse roller threedimensional model display area scrolling realization to the scaling of model, the right button of mouse is realized pair
The rotation of model obtains mouse in the rolling and key-press event of threedimensional model display area in C# script, controls unity3d
The position of video camera relative model can visually realize the scaling and rotation function to model in scape.
Claims (4)
1. a kind of carpal bone system of virtual operation and method based on unity3d specifically includes that medical image in the aobvious of screen terminal
Show method, three-dimensionalreconstruction image in VR displaying method of terminal and preoperative carpal bone surgery simulation system.
2. the carpal bone system of virtual operation and method according to claim 1 based on unity3d, it is characterised in that: described
Image in the display methods of screen terminal include screen to the aobvious of the display methods of two-dimensional medical images and three-dimensionalreconstruction image
Show method；
The display methods of the two-dimensional medical images includes medical image parsing, the tune window method of image, image filtering, image
Measurement；The medical image parsing parses image data, pixel spacing information for reading medical image, and according to tag value
Deng；The tune window method is used to show the image in different gray scale stages；The image filtering is for removing medical image
It makes an uproar, image is allowed to be more clear；The image measurement is mainly used for detecting lesion by way of mouse interaction in screen terminal
Actual size；
The display methods of the three-dimensionalreconstruction image is made of improved light projecting algorithm, is calculated compared to projecting with traditional light
Method more rapidly can carry out three-dimensionalreconstruction to sequence two-dimensional medical images, and image is shown to screen terminal；Three-dimensionalreconstruction
The display methods of image further includes the functions such as the 3-D image after reconstruct being rotated and being amplified by way of mouse interaction.
3. the carpal bone system of virtual operation and method according to claim 1 based on unity3d, it is characterised in that: described
3 d medical images in display methods of the VR displaying method of terminal based on three-dimensionalreconstruction image as claimed in claim 2, will scheme
As being passed in VR device end, enables doctor to check the medical image after three-dimensionalreconstruction in the virtual world, realize simultaneously
Interact function.
4. the carpal bone system of virtual operation and method according to claim 1 based on unity3d, it is characterised in that: described
Pre-operative surgical simulation system include carpal bone operating robot threedimensional model, carpal bone operating robot model cootrol module, interaction
Formula operation pathway planning module, carpal bone operating robot intelligently puncture rotational speed control module；
The three dimensional model for robot is by solidworks 3 d modeling software with according to true designed carpal bone hand out
Art robot size draws corresponding three-dimensional display model, is then introduced into 3dmax software, finally imported into unity3d three-dimensional
In scene, to constitute the carpal bone operating robot threedimensional model in system of virtual operation；
The carpal bone operating robot model cootrol module, is interacted by doctor and xbox handle, to control operation
The position and posture of the execution end of robot model, and it is each according to Kinematics analysis algorithm to calculate carpal bone operating robot
Then the angle in joint makes carpal bone operating robot by the rotational angle of each mechanical arm of Script controlling carpal bone operating robot
It executes end and reaches desired position and posture；
The pose of the virtual hollow needle that the interactive intra-procedural path planning module is placed according to doctor generate one or
Mulitpath, it is ensured that each component of operating robot and the equipment in environment and patient have certain safe distance；
The carpal bone operating robot intelligently punctures rotational speed control module carpal bone bone density number according to measured by other equipment
Accordingly and the age information of patient, the revolving speed of carpal bone operating robot tip penetration equipment is intelligently calculated, to greatly drop
Low wound of the puncture force to the poor patient of sclerotin.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710785586.0A CN109419554A (en) | 2017-09-04 | 2017-09-04 | A kind of carpal bone system of virtual operation and method based on unity3d |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710785586.0A CN109419554A (en) | 2017-09-04 | 2017-09-04 | A kind of carpal bone system of virtual operation and method based on unity3d |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109419554A true CN109419554A (en) | 2019-03-05 |
Family
ID=65513335
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710785586.0A Pending CN109419554A (en) | 2017-09-04 | 2017-09-04 | A kind of carpal bone system of virtual operation and method based on unity3d |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109419554A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110974426A (en) * | 2019-12-24 | 2020-04-10 | 上海龙慧医疗科技有限公司 | Robot system for orthopedic joint replacement surgery |
| CN111113414A (en) * | 2019-12-19 | 2020-05-08 | 长安大学 | Robot three-dimensional space scale prompting method and system based on screen identification |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101530341A (en) * | 2009-04-20 | 2009-09-16 | 马秋野 | Intelligent bone drill and control method thereof |
| CN101849849A (en) * | 2010-06-18 | 2010-10-06 | 北京航空航天大学 | Device for intellectualized monitoring of surgical grinding tool |
| CN102961187A (en) * | 2012-10-26 | 2013-03-13 | 深圳市旭东数字医学影像技术有限公司 | Surgical planning method and system for percutaneous puncture |
| US20170108930A1 (en) * | 2012-09-27 | 2017-04-20 | The Board Of Trustees Of The University Of Illinois | Haptic augmented and virtual reality system for simulation of surgical procedures |
| CN106691600A (en) * | 2016-11-21 | 2017-05-24 | 胡磊 | Spine pedicle screw implanting and locating device |
| CN106901834A (en) * | 2016-12-29 | 2017-06-30 | 陕西联邦义齿有限公司 | The preoperative planning of minimally invasive cardiac surgery and operation virtual reality simulation method |
| CN106974730A (en) * | 2017-04-01 | 2017-07-25 | 上海术理智能科技有限公司 | Surgical simulation method, device and equipment based on virtual reality and medical image |
-
2017
- 2017-09-04 CN CN201710785586.0A patent/CN109419554A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101530341A (en) * | 2009-04-20 | 2009-09-16 | 马秋野 | Intelligent bone drill and control method thereof |
| CN101849849A (en) * | 2010-06-18 | 2010-10-06 | 北京航空航天大学 | Device for intellectualized monitoring of surgical grinding tool |
| US20170108930A1 (en) * | 2012-09-27 | 2017-04-20 | The Board Of Trustees Of The University Of Illinois | Haptic augmented and virtual reality system for simulation of surgical procedures |
| CN102961187A (en) * | 2012-10-26 | 2013-03-13 | 深圳市旭东数字医学影像技术有限公司 | Surgical planning method and system for percutaneous puncture |
| CN106691600A (en) * | 2016-11-21 | 2017-05-24 | 胡磊 | Spine pedicle screw implanting and locating device |
| CN106901834A (en) * | 2016-12-29 | 2017-06-30 | 陕西联邦义齿有限公司 | The preoperative planning of minimally invasive cardiac surgery and operation virtual reality simulation method |
| CN106974730A (en) * | 2017-04-01 | 2017-07-25 | 上海术理智能科技有限公司 | Surgical simulation method, device and equipment based on virtual reality and medical image |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111113414A (en) * | 2019-12-19 | 2020-05-08 | 长安大学 | Robot three-dimensional space scale prompting method and system based on screen identification |
| CN111113414B (en) * | 2019-12-19 | 2022-08-30 | 长安大学 | Robot three-dimensional space scale prompting method and system based on screen identification |
| CN110974426A (en) * | 2019-12-24 | 2020-04-10 | 上海龙慧医疗科技有限公司 | Robot system for orthopedic joint replacement surgery |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107656613B (en) | Human-computer interaction system based on eye movement tracking and working method thereof | |
| CN108777081B (en) | Virtual dance teaching method and system | |
| CA2896240C (en) | System and method for role-switching in multi-reality environments | |
| Gallo et al. | 3D interaction with volumetric medical data: experiencing the Wiimote | |
| US8648811B2 (en) | Remote control system for electronic device and remote control method thereof | |
| CN103092432B (en) | The trigger control method of man-machine interactive operation instruction and system and laser beam emitting device | |
| US20190187876A1 (en) | Three dimensional digital content editing in virtual reality | |
| Carvalheiro et al. | User redirection and direct haptics in virtual environments | |
| CN107296650A (en) | Intelligent operation accessory system based on virtual reality and augmented reality | |
| US20110289455A1 (en) | Gestures And Gesture Recognition For Manipulating A User-Interface | |
| CN112618026B (en) | Remote operation data fusion interactive display system and method | |
| CN107016717A (en) | System and method for the see-through view of patient | |
| CN106325509A (en) | Three-dimensional gesture recognition method and system | |
| DE102015206110A1 (en) | SYSTEM AND METHOD FOR PRODUCING COMPUTER CONTROL SIGNALS FROM ATATOR ATTRIBUTES | |
| JP2009205556A (en) | User interface device | |
| CN114862992A (en) | Virtual digital human processing method, model training method and device thereof | |
| CN112348942B (en) | Body-building interaction method and system | |
| CN109419554A (en) | A kind of carpal bone system of virtual operation and method based on unity3d | |
| CN114711962A (en) | Augmented reality operation planning navigation system and method | |
| CN110660130A (en) | Medical image-oriented mobile augmented reality system construction method | |
| CN113190109A (en) | Input control method and device of head-mounted display equipment and head-mounted display equipment | |
| Zhang et al. | Research on volleyball action standardization based on 3D dynamic model | |
| CN107993720A (en) | Recovery function evaluation device and method based on depth camera and virtual reality technology | |
| WO2020160129A1 (en) | Systems and methods for facilitating insertion of a surgical instrument into a surgical space | |
| CN111312363B (en) | Double-hand coordination enhancement system based on virtual reality |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190305 |
|
| WD01 | Invention patent application deemed withdrawn after publication |