CN106974730A - Virtual reality and medical image-based surgery analogy method, device and equipment - Google Patents

Virtual reality and medical image-based surgery analogy method, device and equipment Download PDF

Info

Publication number
CN106974730A
CN106974730A CN201710214282.9A CN201710214282A CN106974730A CN 106974730 A CN106974730 A CN 106974730A CN 201710214282 A CN201710214282 A CN 201710214282A CN 106974730 A CN106974730 A CN 106974730A
Authority
CN
China
Prior art keywords
file
modeling
simulation
based
virtual environment
Prior art date
Application number
CN201710214282.9A
Other languages
Chinese (zh)
Inventor
王薇
Original Assignee
上海术理智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海术理智能科技有限公司 filed Critical 上海术理智能科技有限公司
Priority to CN201710214282.9A priority Critical patent/CN106974730A/en
Publication of CN106974730A publication Critical patent/CN106974730A/en

Links

Abstract

The invention provides a virtual reality and medical image-based surgery analogy method, device and equipment. The surgery analogy method comprises the following steps of constructing a virtual environment used during the analogy surgery, creating a modeling document used for display and interaction according to the medical image in the virtual environment; constructing a simulation modeling interaction document according to medical requirements; importing the modeling document and the simulation modeling interaction document in the virtual environment; then conducting the analogy surgery based on the virtual environment. The method can provides a clinician with precise and visual pathological anatomy structural information and visual feedbacks, thereby helping the clinician achieve fast and precise diagnoses and treatment.

Description

基于虚拟现实和医学影像的手术模拟方法、装置和设备 Based on virtual reality surgical simulation and medical imaging method, apparatus and equipment

技术领域 FIELD

[0001] 本发明涉及医疗信息化技术领域,具体而言,涉及一种基于虚拟现实和医学影像的手术模拟方法、装置和电子设备。 [0001] The present invention relates to the field of medical information technology, particularly, to a virtual reality surgical simulation and medical imaging method, apparatus and electronic equipment.

背景技术 Background technique

[0002] 目前,虚拟现实技术(Virtual Reality,简称VR)正在逐渐渗透到各个科学领域, 尤其是医药领域,主要是因为借助虚拟现实技术可以根据病患的特点和临床需要实现个性化诊疗,因此虚拟现实技术己成为辅助精准医疗的潜在有力手段。 [0002] Currently, virtual reality technology (Virtual Reality, referred to as VR) is gradually penetrated into all fields of science, especially in the field of medicine, largely because the need to achieve a personalized treatment based on the characteristics of the patient and the clinical use of virtual reality technology, virtual reality technology has the potential to become a powerful tool for precise medical aid.

[0003] 但是,如何能够为临床医生提供精确而直观的病理解剖结构信息和视觉反馈,从而协助临床医生实现快速、精准的诊疗一直是行业内亟待解决的技术问题。 [0003] However, how to provide clinicians with accurate and intuitive pathological anatomy of information and visual feedback to assist clinicians rapid, accurate diagnosis and treatment has been the industry's technical problems to be solved.

[0004] 需要说明的是,在上述背景技术部分公开的信息仅用于加强对本发明的背景的理解,因此可以包括不构成对本领域普通技术人员己知的现有技术的信息。 [0004] Incidentally, in the above information disclosed in this Background section is only for enhancement of understanding of the background of the present invention therefore may include information not constitute prior art to those of ordinary skill in the art is known.

发明内容 SUMMARY

[0005] 本发明的目的在于提供一种基于虚拟现实和医学影像的手术模拟方法、装置和电子设备,进而至少在一定程度上克服由于相关技术的限制和缺陷而导致的一个或者多个问题。 [0005] The object of the present invention is to provide a virtual reality simulation method based surgical and medical images, electronic equipment and apparatus, thereby overcoming one or more of the problems due to limitations and disadvantages of the related art is caused at least to some extent.

[0006] 本发明的其他特性和优点将通过下面的详细描述变得显然,或部分地通过本发明的实践而习得。 [0006] Other characteristics and advantages of the invention will become apparent, in part or may be learned by practice of the present invention by the following detailed description.

[0007] 根据本发明的第一方面,提供了一种基于虚拟现实和医学影像的手术模拟方法, 包括:构建用于模拟手术过程的虚拟环境;基于医学影像生成在所述虚拟环境中进行显示及交互的建模文件;根据临床需求,建立仿真建模互动文件;将所述建模文件和所述仿真建模互动文件导入所述虚拟环境;在将所述建模文件和所述仿真建模互动文件导入所述虚拟环境后,基于所述虚拟环境模拟手术过程。 [0007] According to a first aspect of the present invention, there is provided a simulation method based on virtual reality surgical and medical imaging, comprising: building a virtual environment for simulating a surgical procedure; generating a display in the virtual environment based on Medical Imaging and interactive modeling files; based on clinical needs and build interactive simulation modeling file; the model file and the simulation modeling files into the interactive virtual environment; in the modeling and the simulation file built after the interactive mode file into the virtual environment, the virtual environment based on simulated surgical procedure.

[0008] 在本发明的一些实施例中,基于前述方案,所述的手术模拟方法还包括:配置所述虚拟环境的场景参数,以及所述虚拟环境对应的输入管理器和/或输出管理器。 [0008] In some embodiments of the present invention, based on the foregoing embodiment, the surgical simulation method further comprises: configuring the virtual environment scene parameters, and the virtual environment manager corresponding input and / or output manager .

[0009] 在本发明的一些实施例中,基于前述方案,所述场景参数包括以下至少之一或多个的组合:观察场、视野角度、镜头方向、照明方向、坐标系。 [0009] In some embodiments of the present invention, based on the foregoing embodiment, the scene parameter includes a combination of at least one or more of the following: observation field, viewing angle, the lens direction, the illumination direction, the coordinate system.

[0010] 在本发明的一些实施例中,基于前述方案,所述的手术模拟方法还包括:配置仿真介入点的交互操作对所述虚拟环境中的组织或器官产生的形变模型。 [0010] In some embodiments of the present invention, based on the foregoing embodiment, the surgical simulation method further comprises: interworking configuration deformable model simulation of the intervention point of the virtual environment generated by the tissue or organ.

[0011] 在本发明的一些实施例中,基于前述方案,所述的手术模拟方法还包括:获取真实介入物在介入真实组织或器官时,对所述真实组织或器官产生的应力形变数据和/或实质性状态的形变数据;根据所述应力形变数据和/或所述实质性状态的形变数据配置所述形变模型。 [0011] In some embodiments of the present invention, based on the foregoing embodiment, the surgical simulation method further comprises: acquiring real deformation stress when intervening interposer real tissue or organ, the tissue or organ to produce the real and / or substantial deformation data state; strain stress strain data according to the data and / or status of the substantive configuring the deformable model.

[0012] 在本发明的一些实施例中,基于前述方案,所述的手术模拟方法还包括:配置仿真介入点与所述虚拟环境中的组织或器官之间的安全距离。 [0012] In some embodiments of the present invention, based on the foregoing embodiment, the surgical simulation method further comprises: configuring the virtual simulation intervention point safety distance between the tissue or organ environment.

[0013]在本发明的一些实施例中,基于前述方案,基于医学影像生成在所述虚拟环境中进行显示及交互的建模文件的步骤,包括:根据临床需求和需要呈现在虚拟环境中的病灶和/或器官的信息,通过第一影像设备采集包含所述病灶和/或器官的解剖和功能信息的第一影像文件;基于所述第一影像文件进行三维重建,生成重建文件;将所述重建文件的格式转换为所述虚拟环境支持的文件格式,以得到所述建模文件。 [0013] In some embodiments of the present invention, based on the foregoing embodiment, the step of modeling based on the medical image file display and interacting in the virtual environment generation, comprising: based on clinical needs and requirements presented in the virtual environment lesions and / or information organ, by the first image acquisition apparatus comprising the lesion and / or organs of a first image file of the anatomical and functional information; three-dimensional reconstruction image based on the first file, generate a reconstructed documents; the format converting file formats supported by the virtual environment to the reconstruction of said file, to obtain the model file.

[0014]在本发明的一些实施例中,基于前述方案,基于所述第一影像文件进行三维重建, 生成重建文件的步骤,包括:根据所述第一影像文件的图像特性,对所述第一影像文件进行感兴趣区域的划分;基于划分的所述感兴趣区域对所述第一影像文件进行图像分割及三维重建,以得到针对每个所述感兴趣区域的重建文件。 [0014] In some embodiments of the present invention, based on the aforementioned scheme, based on three-dimensional reconstruction of the first image file, step of reconstructing a file, including generating: an image characteristic of the first image file, the first an image file is divided region of interest; image segmentation and reconstruction of the first video file based on the divided region of interest, to obtain a reconstructed file for each area of ​​interest.

[0015] 在本发明的一些实施例中,基于前述方案,所述的手术模拟方法还包括:通过第一影像设备采集所述病灶和/或器官处于不同相位期的多个第一影像文件,和/或通过第二影像设备采集包含所述病灶和/或器官的解剖和功能信息的第二影像文件;对根据所述多个第一影像文件生成的重建文件进行基于解剖结构和相位时间的图像配准处理,和/或对所述第二影像文件的重建文件和所述第一影像文件的重建文件进行基于解剖特征点的图像配准处理。 [0015] In some embodiments of the present invention, based on the foregoing embodiment, the surgical simulation method further comprising: a first image capture device of the lesion and / or a first plurality of image files in a different phase of the organ, and / or comprising the lesion and / or the second image file organ anatomical and functional information acquired by the second imaging equipment; to reconstruct the file according to the first plurality of image files generated based on the phase and the time of anatomy image registration processing, and / or image registration processing based on an anatomical characteristic point of the second reconstruction image file document file and reconstruction of the first image file.

[0016] 在本发明的一些实施例中,基于前述方案,建立仿真建模互动文件的步骤,包括: 根据临床需求,确定需要使用的器械;基于所述器械的外部特征、物理功能和介入角度,生成所述仿真建模互动文件。 [0016] In some embodiments of the present invention, based on the foregoing embodiment, the step of establishing interactive simulation modeling file, comprising: based on clinical needs, the need to determine the instrument; instrument based on the external characteristics, physical function, and the angle involved generating the interactive simulation modeling file.

[0017] 在本发明的一些实施例中,基于前述方案,基于所述虚拟环境模拟手术过程的步骤,包括:确定所述虚拟环境中的跟踪点或观察点;计算并显示所述跟踪点或观察点与所述建模文件中的病灶的坐标相对位置和距离,和/或所述跟踪点或观察点与其周围组织的坐标相对位置和距离。 [0017] In some embodiments of the present invention, based on the foregoing embodiment, the step of the virtual environment simulation surgical procedures, including those based on: determining the tracking point of the virtual environment or watchpoint; calculating and displaying the tracking points, or the relative position and distance of the coordinates of the observation point of the lesion modeling file coordinate distance and the relative position and / or the tracking point or observation points with the surrounding tissue.

[0018] 在本发明的一些实施例中,基于前述方案,所述跟踪点或观察点包括:所述仿真建模互动文件与所述建模文件的第一接触点。 [0018] In some embodiments of the present invention, based on the foregoing embodiment, the tracking point or observation point comprising: a first contact point of the simulation model to interact with the document model file.

[0019] 在本发明的一些实施例中,基于前述方案,基于所述虚拟环境模拟手术过程的步骤,包括:在所述仿真建模文件介入所述建模文件的过程中,计算并显示所述仿真建模互动文件与所述建模文件之间的交互关系及相关参数。 [0019] In some embodiments of the present invention, based on the foregoing embodiment, the step of the virtual environment based on simulated surgical procedure, comprising: modeling the file involved in the process simulation modeling file, and displays the calculated said simulation modeling the interaction between the interactive modeling files and files related parameters.

[0020] 在本发明的一些实施例中,基于前述方案,所述交互关系及相关参数包括:所述仿真建模互动文件的各个边界与其周围组织的坐标相对位置和距离,和/或所述仿真建模互动文件与所述建模文件中的病灶的坐标相对位置和距离。 [0020] In some embodiments of the present invention, based on the foregoing embodiment, the relationship and interaction parameters comprising: coordinates of the boundary with the surrounding tissue interaction simulation modeling file relative position and distance, and / or the interactive simulation modeling the coordinates of the lesion with the document file modeling the relative position and distance.

[0021] 在本发明的一些实施例中,基于前述方案,基于所述虚拟环境模拟手术过程的步骤,包括:在所述仿真建模文件介入所述建模文件的过程中,计算并显示所述仿真建模互动文件与所述建模文件中的敏感性组织的安全范围,和/或由于所述仿真建模互动文件的介入而对所述建模文件中的组织造成的形变信息。 [0021] In some embodiments of the present invention, based on the foregoing embodiment, the step of the virtual environment based on simulated surgical procedure, comprising: modeling the file involved in the process simulation modeling file, and displays the calculated security range above simulation modeling interactive modeling file with the file in sensitive tissues, and / or due to the intervention of the simulation modeling interactive documents resulting from the model file tissue deformation information.

[0022] 在本发明的一些实施例中,基于前述方案,基于所述虚拟环境模拟手术过程的步骤,包括:计算所述仿真建模互动文件介入所述建模文件的可能路径和/或最优路径;显示所述可能路径和/或所述最优路径。 [0022] In some embodiments of the present invention, based on the foregoing embodiment, the step of the virtual environment based on simulated surgical procedure, comprising: calculating a simulation modeling of the interaction document model file may intervene path and / or the most shortest paths; displaying said possible paths and / or the optimal path.

[0023] 在本发明的一些实施例中,基于前述方案,计算所述可能路径和/或所述最优路径所依据的参数包括:所述仿真建模互动文件介入所述建模文件的介入点及介入角度、所述仿真建模互动文件的结构、所述仿真建模互动文件对所述建模文件中的敏感性组织的最小伤害、所述仿真建模互动文件到达所述建模文件中的病灶的最短距离。 [0023] In some embodiments of the present invention, based on the foregoing embodiment, the calculating possible paths and / or the optimal path based on the parameters comprising: the simulation modeling of the interaction document involved intervention modeling files point and intervention perspective, the structural modeling interactive simulation file, the file simulation modeling interactive modeling minimal damage to the sensitive tissues of the file, the file arrives the simulation modeling interactive modeling files the shortest distance lesions.

[0024] 在本发明的一些实施例中,基于前述方案,基于所述虚拟环境模拟手术过程的步骤,包括:在所述互动文件与所述影像建模文件中的敏感性组织接触时,进行预警提示。 [0024] In some embodiments of the present invention, based on the foregoing embodiment, the step of the virtual environment based on simulated surgical procedures, comprising: tissue sensitivity of the interaction with the image file in the file modeled brought into contact, for warning prompt.

[0025] 在本发明的一些实施例中,基于前述方案,所述的手术模拟方法还包括:根据所述建模文件、所述虚拟环境和在仿真计算时的复杂程度,确定运行所述虚拟环境的设备类型; 基于所述设备类型,生成运行所述虚拟环境的应用程序。 [0025] In some embodiments of the present invention, based on the foregoing embodiment, the surgical simulation method further comprises: modeling based on the file, and the complexity of the virtual environment during the simulation, determining the virtual operating device type environment; based on the device type, the operation of generating a virtual application environment.

[0026] 根据本发明的第二方面,提供了一种基于虚拟现实和医学影像的手术模拟装置, 包括:构建单元,构建用于模拟手术过程的虚拟环境;生成单元,用于基于医学影像生成在所述虚拟环境中进行显示及交互的建模文件;建立单元,用于根据临床需求,建立仿真建模互动文件;导入单元,用于将所述建模文件和所述仿真建模互动文件导入所述虚拟环境;处理单元,用于在所述导入单元将所述建模文件和所述仿真建模互动文件导入所述虚拟环境后,基于所述虚拟环境模拟手术过程。 [0026] According to a second aspect of the present invention, there is provided a virtual reality surgical simulation and medical imaging apparatus, comprising: a construction unit, build a virtual environment for simulating a surgical procedure; generating means for generating a medical image based on displaying and interacting in the virtual environment modeling files; establishing means, according to clinical need, simulation modeling to establish interactive file; introducing means for modeling the interaction simulation modeling file and the file introducing the virtual environment; a processing unit, after said introducing means for introducing said modeling simulation modeling file and the file to interact with the virtual environment, the virtual environment based on simulated surgical procedure.

[0027] 根据本发明的第三方面,提供了一种电子设备,包括:处理器和存储器,所述存储器存储有可执行指令,所述处理器用于调用所述存储器存储的可执行指令执行如上述第一方面中所述的方法。 [0027] According to a third aspect of the present invention, there is provided an electronic device, comprising: a processor and a memory, the memory storing executable instructions, the processor is configured to invoke the memory storing executable instructions to perform the the method described in the above first aspect.

[0028] 在本发明的一些实施例所提供的技术方案中,通过将医学影像与虚拟现实技术相结合来进行手术过程的模拟,并通过对医学影像进行R〇I (Region of Interest,感兴趣区±或)的划分、图像分割、建模文件的生成、配准优化等处理,使得能够建立标准化的、精准的包含病理信息的解剖模型,并基于虚拟平台来将解剖结构的建模文件逐一展现出来,进而能为临床医生提供精确而直观的病理解剖结构信息和视觉反馈,从而有助于协助医生实现快速、精准的诊疗。 [0028] In some embodiments of the technical solutions provided by the present invention, a surgical procedure is performed by the simulated medical imaging technology with virtual reality, and R〇I (Region of Interest by medical imaging, interest or ± area) is divided, image segmentation, modeling file generated, the registration process optimization, making it possible to establish a standardized, accurate anatomical pathology model contains information and the file to be modeled based on an anatomical structure by one virtual platform show up, and then be able to provide accurate and intuitive pathological anatomy of information and visual feedback to clinicians to help doctors help achieve rapid, accurate diagnosis and treatment. 可见,本发明的技术方案对推进精准医疗及实现患者个性化治疗具有重要的现实意义和社会效益。 Visible, aspect of the present invention has important practical significance for promoting social and medical precision and personalized treatment of patients.

[0029] 应当理解的是,以上的一般描述和后文的细节描述仅是示例性和解释性的,并不能限制本发明。 [0029] It should be understood that both the foregoing general description and the details described hereinafter are merely exemplary and explanatory and are not intended to limit the present invention.

附图说明 BRIEF DESCRIPTION

[0030] 此处的附图被并入说明书中并构成本说明书的一部分,示出了符合本发明的实施例,并与说明书一起用于解释本发明的原理。 [0030] The accompanying drawings, which are incorporated herein and constitute a part of this specification, illustrate embodiments consistent with the present invention, and together with the description serve to explain the principles of the invention. 显而易见地,下面描述中的附图仅仅是本发明的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其他的附图。 Apparently, the drawings in the following description are only some embodiments of the present invention, those of ordinary skill in the art is concerned, without creative efforts, we can derive from these drawings other drawings. 在附图中: In the drawings:

[0031] 图1示意性示出了根据本发明的第一个实施例的基于虚拟现实和医学影像的手术模拟方法的流程图; [0031] Figure 1 schematically shows a flowchart of virtual reality surgical simulation and medical imaging method of a first embodiment of the present invention;

[0032] 图2示意性示出了根据本发明的一个实施例的基于医学影像生成建模文件的流程图; [0032] FIG. 2 schematically shows a flowchart of an embodiment of the present invention based on the medical image file generated modeling;

[0033]图3示意性示出了根据本发明的第二个实施例的基于虚拟现实和医学影像的手术模拟方法的流程图; [0033] FIG. 3 schematically shows a flowchart of a second method of virtual reality surgical simulation and medical imaging according to the present embodiment of the invention;

[0034]图4示意性示出了根据本发明的第一个实施方式的基于医学影像的三维建模的流程图; [0034] FIG 4 schematically illustrates a three-dimensional model based on medical imaging in a flowchart of a first embodiment of the present invention;

[0035]图5示意性示出了根据本发明的第二个实施方式的基于医学影像的三维建模的流程图; [0035] FIG. 5 schematically shows a flow diagram of the medical image based on three-dimensional model according to a second embodiment of the present invention;

[0036]图6示意性示出了根据本发明的一个实施例的搭建VR模拟环境的流程图; [0036] Figure 6 schematically shows a flowchart of the simulation environment built VR in accordance with an embodiment of the present invention;

[0037]图7示意性示出了根据本发明的实施例的搭建出的虚拟环境及基本场景参数的设置的界面示意图; [0037] FIG 7 schematically shows a schematic view of the interface provided to build a virtual environment in which embodiments of the present invention, and the basic parameters of the scene;

[0038]图8示意性示出了根据本发明的实施例的跟踪点/观察点在组织或器官内的示意图; [0038] FIG. 8 schematically shows a schematic view of an embodiment of the tracking point of the present invention / watchpoint tissue or organ;

[0039]图9示意性示出了根据本发明的实施例的将虚拟交互环境在手机APP上运行的示意图; [0039] FIG 9 schematically shows a schematic view of the interactive virtual environment in which embodiments of the present invention, the APP running on the phone;

[0040] 图10示意性示出了根据本发明的实施例的通过虚拟眼镜或头盔等显示的虚拟交互环境示意图; [0040] FIG. 10 schematically shows a schematic view of the virtual environment through a virtual interactive eyeglasses or helmets and other embodiments of the present invention shown;

[0041] 图11示意性示出了根据本发明的一个实施例的基于虚拟现实和医学影像的手术模拟装置的框图。 [0041] FIG. 11 schematically shows a block diagram based on virtual reality surgical simulation apparatus and a medical image of the embodiment according to the present invention.

具体实施方式 Detailed ways

[0042] 现在将参考附图更全面地描述示例实施方式。 [0042] Reference will now be described more fully hereinafter with exemplary embodiments. 然而,示例实施方式能够以多种形式实施,且不应被理解为限于在此阐述的范例;相反,提供这些实施方式使得本发明将更加全面和完整,并将示例实施方式的构思全面地传达给本领域的技术人员。 However, exemplary embodiments can be implemented in various forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the present disclosure will be thorough and complete, and the concept of example embodiments will fully convey the to those skilled in the art.

[0043] 此外,所描述的特征、结构或特性可以以任何合适的方式结合在一个或更多实施例中。 [0043] Furthermore, the described features, structures or characteristics may be combined in one or more embodiments in any suitable manner. 在下面的描述中,提供许多具体细节从而给出对本发明的实施例的充分理解。 In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the present invention. 然而, 本领域技术人员将意识到,可以实践本发明的技术方案而没有特定细节中的一个或更多, 或者可以采用其它的方法、组元、装置、步骤等。 However, those skilled in the art will appreciate that the present invention may be practiced aspect without the specific details of one or more, or other methods may be used, component, means, step, etc.. 在其它情况下,不详细示出或描述公知方法、装置、实现或者操作以避免模糊本发明的各方面。 In other instances, not shown in detail or described in the known method, apparatus, or achieve the operation to avoid obscuring aspects of the present invention.

[0044]附图中所示的方框图仅仅是功能实体,不一定必须与物理上独立的实体相对应。 [0044] The block diagram shown in the drawings are purely functional entities, does not necessarily correspond to physically separate entities. 艮P,可以采用软件形式来实现这些功能实体,或在一个或多个硬件模块或集成电路中实现这些功能实体,或在不同网络和/或处理器装置和/或微控制器装置中实现这些功能实体。 Gen P, may take the form of software to implement these functional entities, these functional entities or implement one or more hardware modules, or an integrated circuit, or implemented in these different networks and / or processor means and / or a microcontroller device functional entity. [0045]附图中所示的流程图仅是示例性说明,不是必须包括所有的内容和操作/步骤,也不是必须按所描述的顺序执行。 The flowchart shown in [0045] the accompanying drawings are merely illustrative, not necessarily include all the contents and operations / steps, not necessary performed in the order described. 例如,有的操作/步骤还可以分解,而有的操作/步骤可以合并或部分合并,因此实际执行的顺序有可能根据实际情况改变。 For example, some operations / steps may be decomposed and some operations / steps may be combined or partially combined, so the actual execution order may change according to the actual situation.

[0046]图1示意性示出了根据本发明的第一个实施例的基于虚拟现实和医学影像的手术模拟方法的流程图。 [0046] Figure 1 schematically shows a flowchart of virtual reality surgical simulation and medical imaging method of a first embodiment of the present invention.

[0047]参照图1,根据本发明的第一个实施例的基于虚拟现实和医学影像的手术模拟方法,包括以下步骤: [0047] Referring to FIG. 1, a method based on virtual reality surgical simulation and medical imaging according to the first embodiment of the present invention, comprising the steps of:

[0048] 步骤S10,构建用于模拟手术过程的虚拟环境。 [0048] Step S10, the virtual environment is constructed for simulating a surgical procedure.

[0049] 在本发明的实施方式中,虚拟环境(Virtual Environment)是由计算机生成的,通过视、听、触觉等作用于用户,使之产生身临其境的感觉的交互式视境仿真。 [0049] In an embodiment of the present invention, the virtual environment (Virtual Environment) is generated by the computer, by the action of vision, hearing, touch and the like to the user, so that the feeling of immersive interactive visual simulation environment. 在构建用于模拟手术过程的虚拟环境时,可以利用虚拟引擎来进行构建,例如,可以基于Unity引擎来构建虚拟环境。 When building a virtual environment for simulating a surgical procedure, may be utilized to construct a virtual engine, for example, can be constructed based on the engine Unity virtual environment.

[0050] 在构建虚拟环境之后,还需要配置所述虚拟环境的场景参数,以及所述虚拟环境对应的输入管理器和/或输出管理器。 [0050] After building the virtual environment, but also need to configure a scene of the virtual environment parameters, and the virtual environment manager corresponding input and / or output manager.

[0051] 在本发明的实施方式中,虚拟环境的场景参数包括但不限于:观察场、视野角度、 镜头方向、照明方向、坐标系。 [0051] In an embodiment of the present invention, the virtual environment scene parameters include but are not limited to: observation field, viewing angle, the lens direction, the illumination direction, the coordinate system. 配置输入管理器主要是设置输入给虚拟环境的指令和交互信息;配置输出管理器主要是设置虚拟环境输出的指令和交互信息。 Configuration input manager primarily to set the input instruction and the virtual environment interaction information; Configuration Output Manager virtual environment mainly set output instruction and interaction information.

[0052] 在本发明的实施方式中,由于正常人体的组织和器官会因为外界物体的进入而产生相应的形变,为了在虚拟环境中模拟这种形变,需要在构建虚拟环境之后,配置仿真介入点的交互操作对所述虚拟环境中的组织或器官产生的形变模型。 [0052] In an embodiment of the present invention, the body's normal tissues and organs because of external objects into the corresponding deformation due generated, this deformation in order to simulate a virtual environment, the virtual environment is constructed after the need, the configuration simulation intervention deformable models interactions in the virtual environment produced by the organ or tissue site.

[0053]具体来说,可以先获取真实介入物在介入真实组织或器官时,对所述真实组织或器官产生的应力形变数据和/或实质性状态的形变数据;然后根据所述应力形变数据和/或所述实质性状态的形变数据配置所述形变模型。 [0053] Specifically, when the first acquired real intervening tissue or organ stress of the real deformation data produced by the organ or tissue and / or substantial deformation data of the real state of the interposer; stress and strain data according to the and / or the state of substantial deformation of the deformable model configuration data.

[0054]需要说明的是,上述的应力形变数据主要是真实介入物介入真实组织或器官时, 由于真实介入物的结构、形状、大小等而对真实组织或器官产生的机械形变的数据;上述的实质性状态的形变数据主要是由于真实介入物的物理功能(如烧灼)而对真实组织或器官产生的实质性状态的变化。 [0054] Incidentally, the above-mentioned stress strain data mainly mechanical deformation when real intervention was involved real tissue or organ, due to the structure, shape, size, real interposer or the like generated real tissue or organ data; and the deformation data substantive change in the state is mainly due to substantial state intervention was real physical features (such as burning) and real tissue or organ produced.

[0055]此外,在本发明的一些实施例中,在构建虚拟环境之后,还可以配置仿真介入点与所述虚拟环境中的组织或器官之间的安全距离。 [0055] Further, in some embodiments of the present invention, after constructing the virtual environment, may also be arranged safety distance between the entrance point of the simulated virtual environment of the tissue or organ. 这样可以在虚拟环境中模拟手术过程时, 能够根据配置的安全距离来进行安全范围的预警。 This simulation procedure can be carried out in accordance with the scope of early warning safety safe distance configured in the virtual environment.

[0056]步骤S12,基于医学影像生成在所述虚拟环境中进行显示及交互的建模文件。 [0056] step S12, based on the generated medical image display and interactive modeling file in the virtual environment.

[0057]根据本发明的示例性实施例,如图2所示,步骤S12具体包括: [0057] According to an exemplary embodiment of the present invention, as shown in Step S12 2 comprises:

[0058]步骤S121,根据临床需求和需要呈现在虚拟环境中的病灶和/或器官的信息,通过第一影像设备采集包含所述病灶和/或器官的解剖和功能信息的第一影像文件。 [0058] step S121, the based on clinical needs and requirements and presentation information in a virtual environment, or lesion / organ, by the first image acquisition apparatus comprising the lesion and / or organ of the first image file anatomical and functional information.

[0059] 在本发明的实施例中,第一影像设备可以是CT (Computed Tomography,计算机断层扫描)设备。 [0059] In an embodiment of the present invention, the first imaging device may be a CT (Computed Tomography, Computed Tomography) apparatus.

[0060]步骤S122,基于所述第一影像文件进行三维重建,生成重建文件。 [0060] step S122, the three-dimensional image reconstruction based on the first file, generate a reconstructed file.

[0061]根据本发明的示例性实施例,步骤S122具体包括:根据所述第一影像文件的图像特性,对所述第一影像文件进行感兴趣区域的划分;基于划分的所述感兴趣区域对所述第一影像文件进行图像分割及三维重建,以得到针对每个所述感兴趣区域的重建文件。 [0061] According to an exemplary embodiment of the present invention, comprises the step S122: The image characteristics of the first image file, the image file is divided first region of interest; based on the divided region of interest the first image file for image segmentation and reconstruction, the reconstruction to obtain a file for each area of ​​interest.

[0062]需要说明的是,由于能够对第一影像文件进行感兴趣区域的划分,因此能够将每一个器官、功能块、病灶等独立划分为一个感兴趣区域,进而能够使每一个器官、功能块、病灶等独立形成一个重建文件。 [0062] Incidentally, since the divided region of interest of the first image file, it is possible to every organ, function blocks, and other independent lesions divided into regions of interest, and thus enables each organ function block, and other lesions formed independently a reconstructed file.

[0063]步骤S1M,将所述重建文件的格式转换为所述虚拟环境支持的文件格式,以得到所述建模文件。 [0063] Step S1M, file format conversion to the format of the virtual environment supports the reconstructed file, to obtain the model file.

[00M]在本发明的实施例中,虚拟环境支持的文件格式比如可以是“.OBJ”格式。 [00M] In an embodiment of the present invention, the virtual environment supported file formats such as may be ".OBJ" format. 在格式转换的过程中,需要确保解剖信息不扭曲、不丢失,并且可以针对建模文件在虚拟环境中进行材质、颜色、折射、透明度、光泽的定义和设置。 In the format conversion process, the need to ensure that the information is not distorted anatomy, not lost, and can be material, color in a virtual environment for modeling files, refraction, transparency, gloss and definition of settings.

[0065]在本发明的实施例中,为了向临床医生提供更加清晰的解剖结构功能信息,上述针对每个器官、功能块、病灶等的多个重建文件可以基于不同的影像设备采集的原始数据或者DIC0M影像文件经过一系列数据和图像处理算法重建生成,或者基于同一影像设备采集的不同时间相位期的原始数据或者DIC0M影像文件处理经过一系列数据和图像处理算法重建生成。 [0065] In an embodiment of the present invention, in order to provide clearer information anatomical features to the clinician the original data can be reconstructed for each of the plurality of files organs, functional blocks, such as the lesion based on different image acquisition devices DIC0M image file or a series of data processing and image reconstruction algorithms to generate, based on the original data at different times or phases of the same device to capture an image or a series DIC0M image data file processing and image processing algorithms to generate reconstructed. 此外,还可以对生成的多个重建文件进行图像配准处理,从而实现清晰精准的重现解剖结构和器官功能信息。 In addition, image registration may also be processed to reconstruct a plurality of files generated in order to achieve a clear and precise reproduction organ anatomy information. 具体地,本发明实施例提供了如下三种图像配准方案: In particular, embodiments of the present invention provides the following three image registration scheme:

[0066] 图像配准方案一: [0066] Image registration program:

[0067]通过第一影像设备采集所述病灶和/或器官处于不同相位期的多个第一影像文件;对根据所述多个第一影像文件生成的重建文件进行基于解剖结构和相位时间的图像配准处理。 [0067] by the first image capture device lesion and / or a first plurality of image files in a different phase of the organ; to reconstruct the file according to the first plurality of image files generated based on the phase and the time of anatomy image registration process.

[0068] 图像配准方案二: [0068] Image Registration Scheme II:

[0069]通过第二影像设备采集包含所述病灶和/或器官的解剖和功能信息的第二影像文件;对所述第二影像文件的重建文件和所述第一影像设备采集的第一影像文件的重建文件进行基于解剖特征点的图像配准处理。 [0069] comprising the lesion and / or the second image file organ anatomical and functional information acquired by the second imaging device; reconstruct the file to the second file image and the first image of the first image acquisition device file rebuild the image registration process based on anatomical feature points.

[0070] 图像配准方案三: [0070] Image Registration Program III:

[0071]该图像配准方案为上述图像配准方案一和图像配准方案二的结合方案,即通过第一影像设备采集所述病灶和/或器官处于不同相位期的多个第一影像文件,并通过第二影像设备采集包含所述病灶和/或器官的解剖和功能信息的第二影像文件,然后基于第二影像文件的重建文件和多个第一影像文件的重建文件进行图像配准处理。 [0071] The image registration program is a program of the image and the registration image registration program combines the second solution, i.e., by a first device to capture images of the lesion and / or a first plurality of image files in a different phase of the organ and the second image acquisition apparatus comprising the lesion and / or the second information image file anatomical and functional organ, and then rebuild the file-based image registration file and the second image file of the first plurality of reconstructed image file deal with.

[0072] 在一种可能的实施方式中,可以先对第一影像设备采集的多个第一影像文件的重建文件进行基于解剖结构和相位时间的图像配准处理,然后再对根据多个第一影像文件的重建文件配准后的结果和第二影像设备采集的第二影像文件的重建文件进行基于解剖特征点的图像配准处理。 [0072] In one possible embodiment, the first plurality may be reconstructed image file of the first file a first image capture device performing image registration process anatomy and time-based phase, and then to a plurality of This is resulting in an image file and the second file registration reconstructed image capture device a second image file is a file based on the anatomical feature point image registration processing. 当然,本发明的实施方式并不限于此。 Of course, embodiments of the present invention is not limited thereto.

[0073] 需要说明的是,上述的第一影像设备可以是MR (Magnetic Resonance,磁共振)设备。 [0073] Note that the first image may be a device MR (Magnetic Resonance, MRI) device.

[0074] 步骤S14,根据临床需求,建立仿真建模互动文件。 [0074] step S14, based on clinical needs and build interactive simulation modeling file.

[0075] 根据本发明的示例性实施例,步骤S14包括:根据临床需求,确定需要使用的器械; 基于所述器械的外部特征、物理功能和介入角度,生成所述仿真建模互动文件。 [0075] According to an exemplary embodiment of the present invention, the step S14 comprises: according to clinical needs, the need to determine the instrument; the instrument based on external characteristics, physical function, and the angle involved, generating the interactive simulation modeling file.

[0076] 步骤sie,将所述建模文件和所述仿真建模互动文件导入所述虚拟环境。 [0076] Step sie, the modeling file and the interactive simulation modeling file into the virtual environment.

[0077] 步骤S18,在将所述建模文件和所述仿真建模互动文件导入所述虚拟环境后,基于所述虚拟环境模拟手术过程。 [0077] step S18, after the model file and the simulation model file into the virtual interactive environment, the virtual environment based on simulated surgical procedure.

[0078] 对于步骤S18,以下列举几个模拟手术过程的实施方式,当然本发明的实施方式也不限于此: [0078] For the step S18, the following embodiments include several simulated surgical procedure, of course, embodiments of the present invention is not limited thereto:

[0079] 实施方式一: [0079] The first embodiment:

[0080] 步骤S18包括:确定所述虚拟环境中的跟踪点或观察点;计算并显示所述跟踪点或观察点与所述建模文件中的病灶的坐标相对位置和距离,和/或所述跟踪点或观察点与其周围组织的坐标相对位置和距离。 [0080] Step S18 comprises: determining the tracking point of the virtual environment or watchpoint; calculates and displays the coordinates of the tracking point or focal point of the observation model file relative position and distance, and / or the coordinates of said tracking point or points with the surrounding tissue was observed in the relative position and distance.

[0081] 根据本发明的示例性实施例,可以将仿真建模互动文件与建模文件的第一接触点作为上述的跟踪点或观察点,但是本发明的实施例并不限于此。 [0081] According to an exemplary embodiment of the present invention, the first point of contact with the modeling simulation modeling interaction document file as the tracking point or observation point, embodiments of the present invention are not limited thereto.

[0082] 实施方式二: [0082] Second Embodiment:

[0083] 步骤SIS包括:在所述仿真建模文件介入所述建模文件的过程中,计算并显示所述仿真建模互动文件与所述建模文件之间的交互关系及相关参数。 [0083] SIS step comprises: modeling said file involved in the process simulation modeling file, the computing and displaying the simulation modeling the interaction between the interactive modeling files and files related parameters.

[0084]根据本发明的示例性实施例,所述交互关系及相关参数包括:所述仿真建模互动文件的各个边界与其周围组织的坐标相对位置和距离,和/或所述仿真建模互动文件与所述建模文件中的病灶的坐标相对位置和距离。 [0084] According to an exemplary embodiment of the present invention, the interactions and related parameters including: coordinates of the boundary with the surrounding tissue interaction simulation modeling file relative position and distance, and / or the Interactive Simulation Model the coordinates of the lesion in the file with the file modeling the relative position and distance.

[0085] 实施方式三: [0085] Embodiment three:

[0086] 步骤S18包括:在所述仿真建模文件介入所述建模文件的过程中,计算并显示所述仿真建模互动文件与所述建模文件中的敏感性组织的安全范围,和/或由于所述仿真建模互动文件的介入而对所述建模文件中的组织造成的形变信息。 [0086] Step S18 comprises: modeling process involves the simulation modeling file in the file, the computing and display range of the security document to interact with the simulation model modeling the sensitive tissues of the file, and / or due to the intervention of the simulation modeling interactive documents resulting from the model file tissue deformation information.

[0087] 实施方式四: [0087] Embodiment four:

[0088] 步骤S18包括:计算所述仿真建模互动文件介入所述建模文件的可能路径和/或最优路径;显示所述可能路径和/或所述最优路径。 [0088] Step S18 comprises: calculating a simulation modeling interactive modeling file path of the file may be involved and / or optimal path; displaying said possible paths and / or the optimal path.

[0089] 根据本发明的示例性实施例,计算所述可能路径和/或所述最优路径所依据的参数包括但不限于:所述仿真建模互动文件介入所述建模文件的介入点及介入角度、所述仿真建模互动文件的结构、所述仿真建模互动文件对所述建模文件中的敏感性组织的最小伤害、所述仿真建模互动文件到达所述建模文件中的病灶的最短距离。 [0089] According to an exemplary embodiment of the present invention, the calculation of possible paths and / or the optimal path based on the parameters include but are not limited to: interact with the simulation modeling file intervention intervention point of the modeling file and intervention point of view, the structural modeling interactive simulation file, the file simulation modeling interactive modeling minimal damage to the sensitive tissues of the file, the file arrives interactive simulation modeling in the modeling file the shortest distance lesions.

[0090] 实施方式五: [0090] Embodiment five:

[0091] 步骤S18包括:在所述互动文件与所述影像建模文件中的敏感性组织接触时,进行预警提示。 [0091] Step S18 comprises: when the image in contact with the interactive modeling file in the file-sensitive tissues, suggesting early warning.

[0092] 在本发明的实施例中,预警提示的方式包括但不限于:通过声音的方式进行提示、 通过光的方式进行提示、通过显示屏显示的方式进行提示、通过外接设备(如手柄、手套等) 的震动来进行提示等、在虚拟环境中的预警屏幕进行提示。 [0092] In an embodiment of the present invention, the warning prompt manner include but are not limited to: voice prompts through, by way of prompt light, by way of prompt displayed on the screen, with an external device (such as handles, vibration gloves, etc.) to carry out tips, warning screen in a virtual environment prompt.

[0093] 在本发明的一些实施例中,基于前述方案,上述的手术模拟方法还可以包括:根据所述建模文件、所述虚拟环境和在仿真计算时的复杂程度,确定运行所述虚拟环境的设备类型;基于所述设备类型,生成运行所述虚拟环境的应用程序。 [0093] In some embodiments of the present invention, based on the foregoing embodiment, the above-described surgical simulation method may further comprise: modeling based on the file, and the complexity of the virtual environment during the simulation, determining the virtual operating device type environment; based on the device type, the operation of generating a virtual application environment.

[0094]具体来说,若建模文件、虚拟环境和仿真计算较为复杂,则可以选择在具有较强计算能力的设备(如计算机)上运行;相反地,若建模文件、虚拟环境和仿真计算相对简单,则可以选择在移动设备(如智能手机、平板电脑等)上来运行。 [0094] Specifically, if the document model, simulation and virtual environments more complex, may be selected to run on the device (e.g., a computer) has strong computational power; conversely, if the file modeling, simulation and virtual environments relatively simple to calculate, you can select a mobile device (such as smart phones, tablet computers, etc.) in the run up.

[G095]图3示意性示出了根据本发明的第二个实施例的基于虚拟现实和医学影像的手术模拟方法的流程图。 [G095] FIG. 3 schematically shows a flowchart of a second method of virtual reality surgical simulation and medical imaging according to the present embodiment of the invention.

[00%] 参照图3,根据本发明的第二个实施例的基于虚拟现实和医学影像的手术模拟方法,包括以下步骤: [00%] Referring to Figure 3, surgical simulation and medical imaging based on virtual reality a second embodiment of the present invention, comprising the steps of:

[0097] 步骤S301,建立临床需求。 [0097] step S301, the establishment of clinical needs.

[0098]临床上存在着各种复杂的病例/病灶,需要临床医生针对个体病例设计可行性治疗方案,并基于可行性的基础上选择最优方案。 [0098] There are various complicated cases / focus on clinical need for the clinician to design the feasibility of individual cases of treatment options, and choose the best program on the basis of the feasibility. 虚拟现实技术和临床影像的结合,可以重现病灶和周边敏感性器官,模拟建模手术或者微创介入环境,允许医生在一个仿真虚拟又完全呈现现实解剖病理结构的环境中制定、执行、优化,及重复的搭建手术或者介入的过程, 增加疑难病症的治疗可行性和手术成功率。 Virtual reality technology and combined with clinical images, can reproduce the lesions and the surrounding sensitive organs, or minimally invasive surgery simulation modeling environment that allows doctors in a simulated virtual environment and fully rendered realistic anatomical pathology structures in the development, implementation, optimization, and repeat the process to build a surgery or intervention, therapy increase the feasibility and success rate of incurable diseases.

[00"] 步骤S3〇2,病灶/器官及周围解剖结构基于医学影像的三维建模,得到建模文件。 [00 "] step S3〇2 lesion / organ and surrounding anatomical structures based three-dimensional modeling of medical imaging to obtain a modeling file.

[0100] 对于步骤S302,本发明提出了如下两种具体的实施例: [0100] For the step S302, the present invention proposes the following two specific examples:

[0101] 实施例一: [0101] Example a:

[0102]参照图4所示,在本发明的一个示例性实施例中,步骤S302包括: [0102] Referring to FIG, in one exemplary embodiment of the present invention embodiment, step 4 S302 comprises:

[0103] 步骤S3021,图像及数据采集。 [0103] Step S3021, the image and data collection.

[0104]具体来说,根据临床需求,以及临床医生和影像专家确认的需要呈现在虚拟环境中的病灶和组织器官的信息,采集相对应器官/病灶的解剖和功能信息。 [0104] In particular, according to clinical needs, as well as clinicians and imaging experts confirm the need to present information lesions in organs and tissues in virtual environments, collect corresponding organ / lesions in anatomical and functional information. 例如,若三维建模用于制定手术最优方案,那么建模需要包含:病灶信息、病灶周围组织器官信息、敏感性器官信息、手术过程中/介入过程中的路径信息,以及与病灶相连接的重要不可触碰的器官信息(例如,主动脉,神经)。 For example, if three-dimensional modeling procedure used to develop the optimal solution, then the model needs to include: information lesions, lesions around the tissues and organs of information, sensitive information organs during surgery / interventional procedure in the path information, and connected with lesions untouchable organ important information (eg, aorta, nerve).

[0105]同时,可以通过CT和MR采集包含器官和病灶信息,以精确反应人体解剖结构的影像文件,该文件包括DICOM(Digital Imaging and Communications in Medicine,医学数字成像和通信)文件和裸数据(raw data)文件。 [0105] Also, by CT and MR acquisition comprising organs and lesions information to accurately reflect the image files human anatomy, the document comprising a DICOM (Digital Imaging and Communications in Medicine, Digital Imaging and Communications) files and the bare data ( raw data) file. 为了获得空间分辨率足够高的三维重建图像,会采用高端CT的图像文件作为基础,数据采集层厚一般小于〇.6ram,经过特定的算法使得层厚在0 • 3mm-0 • 6mm之间,时间分辨率需要高于1/2的心动周期,保持xy—z轴方向各向同性。 In order to obtain a sufficiently high spatial resolution image reconstruction, high-end CT image will be used as a base file, the data acquisition is typically less than a layer thickness 〇.6ram, through a particular algorithm such that the thickness of 0 • 3mm-0 • between 6mm, higher than the required time resolution of 1/2 of the cardiac cycle, holding xy-z-axis isotropic.

[0106]需要说明的是:上述的DIC0M是医学图像和相关信息的国际标准(IS0 12052),它定义了质量能满足临床需要的可用于数据交换的医学图像格式。 [0106] Note that: the above-mentioned international standards DIC0M medical images and related information (IS0 12052), which defines the format of the medical image quality to meet clinical needs for data exchange. DI COM被广泛应用于放射医疗,心血管成像以及放射诊疗诊断设备(X射线、CT、核磁共振、超声等),并且在眼科和牙科等其它医学领域得到越来越深入广泛的应用。 DI COM is widely used in radiology, cardiovascular imaging and radiology diagnostic equipment (X-ray, CT, MRI, ultrasound, etc.), and get more in-depth widely used in other medical fields such as ophthalmology and dentistry. 在数以万计的在用医学影像设备中,DIC0M 是部署最为广泛的医疗信息标准之一。 With tens of thousands of medical imaging equipment, DIC0M is one of the most widely deployed standard of medical information.

[0107]上述的裸数据是一个医学影像的专有名词,例如可以指从CT系统里直接提取出来的未做2维重建图像(DIC0M)的直接从探测器上采集的经过人体衰减的数据。 [0107] The bare data is a proper noun medical imaging, for example, may refer to data extracted from the CT system out directly without making a two-dimensional reconstructed images (DIC0M) collected directly from the detector through the body attenuated.

[0108]此外,为了获得更加精准的三维重建文件,采用同一种影像设备中的不同相位期的影像文件也是必须的,例如,在CT成像中为了更清楚的显示动脉和静脉的血管信息,会在患者体内注入一定的碘造影剂,并分别采集在动脉期和静脉期的影像信息,为最终的三维重建提供数据。 [0108] Further, in order to obtain more accurate three-dimensional reconstruction of the file, using the image file of the same imaging apparatus in different phases of also necessary, for example, in CT imaging in order to more clearly show blood vessel information arteries and veins, would injection in a patient in certain iodinated contrast media, and were collected in the image information of the arterial and venous phase, to provide data for the final three-dimensional reconstruction.

[0109] 步骤S3022,图像预处理。 [0109] Step S3022, image preprocessing.

[0110] 步骤S3〇23,数据预处理及2维图像重建。 [0110] Step S3〇23, 2-dimensional image data pre-processing and reconstruction. 步骤S30M,3维图像重建及图像后处理。 Step S30M, 3-dimensional image reconstruction and image processing.

[0112] 步骤S3〇25,基于医学影像的图像建模。 [0112] Step S3〇25, the image model based on medical imaging.

[0113]在本发明的示例性实施例中,对图像的处理可以包括图像预处理,后处理,以及2D 转换3D图像建模。 [0113] In an exemplary embodiment of the present invention, the processing of the image may include an image pre-processing and post-processing, convert the 3D image and 2D modeling. 基于DIC0M图像或者裸数据,根据图像特性,例如CT的HU值和低对比度,对图像进行感兴趣区域划分、图像分割和3D建模,最终使每一个器官,功能块,病灶独立形成一个重建文件。 DIC0M image data based on a bare or, in accordance with image characteristics, e.g. HU values ​​of CT and low contrast, dividing the image region of interest, image segmentation and 3D modeling, and finally to every organ, function block, a reconstructed file form independent foci . 并通过图像配准,例如参考MR下同一器官/病灶的图像,与获得的高空间分辨率的CT图像根据解剖特征点进行配准,得到更加清晰的功能信息。 And by image registration, for example, with reference to the MR images of the same organ / lesions, and to obtain a high spatial resolution CT images anatomical registration feature points, to obtain a clearer function information.

[0114]步骤S3026,生成基于医学影像的病灶/器官及周围解剖结构的可在VR环境下显示和互动的建模文件。 [0114] Step S3026, generated based on the display and interactive modeling file lesions Medical Imaging / organ and surrounding anatomical structures in the VR environment.

[0115]具体来说,将基于医学影像的每个病灶和器官及周边组织的3D建模文件转换成虚拟现实环境可接受文件,如“.OBJ”文件(一种标准的3D模型文件格式)。 [0115] In particular, it will be converted into virtual reality environment acceptable file based on medical images of each lesion and 3D modeling files organs and surrounding tissues, such as ".OBJ" file (a standard 3D model file format) . 并在转换过程中确保解剖信息不扭曲、不丢失,并对每个3D建模文件,在虚拟现实的环境中的进行材质,颜色, 折射,透明度,光泽的定义和设置。 And to ensure that anatomical information during the conversion process is not distorted, not lost, and each 3D modeling files, virtual reality environments are material, color, refraction, transparency, gloss and definition of settings.

[0116] 实施例二: [0116] Example II:

[0117] 参照图5所示,在本发明的二个示例性实施例中,主要是基于医学影像DIC0M文件和/或裸数据生成建模文件。 As shown in [0117] Referring to FIG 5, in the two exemplary embodiments of the present invention, the main model is generated based on medical imaging DIC0M document file and / or raw data.

[0118] 具体地,根据临床医生和影像专家工程师确认的需要呈现在虚拟环境中的病灶和组织器官的消息,采集临床医学影像数据,对其进行图像预处理、后处理、重建及格式转换和优化等一系列影像处理技术,形成可被虚拟现实环境接受的文件格式。 [01] Specifically, according to clinicians and imaging expert engineers recognized the need to present a message lesions in organs and tissues in a virtual environment, collecting clinical image data, its image pre-processing, post-processing, and format conversion and reconstruction optimization and a series of image processing technology, the formation can be accepted virtual reality environment file format. 下面结合图5对其进行说明。 Below in connection with FIG. 5 it is described.

[0119] 首先,进行图像和数据采集。 [0119] First, image and data acquisition.

[0120] 在本发明的一个实施例中,可以获取DIC0M图像。 [0120] In one embodiment of the present invention, images can be acquired DIC0M. 在本发明的另一实施例中,在另一实施例中,可以获取裸数据,然后对该裸数据进行数据预处理和图像重建。 Embodiment, in another embodiment, data may be acquired bare, then the data pre-processing and image reconstruction data naked In another embodiment of the present invention. 例如:降噪、去伪影、感兴趣区域选择(R〇I (region of interest) info)、迭代重建增强等等。 For example: noise reduction, to the artifact, the region of interest is selected (R〇I (region of interest) info), iterative reconstruction enhancement and the like.

[0121] 其次,对上述采集的图像和数据进行图像处理。 [0121] Next, the above-described acquired image data and performs image processing.

[0122] 在图5所示的实施方式中,图像处理可以包括图像预处理、图像后处理和图像建模。 [0122] In the embodiment shown in FIG. 5, the image processing may include image preprocessing, image after image processing and modeling.

[0123] 其中,图像预处理例如可以包括:图像配准、感兴趣区域增强,骨提取、肌肉提取、 图像伪影消除等。 [0123] wherein, the image pre-processing may include, for example: image registration, the region of interest enhancement, extraction of bone, muscle extracts, and the like to eliminate image artifacts. 图像后处理例如可以包括:图像分裂,过滤,扩散,量化,转换等。 After the image processing, for example, may comprise: an image splitting, filtering, diffusing, quantization, conversion. 图像建模例如可以包括:切割,过滤,标识等。 Image-based modeling include, for example: cutting, filtered and logo.

[0124] 其中,这里的图像建模可以采用三维图像重建及显示:利用经典三维重建算法,对获取的序列图像在计算机上进行三维重建;并利用可视化技术显示出模体数据中各个断层图像数据,同时包括了裁剪、缩放图像处理功能,实现了对图像的更多的交互处理。 [0124] wherein image-based modeling herein may be employed a three-dimensional image reconstruction and display: using classical three-dimensional reconstruction algorithm, sequence of images acquired by three-dimensional reconstruction on the computer; and visualization technique shows each tomographic image data phantom data while including cropping, scaling, image processing functions to achieve a more interactive processing of the image.

[0125] 通过对上述图像处理的迭代过程,并经影像专家工程师和临床医生确保图像/模体精准的呈现器官和病灶信息,生成建模文件。 [0125] through an iterative process of the image processing, image experts and by engineers and clinicians to ensure accurate image rendering / mold body organs and lesions information generated modeling files.

[0126] 继续参照图3,所述的手术模拟方法还包括: [0126] With continued reference to FIG. 3, the operation of the simulation method further comprises:

[0127] 步骤S303,搭建VR模拟环境。 [0127] Step S303, the simulation environment built VR.

[0128] 根据本发明的示例性实施例,可以利用虚拟引擎搭建虚拟环境,例如,可以通过Unity引擎和C-sharp语言编辑功能函数实现虚拟环境的搭建和输入/输出管理器的设置。 [0128] According to an exemplary embodiment of the present invention may be utilized to build a virtual environment virtualization engine, for example, may be implemented to build and set the input / output manager of the virtual environment through Unity engine and C-sharp language editing functions.

[0129] 具体过程如图6所示,包括: [0129] In particular the process shown in FIG. 6, comprising:

[0130] 步骤S3031,建立观察场、视野角度/镜头方位、照明方向、坐标系等基本场景参数。 [0130] Step S3031, the establishment of the basic parameters of the scene observation field, the viewing angle / orientation of the camera, the illumination direction, the coordinate system and the like. 具体地,图7示出了根据本发明的实施例的搭建出的虚拟环境及基本场景参数的设置的界面示意图。 In particular, FIG. 7 shows a schematic view of the interface according to the setting out to build a virtual environment in which embodiments of the present invention, and the basic parameters of the scene.

[0131] 步骤S3032,进行输入管理器设计和配置。 [0131] Step S3032, the input manager for design and configuration.

[0132] 其中,输入管理器设计和配置主要是配置输入给VR场景的指令和交互信息。 [0132] wherein, the input manager is to configure the design and configuration of the main input to the interactive information and instructions VR scenes. 例如: 位置定义和输入指令、引导定义和输入指令、敏感度定义和输入指令、动作定义和输入指令等,以及各个外接设备,例如手柄,手套,头盔的输入设置。 For example: define the location and input instructions, and input commands define the guide, and the sensitivity is defined as the input command, and the action definition command input, etc., and various external devices, such as input a handle, gloves, helmet. 根据本发明的示例性实施例,可以通过配置界面来进行输入管理的设计和配置。 According to an exemplary embodiment of the present invention, it may be input by the design and configuration management configuration interface.

[0133] 步骤S3033,进行输出管理器设计和配置。 [0133] Step S3033, the output manager for design and configuration.

[0134] 其中,输出管理器设计和配置主要是配置VR场景输出的指令和交互信息。 [0134] wherein the output manager primarily designed and configured and arranged to interact instruction information output VR scene. 例如,场景中交互文件的位置坐标显示、安全空间显示、建模文件碰撞输出设置等,以及各个外接设备,例如手柄,手套的输出设置。 For example, the position coordinates of the scene file is displayed interactively, security display space, the modeling output setting file collision, etc., and various external devices, such as a handle, a glove output settings.

[0135] 步骤S3034,设置仿真介入点介入产生的周围组织/器官的形变模型和安全范围。 [0135] Step S3034, set the simulation point of intervention intervention generated around the tissue / organ deformation range of models and security.

[0136] 由于正常人体的组织和器官在外界物体进入的途径中会产生相应的形变,该形变包括根据外界进入物体(例如,手术刀,介入针)的结构、形状、大小等的不同产生的应力形变及物理功能(例如,烧灼)不同而产生的实质性状态的变化,这种形变需要在虚拟环境中提前建模设置。 [0136] Since normal human tissues and organs generated in a corresponding way a foreign object into the deformation of the deformation depending comprising generating enters the external object (e.g., a scalpel, a needle intervention) structure, shape and size of the substantial change of the physical state of stress and deformation function (e.g., burn) generated by the difference, this deformation model provided in advance in the virtual environment. 同样外界进入物体与周围敏感性器官的安全距离,也是需要在虚拟环境中提前定义和设置的。 Also a safe distance from the outside world into the object and the surrounding sensitive organs, but also the need to define in advance and set in the virtual environment. 其中,在步骤S3034中进行设置时,需要知道仿真介入点的尺寸和结构。 Wherein, when set in the step S3034, the simulation needs to know the size and structure of intervention points.

[0137] 步骤S3035,在虚拟环境中进行相互位置坐标显示和接触警示数据提示。 [0137] Step S3035, the mutual positional coordinates of the contact and displays a warning prompt data in a virtual environment.

[0138] 继续参照图3,所述的手术模拟方法还包括: [0138] With continued reference to FIG. 3, the operation of the simulation method further comprises:

[0139] 步骤S304,建立仿真建模互动文件。 [0139] step S304, the establishment of interactive simulation modeling file.

[0140] 在本发明的实施例中,可以根据不同的临床手术的需求,建立一系列的仿真建模互动文件用来表示,例如,手术刀,介入针等治疗器械。 [0140] In an embodiment of the present invention, according to the needs of different clinical procedures, simulation modeling to establish a series of files used to indicate interaction, e.g., a scalpel, a needle or the like interventional therapy device. 在建模互动文件中,包括形状,结构, 大小及物理功能,和介入进入角度。 Interactive modeling file, including the shape, structure, size and physical function, and intervention into the angle. 根据本发明的示例性实施例,仿真建模互动文件的格式可以是“.OBJ”格式。 According to an exemplary embodiment of the present invention, interactive simulation modeling format file may be ".OBJ" format.

[0141] 需要说明的是,上述步骤S302、步骤S303和步骤S304之间的执行顺序可以进行互换。 [0141] Note that the above step S302, the execution sequence between step S303 and step S304 may be interchanged.

[0142] 步骤S305,将建模文件和建模互动文件导入搭建的模拟环境。 [0142] step S305, the modeling files and the files into the interactive modeling simulation environment built.

[0143] 具体来说,在步骤S305中,将影像建模的各个文件(病灶、器官、周围组织、敏感功能性器官等)和建模互动文件同时导入预设的虚拟现实平台。 [0143] Specifically, in step S305, the modeling of each image file (lesions, organs, surrounding tissues, sexual organs and other sensitive functions) and interactive modeling files while importing preset virtual reality platform. 在虚拟环境中,对图像/建模分辨率进行优化达到临床应用的要求同时提高其运行速度,并对各个建模文件在虚拟现实的环境中进行材质,颜色,折射,透明度,光泽的定义和设置。 In a virtual environment, image / modeling resolution required to achieve clinical application optimization while increasing its speed, and various modeling file materials, colors in a virtual reality environment, refraction, transparency, gloss and definition settings.

[0144] 步骤S306,跟踪介入点,根据介入文件的尺寸和结构及介入方式计算并显示介入点和周围建模组织/器官在坐标系的位置关系,提供接触警示和安全范围警示。 [0144] step S306, the tracking point of intervention, intervention points are calculated and displayed modeling and surrounding tissues / organs in the positional relationship between the coordinate system, and providing a contact safety margin warning alerts the size and configuration files involved and interventions.

[0145] 根据本发明的示例性实施例,可以设置建模互动文件(例如,介入针)与影像建模文件的第一接触点(例如:介入针的针头)为跟踪点或观察点。 [0145] According to an exemplary embodiment of the present invention may be provided to interact with the modeling file (e.g., intervening needles) with the first contact point of the modeling image file (for example: interventional needle needle) of the tracking point or observation point. 当然,本发明的实施例并不限于此。 Of course, embodiments of the present invention are not limited thereto.

[0146] 所述的跟踪点或观察点,可以设置为手部动作或者位置跟踪,并通过外界输入(如手势,手柄等)到虚拟环境中显示屏幕参数进行互动的跟踪,同样,本发明的实施方式并不局限于此,可以根据外接设备的设置和输入输出管理器的设置有不同的方式。 Tracking points or observation points [0146] The may be provided to track hand motion or a position, and tracked by interactive external input (e.g. gesture, handles, etc.) to the display screen a virtual environment parameters, similarly, the present invention is not limited to this embodiment, there may be provided in different ways according to the external input and output setting device manager.

[0147] 步骤S3〇7,跟踪介入点,计算并显示介入点和病灶在坐标系的位置关系,并建议最优进入路径。 [0147] Step S3〇7, the tracking point of intervention, intervention points are calculated and displayed and the lesion in the coordinate system of the positional relationship, and suggested that the optimal path enters. 具体地,图8示出了根据本发明的实施例的跟踪点/观察点在组织或器官内的示意图。 In particular, FIG. 8 shows a schematic diagram of an embodiment of the tracking point of the present invention / observation point in the tissue or organ.

[0148] 根据本发明的示例性实施例,可以根据互动文件的进入角度(互动文件与影像建模文件的夹角)、互动文件的结构、互动文件对建模文件中的敏感性组织的最小伤害、互动文件到达建模文件中的病灶的最短距离等,计算并显示互动文件的可能进入路径及最优进入路径。 [0148] According to an exemplary embodiment of the present invention, it can interact with the file entry angles (angle of the image file interaction modeling file) structure, minimal interaction document file interactive modeling file sensitive tissues injury, interactive file the shortest distance lesion modeling files, etc., may be calculated and displayed into the path of interactive documents and enter the optimal path.

[0149]为了进一步说明如何计算可能进入路径和最优进入路径,本发明实施例给出了如下两个例子:1、若介入针长度为2〇cm,如果需要的最短直线距离已经大于20cm,则该进入方案的可行性为零;2、如果介入针在进入过程中插中动脉大血管,则该进入方案的可行性为零。 [0149] To further illustrate the calculation of the optimal path and may enter into the path, embodiments of the present invention gives the following two examples: 1, if the length of the needle 2〇cm intervention, if the shortest straight line distance is greater than has been required 20cm, feasibility of the program enters zero; 2, if the needle is inserted in the artery intervention large vessels entering the process enters the feasibility zero. 需要说明的是,上述的两个例子仅为示例,本发明的实施方式并不局限于此。 Incidentally, the above-described two examples are merely exemplary embodiments of the present invention is not limited thereto.

[0150]步骤S30S,判断仿真建模是否复杂,若是,则执行步骤S311;否则,执行步骤S309。 [0150] Step S30S, simulation modeling to determine whether the complex, and if so, to step S311; otherwise, to step S309.

[0151]具体来说,可以根据仿真建模文件、虚拟环境,以及后台仿真计算的复杂程度,确定将生成的交互环境程序在计算机上运行或者生成智能设备(如智能手机)的APP并下载到智能设备中运行。 [0151] Specifically, according to the simulation modeling files, virtual environments, as well as the complexity of the background of simulation, interactive environment to determine the programs will run on a computer-generated or generation of smart devices (such as smartphones) and downloaded to the APP the operation of smart devices.

[0152] 步骤S309,生成交互环境APP。 [0152] step S309, the generated interactive environment APP.

[0153] 步骤S310,将APP导入智能设备,并链接输入输出设备。 [0153] step S310, the APP introducing the smart device, input-output devices and links.

[0154]具体来说,对于简单的虚拟环境和简单仿真建模都可以在智能设备上运行。 [0154] In particular, you can run virtual environment for simple and simple simulation modeling on the smart device. 比如可以生成手机APP,然后下载到智能手机内,再将手机安装在虚拟眼镜中,用户就可以开始与虚拟环境互动了。 For example, you can generate a mobile phone APP, then downloaded to smart phones, mobile phone and then installed in the virtual glasses, users can begin to interact with a virtual environment. 如图9所示,为本发明实施例的将虚拟交互环境在手机APP上进行运行的一个示例。 9, the embodiment of a virtual interactive environment for example APP running on the mobile phone of the present invention.

[0155] 步骤S311,生成交互环境计算机应用程序。 [0155] Step S311, generates the interactive environment of a computer application.

[0156] 步骤S312,在计算机运行应用程序,链接输入输出设备。 [0156] step S312, the computer running the application, the link input and output devices.

[0157] 具体来说,对于具有复杂的后台计算和大量的仿真建模文件的虚拟环境,由于对计算的要求度高,因此可以在计算机上运行,并通过外接设备和传输导线将虚拟环境展示在链接的眼镜或者头盔上显示给用户。 [0157] In particular, for virtual computing environment with complex background and a lot of simulation modeling files, because it can run on a computer with high computing requirements, and through external devices and transmission line will show a virtual environment displayed to the user on the glasses or helmet link. 如图10所示,为本发明实施例的通过虚拟眼镜或头盔等显示的虚拟交互环境示意图。 10, the virtual interactive environment is a schematic diagram showing a helmet or the like virtual glasses embodiment of the present invention.

[0158]上述的输入输出设备主要是指虚拟眼镜/头盔、手柄、手套、外界摄像设备及虚拟环境程序的运行设备。 [0158] The main input and output devices is a virtual glasses / helmet, handles, gloves, the external operating device and the imaging equipment virtual environment program. 具体可以根据虚拟环境程序的复杂程度、用户的习惯,以及数入输出管理器的指令定义和设置进行不同的链接编码。 DETAILED concatenated coding may be different depending on the complexity of the virtual application environment, the user's habits, and the number of the output manager to define and set instruction.

[0159] 步骤S313,利用跟踪介入点和手柄坐标点定位及交互行为进行医生与虚拟环境的互动和信息反馈。 [0159] step S313, and interact with the virtual environment of the information doctors use to track feedback intervention point and the coordinates of point positioning and handle interactions.

[0160]图11示意性示出了根据本发明的一个实施例的基于虚拟现实和医学影像的手术模拟装置的框图。 [0160] FIG. 11 schematically shows a block diagram based on virtual reality surgical simulation apparatus and a medical image of the embodiment according to the present invention.

[0161] 参照图11,根据本发明的一个实施例的基于虚拟现实和医学影像的手术模拟装置1100,包括:构建单元1101、生成单元1102、建立单元1103、导入单元1104和处理单元1105。 [0161] Referring to FIG 11, in accordance with one embodiment of the present invention based on virtual reality surgical simulation and medical imaging apparatus 1100, comprising: a constructing unit 1101, generating means 1102, establishing unit 1103, unit 1104 and introducing the processing unit 1105. [0162] 具体地,构建单元1101构建用于模拟手术过程的虚拟环境;生成单元1102用于基于医学影像生成在所述虚拟环境中进行显示及交互的建模文件;建立单元1103用于根据临床需求,建立仿真建模互动文件;导入单元1104用于将所述建模文件和所述仿真建模互动文件导入所述虚拟环境;处理单元1105用于在所述导入单元将所述建模文件和所述仿真建模互动文件导入所述虚拟环境后,基于所述虚拟环境模拟手术过程。 [0162] Specifically, the construction unit 1101 constructs a virtual environment for simulating a surgical procedure; generating unit 1102 for generating a display and interactive modeling file in the virtual environment based on the medical image; establishing unit 1103 according to the clinical demand, interactive simulation modeling to establish a file; import unit 1104 for the file and the simulation model modeling file into the virtual environment interaction; processing unit 1105 for the file in the import modeling unit the interactive simulation and modeling file import the virtual environment, the virtual environment based on simulated surgical procedure.

[0163]需要说明的是,上述基于虚拟现实和医学影像的手术模拟装置中各模块/单元的具体细节已经在对应的基于虚拟现实和医学影像的手术模拟方法中进行了详细的描述,因此此处不再赘述。 [0163] Incidentally, the above-described already in the corresponding detailed description is based surgical simulation methods of virtual reality and the medical image based on the details of the surgical simulator virtual reality and medical imaging of the modules / units, and therefore this at not repeat them.

[0164] 此外,本发明的实施方式还提供一种电子设备,可以包括:处理器和存储器,其中, 所述存储器存储有可执行指令,所述处理器用于调用所述存储器存储的可执行指令执行如本发明上述实施例中所述的基于虚拟现实和医学影像的手术模拟方法。 [0164] Further, embodiments of the present invention also provides an electronic device may include: a processor and a memory, wherein the memory stores executable instructions for invoking the memory storing processor-executable instructions performing surgery simulation method according to the above embodiment of the present invention based on the medical images and virtual reality.

[0165] 在示例性实施例中,还提供了一种包括指令的非临时性计算机可读存储介质,例如包括指令的存储器,上述指令可由电子设备的处理器执行以完成在示例性实施例中的上述技术方案。 [0165] In an exemplary embodiment, further comprising instructions provided a non-transitory computer-readable storage medium such as a memory comprising instructions, the command executable by the processor to complete the electronic device in an exemplary embodiment the above technical solutions. 例如,所述非临时性计算机可读存储介质可以是R0M、随机存取存储器(RAM)、 CD-ROM、磁带、软盘和光数据存储设备等。 For example, the non-transitory computer-readable storage medium may be a R0M, a random access memory (RAM), CD-ROM, magnetic tapes, floppy disks, and optical data storage devices.

[0166] 应当注意,尽管在上文详细描述中提及了用于动作执行的设备的若干模块或者单元,但是这种划分并非强制性的。 [0166] It should be noted that although a number of modules or unit of a device for performing an operation in the detailed description above, but this division is not mandatory. 实际上,根据本发明的实施方式,上文描述的两个或更多模块或者单元的特征和功能可以在一个模块或者单元中具体化。 In fact, according to an embodiment of the present invention, two or more features and functions of the modules or units described above may be embodied in a module or unit. 反之,上文描述的一个模块或者单元的特征和功能可以进一步划分为由多个模块或者单元来具体化。 Conversely, features and functions of a cell or module described above may be further divided by a plurality of modules or units of concrete.

[0167] 通过以上的实施方式的描述,本领域的技术人员易于理解,这里描述的示例实施方式可以通过软件实现,也可以通过软件结合必要的硬件的方式来实现。 [0167] By the above described embodiments, those skilled in the art will readily appreciate that the example embodiments described herein may be implemented by software, may also be implemented in conjunction with the necessary hardware manner by the software. 因此,根据本发明实施方式的技术方案可以以软件产品的形式体现出来,该软件产品可以存储在一个非易失性存储介质(可以是⑶-R0M,U盘,移动硬盘等)中或网络上,包括若干指令以使得一台计算设备(可以是个人计算机、服务器、触控终端、或者网络设备等)执行根据本发明实施方式的方法。 Thus, According to the embodiment of the present invention may be embodied in a software product out, the software product may be stored in a nonvolatile storage medium (which may be ⑶-R0M, U disk, mobile hard disk, etc.) on the network or It includes several instructions to enable a computing device (may be a personal computer, a server, a touch terminal, or a network device) to perform a method according to embodiments of the present invention.

[0168]本领域技术人员在考虑说明书及实践这里公开的发明后,将容易想到本发明的其它实施方案。 [0168] Those skilled in the art upon consideration of the specification and practice of the invention disclosed herein, will readily appreciate other embodiments of the present invention. 本申请旨在涵盖本发明的任何变型、用途或者适应性变化,这些变型、用途或者适应性变化遵循本发明的一般性原理并包括本发明未公开的本技术领域中的公知常识或惯用技术手段。 This application is intended to cover any variations, uses, or adaptations of the present invention encompasses these variations, uses, or adaptations of the invention following the general principles of the common general knowledge and comprises the art of the present invention are not disclosed in the conventional techniques or . 说明书和实施例仅被视为示例性的,本发明的真正范围和精神由下面的权利要求指出。 The specification and examples be considered as exemplary only, with a true scope and spirit of the invention indicated by the following claims claim.

[0169]应当^解的是,本发明并不局限于上面已经描述并在附图中示出的精确结构,并且可以在不脱离其范围进行各种修改和改变。 [0169] ^ solution should be that the present invention is not limited to the above has been described and illustrated in the drawings precise structure, and may be carried out without departing from the scope of the various modifications and changes. 本发明的范围仅由所附的权利要求来限制。 Scope of the invention be limited only by the appended claims.

Claims (21)

1.一种基于虚拟现实和医学影像的手术模拟方法,其特征在于,包括: 构建用于模拟手术过程的虚拟环境; 基于医学影像生成在所述虚拟环境中进行显示及交互的建模文件; 根据临床需求,建立仿真建模互动文件; 将所述建模文件和所述仿真建模互动文件导入所述虚拟环境; 在将所述建模文件和所述仿真建模互动文件导入所述虚拟环境后,基于所述虚拟环境模拟手术过程。 1. A surgical simulation and virtual reality based on medical imaging, characterized by comprising: constructing a surgical procedure for simulating a virtual environment; generate and display interactive modeling file in the virtual environment based on the medical image; according to clinical needs, establish interactive simulation modeling file; the model file and the simulation modeling files into the interactive virtual environment; in the model file and the simulation modeling files into the interactive virtual after the environment, the virtual environment based on simulated surgical procedure.
2.根据权利要求1所述的手术模拟方法,其特征在于,还包括: 配置所述虚拟环境的场景参数,以及所述虚拟环境对应的输入管理器和/或输出管理器。 The surgical simulation method according to claim 1, characterized in that, further comprising: configuring parameters of a scene of the virtual environment, the virtual environment and manager corresponding input and / or output manager.
3.根据权利要求2所述的手术模拟方法,其特征在于,所述场景参数包括以下至少之一或多个的组合: 观察场、视野角度、镜头方向、照明方向、坐标系。 3. The surgical simulation method according to claim 2, wherein the scene parameter includes a combination of at least one or more of the following: observation field, viewing angle, the lens direction, the illumination direction, the coordinate system.
4. 根据权利要求1所述的手术模拟方法,其特征在于,还包括: 配置仿真介入点的交互操作对所述虚拟环境中的组织或器官产生的形变模型。 The surgical simulation method according to claim 1, characterized in that, further comprising: configuring deformation model interaction simulation of the intervention point of the virtual environment generated by the tissue or organ.
5. 根据权利要求4所述的手术模拟方法,其特征在于,还包括: 获取真实介入物在介入真实组织或器官时,对所述真实组织或器官产生的应力形变数据和/或实质性状态的形变数据; 根据所述应力形变数据和/或所述实质性状态的形变数据配置所述形变模型。 The surgical simulation method according to claim 4, characterized in that, further comprising: acquiring real deformation stress when intervening interposer real tissue or organ, the tissue or organ to produce the real and / or substantial state the deformation data; configuring the deformable model according to the stress-strain data and / or substantial deformation of the state data.
6.根据权利要求1所述的手术模拟方法,其特征在于,还包括: 配置仿真介入点与所述虚拟环境中的组织或器官之间的安全距离。 The surgical simulation method according to claim 1, characterized in that, further comprising: configuring the virtual simulation intervention point safety distance between the tissue or organ environment.
7.根据权利要求1所述的手术模拟方法,其特征在于,基于医学影像生成在所述虚拟环境中进行显示及交互的建模文件,包括: 根据临床需求和需要呈现在虚拟环境中的病灶和/或器官的信息,通过第一影像设备采集包含所述病灶和/或器官的解剖和功能信息的第一影像文件; 基于所述第一影像文件进行三维重建,生成重建文件; 将所述重建文件的格式转换为所述虚拟环境支持的文件格式,以得到所述建模文件。 The surgical simulation method according to claim 1, wherein the medical image is generated based on modeling of the file display and interaction in the virtual environment, comprising: based on clinical needs and requirements presented in the virtual environment of the lesion information and / or organs, by the first image acquisition apparatus comprising the lesion and / or organs of a first image file of the anatomical and functional information; three-dimensional reconstruction image based on the first file, generate a reconstructed file; the convert the file format to format the virtual environment to support the reconstruction of the file to get the model file.
8.根据权利要求7所述的手术模拟方法,其特征在于,基于所述第一影像文件进行三维重建,生成重建文件的步骤,包括: 根据所述第一影像文件的图像特性,对所述第一影像文件进行感兴趣区域的划分; 基于划分的所述感兴趣区域对所述第一影像文件进行图像分割及三维重建,以得到针对每个所述感兴趣区域的重建文件。 8. The surgical simulation method according to claim 7, characterized in that the three-dimensional image reconstruction based on the first file, the file generating step of reconstructing, comprising: an image characteristic of the first image file, the a first image file divided region of interest; image segmentation and reconstruction of the first video file based on the divided region of interest, to obtain a reconstructed file for each area of ​​interest.
9.根据权利要求7所述的手术模拟方法,其特征在于,还包括: 通过第一影像设备采集所述病灶和/或器官处于不同相位期的多个第一影像文件,和/ 或通过第二影像设备采集包含所述病灶和/或器官的解剖和功能信息的第二影像文件; 对根据所述多个第一影像文件生成的重建文件进行基于解剖结构和相位时间的图像配准处理,和/或对所述第二影像文件的重建文件和所述第一影像文件的重建文件进行基于解剖特征点的图像配准处理。 9. The surgical simulation method according to claim 7, characterized in that, further comprising: a first image capture device of the lesion and / or a first plurality of image files in a different phase of the organ, and / or by the second two image acquisition apparatus comprising the lesion and / or the second image file organ anatomical and functional information; based on the anatomy and the image registration processing according to the time phase of the plurality of first reconstruction image file generated files, and / or recreate the second file image file of the first image and the reconstructed image file registration process based on anatomical characteristic points.
10.根据权利要求1所述的手术模拟方法,其特征在于,建立仿真建模互动文件的步骤, 包括: 根据临床需求,确定需要使用的器械; 基于所述器械的外部特征、物理功能和介入角度,生成所述仿真建模互动文件。 10. The surgical simulation method according to claim 1, wherein the step of establishing interactive simulation modeling file, comprising: based on clinical needs, the need to determine the instrument; the instrument based on external characteristics, physical function and Interventional angle, generating the interactive simulation modeling file.
11.根据权利要求1所述的手术模拟方法,其特征在于,基于所述虚拟环境模拟手术过程的步骤,包括: 确定所述虚拟环境中的跟踪点或观察点; 计算并显示所述跟踪点或观察点与所述建模文件中的病灶的坐标相对位置和距离, 和/或所述跟踪点或观察点与其周围组织的坐标相对位置和距离。 11. The surgical simulation method according to claim 1, wherein the step of the virtual environment based on simulated surgical procedure, comprising: determining the tracking point of the virtual environment or watchpoint; calculating and displaying the tracking point or coordinates of the observation point lesions modeling file coordinate positions and relative distances, and / or the tracking points or observation points with the surrounding tissue of the relative position and distance.
12.根据权利要求11所述的手术模拟方法,其特征在于,所述跟踪点或观察点包括:所述仿真建模互动文件与所述建模文件的第一接触点。 12. The operation of the simulation method according to claim 11, wherein said tracking point or observation point comprising: a first contact point of the simulation modeling of the interaction document model file.
13.根据权利要求1所述的手术模拟方法,其特征在于,基于所述虚拟环境模拟手术过程的步骤,包括: 在所述仿真建模文件介入所述建模文件的过程中,计算并显示所述仿真建模互动文件与所述建模文件之间的交互关系及相关参数。 13. The surgical simulation method according to claim 1, wherein the step of the virtual environment based on simulated surgical procedure, comprising: modeling process involves the simulation modeling file in the file, the computing and display the simulation modeling the interaction between the modeling and interactive file documents and related parameters.
14. 根据权利要求13所述的手术模拟方法,其特征在于,所述交互关系及相关参数包括: 所述仿真建模互动文件的各个边界与其周围组织的坐标相对位置和距离,和/或所述仿真建模互动文件与所述建模文件中的病灶的坐标相对位置和距离。 14. The surgical simulation method according to claim 13, characterized in that said interactions and related parameters comprising: interactive simulation modeling the coordinates of the respective boundaries of the file and its surrounding tissue relative position and distance, and / or said modeling simulation modeling of the interaction document file of a lesion in the relative coordinate position and distance.
15. 根据权利要求1所述的手术模拟方法,其特征在于,基于所述虚拟环境模拟手术过程的步骤,包括: 在所述仿真建模文件介入所述建模文件的过程中,计算并显示所述仿真建模互动文件与所述建模文件中的敏感性组织的安全范围,和/或由于所述仿真建模互动文件的介入而对所述建模文件中的组织造成的形变信息。 The surgical simulation method according to claim 1, wherein the step of the virtual environment based on simulated surgical procedure, comprising: modeling process involves the simulation modeling file in the file, the computing and display the range of simulation modeling interactive security file with the modeling files in sensitive tissues, and / or distortion of information due to the intervention of the simulation modeling interactive documents resulting from the model file organization.
16. 根据权利要求1所述的手术模拟方法,其特征在于,基于所述虚拟环境模拟手术过程的步骤,包括: 计算所述仿真建模互动文件介入所述建模文件的可能路径和/或最优路径; 显示所述可能路径和/或所述最优路径。 16. The surgical simulation method according to claim 1, wherein the step of the virtual environment based on simulated surgical procedure, comprising: calculating a simulation modeling of the interaction document model file may intervene path and / or optimal path; displaying the possible path and / or the optimal path.
17. 根据权利要求16所述的手术模拟方法,其特征在于,计算所述可能路径和/或所述最优路径所依据的参数包括: 所述仿真建模互动文件介入所述建模文件的介入点及介入角度、所述仿真建模互动文件的结构、所述仿真建模互动文件对所述建模文件中的敏感性组织的最小伤害、所述仿真建模互动文件到达所述建模文件中的病灶的最短距离。 The surgical simulation method according to claim 16, wherein calculating the possible paths and / or the optimal path based on the parameters comprising: the interactive simulation modeling file involved in the modeling file point of intervention and intervention point of view, the structural modeling interactive simulation file, the file simulation modeling interactive modeling minimal damage to the sensitive tissues of the file, the file reaches the simulation modeling interactive modeling the shortest distance lesion file.
18. 根据权利要求1所述的手术模拟方法,其特征在于,基于所述虚拟环境模拟手术过程的步骤,包括: 在所述互动文件与所述影像建模文件中的敏感性组织接触时,进行预警提示。 The surgical simulation method according to claim 1, wherein the step of the virtual environment based on simulated surgical procedure, comprising: contacting said files interact with the modeling image file in sensitive tissues, early warning prompt.
19. 根据权利要求1至18中任一项所述的手术模拟方法,其特征在于,还包括: 根据所述建模文件、所述虚拟环境和在仿真计算时的复杂程度,确定运行所述虚拟环境的设备类型; 基于所述设备类型,生成运行所述虚拟环境的应用程序。 19. The method of surgery simulation 1 to 18 according to any one of the preceding claims, characterized in that, further comprising: a file based on the modeling, and the complexity of the virtual environment during the simulation, determining the operating device type virtual environment; based on the device type, the operation of generating a virtual application environment.
20. —种基于虚拟现实和医学影像的手术模拟装置,其特征在于,包括: 构建单元,构建用于模拟手术过程的虚拟环境; 生成单元,用于基于医学影像生成在所述虚拟环境中进行显示及交互的建模文件; 建立单元,用于根据临床需求,建立仿真建模互动文件; 导入单元,用于将所述建模文件和所述仿真建模互动文件导入所述虚拟环境; 处理单元,用于在所述导入单元将所述建模文件和所述仿真建模互动文件导入所述虚拟环境后,基于所述虚拟环境模拟手术过程。 20. - kind of virtual reality simulation apparatus based surgical and medical imaging, which is characterized in that, comprising: a construction unit, build a virtual environment for simulating a surgical procedure; generating means for performing in the virtual environment generated on the basis of Medical Imaging display and interactive modeling files; establishing means, according to clinical need, simulation modeling to establish interactive file; introducing means, for modeling the file and the simulation model file into the virtual environment interaction; processing means for after said introducing means introducing said modeling simulation modeling file and the file to interact with the virtual environment, the virtual environment based on simulated surgical procedure.
21. —种电子设备,其特征在于,包括:处理器和存储器,所述存储器存储有可执行指令,所述处理器用于调用所述存储器存储的可执行指令执行如权利要求1至19中任一项所述的手术模拟方法。 21. - electronic apparatus comprising: a processor and a memory, the memory storing executable instructions, the processor-executable instructions for invoking the stored execution memory as in any one of claims 1 to 19, the method of one of the surgical simulation.
CN201710214282.9A 2017-04-01 2017-04-01 Virtual reality and medical image-based surgery analogy method, device and equipment CN106974730A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710214282.9A CN106974730A (en) 2017-04-01 2017-04-01 Virtual reality and medical image-based surgery analogy method, device and equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710214282.9A CN106974730A (en) 2017-04-01 2017-04-01 Virtual reality and medical image-based surgery analogy method, device and equipment

Publications (1)

Publication Number Publication Date
CN106974730A true CN106974730A (en) 2017-07-25

Family

ID=59344434

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710214282.9A CN106974730A (en) 2017-04-01 2017-04-01 Virtual reality and medical image-based surgery analogy method, device and equipment

Country Status (1)

Country Link
CN (1) CN106974730A (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6256529B1 (en) * 1995-07-26 2001-07-03 Burdette Medical Systems, Inc. Virtual reality 3D visualization for surgical procedures
CN101393654A (en) * 2008-10-07 2009-03-25 华南师范大学;南方医科大学珠江医院 Computer assistant organ operation simulating system
CN104463965A (en) * 2014-12-17 2015-03-25 中国科学院自动化研究所 Training scene simulation system and method for minimally invasive cardiovascular interventional operation
CN104739519A (en) * 2015-04-17 2015-07-01 中国科学院重庆绿色智能技术研究院 Force feedback surgical robot control system based on augmented reality
CN105264459A (en) * 2012-09-27 2016-01-20 沉浸式触感有限公司 Haptic augmented and virtual reality system for simulation of surgical procedures
CN106200982A (en) * 2016-07-26 2016-12-07 南京意斯伽生态科技有限公司 Medical surgery training system based on virtual reality technology

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6256529B1 (en) * 1995-07-26 2001-07-03 Burdette Medical Systems, Inc. Virtual reality 3D visualization for surgical procedures
CN101393654A (en) * 2008-10-07 2009-03-25 华南师范大学;南方医科大学珠江医院 Computer assistant organ operation simulating system
CN105264459A (en) * 2012-09-27 2016-01-20 沉浸式触感有限公司 Haptic augmented and virtual reality system for simulation of surgical procedures
CN104463965A (en) * 2014-12-17 2015-03-25 中国科学院自动化研究所 Training scene simulation system and method for minimally invasive cardiovascular interventional operation
CN104739519A (en) * 2015-04-17 2015-07-01 中国科学院重庆绿色智能技术研究院 Force feedback surgical robot control system based on augmented reality
CN106200982A (en) * 2016-07-26 2016-12-07 南京意斯伽生态科技有限公司 Medical surgery training system based on virtual reality technology

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
王子罡、唐泽圣、王田苗等: "基于虚拟现实的计算机辅助立体定向神经外科手术系统", 《计算机学报》 *

Similar Documents

Publication Publication Date Title
CN103959179B (en) For medical procedures holographic user interface
CA2867839C (en) Method and system for providing information from a patient-specific model of blood flow
Vidal et al. Principles and applications of computer graphics in medicine
CN102368972B (en) The use of patient-specific model of image-guided procedure computerized simulation system and method
Andriole et al. Optimizing analysis, visualization, and navigation of large image data sets: one 5000-section CT scan can ruin your whole day
US20070247454A1 (en) 3D visualization with synchronous X-ray image display
Bernhardt et al. The status of augmented reality in laparoscopic surgery as of 2016
CN102254476B (en) Endoscopic minimally invasive surgery simulation training method and system
Noecker et al. Development of patient-specific three-dimensional pediatric cardiac models
Robb Virtual endoscopy: development and evaluation using the Visible Human datasets
CN104246855B (en) Training tumor ablation system
US8149236B2 (en) Information processing apparatus and program
Schendel et al. Three-dimensional imaging and computer simulation for office-based surgery
JP4850892B2 (en) Fundus image display apparatus, control method therefor, and computer program
US9375191B2 (en) Method and apparatus for determining three-dimensional reconstruction of an object
Riener et al. Virtual reality in medicine
US20110236868A1 (en) System and method for performing a computerized simulation of a medical procedure
Heng et al. Application to anatomic visualization and orthopaedics training
CN103371870B (en) A multi-mode image based surgical navigation system
EP2612294A1 (en) Medical information display apparatus, operation method and program
JP2003271924A (en) Image processing system, and medical image processing system
US10231704B2 (en) Method for acquiring ultrasonic data
Lopez-Perez et al. Three-dimensional cardiac computational modelling: methods, features and applications
CN1973779A (en) Method and system for cardiac imaging and catheter guidance for radio frequency (RF) ablation
JP2004520149A (en) How to create a medical report, system and computer program

Legal Events

Date Code Title Description
PB01 Publication
SE01 Entry into force of request for substantive examination