CN116206735A - Medical data visualization method based on histogram and nonlinear embedded transfer function - Google Patents
Medical data visualization method based on histogram and nonlinear embedded transfer function Download PDFInfo
- Publication number
- CN116206735A CN116206735A CN202310174362.1A CN202310174362A CN116206735A CN 116206735 A CN116206735 A CN 116206735A CN 202310174362 A CN202310174362 A CN 202310174362A CN 116206735 A CN116206735 A CN 116206735A
- Authority
- CN
- China
- Prior art keywords
- data
- transfer function
- nonlinear
- points
- histogram
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H30/00—ICT specially adapted for the handling or processing of medical images
- G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Public Health (AREA)
- Epidemiology (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Primary Health Care (AREA)
- Radiology & Medical Imaging (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Processing Or Creating Images (AREA)
Abstract
Description
技术领域technical field
本发明属于医学数据解释可视化技术领域,具体提出一种基于直方图和非线性内嵌式传递函数的医学数据可视化方法。The invention belongs to the technical field of medical data interpretation and visualization, and specifically proposes a medical data visualization method based on a histogram and a nonlinear embedded transfer function.
背景技术Background technique
传递函数设计是一种传统的体积可视化方法,为体积数据中每个体素分配不同的颜色和透明度方案。近年来,体积可视化正在石油/天然气勘探、大气和海洋模拟、医学诊断等多个科学和工程领域已有广泛的应用,可帮助理解复杂的观察或模拟体积数据。随着图形硬件和体积可视化算法的发展,体积渲染变得更快、更精确。因此,体绘制的重点已经转移到传递函数的设计上。Transfer function design is a traditional approach to volume visualization that assigns each voxel in the volume data a different color and transparency scheme. In recent years, volume visualization has been widely used in many scientific and engineering fields such as oil/gas exploration, atmospheric and ocean simulation, medical diagnosis, etc., and can help understand complex observed or simulated volume data. With advances in graphics hardware and volume visualization algorithms, volume rendering has become faster and more precise. Therefore, the focus of volume rendering has shifted to the design of transfer functions.
近年来,为了提高体积可视化的效率和准确性,一般有两种传递函数的设计方法,分别为以数据为中心的方法和以图像为中心的方法。但这两种传统的方法每个探索的时间只能加载一个TF或TF-let,不能在不同的可视化结果之间切换。并且,在改变对象的透明度和清晰度时,需要移动所有的结点,操作复杂,迫切需要更完善的传递函数设计。In recent years, in order to improve the efficiency and accuracy of volumetric visualization, there are generally two methods for designing transfer functions, namely data-centric methods and image-centric methods. However, these two traditional methods can only load one TF or TF-let at each exploration time, and cannot switch between different visualization results. Moreover, when changing the transparency and clarity of an object, all the nodes need to be moved, and the operation is complicated, so a more complete transfer function design is urgently needed.
总体而言,目前的传递函数设计还有很多局限性,例如:效率不高,用户探索时单调乏味,每次只能加载一个TF,不能在不同的可视化结果之间切换,改变组织透明度和清晰度时操作复杂。有以上局限性主要是由于人体组织的复杂性和多样性,以及用户观察和分析组织的需求多样性。Overall, the current transfer function design still has many limitations, such as: not high efficiency, tedious when users explore, can only load one TF at a time, cannot switch between different visualization results, change the organization transparency and clarity The operation is complicated at a certain time. The above limitations are mainly due to the complexity and diversity of human tissues, as well as the diversity of user needs for observing and analyzing tissues.
发明内容Contents of the invention
发明目的:本发明提出了一种基于直方图和非线性内嵌式传递函数的医学数据可视化方法,根据不同专家的领域知识,协同编辑一个或多个组织,提高效率和准确性。Purpose of the invention: The present invention proposes a medical data visualization method based on histogram and nonlinear embedded transfer function, according to the domain knowledge of different experts, one or more organizations can be collaboratively edited to improve efficiency and accuracy.
技术方案:本发明旨在一种基于直方图和非线性内嵌式传递函数的医学数据可视化方法,具体包括以下步骤:Technical solution: The present invention aims at a medical data visualization method based on histogram and nonlinear embedded transfer function, which specifically includes the following steps:
(1)对预先获取的医学人体组织数据进行预处理,得到三维体数据;(1) Preprocessing the pre-acquired medical human tissue data to obtain three-dimensional volume data;
(2)为给用户更有效地编辑传递函数,在构建直方图非线性内嵌式传递函数背景中使用非线性直方图和非均匀网格的设计;(2) In order to edit the transfer function more effectively for users, the design of nonlinear histogram and non-uniform grid is used in the background of constructing the histogram nonlinear embedded transfer function;
(3)在TF编辑器中,一个属性对应于一个类似于三角状小波的传递函数,提取医学数据中组织的TF-let;上下同步移动TF-let的几个节点以调整每个组织图片的透明度;控制点之间每个点的颜色和透明度由两个最近的控制点差值得到;左右同步移动TF-let的节点以调整每个组织图片的清晰度;控制点之间每个点的颜色和清晰度由两个最近的控制点差值得到;(3) In the TF editor, an attribute corresponds to a transfer function similar to a triangular wavelet to extract the TF-let of the tissue in the medical data; move several nodes of the TF-let up and down synchronously to adjust the texture of each tissue image Transparency; the color and transparency of each point between control points are obtained from the difference between the two nearest control points; move the TF-let nodes synchronously left and right to adjust the clarity of each tissue image; the color of each point between control points and sharpness are obtained from the difference between the two nearest control points;
(4)设计基于传递函数的传递函数方法TF-of-TF为用户提供探索的视觉线索,所述TF-of-TF为简单地单击相应的TF-let节点来融合多个TF-let;(4) Design the transfer function method TF-of-TF based on the transfer function to provide users with visual cues for exploration, the TF-of-TF is to simply click the corresponding TF-let node to fuse multiple TF-lets;
(5)具有不同领域知识的多个用户能够协作浏览单个医疗数据,编辑其子区域,提高医疗数据探索的效率和准确性。(5) Multiple users with different domain knowledge can collaboratively browse a single medical data, edit its sub-regions, and improve the efficiency and accuracy of medical data exploration.
进一步地,所述步骤(1)实现过程如下:Further, the implementation process of the step (1) is as follows:
将医学人体组织体数据,进行切片整理,每个切片之间的反射时间间隔为1毫秒;根据反射时间排列井数据将所有数据切片与实际物理空间一一对齐,对齐之后的数据即为三维体数据;将三维体数据从CPU传输到GPU上。The medical human body tissue data is sliced and sorted, and the reflection time interval between each slice is 1 millisecond; according to the reflection time alignment data, all data slices are aligned with the actual physical space one by one, and the aligned data is a three-dimensional volume Data; transfer 3D volume data from the CPU to the GPU.
进一步地,所述步骤(2)实现过程如下:Further, the implementation process of the step (2) is as follows:
直方图纵轴代表不透明度,水平轴采用非线性映射设计;将直方图渲染成网格,以减少视觉混乱;在对数据进行预处理时计算数据的bin的大小,用于绘制直方图;组距计算公式为:The vertical axis of the histogram represents opacity, and the horizontal axis adopts a nonlinear mapping design; render the histogram into a grid to reduce visual confusion; calculate the bin size of the data when preprocessing the data, and use it to draw the histogram; group The distance calculation formula is:
其中,bi表示每个bin的大小,i代表的是第几个组;在低尺度值时放大垂直轴的宽度,用如下公式计算:Among them, b i represents the size of each bin, and i represents the number of groups; when the scale value is low, the width of the vertical axis is enlarged, and the following formula is used to calculate:
非线性TF编辑器的纵轴代表不透明度,即α值;The vertical axis of the nonlinear TF editor represents the opacity, that is, the alpha value;
在低比例值处放大垂直轴的宽度,控制点与非线性坐标的变换如下:Enlarging the width of the vertical axis at low scale values, the control points are transformed to non-linear coordinates as follows:
其中,α表示放大程度;α越大,非线性tf的低值部分越宽。Among them, α represents the degree of amplification; the larger α is, the wider the low value part of the nonlinear tf is.
进一步地,所述步骤(3)实现过程如下:Further, the implementation process of the step (3) is as follows:
当用户通过TF-let获得最佳渲染结果时,借助快速加载和重新加载的功能将所有颜色方案序列化到磁盘中,以便后续加载;When the user obtains the best rendering result through TF-let, all color schemes are serialized to disk with the function of fast loading and reloading for subsequent loading;
当用户决定序列化TF控制点数据并加载它时,会出现一个TF let节点,每个节点的垂直坐标表示对应TF let中控制点的最大不透明度;When the user decides to serialize TF control point data and load it, a TF let node will appear, and the vertical coordinates of each node represent the maximum opacity of the control point in the corresponding TF let;
TF-let与组织体数据相关,将所有TF-let序列化,以便重新加载和后续分析;使用TF编辑器的TF为传递函数设计提供了TF的每个节点对应于原始非线性传递函数的一个TF-let;在进行TF编辑时,体积数据根据其在特征空间中的分布进行分类。TF-lets are associated with tissue data, all TF-lets are serialized for reloading and subsequent analysis; TF using the TF editor provides transfer function design with each node of TF corresponding to one of the original nonlinear transfer function TF-let; when doing TF editing, volumetric data are classified according to their distribution in feature space.
进一步地,所述步骤(4)实现过程如下:Further, the implementation process of the step (4) is as follows:
(41)将同组织中的点相互绑定,同组织区域的编辑器中有abc三个点,用单击的方式将这三个点进行绑定,并且拖动这三个点中的最高点会使绑定的三个点同时左右移动,而不改变该组织上三个点的相对位置,提高改变图像清晰度和透明度的效率;这三个点所绑定成的是TF-of-TF方法中的第一个TF;(41) Bind the points in the same organization to each other. There are three points abc in the editor of the same organization area. Click to bind these three points, and drag the highest of these three points The point will make the bound three points move left and right at the same time without changing the relative position of the three points on the tissue, improving the efficiency of changing image clarity and transparency; what these three points are bound into is TF-of- The first TF in the TF method;
(42)将步骤(41)中绑定在一起的abc三点取其中y轴值最高的b点,b的坐标是(x,y),对应设置第二个TF点(x,-y)为绑定的abc三点的关联点,移动该TF点可以控制abc三点的移动,该TF就是TF-of-TF方法中的第二个TF;(42) Take the three points abc bound together in step (41) and take the b point with the highest y-axis value. The coordinates of b are (x, y), and the second TF point (x, -y) is correspondingly set It is the associated point of the bound abc three points, moving the TF point can control the movement of the abc three points, this TF is the second TF in the TF-of-TF method;
(43)用户使用鼠标按下事件功能立即响应鼠标消息,并增加活动变量以确定鼠标是否单击控制点;使TF-let中TF节点的内部关系保持不变,并作为一个整体移动,上下拖动控制点也能够更改整个点的值,从而更改与TF let相对应的组织的透明度。(43) The user uses the mouse press event function to immediately respond to the mouse message, and increases the activity variable to determine whether the mouse clicks the control point; keeps the internal relationship of the TF nodes in the TF-let unchanged, and moves as a whole, dragging up and down Animating a control point is also able to change the value of the entire point, thus changing the transparency of the tissue corresponding to the TF let.
有益效果:与现有技术相比,本发明的有益效果:Beneficial effect: compared with prior art, the beneficial effect of the present invention:
本发明能够快速加载TF-let,用户只要保存之前对应的TF-Let,就能够得到序列化后的TF-Let渲染结果;通过单击TF-Let节点,用户能够有效地查看数据集中的任何组织,当加载一个新的数据集时,用户需要探索数据并找到所有组织的单个TF-Let;The invention can quickly load TF-let, and the user can obtain the serialized TF-Let rendering result as long as the user saves the previous corresponding TF-Let; by clicking the TF-Let node, the user can effectively view any organization in the data set , when loading a new dataset, the user needs to explore the data and find a single TF-Let for all tissues;
本发明实现了TF-of-TF垂直方向上的结果编辑:用户能够通过上下移动单个节点来进行组合调整;当需要仔细而清晰地观察其中一部分时,用户就要修改血管的透明度;此时,只需要通过鼠标点击、上下移动就能够改变透明度,比传统方法更加省时高效;此外,用户还能够通过传递函数按比例拉伸多个组织以获得更好的观察效果;The present invention realizes the result editing in the vertical direction of TF-of-TF: the user can adjust the combination by moving a single node up and down; when it is necessary to observe a part of it carefully and clearly, the user needs to modify the transparency of the blood vessel; at this time, The transparency can be changed only by clicking and moving up and down with the mouse, which is more time-saving and efficient than traditional methods; in addition, users can stretch multiple tissues proportionally through the transfer function to obtain better observation effects;
本发明也实现了TF-of-TF水平方向上的结果编辑:用户能够通过左右移动TF-of-TF点来控制显示图像的分辨率,如手部数据集的皮肤、血管和骨骼等;单击鼠标、调整蒙皮点的上下移动能够将需要的部位突出显示,便于观察;此外,用户能够通过左右拖动TF-let来快速切换不同的医疗组织,操作方便;The present invention also realizes the result editing in the TF-of-TF horizontal direction: the user can control the resolution of the displayed image by moving the TF-of-TF point left and right, such as the skin, blood vessels and bones of the hand data set; Clicking the mouse and adjusting the up and down movement of the skin points can highlight the required parts for easy observation; in addition, users can quickly switch between different medical organizations by dragging the TF-let left and right, which is easy to operate;
本发明根据不同专家的领域知识,协同编辑一个或多个组织,能够提高效率和准确性;用户还能够加载更新后的文件,得到具体的融合视觉效果,以便他们能够更好地了解相应的组织部位;多用户操作的实现使得多个用户能够充分利用发挥他们的不同领域知识背景。According to the domain knowledge of different experts, the present invention collaboratively edits one or more organizations, which can improve efficiency and accuracy; users can also load updated files to obtain specific fusion visual effects, so that they can better understand the corresponding organizations Part; the realization of multi-user operation enables multiple users to make full use of their knowledge background in different fields.
附图说明Description of drawings
图1为本发明的流程图;Fig. 1 is a flowchart of the present invention;
图2为本发明中基于直方图的非线性传递函数编辑器示意图;Fig. 2 is a schematic diagram of a nonlinear transfer function editor based on a histogram in the present invention;
图3为本发明中TF-of-TF的说明示意图;Fig. 3 is a schematic illustration of TF-of-TF in the present invention;
图4为本发明中多用户融合的方法的示意图;FIG. 4 is a schematic diagram of a method for multi-user fusion in the present invention;
图5为本发明利用函数编辑器进行立体渲染的流程示意图;Fig. 5 is a schematic flow chart of stereoscopic rendering using a function editor in the present invention;
图6为数据集CHEST中不同部位的单个TF-let和渲染结果示意图;其中,(a)为胸部的单个TF-let示意图;(b)为TF-let对胸部的渲染结果示意图;(c)为胸骨的单个TF-let示意图;(d)为TF-let对胸骨的渲染结果示意图;(e)为手骨的单个TF-let示意图;(f)为TF-let对手骨的渲染结果示意图;(g)为头骨的单个TF-let示意图;(h)为TF-let对头骨的渲染结果示意图;Figure 6 is a schematic diagram of a single TF-let and rendering results of different parts in the data set CHEST; where (a) is a schematic diagram of a single TF-let of the chest; (b) is a schematic diagram of the rendering results of TF-let on the chest; (c) A schematic diagram of a single TF-let of a sternum; (d) a schematic diagram of the rendering result of TF-let on a sternum; (e) a schematic diagram of a single TF-let of a hand bone; (f) a schematic diagram of the rendering result of a TF-let hand bone; (g) is a schematic diagram of a single TF-let of the skull; (h) is a schematic diagram of the rendering result of TF-let on the skull;
图7为本发明中数据集HEAD和数据集CHEST的评估图,其中(a)为使用传统线性传递函数方法对数据集HEAD的渲染结果示意图;(b)为使用所提出方法对数据集HEAD进行渲染的结果示意图;(c)为数据集CHEST使用线性方法的渲染结果示意图;(d)为使用本发明所提出的方法对数据集CHEST进行渲染的结果示意图;Fig. 7 is the evaluation chart of data set HEAD and data set CHEST in the present invention, wherein (a) is the rendering result schematic diagram of data set HEAD using traditional linear transfer function method; (b) uses the proposed method to carry out data set HEAD A schematic diagram of the rendering result; (c) is a schematic diagram of the rendering result of the dataset CHEST using a linear method; (d) is a schematic diagram of the rendering result of the dataset CHEST using the method proposed by the present invention;
图8本发明中数据集HAND的评估案例;其中(a)(b)为两个用户为数据集HAND展示的传统线性传递函数编辑结果示意图;(c)为用户使用本发明所提出的方法来寻找血管的示意图;The evaluation case of data set HAND among Fig. 8 the present invention; Wherein (a) (b) is the traditional linear transfer function editing result schematic diagram that two users show for data set HAND; (c) uses the method that the present invention proposes for the user to Find a schematic diagram of blood vessels;
图9为通过鼠标点击移动皮肤点来改变皮肤的透明度和清晰度的示意图;其中(a)为原图;(b)为本发明用户通过鼠标将第一个皮肤点向上移动来降低皮肤的透明度的示意图;(c)为本发明用户通过鼠标将第一个皮肤点向上移动来降低皮肤的透明度的示意图;(d)为本发明用户通过鼠标将第二个皮肤点向上移动来降低皮肤的透明度的示意图;(e)为本发明用户通过鼠标将第二个皮肤点向右移动来提高皮肤的清晰度的示意图;(f)为本发明用户通过鼠标将第三个皮肤点向左移动来提高皮肤的清晰度的示意图;(g)为本发明用户通过鼠标将第三个皮肤点向下移动来增加皮肤的透明度的示意图;Figure 9 is a schematic diagram of changing the transparency and clarity of the skin by clicking on the mouse to move the skin point; where (a) is the original picture; (b) is the user of the present invention moves the first skin point upwards to reduce the transparency of the skin (c) is a schematic diagram that the user of the present invention moves the first skin point upwards to reduce the transparency of the skin; (d) is the user of the present invention moves the second skin point upwards to reduce the transparency of the skin (e) is a schematic diagram for the user of the present invention to improve the clarity of the skin by moving the second skin point to the right with the mouse; (f) is a schematic diagram for the user of the present invention to improve the clarity of the skin by moving the third skin point to the left with the mouse A schematic diagram of the clarity of the skin; (g) is a schematic diagram of increasing the transparency of the skin by the user of the present invention moving the third skin point downward through the mouse;
图10为本发明用户通过调整TF-of-TF中的结点来查看数据集HAND中的血管示意图;Fig. 10 is a schematic diagram of blood vessels in the data set HAND viewed by users of the present invention by adjusting the nodes in TF-of-TF;
图11为本发明基于数据集HAND的皮肤、血管和骨骼多用户融合结果示意图;Figure 11 is a schematic diagram of the multi-user fusion results of skin, blood vessels and bones based on the data set HAND in the present invention;
图12为采用本发明的融合示意图;其中(a)为数据集HEAD的皮肤和骨骼的融合结果示意图;(b)为数据集CHEST的胸部(连同肺)和胸骨的融合结果示意图。Fig. 12 is a schematic diagram of fusion using the present invention; wherein (a) is a schematic diagram of fusion results of skin and bones in data set HEAD; (b) is a schematic diagram of fusion results of chest (together with lungs) and sternum in data set CHEST.
具体实施方式Detailed ways
下面结合附图对本发明作进一步详细说明。The present invention will be described in further detail below in conjunction with the accompanying drawings.
本发明提供一种基于直方图和非线性内嵌式传递函数的医学数据可视化方法,如图1所示,首先,原始数据被预处理成一个卷;然后用户使用非线性直方图提供的视觉提示来调整非线性传递函数;调整之后所有TF-let都被序列化到磁盘中,它们将被反序列化来进行快速地重新加载。最后几个TF-let能够以focus-and-contexts方式融合成最终的非线性传递函数。具体包括以下步骤:The present invention provides a medical data visualization method based on histogram and nonlinear embedded transfer function, as shown in Figure 1, first, the raw data is preprocessed into a volume; then the user uses the visual cue provided by the nonlinear histogram to adjust the nonlinear transfer function; after adjustment all TF-lets are serialized to disk, and they will be deserialized for fast reloading. The last few TF-lets can be fused into the final nonlinear transfer function in a focus-and-contexts manner. Specifically include the following steps:
步骤1:对预先获取的医学人体组织数据进行预处理,得到三维体数据。Step 1: Preprocessing the pre-acquired medical human tissue data to obtain 3D volume data.
将患者的医学人体组织数据变成体数据预处理,包括医学数据中的不同的属性(组织)血液、脂肪、软组织和骨骼;将所有切片按照真实的物理空间排列,从而得到三维体数据;系统将处理好的三维体数据加载到系统中,为后续的医学数据可视化与探索提供数据支持。Transform the patient's medical human tissue data into volume data preprocessing, including different attributes (tissue) blood, fat, soft tissue and bone in medical data; arrange all slices according to the real physical space to obtain three-dimensional volume data; system Load the processed 3D volume data into the system to provide data support for subsequent medical data visualization and exploration.
步骤2:为给用户更有效地编辑传递函数,在构建直方图非线性内嵌式传递函数背景中使用非线性直方图和非均匀网格的设计。直方图容器尺寸越大,在背景中绘制的容器宽度就越宽。Step 2: In order to allow users to edit the transfer function more efficiently, the design of nonlinear histogram and non-uniform grid is used in the background of constructing histogram nonlinear embedded transfer function. The larger the histogram bin size, the wider the bin width is drawn in the background.
如图2所示,相应的bin大小计算出直方图的bin宽度。图中的TF-let能够表示体数据的一个属性(例如,一个组织)。最左边的TF-let代表空气,最右边的TF-let代表骨架。每个TF-let都有自己的TF-node,用于快速加载和融合。As shown in Figure 2, the corresponding bin size computes the bin width of the histogram. A TF-let in a graph can represent an attribute of volume data (eg, an organization). The leftmost TF-let represents the air, and the rightmost TF-let represents the skeleton. Each TF-let has its own TF-node for fast loading and fusion.
直方图纵轴代表不透明度,水平轴采用非线性映射设计;具体实现方法是:将直方图渲染成网格,以减少视觉混乱。在对数据进行预处理时计算数据的bin的大小,用于绘制直方图。组距计算公式为:The vertical axis of the histogram represents opacity, and the horizontal axis adopts a nonlinear mapping design; the specific implementation method is: render the histogram into a grid to reduce visual clutter. When preprocessing the data, calculate the size of the bin of the data, which is used to draw the histogram. The group distance calculation formula is:
其中bi表示每个bin的大小,用如下公式计算:Where bi represents the size of each bin, calculated with the following formula:
在低比例值处放大垂直轴的宽度,控制点与非线性坐标的变换如下:Enlarging the width of the vertical axis at low scale values, the control points are transformed to non-linear coordinates as follows:
步骤3:在TF编辑器中,一个属性对应于一个类似于三角状小波的传递函数,提取医学数据中组织的TF-let;上下同步移动TF-let的几个节点以调整每个组织图片的透明度;控制点之间每个点的颜色和透明度由两个最近的控制点差值得到;左右同步移动TF-let的节点以调整每个组织图片的清晰度;控制点之间每个点的颜色和清晰度由两个最近的控制点差值得到。Step 3: In the TF editor, an attribute corresponds to a transfer function similar to a triangular wavelet to extract the TF-let of the tissue in the medical data; move several nodes of the TF-let up and down synchronously to adjust the image of each tissue Transparency; the color and transparency of each point between control points are obtained from the difference between the two nearest control points; move the TF-let nodes synchronously left and right to adjust the clarity of each tissue image; the color of each point between control points and sharpness are obtained from the difference between the two nearest control points.
当用户通过TF-let获得最佳渲染结果时,他们能够借助快速加载和重新加载的功能将所有颜色方案序列化到磁盘中,以便后续加载。快速加载功能的界面设计为二维区域。When users achieve the best rendering results with TF-let, they can serialize all color schemes to disk for subsequent loading with the help of fast loading and reloading functions. The interface of the quick loading function is designed as a two-dimensional area.
当用户决定序列化TF控制点数据并加载它时,会出现一个TF let节点,每个节点的垂直坐标(带有绝对值)表示对应TF let中控制点的最大不透明度。When the user decides to serialize the TF control point data and load it, a TF let node will appear, and the vertical coordinate (with absolute value) of each node represents the maximum opacity of the control point in the corresponding TF let.
TF-let与组织体数据相关,所有TF-let都将被序列化,以便重新加载和后续分析。使用TF编辑器的TF为传递函数设计提供了TF的每个节点对应于原始非线性传递函数的一个TF let;在进行TF编辑时,体积数据根据其在特征空间中的分布进行分类。TF-lets are associated with tissue data, and all TF-lets will be serialized for reloading and subsequent analysis. TF using the TF editor provides transfer function design with each node of TF corresponding to a TF let of the original nonlinear transfer function; when doing TF editing, the volume data is classified according to its distribution in the feature space.
步骤4:设计一个类似小波的短传递函数(TF-let),可以在后续的探索中快速序列化和重新加载。此外,设计了一种TF-let融合方法,通过简单地单击相应的TF-let节点来融合多个TF-let。这种融合多个TF-let的方法,称为基于传递函数的传递函数方法(TF-of-TF),为用户提供探索的视觉线索。Step 4: Design a wavelet-like short transfer function (TF-let) that can be quickly serialized and reloaded in subsequent explorations. Furthermore, a TF-let fusion method is designed to fuse multiple TF-lets by simply clicking the corresponding TF-let nodes. This method of fusing multiple TF-lets, called transfer function-based transfer function approach (TF-of-TF), provides visual cues for users to explore.
用户能够通过非线性TF编辑器调整可视化结果,用直方图非线性内嵌式传递函数,即即基于传递函数的传递函数方法能够为用户提供探索的视觉线索。在TF编辑器中,一个属性对应于一个类似小波的传递函数,在本发明中将其称为TF let。如图3所示,首先,能够通过上下拖动TF-of-TF结点来调整组织的透明度。然后能够通过左右拖动TF-of-TF结点来调整组织的清晰度。因此,用户能够在不改变原始TF-let的所有控制点的情况下获得最终的可视化结果。Users can adjust the visualization results through the nonlinear TF editor, and use the histogram nonlinear embedded transfer function, that is, the transfer function method based on the transfer function can provide users with visual clues for exploration. In the TF editor, an attribute corresponds to a wavelet-like transfer function, which is called a TF let in the present invention. As shown in Figure 3, first, the transparency of the tissue can be adjusted by dragging the TF-of-TF node up and down. You can then adjust the clarity of the tissue by dragging the TF-of-TF node left and right. Therefore, users are able to obtain the final visualization results without changing all the control points of the original TF-let.
将同组织中的点相互绑定,比如同组织区域的编辑器中有abc三个点,用单击的方式将这三个点进行绑定,并且拖动这三个点中的最高点会使这种设计使绑定的三个点同时左右移动,而不改变该组织上三个点的相对位置,从而提高改变图像清晰度和透明度的效率。这三个点所绑定成的是TF-of-TF方法中的第一个TF。Bind the points in the same organization, for example, there are three points abc in the editor of the same organization area, bind these three points by clicking, and drag the highest point of the three points to This design enables the three bound points to move left and right simultaneously without changing the relative positions of the three points on the tissue, thereby improving the efficiency of changing image clarity and transparency. These three points are bound into the first TF in the TF-of-TF method.
将绑定在一起的abc三点取其中y轴值最高的b点,假设b的坐标是(x,y),对应设置第二个TF点(x,-y)为绑定的abc三点的关联点,移动该TF点可以控制abc三点的移动,该TF就是TF-of-TF方法中的第二个TF。Take the point b with the highest y-axis value among the three points of abc bound together, assuming that the coordinates of b are (x, y), and correspondingly set the second TF point (x, -y) as the bound three points of abc , moving this TF point can control the movement of the three points abc, this TF is the second TF in the TF-of-TF method.
用户能够能够使用鼠标按下事件功能立即响应鼠标消息,并增加活动变量以确定鼠标是否单击控制点。它使TF let中TF节点的内部关系保持不变,并作为一个整体移动,上下拖动控制点也能够更改整个点的值,从而更改与TF let相对应的组织的透明度。Users can use the mouse down event function to immediately respond to mouse messages and increment the active variable to determine whether the mouse clicked on a control point. It keeps the internal relationship of TF nodes in TF let unchanged and moves as a whole, and dragging the control point up and down can also change the value of the whole point, thereby changing the transparency of the organization corresponding to TF let.
步骤5:具有不同领域知识的多个用户能够协作浏览单个医疗数据,编辑其子区域,提高医疗数据探索的效率和准确性。如图4所示,多个用户编辑他们感兴趣的组织所对应的TF-of-TF结点;能够通过融合TF-of-TF的结点来融合最终的渲染效果。Step 5: Multiple users with different domain knowledge are able to collaboratively browse a single medical data, edit its sub-regions, and improve the efficiency and accuracy of medical data exploration. As shown in Figure 4, multiple users edit the TF-of-TF nodes corresponding to the tissues they are interested in; the final rendering effect can be fused by fusing the TF-of-TF nodes.
比如多个医生用户想要分别编辑手臂组织中的血液、骨骼、皮肤的透明度和清晰度,通过目前的软件可以实现这种功能。在TF-let融合阶段,用户能够获得多个TF-let渲染结果的任意组合。能够使用焦点上下文技术同时查看多个焦点组织及其上下文环境。此外,他们还能够通过双击TF-let节点删除任何组织。For example, multiple doctor users want to edit the transparency and clarity of blood, bone, and skin in arm tissue separately, and this function can be realized through the current software. In the TF-let fusion stage, users can obtain any combination of multiple TF-let rendering results. Ability to view multiple focus groups and their contexts simultaneously using focus context technology. Also, they were able to delete any tissue by double-clicking on the TF-let node.
焦点和上下文可视化技术通过TF-lets融合来探索体数据。为了提高探测效率,设计了多用户融合功能。以医学数据为例,如果用户在查看血管(focus属性)时想要探索相邻的软组织(上下文),能够保存相应的TF let节点,将其添加到最终融合的TF中。多用户融合过程,例如,用户#01将皮肤对应的TF let保存到特定文件夹中,用户#02将血液对应的TFlet保存到特定文件夹中,用户#03将骨骼对应的其他TF let保存到同一文件夹中。三个预先保存的TF let能够快速加载和融合。当用户不希望在最终渲染结果中看到相应的属性时,用户能够从最终融合的TF中删除多个TF let节点中的任意一个。Focal and contextual visualization techniques explore volumetric data through TF-lets fusion. In order to improve the detection efficiency, a multi-user fusion function is designed. Taking medical data as an example, if the user wants to explore the adjacent soft tissue (context) while viewing blood vessels (focus attribute), the corresponding TF let node can be saved and added to the final fused TF. Multi-user fusion process, for example,
图6中(a)至(h)为本发明数据集中各个组织的单个TF-let示意图和最终的渲染结果。用户如果之前保存了对应的TF-lets,就能够得到序列化的TF-lets渲染出来的渲染结果。以医疗数据为例。所有组织的TF-let都能够序列化并单独保存到磁盘。用户能够通过单击TF-let结点有效地查看数据集中地任何组织。但是,当用户加载新数据集时,需要用户探索数据并找到所有组织的单个TF-let。图6中(a)显示了数据集CHEST中胸部的单个TF-let。通过点击对应的TF-let结点,加载出如图6(b)所示的渲染结果。图6中(c)、图6中(e)和图6中(g)分别为胸骨、手骨和头骨的三个单个TF-let,图6中(d)、图6(f)和图6中(h)分别为其相应的渲染结果。(a) to (h) in Fig. 6 are schematic diagrams of a single TF-let of each organization in the dataset of the present invention and the final rendering results. If the user has saved the corresponding TF-lets before, he can get the rendering result rendered by the serialized TF-lets. Take medical data, for example. All organized TF-lets can be serialized and saved to disk individually. Users can effectively view any organization in the dataset by clicking on a TF-let node. However, when a user loads a new dataset, the user is required to explore the data and find all organized individual TF-lets. Figure 6(a) shows a single TF-let for chest in the dataset CHEST. By clicking the corresponding TF-let node, the rendering result shown in Figure 6(b) is loaded. Figure 6(c), Figure 6(e) and Figure 6(g) are three single TF-lets of sternum, hand bone and skull respectively, Figure 6(d), Figure 6(f) and Figure 6 (h) in 6 are their corresponding rendering results.
图7中(a)-(d)为本发明中数据集HEAD和数据集CHEST的评估案例。通过所提出的方法很容易得到一些有趣的结果,而用户通过线性方法很难得到类似的结果。图7中(a)-(b)分别显示了传统线性传递函数方法和本发明所提出的方法呈现的相应结果。在数据集CHEST中很难找到可能是胸部表面某些病变的团块,因为这些组织的属性范围很小,如图7中(c)-(d)所示。图7中(a)数据集HEAD使用传统线性传递函数的渲染结果,图7中(b)为数据集HEAD使用本发明所提出的方法的渲染结果,红圈中各种组织的轮廓要清晰得多。图7中(c)为数据集CHEST使用线性方法的渲染结果,图7中(d)为数据集CHEST使用本发明所提出的方法的渲染结果,用户能够在胸部表面的红色圆圈中找到一些可能是病变的团块。(a)-(d) in Fig. 7 are the evaluation cases of the data set HEAD and the data set CHEST in the present invention. It is easy to obtain some interesting results by the proposed method, while it is difficult for users to obtain similar results by linear methods. (a)-(b) in Fig. 7 respectively show the corresponding results presented by the traditional linear transfer function method and the method proposed by the present invention. It is difficult to find clumps that may be some lesions on the surface of the chest in the dataset CHEST because the attributes of these tissues are in a small range, as shown in (c)-(d) in Figure 7. In Fig. 7 (a) data set HEAD uses the rendering result of traditional linear transfer function, in Fig. 7 (b) is the data set HEAD uses the rendering result of the method proposed by the present invention, the outline of various tissues in the red circle should be clear many. (c) in Figure 7 is the rendering result of the data set CHEST using the linear method, and (d) in Figure 7 is the rendering result of the data set CHEST using the method proposed by the present invention. The user can find some possibilities in the red circle on the chest surface. It is a mass of disease.
图8中(a)-(c)为本发明中数据集HAND的评估案例。图8中(a)-(b)为两个用户展示的数据集HAND用传统线性传递函数编辑的结果。他们试图以试错的方式获取血管,但他们发现很难获得最佳结果,得到这两个结果是很费时间的。图8中(c)为用户使用本发明所提出的方法来寻找血管。与传统方法提取的结果相比,用户很容易得到非常清晰的结果。此外,通过图8(a)-(c)(上图),还发现血管上的contexts更加清晰(即骨骼的轮廓和结构)。(a)-(c) in Fig. 8 are evaluation cases of the data set HAND in the present invention. (a)-(b) in Figure 8 are the results of editing the data set HAND shown by two users with a traditional linear transfer function. They tried to get the blood vessels by trial and error, but they found it difficult to get the best results, and it was time-consuming to get both results. (c) in FIG. 8 shows that the user uses the method proposed by the present invention to find blood vessels. Compared with the results extracted by traditional methods, users can easily get very clear results. In addition, through Figure 8(a)-(c) (above), it is also found that the contexts on blood vessels are clearer (ie, the outline and structure of bones).
对于普通用户来说,如果要将每一张纸巾都调整到容易看清的状态,就需要从外到内进行调整。为用户提供了一种通过鼠标点击移动皮肤点来改变皮肤的透明度和清晰度的方法。图9中(a)-(b)为用户通过鼠标将第一个皮肤点向上移动来降低皮肤的透明度。图9中(b)-(c)为用户通过鼠标将第一个皮肤点向上移动来降低皮肤的透明度。图9中(c)-(d)为用户通过鼠标将第二个皮肤点向上移动来降低皮肤的透明度。图9中(d)-(e)为用户通过鼠标将第二个皮肤点向右移动来提高皮肤的清晰度。图9中(e)-(f)为本发明用户通过鼠标将第三个皮肤点向左移动来提高皮肤的清晰度。图9中(f)-(g)为本发明用户通过鼠标将第三个皮肤点向下移动来增加皮肤的透明度。图9中(a)-(g)(上图)展示了皮肤透明度和清晰度的变化,图9中(a)-(g)(下图)为用户移动皮肤点来改变皮肤的透明度和清晰度。For ordinary users, if you want to adjust each tissue to an easy-to-see state, you need to adjust it from the outside to the inside. Provides a way for the user to change the transparency and clarity of the skin by moving the skin points with mouse clicks. (a)-(b) in Fig. 9 shows that the user moves the first skin point up with the mouse to reduce the transparency of the skin. (b)-(c) in Fig. 9 shows that the user moves the first skin point up with the mouse to reduce the transparency of the skin. (c)-(d) in FIG. 9 is that the user moves the second skin point upwards through the mouse to reduce the transparency of the skin. (d)-(e) in FIG. 9 shows that the user moves the second skin point to the right through the mouse to improve the clarity of the skin. (e)-(f) in FIG. 9 shows that the user of the present invention moves the third skin point to the left through the mouse to improve the clarity of the skin. (f)-(g) in Fig. 9 shows that the user of the present invention moves the third skin point downward through the mouse to increase the transparency of the skin. Figure 9 (a)-(g) (upper picture) shows the change of skin transparency and clarity, and Figure 9 (a)-(g) (lower picture) shows that the user moves the skin point to change the transparency and clarity of the skin Spend.
除了服务于普通用户,本发明还满足了专家查看部分组织的需求。专家如果想看到某个血管,需要调整皮肤的透明度和清晰度,首先能够通过鼠标左右移动来调整皮肤的清晰度,如图10(左下图和中下图)所示。然后能够通过鼠标上下移动来调整皮肤的透明度,如图10(中下图和右下图)所示。如图10(上图)所示,需要观察的血管被突出显示,便于观察。假设用户想要查看数据集HAND中的血管,只需要调整TF-of-TF(TF编辑器底部)中的结点即可。In addition to serving ordinary users, the present invention also meets the needs of experts to view part of the organization. If an expert wants to see a certain blood vessel, he needs to adjust the transparency and clarity of the skin. First, he can adjust the clarity of the skin by moving the mouse left and right, as shown in Figure 10 (lower left and lower middle). Then the transparency of the skin can be adjusted by moving the mouse up and down, as shown in Figure 10 (lower middle and lower right). As shown in Figure 10 (upper image), the blood vessels that need to be observed are highlighted for easy observation. Suppose the user wants to view the blood vessels in the dataset HAND, just adjust the nodes in TF-of-TF (at the bottom of the TF editor).
图11为本发明数据集HAND基于不同用户编辑的多个单一结果的融合结果。图11(左侧)的三个单一结果涉及皮肤、血管和手部骨骼的组织。图11(右上)为这三个单一结果的融合结果。此外,如果用户想要探索带有血管的骨骼,而不想探索皮肤,他们能够通过简单地点击传递函数中对应的TF-of-TF结点来移除皮肤属性的编辑,如图11(右下)。Fig. 11 is the fusion result of multiple single results edited by different users based on the data set HAND of the present invention. The three single results in Figure 11 (left side) relate to the organization of the skin, blood vessels, and bones of the hand. Figure 11 (top right) shows the fusion of these three single results. Furthermore, if users want to explore bones with blood vessels instead of skin, they can remove the skin property edits by simply clicking on the corresponding TF-of-TF node in the transfer function, as shown in Figure 11 (bottom right ).
如图12中(a)所示,数据集HAND的皮肤(左上图)和骨骼(左下图)最终的融合结果如图12中(a)(右上图)所示。如图12中(b)所示,数据集CHEST的胸部(左上图)和胸骨(左下图)最终的融合结果如图12(b)(右上图)所示。实际上,用户能够通过有效地指定多个焦点组织和多个context来获得任意的融合组合,他们只需要点击相应的TF-let-TF结点即可实现,即图12中(a)用户只需要点击皮肤和骨骼的两个TF-let结点即可实现最终的融合结果;图12中(b)用户也只需要点击胸骨和胸骨的两个TF-let结点即可获得最终的融合结果。As shown in (a) in Figure 12, the final fusion result of the skin (upper left image) and bones (lower left image) of the data set HAND is shown in (a) (upper right image) in Figure 12. As shown in Figure 12(b), the final fusion results of the chest (upper left image) and sternum (lower left image) of the dataset CHEST are shown in Figure 12(b) (upper right image). In fact, users can effectively specify multiple focus organizations and multiple contexts to obtain arbitrary fusion combinations, and they only need to click on the corresponding TF-let-TF node to achieve, that is, in Figure 12 (a) the user only needs to You need to click the two TF-let nodes of the skin and bones to achieve the final fusion result; in (b) in Figure 12, the user only needs to click the two TF-let nodes of the sternum and the sternum to obtain the final fusion result .
Claims (5)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310174362.1A CN116206735A (en) | 2023-02-28 | 2023-02-28 | Medical data visualization method based on histogram and nonlinear embedded transfer function |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN202310174362.1A CN116206735A (en) | 2023-02-28 | 2023-02-28 | Medical data visualization method based on histogram and nonlinear embedded transfer function |
Publications (1)
Publication Number | Publication Date |
---|---|
CN116206735A true CN116206735A (en) | 2023-06-02 |
Family
ID=86514428
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN202310174362.1A Pending CN116206735A (en) | 2023-02-28 | 2023-02-28 | Medical data visualization method based on histogram and nonlinear embedded transfer function |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN116206735A (en) |
-
2023
- 2023-02-28 CN CN202310174362.1A patent/CN116206735A/en active Pending
Similar Documents
Publication | Publication Date | Title |
---|---|---|
CN102222352B (en) | Image processing method and image processing apparatus | |
JP4774187B2 (en) | Occlusion reduction transform for observing details in a three-dimensional context | |
Huang et al. | Rgvis: Region growing based techniques for volume visualization | |
Robb et al. | ANALYZE: A software system for biomedical image analysis | |
US7529396B2 (en) | Method, computer program product, and apparatus for designating region of interest | |
JP2001502453A (en) | State-of-the-art diagnostic viewer | |
JP2004534584A (en) | Image processing method for interacting with 3D surface displayed on 3D image | |
Bornik et al. | Integrated computer-aided forensic case analysis, presentation, and documentation based on multimodal 3D data | |
Huang et al. | Visualizing industrial CT volume data for nondestructive testing applications | |
Brady et al. | Crumbs: a virtual environment tracking tool for biological imaging | |
WO2021081771A1 (en) | Vrds ai medical image-based analysis method for heart coronary artery, and related devices | |
Liimatainen et al. | Virtual reality for 3D histology: multi-scale visualization of organs with interactive feature exploration | |
Zhang | Virtual reality technology | |
CN114783591A (en) | A three-dimensional visualization auxiliary diagnosis system for brain tumors | |
JP2024516930A (en) | METHOD FOR VISUALIZING AT LEAST A ZONE OF AN OBJECT IN AT LEAST ONE INTERFACE - Patent application | |
AU2019431568B2 (en) | Method and product for processing of vrds 4d medical images | |
Islam et al. | Volume splitting and its applications | |
CN116206735A (en) | Medical data visualization method based on histogram and nonlinear embedded transfer function | |
WO2020168694A1 (en) | Vrds 4d medical image-based ai processing method and product for tumors | |
Niu et al. | Image processing, geometric modeling and data management for development of a virtual bone surgery system | |
Gao et al. | Histogram-based nonlinear transfer function edit and fusion | |
CN106920282B (en) | A method and system for editing a blood vessel digital model | |
Galeano et al. | Alternative Tool for the Diagnosis of Diseases Through Virtual Reality | |
Montilla et al. | Computer assisted planning using dependent texture mapping and multiple rendering projections in medical applications | |
JP7697795B2 (en) | Medical image processing device, medical image processing method and program |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination |