WO2012161658A1

WO2012161658A1 - A method and a device for generating a digital image

Info

Publication number: WO2012161658A1
Application number: PCT/SG2012/000178
Authority: WO
Inventors: Yiyu Cai; Jianmin ZHENG; Wei Yin Patricia CHIANG; Koon-Hou Mak; Nadia THALMANN
Original assignee: Nanyang Technological University
Current assignee: Nanyang Technological University
Priority date: 2011-05-20
Filing date: 2012-05-21
Publication date: 2012-11-29
Anticipated expiration: 2013-11-20

Abstract

Various embodiments provide a method for generating a new digital image from at least one new data point, a plurality of initial data points and an initial digital image, the initial digital image including a plurality of initial vertices, each initial vertex having at least one corresponding initial edge and a corresponding initial constraint, the corresponding initial constraint based at least on the coordinates of the initial vertex, wherein the method includes: determining a plurality of matched vertices by matching each initial data point and each new data point to an initial vertex based on a distance; generating a first intermediate digital image including the plurality of initial vertices and a plurality of additional vertices; generating a second intermediate digital image including a plurality of intermediate vertices and a plurality of intermediate edges; and deforming the intermediate digital image to generate the new digital image, wherein the deformation includes a comparison between the intermediate constraints and the initial constraints. Various embodiments also provide a corresponding device and a computer program product.

Description

A METHOD AND A DEVICE FOR GENERATING A DIGITAL IMAGE

Cross-Reference to Related Application

[0001] This application claims priority from United States of America provisional patent application number 61/488,283 filed 20 May 201 1, the content of it being hereby incorporated by reference in its entirety for all purposes.

Technical Field

[0002] Various embodiments relate generally to a method and a device for generating a digital image.

Background

[0003] Robust methods and corresponding devices that reconstruct a discrete surface with minimal latency are desirable for interventional procedures in the general field of biomedical imaging. Further, it would be desirable for such methods and devices to be compatible with presently used medical imaging modalities such as ultrasound, computed tomography and magnetic resonance imaging in order for efficient digital image reconstruction of commonly imaged human organs such as the heart, the lungs, and the brain.

Summary

[0004] Various embodiments provide a method for generating a new digital image from at least one new data point, a plurality of initial data points and an initial digital image, the initial digital image including a plurality of initial vertices, each initial vertex having at least one corresponding initial edge and a corresponding initial constraint, the corresponding initial constraint based at least on the coordinates of the initial vertex, wherein the method includes: determining a subset of a plurality of matched vertices, each matched vertex having at least one corresponding matched edge, by matching each initial data point to an initial vertex based on a distance between each initial data point and each initial vertex; determining, for each new data point, a subset of the plurality of matched vertices by matching the new data point to an initial vertex based on a distance between the new data point and each initial vertex; generating a first intermediate digital image including the plurality of initial vertices and a plurality of additional vertices, each additional vertex having at least one corresponding additional edge, wherein generating an additional vertex includes partitioning at least either an initial edge or a matched edge, and wherein generating its corresponding additional edge includes connecting the additional vertex to either an initial vertex or another additional vertex; generating a second intermediate digital image including a plurality of intermediate vertices and a plurality of intermediate edges, wherein each intermediate vertex includes a vertex of the first intermediate digital image, and wherein generating an intermediate edge includes disconnecting an edge of the first intermediate digital image and connecting a vertex of the first intermediate digital image to another vertex of the first intermediate digital image, each vertex corresponding to a face subtended by the disconnected edge; calculating a corresponding intermediate constraint for each intermediate vertex based at least on the coordinates of the intermediate vertex; and deforming the intermediate digital image to generate the new digital image, wherein the deformation includes a comparison between the intermediate constraints and the initial constraints. Various embodiments also provide a corresponding device and a computer program product.

Brief Description of the Drawings

[0005] In the drawings, like reference characters generally refer to the same parts throughout the different views. The drawings are not necessarily to scale, emphasis instead generally being placed upon illustrating the principles of the invention. In the following description, various embodiments of the invention are described with reference to the following drawings, in which:

[0006] Figure 1 shows a system for data acquisition with at least one sensor and for 3D digital reconstruction of an organ surface according to an embodiment.

[0007] Figure 2 shows a flowchart of a method for generating a new digital image from at least one new data point, a plurality of initial data points and an initial digital image according to an embodiment.

[0008] Figure 3 shows an initial digital image, a new data point and a plurality of initial data points according to an embodiment.

[0009] Figure 4 shows the generation of a first intermediate digital image according to an embodiment.

[0010] Figure 5 shows the generation of additional vertices according to an embodiment.

[0011] Figure 6 shows the generation of a plurality of additional edges for a first intermediate digital image according to an embodiment. [0012] Figure 7 which shows the generation of a second intermediate digital image according to an embodiment.

[0013] Figure 8 shows a flowchart of a method for generating a new digital image from a new data point, a plurality of initial data points, a generic digital image and an initial digital image according to an embodiment.

Detailed Description

[0014] The following detailed description refers to the accompanying drawings that show, by way of illustration, specific details and embodiments in which the invention may be practised. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. Other embodiments may be utilized and structural, logical, and electrical changes may be made without departing from the scope of the invention. The various embodiments are not necessarily mutually exclusive, as some embodiments can be combined with one or more other embodiments to form new embodiments. _r

[0015] The advance of stem-cell research has offered a promising opportunity to treat life threatening disease, such as cardiac disease. Through electromechanical mapping, it may be possible to combine the procedures of diagnosis and stem-cell therapy for treatment of cardiac disease in an effective manner.

[0016] Three dimensional (3D) discrete reconstructions of organ surfaces (displayed as digital images) from imaging modalities such as ultrasound, computed tomography and magnetic resonance imaging may provide important structural information for surgical planning and diagnosis. In this regard, various planes of imaging may be combined to reconstruct a 3D digital image using data sampled using a sensor placed within the human body whilst the body is imaged using any one of the above-mentioned modalities, or using a combination thereof.

[0017] Compared to interventional procedures, 3D digital reconstruction of organ surfaces may not be constrained by real-time requirements. Interventional procedures, on the other hand, require low-latency between data acquisition with a sensor and 3D digital reconstruction of organ surfaces. Accordingly, there exists a need for a fast digital reconstruction of the entire or a part of the organ of interest.

[0018] Figure 1 shows a system 100 for data acquisition with at least one sensor and for 3D digital reconstruction of an organ surface according to an embodiment. In an embodiment, the system 100 includes at least one sensor (namely, at least one of 102 and 104) for insertion into a human body. Sensors 102, 104 are not drawn to scale and are meant for illustrative purposes only. Accordingly, sensors 102, 104 according to Figure 1 are not meant to be limiting. In an embodiment, the at least one sensor 102, 104 may be a catheter, an endoscope, an ingestible camera, or the like. In an embodiment, at least one sensor 102, 104 may include a device 105 that generates signals that may be used to determine the position and orientation of the sensor within the human body. In an embodiment, at least one sensor 102, 104 may be inserted into the human heart, and thus, the sensor may be able to generate signals that may be used to determine the position and orientation of the sensor within the human heart. In an embodiment, at least one sensor 102, 104 may include appropriate circuitry 107 that may enable the physician or surgeon to steer and/or guide the sensor within the human body, for example, within the human heart. In an embodiment, each of the sensors 102, 104 may include a cable 109 that communicatively couples each sensor 102, 104 to a computer 106. In an embodiment, the computer 106 may be configured to receive position and orientation data from at least one sensor 102, 104 and store it in memory 110. In an embodiment, the memory 1 10 may be local to the computer 106. In an embodiment, the computer 106 may pass the position and orientation data along to an external memory 110 for subsequent storage. In an embodiment, the computer 106 may include a display device 108, such as a computer screen, an LCD projector, or the like. In an embodiment, the computer 106 may be configured to display the position and orientation of at least one sensor 102, 104 whilst data (namely, position and orientation of the human organ of interest) is collected from the human body, such as within the human heart. In an embodiment, the data collected may be displayed on the display device 108 as a digital image in the form of a 3D discrete surface reconstruction of the organ surface, namely a digital description of a surface.

[0019] A difficulty of 3D discrete surface reconstruction may lie in the progressive acquisition of data, namely, the acquisition of one data point after another. When the data points are acquired one by one, the discrete surface may have to be reconstructed again with each new data point acquisition. Further, during initial acquisition of data, a reasonable discrete surface must be reconstructed with relatively few initial data points. Subsequently, as each new data point is acquired, the initial discrete surface may have to deform and progress towards a true surface representation with acceptable transitional discrete surfaces or images. Accordingly, there must be little discrepancy between the reconstructed discrete surfaces with n data points, n+1 data points, n+2 data points and so on. [0020] Consequently, various exemplary embodiments provide a method for generating a new digital image from at least one newly acquired data point, a plurality of initial data points and an initial digital image. In various exemplary embodiments, the initial digital image includes a plurality of initial vertices, each initial vertex having at least one corresponding initial edge and a corresponding initial constraint. The word "exemplary" is used herein to mean "serving as an example, instance, or illustration". Any embodiment or design described herein as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments or designs.

[0021] Various embodiments provide a method which includes: determining a subset of a plurality of matched vertices, each matched vertex having at least one corresponding matched edge, by matching each initial data point to an initial vertex based on a distance between each initial data point and each initial vertex; determining, for each new data point, a subset of the plurality of matched vertices by matching the new data point to an initial vertex based on a distance between the new data point and each initial vertex; generating a first intermediate digital image including the plurality of initial vertices and a plurality of additional vertices, each additional vertex having at least one corresponding additional edge, wherein generating an additional vertex includes partitioning at least either an initial edge or a matched edge, and wherein generating its corresponding additional edge includes connecting the additional vertex to either an initial vertex or another additional vertex; generating a second intermediate digital image including a plurality of intermediate vertices and a plurality of intermediate edges, wherein each intermediate vertex includes a vertex of the first intermediate digital image, and wherein generating an intermediate edge includes disconnecting an edge of the first intermediate digital image and connecting a vertex of the first intermediate digital image to another vertex of the first intermediate digital image, each vertex corresponding to a face subtended by the disconnected edge; calculating a corresponding intermediate constraint for each intermediate vertex based at least on the coordinates of the intermediate vertex; and deforming the intermediate digital image to generate the new digital image, wherein the deformation includes a comparison between the intermediate constraints and the initial constraints.

[0022] Various exemplary embodiments provide a device for generating a new digital image from a new data point, a plurality of initial data points and an initial digital image.

[0023] Various exemplary embodiments provide a computer program product, which, when executed by a computer, makes the computer perform a method for generating a new digital image from a new data point, a plurality of initial data points and an initial digital image.

[0024] Figure 2 shows a flowchart of a method 200 for generating a new digital image 214 from at least one new data point 202, a plurality of initial data points 204 and an initial digital image including a plurality of initial vertices 206 according to an embodiment. In an embodiment, a method 200 may include determining a set of matched vertices (step 208 in method 200), wherein the set of matched vertices may include a plurality of matched vertices. In an embodiment, determining a set of matched vertices 208 may require processing each of the at least one new data point 202, the plurality of initial data points 204 and the initial digital image 206. The step of determining a set of matched vertices 208 may be illustrated with Figure 3. [0025] Figure 3 shows an initial digital image, a new data point 304 and a plurality of initial data points 306, 308, 310, 312, 314 according to an embodiment. In an embodiment, the initial digital image may include a plurality of initial vertices 302, 316, 318, 320, 322, 324, 326, 328 wherein each initial vertex may be associated with at least one corresponding initial edge, where a corresponding initial edge is an edge connected to the initial vertex. In the embodiment of Figure 3, 316a and 316b are corresponding initial edges in respect of initial vertex 3 16; 318a and 318b are corresponding initial edges in respect of initial vertex 318, and so on. In an embodiment, each initial vertex (namely, each of 302, 316, 318, 320, 322, 324, 326, 328) is associated with a corresponding initial constraint (not shown in Figure 3). In an embodiment, the corresponding initial constraint of an initial vertex may be a local mean curvature based at least on the coordinates of the initial vertex. In an embodiment, the corresponding initial constraint may be further based on either one of a cotangent weighting, a uniform weighting, an inverse length weighting, or a normalized weighting. Specifications relating to the corresponding initial constraint will be presented in a subsequent paragraph of this description.

[0026] In an embodiment, initial vertices that are adjacent to each other may share an initial edge. Illustratively, in the embodiment of Figure 3, initial vertices 302 and 308 share a corresponding initial edge 318b. In an embodiment, each initial vertex (302, 316, 318, 320, 322, 324, 326, 328) of the initial digital image may be displayed with at least one image pixel, depending on the resolution of the initial digital image or of an image display device. In an embodiment, each initial edge (316a, 316b, 318a, 318b, 320a, 322a, 324a, 324b) of the initial digital image may be also displayed with at least one image pixel, depending on the resolution of the initial digital image or of the image display device.

[0027] In an embodiment, a matched vertex may be determined by matching each initial data point (namely, each of 306, 308, 310, 312, 314) to an initial vertex (namely, either one of 316, 318, 320, 322, 324, 326, 328) based on a distance between each initial data point and each initial vertex. In an embodiment, the distance is a Euclidean distance. In an embodiment, the Euclidean distance is the smallest Euclidean distance.

[0028] In the embodiment of Figure 3, if the matching is based on the smallest Euclidean distance, then initial data point 314 may be matched to initial vertex 316, initial data point 306 may be matched to initial vertex 318, initial data point 308 may be matched to initial vertex 320, initial data point 310 may be matched to initial vertex 322, and initial data point 312 may be matched to initial vertex 324.

[0029] In an embodiment, a matched vertex may be further determined by matching each of the at least one new data point to an initial vertex based on a distance between the new data pointand each initial vertex. Illustratively, in the embodiment of Figure 3, if the matching is based on the smallest Euclidean distance, then new data point 304 may be matched to initial vertex 316. When there are more than one new data point, this matching between a new data point and an initial vertex is performed for each new data point. In an embodiment, the matched vertex corresponding to one initial data point may be identical to the matched vertex corresponding to another initial data point. In like manner, the matched vertex corresponding to a new data point may be identical to the matched vertex corresponding to an initial data point. For example, in the embodiment of Figure 3, each of new data point 304 and initial data point 314 is matched to initial vertex 316. In an embodiment, the matched vertex corresponding to one initial data point may not be identical to the matched vertex corresponding to another initial data point. In like manner, in an embodiment, the matched vertex corresponding to a new data point may not be identical to the matched vertex corresponding to an initial data point.

[0030] In an embodiment, the result of matching each initial data point and the at least one new data point to an initial vertex is a plurality of matched vertices. Illustratively, the plurality of matched vertices in Figure 3 includes initial vertices 316, 318, 320, 322, 324. The plurality of matched edges in Figure 3 includes edges 316a, 316b, 318a, 318b, 320a, 322a, 324a, 324b. The set of matched vertices of Figure 3 does not include initial vertices 326, 328 and 302.

[0031] In an embodiment, this matching may be performed by a search for each data point with coordinate to locate the nearest vertex PidxQ). In the embodiment where the matching is based on the smallest Euclidean distance, this search may be expressed as idx(j) - arg - , where P is the set of initial vertices.

[0032] In an embodiment, determining a plurality of matched vertices (step 208 in method 200) may serve to minimize the subsequent deformation 218 required by the method 200 to generate the new digital image. In an embodiment, the initial digital image may be a fine image having numerous initial vertices that are spaced in close proximity to each other, namely a finer mesh or a high resolution initial digital image. Whilst a high resolution initial digital image may allow a more precise matching of data points (either new or initial) to initial vertices, this may consume more computation time. Accordingly, this may be unsuitable for a real-time deformation to generate the new digital image, especially in the application to interventional treatment of pathology and disease.

[0033] Consequently, in an embodiment, a low resolution initial digital image or a coarser mesh may be first utilized to determining the set of matched vertices, and a subsequent step may be added to further refine the set of matched vertices such that the subsequent deformation 218 required by the method may have reduced computation complexity.

[0034] In an embodiment, the method 200 may further include generating a first intermediate digital image comprising the plurality of initial vertices and a plurality of additional vertices (step 210 of method 200). Figure 4 shows the generation of a first intermediate digital image 400b from an initial digital image 400a according to an embodiment. In the embodiment of Figure 4, the intermediate digital image includes the plurality of initial vertices (401, 402, 403, 404, 405, 406, 407, 408) and a plurality of additional vertices (410, 412, 414, 416, 418) .

[0035] In an embodiment, generating an additional vertex may include partitioning an initial edge that is a matched edge. In the embodiment of Figure 4, new data point 304 may be matched to initial vertex 402 based on the smallest Euclidean distance. Accordingly, in this embodiment, generating the additional vertex 412 includes partitioning an initial edge that is a matched edge (namely, edge connecting 402 and 406). In an embodiment, generating an additional vertex may include partitioning an initial edge that is not a matched edge. In the embodiment of Figure 4, generating each of the additional vertices 416 and 418 may include partitioning an initial edge that is not a matched edge (namely, edge connecting 406 and 408 in respect of additional vertex 418; edge connecting 404 and 406 in respect of additional vertex 416). Accordingly, in an embodiment, generating an additional vertex comprises partitioning at least either an initial edge or a matched edge.

[0036] In an embodiment, partitioning the matched edge connecting 402 to 406 may provide an additional vertex 412 that is closer in Euclidean distance to the new data point 304 than a vertex obtained from partitioning any of the other matched edges (namely, edge connecting 402 and 401, edge connecting 402 and 408, edge connecting 402 and 404, edge connecting 402 and 403). Accordingly, in an embodiment, an additional vertex may be generated by partitioning the matched edge connecting 402 to 406. The partitioning may also be refered to as a sub-division process. The sub-division process may provide a refinement of the initial digital image in the vicinity of each matched vertex 402. In an embodiment, partitioning the matched edge may be performed according to at least one of the following sub-division schemes: a Catmull-Clark subdivision, a Doo-Sabin sub-division, a Loop sub-division, a Mid-Edge sub-division, and a > ? sub-division. In an embodiment, the sub-division schemes may partition at least either an initial edge or a matched edge. Of these schemes, the loop sub-division scheme may be a simple scheme that is computationally efficient. Further, the loop sub-division scheme is a technique that generates a new vertex by sub-dividing an edge into 2, and connects the new vertices to form 4 sub-triangles from each triangle. This will be illustrated in the following.

[0037] Figure 5 shows the generation of an additional vertex 412 by partitioning a matched edge (edge joining matched vertices 402 and 406) according to a loop sub- division scheme. In an embodiment, partitioning the matched edge joining initial vertices 402 and 406 may occur at the mid-point of the edge joining these vertices. In another embodiment, partitioning the matched edge depends on the coordinates of neighbouring vertices. In such an embodiment, generating the additional vertex 412 includes partitioning the matched edge based on a weighted contribution of initial vertices in a local neighbourhood around the additional vertex 412. In an embodiment, the local neighbourhood of initial vertices around the additional vertex 412 includes initial vertices on the first ring of neighbours of the partitioned edge (namely, the edge connecting 402 and 406). Accordingly, in the embodiment of Figure 5, the first ring of neighbours of the edge connecting 402 and 406 may include initial vertices 402, 404, 406 and 408.. Illustratively, suppose each of matched vertices 402, 404, 406, 408 has coordinates pi, p₄, p₂, p3, respectively, and intermediate vertex 412 has coordinates p9. According to an embodiment, the contribution of each of the matched vertices 402, 404, 406, 408 to the intermediate vertex 412 may follow a linear combination according to [3a 3a b a]^T where 7a+b=l and a <b <3a. An example may be a = 1/8. Accordingly, p₉ = [3/8 3/8 1/8 1/8]^T [ i P2 P3 p₄] = 3p,/8 + 3p₂/8 + p₃/8 + p₄/8.

[0038] In an embodiment, the contribution of each neighbouring initial vertex 402, 404, 406, 408 to the position of the additional vertex 412 may be based on a distance between the additional vertex 412 to each of its neighbouring initial vertices 402, 404, 406, 408. Based on this linear combination, initial vertices 402 and 406 may have equal contribution since they may be equally spaced from additional vertex 412. Similarly, initial vertices 404 and 408 have equal contribution to the position of additional vertex 412 since they may be equally spaced from additional vertex 412. In addition, the contribution of initial vertex 402 (or 406) may be greater than the contribution of initial vertex 404 (or 408) since any partitioning on the edge joining vertices 402 and 406 may be closer to either one of 402 or 406 than to either one of vertices 404 or 408. Accordingly, the result of the partitioning with a = 1/8 yields the position of additional vertex 412 as shown in Figure 5.

[0039] In a similar manner, Figure 5 shows the generation of an additional vertex 410 by partitioning an initial edge (edge joining initial vertices 402 and 408). In an embodiment, partitioning the initial edge may be performed according to at least one of the following sub-division schemes: a Catmull-Clark sub-division, a Doo-Sabin subdivision, a Loop sub-division, a Mid-Edge sub-division, and a

sub-division. In an embodiment, the partitioning is performed according to a loop sub-division scheme. In an embodiment, generating an additional vertex by partitioning an initial edge may be limited to partitioning initial edges in a local neighbourhood of the partitioned edge yielding the additional vertex 412 (namely, the edge between 402 and 406). In an embodiment, the local neighbourhood of initial edges around the partitioned edge yielding the additional vertex 412 may include initial edges between 402 and 408, between 406 and 408, between 404 and 406, and between 402 and 404.

[0040] In an embodiment, generating the additional vertex 410 may include partitioning the initial edge based on a weighted contribution of initial vertices in a local neighbourhood of the additional vertex. In an embodiment, the local neighbourhood of initial vertices around the additional vertex 410 includes initial vertices on the first ring of neighbours of the partitioned edge yielding additional vertex 410 (namely, edge joining 402 and 408). In the embodiment of Figure 4, this may include initial vertices 401, 402, 406 and 408. Illustratively, suppose each of initial vertices 401, 402, 406, 408 has coordinates p5, pi, p2, p3, respectively, and additional vertex 410 has coordiates PA. According to an embodiment, the contribution of each of the initial vertices 401, 402, 406, 408 to the additional vertex 410 follows a linear combination according to [3a 3a b a]^T where 7a+b=l and a b <3a. An example may be a = 1/9. Accordingly, p_A = [1/3 1/3 2/9 1/9]^T [p, p3 P5 p₂] = Pi/3 + p₃/3 + 2p₅/9 + p₂/9.

[0041] In an embodiment, the contribution of each neighbouring initial vertex to the position of the additional vertex may be based on a distance between the additional vertex to each of its neighbouring initial vertices. Based on the above-described linear combination, initial vertices 402 and 408 may have equal contribution since they may be equally spaced from additional vertex 410. Similarly, initial vertex 401 may have a larger contribution than initial vertex 406 since initial vertex 401 may be closer proximity to additional vertex 410 than initial vertex 406. In addition, the contribution of initial vertex

401 (and 406) may be smaller than the contribution of initial vertex 402 (and 408) since any partitioning on the edge joining vertices 402 and 408 may be closer to either one of

402 or 408 than to either one of vertices 401 or 406. Accordingly, the result of the partitioning with a = 1/9 yields the position of additional vertex 410 as shown in Figure 5.

[0042] In an embodiment, generating a first intermediate digital image (step 210 of method 200) may further include generating at least one corresponding additional edge for each additional vertex. Figure 6 shows the generation of a plurality of additional edges for the first intermediate digital image, wherein generating an additional edge includes connecting an additional vertex (such as, additional vertex 410) to another additional vertex (such as, additional vertex 418) or an initial vertex (such as, initial vertex 408). In an embodiment, connecting each vertex (initial or additional) to another vertex (initial or additional) forms 4 sub-triangles (sub-triangle subtended by 402, 410, 412; sub-triangle subtended by 410, 412, 418; sub-triangle subtended by 408, 410, 418; sub-triangle subtended by 406, 412, 418) from each triangle (triangle subtended by 402, 406, 408).

[0043] In an embodiment, the method 200 may further include generating a second intermediate digital image comprising a plurality of intermediate vertices and a plurality of intermediate edges (step 212 of method 200). In an embodiment, generating a second intermediate digital image may improve the geometry of the first intermediate digital image by changing the connectivity of the first intermediate digital image. This is further illustrated in Figure 7 which shows generating a second intermediate digital image 700b comprising a plurality of intermediate vertices (701, 702, 703, 704, 705, 706, 707, 708) and a plurality of intermediate edges (lines connecting vertices 701 to 707) according to an embodiment. In an embodiment, each intermediate vertex of the second intermediate digital image 700b comprises a vertex of the first intermediate digital image 700a (namely, each of the vertices 701 to 708). In an embodiment, generating an intermediate edge of the second intermediate digital image (such as, the edge connecting 702 and 706) includes disconnecting an edge of the first intermediate digital image (such as, the edge connecting 704 and 708) and connecting a vertex of the first intermediate digital image (702) to another vertex of the first intermediate digital image (708). In an embodiment, each vertex 702 and 708 may correspond to a face subtended by the disconnected edge. In the embodiment of Figure 7, a face subtended by the disconnected edge e_!2 includes vertices 702, 704 and 708, and another face subtended by the disconnected edge ei₂ includes vertices 704 706 and 708. In an embodiment, generating an intermediate edge occurs if a metric is minimized. In an embodiment, the metric is either one of a local mean curvature of a surface subtended by the corresponding intermediate vertex or a local area of a surface subtended by the corresponding intermediate vertex. With reference to Figure 7, the edge e_n (comprising vertices 704 and 708 in 700a) is flipped within the quadrilateral to form edge e₀₃ (comprising vertices 702 and 706 in 700b) if a swap criterion is satisfied. Several edge swap criteria may be are defined. A reasonably good criterion for this application may be the minimization of surface area. Defining _t as the area of triangle T_i , the edge e_y (comprising vertices p_t and p . ) is flipped if and only if area sum of swap configuration is smaller than existing configuration i.e. (A_Q + A₃ ) < (A_] + A₂ ) . Another good criterion for improving the geometry of the first intermediate digital image may be the minimization of local mean curvature. This criterion suppresses the formation of local depression or protrusion and enables global smoothness. For each edge e_t. , the curvature is computed as the angle 9_tj between the normals of 2 adjacent triangles T_i and T_} . For simplicity, the dot product of the normals, cos^. is computed and an edge swap is performed if the swapped edge is locally

Delaunay and the sum of the local inverse curvatures cos θ_υ of edges associated with T_t and T_j is larger for swap configuration i.e

(cos#₀₃ + cos#43 + cos#₅₀ + cos#₆₃ + cos#₇₀ ) > (cos#₁₂ + cos#₄₁ + cos#₅₁ + cos#₆₂ + cos#₇₂) .

An edge is locally Delaunay if the sum of the opposite angles a and β is not greater than π or the Laplacian cotangent weight y₂ (cot a + cot ?) associated with the edge is non-negative. The Delaunay condition maximizes the minimal angle and pre-empts the formation of elongated triangles.

[0044] In an embodiment, the subsequent deformation 218 required by the method 200 to generate the new digital image may require smooth, non-abrubt transitions between the second intermediate digital image and the new digital image. To achieve this, an embodiment of the method 200 may generate a globally (or regionally) continuous and smooth new digital image, whilst preserving certain local characteristics of the second intermediate digital image. Accordingly, an embodiment of the method 200 may include measuring local characteristics of the second intermediate digital image. In an embodiment, measuring local characteristics of the second intermediate digital image may include calculating a corresponding intermediate constraint for each intermediate vertex of the second intermediate digital image (step 214 in method 200 of Figure 2). In an embodiment, the calculation of the corresponding intermediate constraint (and hence, the local characteristics of the intermediate digital image) may be based on the coordinates of a corresponding intermediate vertex of the second intermediate digital image. In an embodiment, the corresponding intermediate constraint may be further based on either one of a cotangent weighting of a corresponding intermediate edge, a uniform weighting of a corresponding intermediate edge, an inverse length weighting of a corresponding intermediate edge, or a normalized weighting of a corresponding intermediate edge. These are equally applicable to the intial constraints associated with the initial digital image.

[0045] In an embodiment, a recursive Laplacian deformation may generate a globally or regionally continuous and smooth new digital image, whilst preserving local Laplacian characteristics of the second intermediate digital image. In an embodiment, the local characteristic of the second intermediate digital image (and hence, the corresponding intermediate constraint) may be computed by a weighted Laplacian differential. The Laplacian differential vector of an intermediate vertex having coordinates (or vector representation) p, may be defined as: δ^*ϊ = ∑j_£N(i) (Oijip_j-pd, where δ] may approximate a local mean curvature, jeN(i) may refer to the first ring of neighbours having coordinates P_j of the intermediate vertex having coordinates p _, and coj_j may be a weight corresponding to each intermediate edge between intermediate vertex having coordinates p_t and its neighbour having coordinates p_j.

[0046] Several types of weights corresponding to each intermediate edge between the vertex having coordinates pi and its neighbour having coordinates p_j may be used. In an embodiment, the Laplacian weighting scheme may use a uniform weighting of a corresponding intermediate edge (namely, the edge connecting p_t andpj , namely, co_y = 1. In another embodiment, an inverse length weighting of a corresponding intermediate edge may be used, wherein ω_¾· = 1/|

In another embodiment, a cotangent weighting of a corresponding intermediate edge may be used, namely coy = cot o¾ + cot /¾, where expand jSj_j are opposite angles of a corresponding intermediate edge. In an embodiment, the cotangent weighting may preserve topology and geometric details of the local characteristics of the intermediate digital image.

[0047] In an embodiment, the set of all differential Laplacian vectors may be expressed as a matrix. Suppose there are n intermediate vertices and P„a = [pid, P2d, ■■·, Pnd]^J with de{x,y,z} the n-by- 1 vector containing the x, y, or z coordinates of the n intermediate vertices. Then, the x, y, and z coordinates of the laplacian deformation weights L_nd = [<S_Id, ¾_/, may be calculated separately as L_nd = LP_nd, where

Ly ⁼ ∑jeN(i) ©i_j if i=j;

Ly = co_y if (ij) e E;

Ly = 0 otherwise. where E is the set of edges in the second intermediate digital image.

[0048] In an embodiment, the method 200 may include deforming the intermediate digital image to generate the new digital image (step 218 in method 200 of Figure 2). In an embodiment, the deformation includes a comparison between the intermediate constraints and the initial constraints. Accordingly, in an embodiment, the deformation 218 may include a comparison between the weighted Laplacian of a corresponding intermediate vertex and a weighted Laplacian of a corresponding initial vertex. In an embodiment, the deformation 218 may minimize the mean square difference between the intermediate constraints and the initial constraints. Accordingly, in an embodiment, the deformation 218 may minimize the mean sequare difference between the weighted Laplacians of the intermediate digital image and the weighted Laplacian of the initial digital image.

[0049] In an embodiment, the vertices of the new digital image may be determined in a least square optimization that minimizes the mean square difference between the computed weighted Laplacian of the intermediate digital image and the computed weighted Laplacian of a previous deformation, namely, the previous deformation that yielded the initial digital image. In an embodiment, the linear system L_nd = LP_nd may be overdetermined. Accordingly, the set of vertices of the new digital image P_n+1 is the solution to the following minimization problem: min (L_nd - LP_nd)² subject to IP„^* = S where P_(n+i)d = [fid, P2d, Pnd. P(n+i)d , with de{x,y,zj, is the (n+l)-by-l vector containing the x, y, or z coordinates of the n+1 vertices of the new digital image, and where / is the identity matrix of degree n, and where P_n ^* is a vector containing the coordinates of the set of intermediate vertices, and where S may be the coordinates of the set of n initial data points and the (n+1) new data point.When there are more than one new data point, (for example, (η+1)^ώ (η+2)^Λ (η+3)^Λ) then the set of vertices of the new digital image P_n+3 is the solution to the following minimization problem: min (L_nd - LP_(n+3)d)² subject to = S where P(„+₃)d =

Pid, -, Pnd. P(n+i)d , P(_n+2)d, P(n+3)df , with de{x,y,z}, is the (n+3)-by-l vector containing the x, y, or z coordinates of the n+3 vertices of the new digital image, and where / is the identity matrix of degree n, and where P„^* is a vector containing the coordinates of the set of intermediate vertices, and where S may be the coordinates of the set of « initial data points and the (n+l)^th, (η+2)^Λ, (n+3)^th new data points.

[0050] Figure 8 shows a flowchart of a method for generating a new digital image from a new data point, a plurality of initial data points and an initial digital image according to an embodiment. In an embodiment, the initial digital image 81 1 may be generated by an alignment 802 of a generic digital image 801 to landmark points of the plurality of initial data points 809, wherein the generic digital image 801 may include a plurality of generic vertices. As used herein, landmark points refer to a subset of the plurality of initial data points that may be particular features identified for a particular organ. For example, landmark points corresponding to a human heart may include the aortic valve center, the mitral valve center, and the apex. In an embodiment, the landmark points may be used for image registration. In an embodiment, the generic digital image 801 may be an approximation of a surface to be reconstructed. Accordingly, the choice of the generic digital image 801 may be associated with the geometry of the surface to be reconstructed. In an embodiment, image regularities, the non-occurrence of high degree, the absense of sharp corners may be typical properties desired for reconstructing surfaces. In an embodiment, the surface to be reconstructed may be an endocardial surface of a human heart. Accordingly, in an embodiment, the generic digital image 801 may be a prolate spheroid.

[0051] In an embodiment, the generic digital image, namely, the prolate spheroid approximating the endocardial surface of the human heart, may be given by the following x, y, z coordinates: x = a sinh ξ sin η cos φ y = a sinh ξ sin η sin φ z = a cosh ξ cos η, where where φ represents the circumferential angular co-ordinate, η represents the longitudinal angular co-ordinate, ξ represents the radial co-ordinate and a represents the focal length. An exemplary parameter set may be given by 0 < φ < 2π; τ/3 <η < π; ξ = 0.5 and a = 50. In an embodiment, the generic digital image may be obtained by digitally sampling the prolate spheroid by varying the pair (η, φ).

[0052] In an embodiment, the alignment 802 may include at least one of a rotation, a scaling, and a translation. Illustratively, if the surface to be reconstructed includes an endocardial wall of a human heart, the landmark points 809 may include the apex of the heart, the mitral valve center, the aortic valve center. In an embodiment, each landmark point (and each initial and at least one new data point) may be acquired in at least one of a fluoroscopy system, an ultrasound system, a computed tomography scan system, an X- ray system, a magnetic resonance imaging system, an electrophysiological mapping system and an electromechanical mapping system. In an embodiment, each of these data points may be acquired with a sensor introduced into a human body.

[0053] In an embodiment, points in the generic digital image 801 corresponding to each of the landmark points 809, namely, the apex of the heart, the mitral valve center, the aortic valve cente may be identified. In an embodiment, these points in the generic digital image may be aligned to the landmark points according to f(P_j) = c X_jl j {_Pi Vy/( I V_y \^k + e)}] /[∑i {_Pi/( I Vj_j \^k + c)}].

Here, p_j denotes the coordinates of generic vertex j of the generic digital image which may be adjusted according to f(p_j) and Vy represents the vector from vertex j of the generic digital image to the i'^h landmark point. In an embodiment, the adjustment may be a weighted sum of inverse distances |Vy| from the i'^h landmark point and its corresponding density />,. In an embodiment, the adjustment may be further modulated by a distribution , of the landmark point around the generic vertex. [0054] In an embodiment, the initial digital image 811 obtained from the above- mentioned alignment of a generic digital image 801 to the landmark points 809, which themselves may be a subset of the initial data points, may then be used for determining a set of matched vertices. Accordingly, the further features described above with reference to determining a set of matched vertices in the embodiment of Figure 2 (step 208 of method 200) are equally applicable, and hereby restated, in respect of determining a set of matched vertices (703) in the embodiment of Figure 8.

[0055] In an embodiment, the method for generating a new digital image from at least one new data point, a plurality of initial data points and an initial digital image may include local sub-division (804) and edge swapping (805). Accordingly, the further features described above with reference to generating a first intermediate digital image (step 210 of method 200) are equally applicable, and hereby restated, in respect of subdivision 804. Similarly, the further features described above with reference to generating a second intermediate digital image (step 212 of method 200) are equally applicable, and hereby restated, in respect of edge swapping.

[0056] In an embodiment, calculating an intermediate constraint for each vertex of the generated second intermediate digital image may include calculating a weighted laplacian (806). Accordingly, the further features described above with reference to calculating an intermediate constraint for each vertex of the second generated intermediate digital image in the embodiment of Figure 2 (step 214 of method 200) are equally applicable, and hereby restated, in respect of calculating an intermediate constraint for each vertex of the generated second intermediate digital image (806). [0057] In an embodiment, deforming the intermediate digital image to generate the new digital image may include a least square optimization (807). Accordingly, the further features described above with reference to deforming the intermediate digital image to generate the new digital image in the embodiment of Figure 2 (step 218 of method 200) are equally applicable, and hereby restated, in respect of generating the new digital image in the embodiment of Figure 8 namely, the custom mesh of 808).

[0058] In an embodiment, a further data point may be acquired 810 after the generation of the new digital image 808. Accordingly, in an embodiment, the method may be repeated in respect of the further acquired data point (being the new data point in the further iteration) and the new digital image (being the initial digital image of the further iteration)

[0059] Various embodiments provide a device for generating a new digital image from at least one new data point, a plurality of initial data points and an initial digital image, the initial digital image including a plurality of initial vertices, each initial vertex having at least one corresponding initial edge and a corresponding initial constraint, the corresponding initial constraint based at least on the coordinates of the initial vertex, wherein the device includes: a first determining circuit configured to determine a subset of a plurality of matched vertices, each matched vertex having at least one corresponding matched edge, by matching each initial data point to an initial vertex based on a distance between each initial data point and each initial vertex; a second determining circuit configured to determine, for each new data point, a subset of the plurality of matched vertices by matching the new data point to an initial vertex based on a distance between the new data point and each initial vertex; a first generating circuit configured to generate a first intermediate digital image including the plurality of initial vertices and a plurality of additional vertices, each additional vertex having at least one corresponding additional edge, wherein generating an additional vertex includes partitioning at least either an initial edge or a matched edge, and wherein generating its corresponding additional edge includes connecting the additional vertex to either an initial vertex or another additional vertex; a second generating circuit configured to generate a second intermediate digital image including a plurality of intermediate vertices and a plurality of intermediate edges, wherein each intermediate vertex includes a vertex of the first intermediate digital image, and wherein generating an intermediate edge includes disconnecting an edge of the first intermediate digital image and connecting a vertex of the first intermediate digital image to another vertex of the first intermediate digital image, each vertex corresponding to a face subtended by the disconnected edge; a calculating circuit configured to calculate a corresponding intermediate constraint for each intermediate vertex based at least on the coordinates of the intermediate vertex; and a deforming circuit configured to deform the intermediate digital image to generate the new digital image, wherein the deformation includes a comparison between the intermediate constraints and the initial constraints.

[0060] The further features descincludribed above with reference to the method are equally applicable, and hereby restated, in respect of the device.

[0061] Various embodiments provide computer program product, which, when executed by a computer, makes the computer perform a method for generating a new digital image from at least one new data point, a plurality of initial data points and an initial digital image, the initial digital image including a plurality of initial vertices, each initial vertex having at least one corresponding initial edge and a corresponding initial constraint, the corresponding initial constraint based at least on the coordinates of the initial vertex, wherein the method comprises: determining a subset of a plurality of matched vertices, each matched vertex having at least one corresponding matched edge, by matching each initial data point to an initial vertex based on a distance between each initial data point and each initial vertex; determining, for each new data point, a subset of the plurality of matched vertices by matching the new data point to an initial vertex based on a distance between the new data point and each initial vertex; generating a first intermediate digital image including the plurality of initial vertices and a plurality of additional vertices, each additional vertex having at least one corresponding additional edge, wherein generating an additional vertex includes partitioning at least either an initial edge or a matched edge, and wherein generating its corresponding additional edge includes connecting the additional vertex to either an initial vertex or another additional vertex; generating a second intermediate digital image including a plurality of intermediate vertices and a plurality of intermediate edges, wherein each intermediate vertex includes a vertex of the first intermediate digital image, and wherein generating an intermediate edge includes disconnecting an edge of the first intermediate digital image and connecting a vertex of the first intermediate digital image to another vertex of the first intermediate digital image, each vertex corresponding to a face subtended by the disconnected edge; calculating a corresponding intermediate constraint for each intermediate vertex based at least on the coordinates of the intermediate vertex; and deforming the intermediate digital image to generate the new digital image, wherein the deformation includes a comparison between the intermediate constraints and the initial constraints.. [0062] The further features described above with reference to the method are equally applicable, and hereby restated, in respect of the computer program product.

[0063] While the invention has been particularly shown and described with reference to specific embodiments, it should be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. The scope of the invention is thus indicated by the appended claims and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced.

Claims

Claims claimed is:

A method for generating a new digital image from at least one new data point, a plurality of initial data points and an initial digital image, the initial digital image comprising a plurality of initial vertices, each initial vertex having at least one corresponding initial edge and a corresponding initial constraint, the

corresponding initial constraint based at least on the coordinates of the initial vertex, wherein the method comprises:

determining a subset of a plurality of matched vertices, each matched vertex having at least one corresponding matched edge, by matching each initial data point to an initial vertex based on a distance between each initial data point and each initial vertex;

determining, for each new data point, a subset of the plurality of matched vertices by matching the new data point to an initial vertex based on a distance between the new data point and each initial vertex;

generating a first intermediate digital image comprising the plurality of initial vertices and a plurality of additional vertices, each additional vertex having at least one corresponding additional edge, wherein generating an additional vertex comprises partitioning at least either an initial edge or a matched edge, and wherein generating its corresponding additional edge comprises connecting the additional vertex to either an initial vertex or another additional vertex;

generating a second intermediate digital image comprising a plurality of intermediate vertices and a plurality of intermediate edges, wherein each intermediate vertex comprises a vertex of the first intermediate digital image, and wherein generating an intermediate edge comprises disconnecting an edge of the first intermediate digital image and connecting a vertex of the first intermediate digital image to another vertex of the first intermediate digital image, each vertex corresponding to a face subtended by the disconnected edge;

calculating a corresponding intermediate constraint for each intermediate vertex based at least on the coordinates of the intermediate vertex; and

deforming the intermediate digital image to generate the new digital image, wherein the deformation comprises a comparison between the intermediate constraints and the initial constraints.

The method according to claim 1 , wherein the initial digital image is generated by an alignment of a generic digital image to the plurality of initial data points, the generic digital image comprising a plurality of generic vertices.

The method according to claim 2, wherein the alignment comprises at least one of a rotation, a scaling, and a translation.

The method according to claim 3, wherein the translation is based on an inverse distance between each generic vertex and each initial data point.

The method according to claim 2, wherein the generic digital image is an approximation of a surface to be reconstructed.

6. The method according to claim 5, wherein the surface to be reconstructed is an endocardial surface of a human heart.

7. The method according to claim 6, wherein the generic digital image is a prolate spheroid.

8. The method according to claim 1 , wherein the distance is a Euclidean distance.

9. The method according to claim 8, wherein the Euclidean distance is the smallest Euclidean distance.

10. The method according to claim 1, wherein each of partitioning the matched edge and partitioning the initial edge comprises at least one of a Catmull-Clark subdivision, a Doo-Sabin sub-division, a Loop sub-division, a Mid-Edge subdivision, and a /3 sub-division.

1 1. The method according to claim 1, wherein generating an intermediate edge occurs if a metric is minimized.

12. The method according to claim 1 1, wherein the metric is either one of a local mean curvature of a surface subtended by the corresponding intermediate vertex or a local area of a surface subtended by the corresponding intermediate vertex.

13. The method according to claim 1 , wherein the deformation minimizes the mean square difference between the intermediate constraints and the initial constraints.

14. The method according to claim 1, wherein each of the corresponding initial

constraint and the corresponding intermediate constraint is further based on either one of a cotangent weighting of a corresponding edge, a uniform weighting of a corresponding edge, an inverse length weighting of a corresponding edge, or a normalized weighting of a corresponding edge.

15. The method according to claim 1, wherein each of the intermediate constraint and the initial constraint is a weighted Laplacian of the corresponding intermediate vertex and a weighted Laplacian of the corresponding initial vertex, respectively.

16. The method according to claim 1, wherein the new data point and each initial data point are acquired in at least one of a fluoroscopy system, an ultrasound system, a computed tomography scan system, an X-ray system, a magnetic resonance imaging system, an electrophysiological mapping system and an

electromechanical mapping system.

17. The method according to claim 1, wherein the new data point and each initial data point are acquired with a sensor introduced into a human body.

18. A device for generating a new digital image from at least one new data point, a plurality of initial data points and an initial digital image, the initial digital image comprising a plurality of initial vertices, each initial vertex having at least one corresponding initial edge and a corresponding initial constraint, the

corresponding initial constraint based at least on the coordinates of the initial vertex, wherein the device comprises:

a first determining circuit configured to determine a subset of a plurality of matched vertices, each matched vertex having at least one corresponding matched edge, by matching each initial data point to an initial vertex based on a distance between each initial data point and each initial vertex;

a second determining circuit configured to determine, for each new data point, a subset of the plurality of matched vertices by matching the new data point to an initial vertex based on a distance between the new data point and each initial vertex;

a first generating circuit configured to generate a first intermediate digital image comprising the plurality of initial vertices and a plurality of additional vertices, each additional vertex having at least one corresponding additional edge, wherein generating an additional vertex comprises partitioning at least either an initial edge or a matched edge, and wherein generating its corresponding additional edge comprises connecting the additional vertex to either an initial vertex or another additional vertex;

a second generating circuit configured to generate a second intermediate digital image comprising a plurality of intermediate vertices and a plurality of intermediate edges, wherein each intermediate vertex comprises a vertex of the first intermediate digital image, and wherein generating an intermediate edge comprises disconnecting an edge of the first intermediate digital image and connecting a vertex of the first intermediate digital image to another vertex of the first intermediate digital image, each vertex corresponding to a face subtended by the disconnected edge;

a calculating circuit configured to calculate a corresponding intermediate constraint for each intermediate vertex based at least on the coordinates of the intermediate vertex; and

a deforming circuit configured to deform the intermediate digital image to generate the new digital image, wherein the deformation comprises a comparison between the intermediate constraints and the initial constraints.

A computer program product, which, when executed by a computer, makes the computer perform a method for generating a new digital image from at least one new data point, a plurality of initial data points and an initial digital image, the initial digital image comprising a plurality of initial vertices, each initial vertex having at least one corresponding initial edge and a corresponding initial constraint, the corresponding initial constraint based at least on the coordinates of the initial vertex, wherein the method comprises: