CN111260547B

CN111260547B - Method, apparatus and computer-readable storage medium for presenting brain image

Info

Publication number: CN111260547B
Application number: CN202010352794.3A
Authority: CN
Inventors: 陈伟; 吴伯阳
Original assignee: Beijing Zhituo Vision Technology Co ltd
Current assignee: Beijing Zhituo Vision Technology Co ltd
Priority date: 2020-04-29
Filing date: 2020-04-29
Publication date: 2020-09-18
Anticipated expiration: 2040-04-29
Also published as: CN111260547A

Abstract

The invention discloses a method, a device and a computer readable storage medium for presenting a brain image, wherein the method comprises the steps of cutting the brain image along a closed curve formed by a plurality of vertexes of a grid forming the brain image to obtain two topological discs; respectively harmoniously mapping the two topological disks onto a half spherical surface; and splicing the two semispherical surfaces along the closed curve to obtain a complete spherical surface so as to map the brain image harmony onto the complete spherical surface. The invention facilitates the acquisition, integration and analysis of brain image data to expand the further application of brain images.

Description

Method, apparatus and computer-readable storage medium for presenting brain image

Technical Field

The present invention generally relates to the field of image analysis. More particularly, the present invention relates to a method, apparatus and computer-readable storage medium for presenting brain images.

Background

Recent advances in brain imaging technology have accelerated the collection and databanking of brain images. Nevertheless, computational problems still arise when integrating and comparing brain data. One method of analysis and comparison is to map brain data into a standard region. However, how to keep the geometric information on the original structure as much as possible while mapping becomes an aspect to be solved.

Disclosure of Invention

To solve at least the above technical problems, the present invention provides a solution for harmonically mapping a brain image to a spherical surface, thereby facilitating the acquisition, integration and analysis of brain image data to expand further applications of the brain image.

In one aspect, the invention provides a method for presenting an image of a brain, comprising:

cutting the brain image along a closed curve formed by a plurality of vertexes of a mesh constituting the brain image to obtain two topological discs;

respectively harmoniously mapping the two topological disks onto a half spherical surface; and

and splicing the two semi-spherical surfaces along the closed curve to obtain a complete spherical surface so as to map the brain image harmony onto the complete spherical surface.

In one embodiment, the aforementioned method further comprises obtaining said closed curve starting from a starting point, going through N intermediate vertices in sequence, and returning to said starting point by performing the following operations:

calculating a function value of each vertex according to the mesh information;

selecting a vertex having a globally smallest function value from all vertices as the starting point of the closed curve;

selecting a vertex having a locally smallest function value as a first intermediate vertex from a plurality of vertices adjacent to the starting point;

for each of the 2 nd to Nth intermediate vertices, performing the following selection operations until returning to the starting point:

selecting a vertex having a locally smallest function value as an Nth intermediate vertex from among a plurality of vertices adjacent to the Nth-1 st intermediate vertex,

wherein N is a positive integer greater than or equal to 2.

In another embodiment, the function value is an eigen function corresponding to a maximum eigenvalue of a laplacian-berlamide matrix based on the grid, the method comprising: calculating a Laplacian-Bell-Lambda matrix based on the mesh according to the topology of the mesh and the side lengths of the mesh edges.

In yet another embodiment, harmonically mapping the two topological discs onto a half sphere, respectively, comprises: respectively harmoniously mapping the two topological disks to a unit disk; and projecting the obtained two unit discs on the half spherical surface respectively through spherical poles.

In a further embodiment, a complete sphere resulting from stitching the two hemispheres is used as an initial mapping of the brain image, the method further comprising optimizing its harmonic energy for the initial mapping to obtain a corrected harmonic mapping.

In further embodiments, optimizing its harmonic energy with respect to the initial mapping includes performing an optimization of the harmonic energy with a gradient descent algorithm to obtain a harmonic mapping with minimized harmonic energy.

In one embodiment, for a sphere having a corrected harmonic mapping, the method further comprises: calculating a Mobius transform using the centroid of the spherical surface to correct the centroid; and optimizing the blending energy until a predetermined energy difference threshold is met, thereby blending mapping the brain image onto the complete sphere.

In another aspect, the present invention provides an apparatus for presenting an image of a brain, comprising:

at least one processor; and

at least one memory for storing program instructions that, when loaded and executed by the at least one processor, cause the apparatus to perform the aforementioned methods and embodiments thereof.

In a further aspect, the invention provides a computer-readable storage medium in which program instructions for presenting brain images are stored, the program instructions being adapted to be loaded by a processor and to carry out the aforementioned method and its various embodiments.

The above-described method, apparatus, and computer-readable storage medium of the present invention can be applied in a variety of scenarios. For example, it may provide canonical space for automatic feature recognition, brain-to-brain registration, brain structure segmentation, brain surface noise reduction, shape analysis, and convenient surface visualization, among others. Further, the scheme of the invention facilitates the acquisition, integration and analysis of brain image data by harmoniously mapping the brain image onto a spherical surface.

Drawings

The above features of the present invention may be better understood, and its numerous objects, features, and advantages made apparent to those skilled in the art by referencing the accompanying drawings, wherein like reference numerals refer to like elements and in which:

FIG. 1 is a simplified flow diagram illustrating a method for presenting brain images according to an embodiment of the present invention;

fig. 2 is a detailed flowchart illustrating a method for presenting a brain image according to an embodiment of the present invention;

fig. 3 is a diagram showing an example brain raw image for presenting a brain image that can be applied to the present invention;

FIG. 4 is a diagram illustrating an exemplary triangular mesh, according to an embodiment of the invention;

fig. 5a is a diagram illustrating a first brain region obtained after incision along a closed curve according to an embodiment of the present invention;

fig. 5b is a diagram illustrating a second brain region obtained after incision along a closed curve according to an embodiment of the present invention;

FIG. 6a is a diagram illustrating a first topological disc obtained in accordance with an embodiment of the present invention;

FIG. 6b is a diagram illustrating a second topological disc obtained in accordance with an embodiment of the present invention;

FIG. 7a is a diagram illustrating a first hemispherical surface obtained according to an embodiment of the present invention;

FIG. 7b is a diagram illustrating a second hemispherical surface obtained according to an embodiment of the present invention;

FIG. 8a is a diagram illustrating a spliced complete sphere obtained according to an embodiment of the present invention;

FIG. 8b is a diagram illustrating a brain image mapped onto a complete sphere by a harmonic mapping according to an embodiment of the present invention; and

fig. 9 is a block diagram illustrating an apparatus for presenting a brain image according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It is to be understood that the terminology used in the description of the disclosure herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification and claims of this disclosure, the singular forms "a", "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be further understood that the term "and/or" as used in the specification and claims of this disclosure refers to any and all possible combinations of one or more of the associated listed items and includes such combinations.

As used in this specification and claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to a determination" or "in response to a detection". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".

Fig. 1 is a simplified flow diagram illustrating a method 100 for presenting brain images in accordance with an embodiment of the present invention. As previously mentioned, the method 100 may be implemented by various types of computing devices including, for example, a computer, and the brain image or brain image data involved therein may be an image obtained by, for example, Magnetic Resonance Imaging ("MRI") techniques or devices, such as the example brain raw image shown in fig. 3.

As shown in fig. 1, at step 102, method 100 slices the brain image along a closed curve formed by a plurality of vertices of a mesh constituting the brain image to obtain two topological discs. In one embodiment, the function value of each vertex may be calculated from the mesh information, and the vertex having the globally smallest function value is selected from all the vertices as the starting point of the aforementioned closed curve. After the starting point of the closed curve is determined, a plurality of intermediate vertexes starting from the starting point and returning to the starting point are sequentially determined. In one embodiment, the function value is an eigen function corresponding to a maximum eigenvalue of a laplacian-berlamide matrix based on the grid. Next, at step 104, the method 100 harmonically maps the two topological disks onto half-spheres, respectively. In one embodiment, harmonically mapping the two topological discs onto the hemispherical surfaces respectively comprises harmonically mapping the two topological discs onto the unit discs respectively, and projecting the resulting two unit discs onto the hemispherical surfaces respectively with the poles respectively, for example, projecting one unit disc onto the unit upper hemispherical surface and the other unit disc onto the unit lower hemispherical surface. Finally, at step 106, the method 100 stitches the two half spheres (the upper and lower spheres as before) along the closed curve to obtain a complete sphere for harmonically mapping the brain image onto the complete sphere.

To achieve the above harmonic mapping of the brain image onto the complete sphere, in one embodiment, the method 100 may use the complete sphere obtained after stitching as an initial mapping of the brain image, and optimize its harmonic energy with respect to the initial mapping to obtain a corrected harmonic mapping. For the spherical surface with the corrected harmonic mapping, the method 100 may also calculate a Mobius transform using the centroid of the spherical surface to correct the centroid; and optimizing the blending energy until a predetermined energy difference threshold is met, thereby blending mapping the brain image onto the complete sphere. Compared with the existing mapping method of the brain image, the scheme of the invention is more stable and has good expansibility.

Fig. 2 is a detailed flow diagram illustrating a method 200 for presenting brain images according to an embodiment of the present invention. It should be understood that the method 200 is a specific implementation of the method 100 shown in fig. 1, and thus the description made with respect to the method 100 is equally applicable to the method 200.

As mentioned in connection with fig. 1, the present invention proposes to form a closed curve along a plurality of vertices of a mesh constituting the brain image in order to dissect the brain image. To this end, as shown in fig. 2, at step 202, the method 200 may compute a mesh-based laplacian-berterra-m matrix according to the topology of the mesh and the side lengths of the mesh edges. The topology of the mesh is understood herein to be the connection relationship of the mesh, specifically, the connection relationship between vertices on the triangular mesh. When the total number of vertices is M, an M × M laplacian matrix is formed. The laplacian-belltremide matrix is described below in conjunction with fig. 4.

Fig. 4 is a diagram illustrating an exemplary triangular mesh according to an embodiment of the present invention. As can be seen from FIG. 4, here two triangular meshes are shown, comprising four vertices

,

,

And are and

. Further, the vertex

And

the edge and the vertex formed between

And

the included angle between the edges formed between the two is

And the vertex is

And

the edge and the vertex formed between

And

the included angle between the edges formed between the two is

. In addition, the vertex can be seen from the figure

And sharing the edges. From the vertices, side lengths, and included angles exemplarily shown here, each element value in the laplacian-berterra-m matrix can be determined by the following formula, i.e., the vertex

Weight (relationship) between:

（1）

wherein:

（2）

the above-mentioned inner edge indicates that the edge is shared by two triangular meshes "

"indicates the cotangent value, and the boundary edge indicates that the edge is not common to both triangular meshes, but is contained by only one triangular mesh.

After the laplacian-berterra-mi matrix based on the grid is calculated according to the topology of the grid and the side length of the grid edge, the matrix can be calculated to obtain the maximum eigenvalue of the matrix, so that the eigenfunction corresponding to the maximum eigenvalue can be determined. Based on the feature function, a function value for each vertex can be obtained. Thus, at step 204, the method 200 may select the vertex with the globally smallest function value from all vertices as the start and end points of a closed curve for separating the brain images.

After determining the start and end points of the closed curve, method 200 also entails determining the N intermediate vertices of the closed curve. Specifically, the method 200 selects a vertex having a locally smallest function value from a plurality of vertices adjacent to the start point as the first intermediate node at step 206. Next, at step 208, the method 200 performs the following selection operations for each of the 2 nd through Nth intermediate vertices, until returning to the starting point (i.e., the ending point): selecting a vertex having a locally smallest function value as an Nth intermediate vertex from a plurality of vertices adjacent to the Nth-1 intermediate vertex, where N may be a positive integer greater than or equal to 2. When the above operations are completed, the method 200 obtains a closed curve starting from the starting point, passing through the N intermediate nodes in sequence, and returning to the starting point at step 210.

After obtaining the closed curve, the method 200 slices the brain image along the closed curve formed by the plurality of vertices of the mesh constituting the brain image at step 212, thereby obtaining a first brain image region as shown in fig. 5a and a second brain image region as shown in fig. 5 b. The two brain image regions may then be mapped onto two topological discs, e.g. a first topological disc as shown in fig. 6a and a second topological disc as shown in fig. 6b, by e.g. a euclidean harmonic mapping. Next, at step 214, the method 200 may harmonically map the two topological disks onto the unit disks, respectively, and at step 216, project the resulting two unit disks, respectively, epipolar into a half-sphere, thereby obtaining a first half-sphere as shown in fig. 7a and a second half-sphere as shown in fig. 7 b.

After obtaining the two hemispherical surfaces described above, at step 218, the method 200 splices the two hemispherical surfaces along the closed curve obtained at step 210 to obtain a complete spherical surface, such as the complete spherical surface shown in fig. 8 a. Next, at step 220, the method 200 takes the resulting complete sphere as an initial mapping of the brain image and optimizes its harmonic energy for the initial mapping to obtain a corrected harmonic mapping.

In order to correct the initial mapping, e.g. to adjust the mapping of the area around the closed curve to meet the conditions of the harmonic mapping, in one embodiment the harmonic energy may be optimized by means of the following gradient descent algorithm

To obtain an overall harmonic map with minimum harmonic energy:

the algorithm is as follows: input (grid)

Step length

Threshold of energy difference

And output (C)

：

) Wherein

The harmonic energy is minimized.

Step S11: order to

= N (N is the initial mapping N of the invention:

) Computing the harmonic energy of the initial mapping

；

Step S12: for each vertex

Calculating the absolute derivative

；

Step S13: by passing

To update

;

Step S14: calculating harmonic energy

；

Step S15: if it is not

Go back to

(ii) a Otherwise, it will

Is assigned to

And steps S12 to S15 are repeated.

Since the harmonic energy has a unique minimum, the algorithm can converge and stabilize quickly.

After obtaining the harmonic mapping with the minimum harmonic energy, the scheme of the invention can further process the sphere. Specifically, at step 222, the method 200 may calculate a Mobius transform using the centroid of the sphere to correct the centroid of the sphere. With the foregoing modifications, method 224 also optimizes the blend energy until a predetermined energy difference threshold is met, thereby blending-mapping the brain image onto a complete sphere. To facilitate a further understanding of

steps

222 and 224, an exemplary algorithm to implement the foregoing steps will be given below:

the algorithm is as follows: input (grid)

Step length

Threshold of energy difference

And output (C)

：

) Wherein

Harmonic energy is minimized and the zero centroid constraint is satisfied.

Step S21: order to

=

（

The corrected harmonic mapping of the present invention

：

) Calculating the harmonic energy

；

Step S22: for each vertex

Calculating the absolute derivative

；

Step S23: by passing

To update

;

Step S24: calculating Mobius transform

So that

（3）

（4）

Wherein

Is that

Upper area unit.

Is the center of mass,

minimizing norm ("norm") under centroid conditions;

step S25: calculating harmonic energy

；

Step S26: if it is not

Go back to

(ii) a Otherwise, it will

Is assigned to

And steps S22 to S26 are repeated.

For step S24, in some application scenarios, the following steps may be substituted:

step S31: calculating the center of mass

；

Step S32: for all

Calculating

；

Step S33: for all

Calculating

。

The above-described alternative steps may monotonically decrease the harmonic energy in each iteration by selecting the appropriate step size.

It can be seen that through the above-described adjustment of the centroid and optimization of the blend energy, the method 200 of the present invention ultimately blends and maps the brain image onto a complete sphere, thereby completing the rendering of the brain image, such as the sphere shown in fig. 8 b.

Fig. 9 is a block diagram illustrating an apparatus 900 for presenting a brain image according to an embodiment of the present invention. As shown in fig. 9, the apparatus 900 for presenting brain images of the present invention may include a CPU 9011, which may be a general-purpose CPU, a dedicated CPU, or an execution unit of other information processing and program execution. Further, the device 900 may further include a mass storage 9012 and a read only memory ("ROM") 9013, wherein the mass storage 9012 may be configured to store various types of data including various brain image data, algorithm data, intermediate result results, and various programs required to run the device 900, and the ROM 9013 may be configured to store a power-on self-test for the device 900, initialization of various functional modules in the system, a driver for basic input/output of the system, and data required to boot the operating system.

Optionally, the apparatus 900 may also include other hardware platforms or components, such as the illustrated TPU (tensor processing unit) 9014, GPU (graphics processing unit) 9015, FPGA (field programmable gate array) 9016, and MLU (machine learning unit) 9017. It is to be understood that although various hardware platforms or components are shown in the apparatus 900, this is by way of illustration and not of limitation, and one skilled in the art can add or remove corresponding hardware as may be desired. For example, the apparatus 900 may comprise only a CPU to implement the method for brain image display of the present invention.

The apparatus 900 of the present invention also includes a communication interface 9018 such that it may connect to a local area network/wireless local area network (LAN/WLAN) 905 via communication interface 9018, which in turn may connect to a local server 906 via LAN/WLAN or to the Internet ("Internet") 907. Alternatively or additionally, the inventive apparatus 900 may also be directly connected to the internet or a cellular network based on wireless communication technology, such as third generation ("3G"), fourth generation ("4G"), or 5 generation ("5G") based wireless communication technology, through communication interface 9018. In some application scenarios, the apparatus 900 of the present invention may also access a server 908 of an external network and possibly a database 909 as needed in order to obtain various known image models, data and modules, and may remotely store various data, such as various types of data used for rendering brain images.

The peripheral devices of the apparatus 900 may include a display device 902, an input device 903, and a data transmission interface 904. In one embodiment, the display device 902 may for example comprise one or more speakers and/or one or more visual displays configured for voice prompting and/or visual display of the computational process of the present invention displaying brain images or the final results. Input device 903 may include, for example, a keyboard, mouse, microphone, gesture capture camera, or other input buttons or controls configured to receive input of brain image data and/or user instructions. The data transfer interface 904 may include, for example, a serial interface, a parallel interface, or a universal serial bus interface ("USB"), a small computer system interface ("SCSI"), serial ATA, FireWire ("FireWire"), PCI Express, and a high-definition multimedia interface ("HDMI"), which are configured for data transfer and interaction with other devices or systems. In accordance with aspects of the present invention, the data transfer interface 904 may receive brain images or brain image data from the MRI device and transmit the brain image data or various other types of data and results to the apparatus 900.

The CPU 9011, the mass memory 9012, the read only memory ROM 9013, the TPU9014, the GPU 9015, the FPGA 9016, the MLU 9017, and the communication interface 9018 of the device 900 of the present invention may be connected to each other through a bus 9019, and implement data interaction with peripheral devices through the bus. Through the bus 9019, the CPU 9011 may control other hardware components and their peripherals in the apparatus 900, in one embodiment.

An apparatus for presenting brain images that may be used to carry out the present invention is described above in connection with fig. 9. It is to be understood that the device configurations herein are merely exemplary, and that the implementations and entities of the invention are not limited thereto, but may be varied without departing from the spirit of the invention.

It should also be appreciated that any module, unit, component, server, computer, terminal, or device executing instructions of the examples of the invention may include or otherwise access a computer-readable medium, such as a storage medium, computer storage medium, or data storage device (removable) and/or non-removable) such as a magnetic disk, optical disk, or magnetic tape. Computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, program modules or other data.

Based on the foregoing, the present invention also discloses a computer readable storage medium having stored therein program instructions adapted to be loaded and executed by a processor: cutting the brain image along a closed curve formed by a plurality of vertexes of a mesh constituting the brain image to obtain two topological discs; respectively harmoniously mapping the two topological disks onto a half spherical surface; and splicing the two semispherical surfaces along the closed curve to obtain a complete spherical surface so as to map the brain image harmony onto the complete spherical surface. Preferably or additionally, the computer readable storage medium may further include program instructions or the like to optimize its harmonic energy for the initial mapping to obtain a corrected harmonic mapping, and to optimize the harmonic energy using, for example, the gradient descent algorithm described previously. In summary, the computer readable storage medium includes program instructions for performing the processing operations described in connection with fig. 1-8 b.

The foregoing may be better understood in light of the following clauses:

clause 1, a method for presenting a brain image, comprising:

Clause 2, the method of clause 1, further comprising obtaining the closed curve from a starting point back to the starting point after sequentially passing through N intermediate vertices by performing the following operations:

calculating a function value of each vertex according to the mesh information;

wherein N is a positive integer greater than or equal to 2.

Clause 3, the method of clause 2, wherein the function value is an eigen function corresponding to a maximum eigenvalue of a laplacian-berlamide matrix based on the grid, the method comprising:

calculating a Laplacian-Bell-Lambda matrix based on the mesh according to the topology of the mesh and the side lengths of the mesh edges.

Clause 4, the method of clause 1, wherein harmonically mapping the two topological discs onto the half-spheres, respectively, comprises:

respectively harmoniously mapping the two topological disks to a unit disk; and

and respectively projecting the obtained two unit discs to the half spherical surface.

Clause 5, the method of any one of clauses 1-4, wherein a complete sphere resulting from stitching the two half spheres is used as an initial mapping for the brain image, the method further comprising:

the harmonic energy of the initial mapping is optimized for the initial mapping to obtain a corrected harmonic mapping.

Clause 6, the method of clause 5, wherein optimizing its reconciliation energy for the initial mapping comprises:

optimization of the harmonic energy is performed using a gradient descent algorithm to obtain a harmonic map with minimized harmonic energy.

Clause 7, the method of clause 6, wherein for a sphere having a corrected harmonic mapping, the method further comprises:

calculating a Mobius transform using the centroid of the spherical surface to correct the centroid; and

optimizing the blend energy until a predetermined energy difference threshold is met, thereby blending mapping the brain image onto the complete sphere.

Clause 8, an apparatus for presenting a brain image, comprising:

at least one processor; and

at least one memory for storing program instructions that, when loaded and executed by the at least one processor, cause the apparatus to perform the method of any of clauses 1-7.

Clause 9, a computer-readable storage medium in which program instructions for presenting a brain image are stored, said program instructions being adapted to be loaded by a processor and to carry out the method according to any one of clauses 1-7.

Although the embodiments of the present invention are described above, the descriptions are only examples for facilitating understanding of the present invention, and are not intended to limit the scope and application scenarios of the present invention. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. A method for presenting brain images, comprising:

stitching the two half spheres along the closed curve to obtain a complete sphere so as to harmonically map the brain image onto the complete sphere,

wherein the closed curve starting from a starting point, passing through N intermediate vertices in order, and returning to the starting point is obtained by performing the following operations:

calculating a function value of each vertex according to the mesh information;

selecting a vertex having a locally smallest function value from a plurality of vertices adjacent to the starting point as a 1 st intermediate vertex; and

selecting, as an ith intermediate vertex, a vertex having a locally smallest function value from among a plurality of vertices adjacent to an i-1 th intermediate vertex, for an ith intermediate vertex of 2 nd to nth intermediate vertices, where N and i are positive integers greater than or equal to 2, until returning to the start point,

wherein the function value is an eigen function corresponding to a maximum eigenvalue of a laplacian-berlamide matrix based on the grid, the method comprising:

2. The method of claim 1, wherein separately harmonic mapping the two topological discs onto a half sphere comprises:

respectively harmoniously mapping the two topological disks to a unit disk; and

3. The method according to claim 1 or 2, wherein a complete sphere resulting from stitching the two hemispheres is used as an initial mapping of the brain image, the method further comprising:

4. The method of claim 3, wherein optimizing its harmonic energy for the initial mapping comprises:

5. The method of claim 4, wherein for a sphere having a corrected harmonic mapping, the method further comprises:

6. An apparatus for presenting brain images, comprising:

at least one processor; and

at least one memory for storing program instructions that, when loaded and executed by the at least one processor, cause the apparatus to perform the method of any of claims 1-5.

7. A computer-readable storage medium, in which program instructions for presenting an image of a brain are stored, the program instructions being adapted to be loaded by a processor and to carry out the method according to any one of claims 1-5.