CN115082586B - Group prior guided thalamus individualized atlas drawing method based on deep learning - Google Patents

Group prior guided thalamus individualized atlas drawing method based on deep learning Download PDF

Info

Publication number
CN115082586B
CN115082586B CN202210822674.4A CN202210822674A CN115082586B CN 115082586 B CN115082586 B CN 115082586B CN 202210822674 A CN202210822674 A CN 202210822674A CN 115082586 B CN115082586 B CN 115082586B
Authority
CN
China
Prior art keywords
thalamus
individual
roi
brain
subject
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210822674.4A
Other languages
Chinese (zh)
Other versions
CN115082586A (en
Inventor
樊令仲
高超宏
吴霞
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Institute of Automation of Chinese Academy of Science
Original Assignee
Institute of Automation of Chinese Academy of Science
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Institute of Automation of Chinese Academy of Science filed Critical Institute of Automation of Chinese Academy of Science
Priority to CN202210822674.4A priority Critical patent/CN115082586B/en
Priority to PCT/CN2022/106161 priority patent/WO2024011647A1/en
Publication of CN115082586A publication Critical patent/CN115082586A/en
Application granted granted Critical
Publication of CN115082586B publication Critical patent/CN115082586B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/002D [Two Dimensional] image generation
    • G06T11/003Reconstruction from projections, e.g. tomography
    • G06T11/005Specific pre-processing for tomographic reconstruction, e.g. calibration, source positioning, rebinning, scatter correction, retrospective gating
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T11/002D [Two Dimensional] image generation
    • G06T11/003Reconstruction from projections, e.g. tomography
    • G06T11/008Specific post-processing after tomographic reconstruction, e.g. voxelisation, metal artifact correction
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/30Determination of transform parameters for the alignment of images, i.e. image registration
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/762Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/70Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10072Tomographic images
    • G06T2207/10088Magnetic resonance imaging [MRI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20092Interactive image processing based on input by user
    • G06T2207/20104Interactive definition of region of interest [ROI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30004Biomedical image processing
    • G06T2207/30016Brain

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Magnetic Resonance Imaging Apparatus (AREA)

Abstract

The invention belongs to the field of brain atlas drawing of magnetic resonance imaging, and particularly relates to a group priori guided thalamus individualized atlas drawing method, system and device based on deep learning, aiming at solving the problem that the accuracy, robustness and repeatability of an individualized thalamus atlas drawn by the prior art are poor. The method comprises the following steps: obtaining a thalamic ROI of a subject individual brain; obtaining a group prior map of the brain of an individual subject; removing cohort prior maps in the thalamic ROI; acquiring individual characteristics, inputting an individualized classification model to obtain a prediction probability value vector of a voxel in each undefined region, and further generating a map of the undefined region; and combining the group prior map and the map of the undefined region to generate a thalamic map individualized by the subject. According to the invention, the individual thalamic subareas and map drawing are guided by using the high-reliability group in a priori manner, so that the accuracy, robustness and repeatability of the drawn individual thalamic map are improved.

Description

Group prior guided thalamus individualized atlas drawing method based on deep learning
Technical Field
The invention belongs to the field of brain atlas drawing of magnetic resonance imaging, and particularly relates to a group priori guided thalamus individualized atlas drawing method, system and device based on deep learning.
Background
The thalamus is a relay nucleus in the brain and participates in brain functional circuits such as hearing, vision, movement, somatosensory, emotion, memory and learning. Clinically, the thalamus is used as a target of Deep Brain Stimulation (DBS) and is involved in the regulation and treatment of nervous system diseases such as Parkinson's disease, epilepsy, multiple sclerosis, plant-induced awakening, schizophrenia and essential tremor. Therefore, the mapping of a fine and accurate thalamic map spectrum is critical to thalamic studies. In the conventional research, researchers have characterized the structural or functional characteristics of the thalamus by histological section staining and magnetic resonance imaging, and have used these as the basis for the thalamus division. In these methods of thalamic zoning, histological section staining is considered as the gold standard for thalamic zones, which can only be performed in ex vivo brain specimens, without reproducibility and relying on manual labeling by the dissector. Rapidly evolving magnetic resonance imaging can non-invasively characterize features within the thalamus, including local microstructures, anatomical and functional connections, and the like. Based on this, the data-driven thalamic partition process is becoming a research hotspot of thalamic partition. According to the modality of magnetic resonance imaging, the existing thalamic partition can be divided into different methods based on three modality data of structural, diffusion and functional images. In the thalamic segmentation method based on these three magnetic resonance modality data, a diffusion magnetic resonance-based thalamic segmentation is constructed closest to the anatomy of the thalamus. The diffusion magnetic resonance image can provide two kinds of diffusion information, namely fiber bundle connection and local diffusion characteristics. In early studies, researchers found that fiber bundle-based thalamic regions also had poor correspondence to the actual thalamic anatomy, while thalamic regions based on local diffusion characteristics were substantially consistent with the anatomy. Therefore, thalamic zoning based on local diffusion characteristics is the most direct method to delineate the local microstructure of the thalamus.
The thalamic region research mostly uses a group of subjects, and based on thalamic region results in individual spaces, different thalamic regions to be tested are mapped to the same space through a manual marking method or an automatic registration method, so that a thalamic map spectrum at a group level is constructed. The method can objectively and unbiased reflect the intrinsic zoning mode of thalamus, such as the number of subregions, and the consistent attributes of the group level, such as the structural connection mode and the functional connection mode of thalamic subtenon and other brain regions. However, as the research progresses, researchers have found that there are significant differences in brain partition patterns between individuals, whether in the cerebral cortex or subcutaneous nuclei, and that individual-specific brain partitions reflect individual characteristics, such as cognitive, developmental, aging, and disease characteristics, more than group-level brain partitions. In addition, in clinical practice, especially in the field of precise medicine, the brain map specific to the subject plays an important role, such as preoperative diagnosis, efficacy prediction, target location, and the like. Therefore, the importance of individualized brain mapping is also becoming a growing concern to researchers. Several methods of individualized atlas mapping have also been developed, roughly divided into three, direct partitioning for single subjects, individual registration for cohort atlases, and group a priori guided individual partitioning. The first method relies on high-quality magnetic resonance imaging data and a robust partition algorithm, and roughly comprises a fiber projection method, a spectral clustering method, an edge detection method, a region growing method and the like; the second method directly registers the group atlas to an individual space, so that the group atlas is regarded as an individual atlas; the third method is that firstly, a group map is constructed, and the group map is regarded as the prior knowledge of the individual partition, so that the subsequent individual partition is assisted. The first segmentation method is applicable to nuclei with specific fiber bundle projections, such as the subthalamic nucleus, the medial globus pallidus, etc. The second method is suitable for subjects with severe brain damage or failure to perform diffusion magnetic resonance imaging, such as brain tumor patients and patients with metal implants in their heads. The third method is suitable for cerebral cortex and nucleus with large volume and rich individual characteristics. Therefore, in order to construct an individualized thalamic atlas accurately, a cohort a priori guided individualized partition strategy may be employed.
With the technical development of magnetic resonance Imaging, high Angular Resolution Diffusion weighted Imaging (High Angular Resolution Diffusion Imaging, HARDI) of multiple b values greatly improves the efficiency and accuracy of modeling and estimation of local Diffusion directions. Multi-b-value high angular resolution diffusion magnetic resonance imaging is becoming a trend, both in neuroscience research and in clinical scans. Meanwhile, in the methodological research of local diffusion direction estimation, the method is different from a classical diffusion tensor model, and the existing multiple q space sampling methods support the construction of a higher-order diffusion model. Such as diffusion kurtosis imaging, diffusion spectroscopy imaging, Q-ball imaging and multi-spherical shell imaging derived therefrom, and the like. Among the methods, the multi-spherical-shell multi-tissue restrictive spherical deconvolution (MSMT-CSD) can directly estimate the diffusion direction distribution function (ODF) of the brain tissue, and the method relies on multi-b-value HARDI, which is the most suitable method for extracting the local diffusion characteristic of the thalamus at present. In addition, deep learning has evolved dramatically in the last decade, with powerful data fitting and classification capabilities making it elegant and varied in various fields including neuroscience. Under the background, an automatic individual thalamus atlas drawing method is gradually possible to be established based on a deep learning model by combining local diffusion characteristics and an individual partitioning strategy guided by group prior.
Disclosure of Invention
In order to solve the above problems in the prior art, that is, to solve the problems that the existing thalamus individualized atlas drawing method cannot construct a high-reliability group prior, cannot guide the drawing of an individualized thalamus atlas by using the group prior, and causes the accuracy, robustness and repeatability of the drawn individualized thalamus atlas to be poor, the first aspect of the invention provides a thalamus individualized atlas drawing method based on deep learning guided by the group prior, which comprises the following steps:
s100, acquiring a cluster thalamus probability map under a first threshold value, and carrying out binarization to obtain a mask of the cluster thalamus probability map, wherein the mask is used as a first mask; registering the first mask to a diffusion magnetic resonance space of the brain of the subject individual to obtain a thalamic ROI of the brain of the subject individual;
s200, acquiring a cluster thalamus probability map under a second threshold value, and registering the cluster thalamus probability map to a diffusion magnetic resonance space of the individual brain of the subject to obtain a cluster prior map of the individual brain of the subject;
s300, in a diffusion magnetic resonance space of the individual brain of the subject, removing the group prior map in the thalamus ROI to obtain a mask of an undefined thalamus region as a second mask;
s400, calculating the coefficient of a 45-dimensional spherical harmonic function and the position coordinate of a 3-dimensional diffusion magnetic resonance space of each voxel in the group prior map by combining the second mask, and combining the coefficient and the position coordinate into a 48-dimensional characteristic vector serving as an individual characteristic; inputting the individual characteristics into a pre-constructed individual classification model to obtain a prediction probability value vector of a voxel in each undefined region, and taking a maximum subregion label corresponding to the prediction probability value vector as a final subregion label of the voxel to further generate a map of the undefined region; the individualized classification model is constructed based on a deep learning neural network;
s500, combining the group prior map and the map of the undefined region to generate a thalamic map of individual subjects;
the construction method of the cluster thalamus probability map comprises the following steps:
a100, acquiring structural nuclear magnetic resonance images and diffusion tensor nuclear magnetic resonance images of brains of N subjects as input images; n is a positive integer;
a200, sequentially carrying out HCP minimum preprocessing, ROI registration and ROI post-processing on the input image to obtain a final thalamus ROI of the individual brain of the subject, and obtaining the local diffusion characteristic of each voxel in the final thalamus ROI of the individual brain of the subject through an ODF estimation algorithm;
a300, calculating the similarity between voxels by combining the local diffusion characteristics obtained in the A200, and clustering to obtain the final clustering result of the voxels in the thalamus ROI of the individual brain of the subject;
a400, registering the clustering result obtained in the step A300 to a standard space, and performing label remapping to obtain a sub-region label corresponding to each voxel in a final thalamus ROI of the individual brains of the subjects, namely obtaining thalamus regions of the individual brains of the N subjects in the standard space;
a500, calculating a corresponding subregion probability value of each voxel in a final thalamus ROI of the individual brain of the subject, removing voxel points of which the maximum subregion probability value is lower than a first threshold value, and then constructing a thalamus probability map of a group level, namely a cohort thalamus probability map, by using the remaining voxel points; and the probability value of the subregions is the ratio of the number of the testees in each subregion to the total number of the testees.
In some preferred embodiments, the input images are sequentially subjected to HCP minimum pre-processing, ROI registration, and ROI post-processing to obtain a final thalamic ROI of the subject individual brain by:
carrying out HCP minimum preprocessing on a structural nuclear magnetic resonance image and a diffusion tensor nuclear magnetic resonance image of the brain of an individual subject to obtain a preprocessed structural nuclear magnetic resonance image and a preprocessed diffusion tensor nuclear magnetic resonance image;
registering a diffusion magnetic resonance space with a structure magnetic resonance space, the structure magnetic resonance space and a standard space by an ROI (region of interest) registration method based on the preprocessed structure nuclear magnetic resonance image and the preprocessed diffusion tensor nuclear magnetic resonance image to obtain an individual thalamus ROI;
calculating fractional anisotropy values corresponding to each voxel point using FSL in a diffusion magnetic resonance space of the brain of the individual subject;
calculating a cerebrospinal fluid probability value corresponding to each voxel point in a structural magnetic resonance space of the brain of the individual subject by using SPM;
removing voxel points with the anisotropy score value larger than a set anisotropy score threshold value or the cerebrospinal fluid probability value larger than a set cerebrospinal fluid probability threshold value from the individual thalamic ROI, and taking the remaining voxel points as the final individual thalamic ROI.
In some preferred embodiments, the local diffusion characteristic of each voxel in the final thalamic ROI of the subject's brain is obtained by:
in a diffusion magnetic resonance space in a final thalamus ROI of the brain of the individual subject, calculating response functions of brain tissues under different diffusion weighting factor b value parameters on diffusion magnetic resonance data by using a dhollander algorithm; and calculating 45-dimensional coefficients of 8-order spherical harmonic functions by using a multi-tissue multi-spherical-shell restrictive spherical deconvolution method in combination with the response function of the brain tissue, and quantifying the local diffusion characteristic of each voxel.
In some preferred embodiments, the similarity between voxels is calculated by:
Figure BDA0003742857540000051
where S (i, j) denotes the similarity between two voxels, E pos (i, j) denotes the Euclidean distance between 3-dimensional coordinates of two voxels in diffusion magnetic resonance space, E odf (i, j) denotes two voxelsOf the 45-dimensional spherical harmonic function of (a) is calculated by the Euclidean distance, w, between the sampling coefficients of the 45-dimensional spherical harmonic function pos And w odf Respectively represent E pos (i,j)、E odf (i, j) corresponding weighting coefficients in calculating the similarity.
In some preferred embodiments, the final thalamic ROI of the subject's brain is obtained by:
reducing the local diffusion characteristics of each voxel in a final thalamus ROI of the individual brains of the subjects and the similarity among the voxels by a spectral clustering method;
based on the local diffusion characteristics after dimensionality reduction and the similarity between voxels, clustering each voxel into K classes by using K-means clustering, and taking the K classes as the clustering result of the voxels in the final thalamus ROI of the individual brain of the subject.
In some preferred embodiments, the subject individual brain's final thalamic ROI is obtained by labeling each voxel in the subject individual brain's final thalamic ROI with a sub-region label obtained by:
calculating N-dimensional label vectors of each voxel point on N subjects, then clustering all voxel points in the final thalamus ROI of the individual brains of the subjects according to the similarity of the label vectors, and taking the clustering result as a partition label;
based on the subarea labels, the clustering result of each voxel in the final thalamic ROI of the individual brain of the subject is re-marked according to a space maximum overlapping method, so that a subarea label corresponding to each voxel in the final thalamic ROI of the individual brain of the subject is obtained;
wherein, calculating N dimension label vector of each subject on N subjects, namely extracting corresponding sub-region label of each subject on the subject, and respectively extracting a sub-region label for N subjects to form the N dimension label vector.
In some preferred embodiments, the individualized classification model is trained by:
obtaining individual characteristics through the methods of S100-S400, and inputting the individual characteristics into a pre-constructed individualized classification model to obtain a prediction probability value vector of a voxel in each undefined region;
based on the prediction probability value vector, combining with the sub-region categories corresponding to the group prior map, obtaining a loss value through a mean square error loss function, and updating model parameters of the individualized classification model;
and circulating the steps until the trained individualized classification model is obtained.
In a second aspect of the present invention, a group prior guided hypothalamic individualized atlas drawing system based on deep learning is provided, which includes: the thalamus ROI acquisition module, the group prior map acquisition module, the elimination processing module, the individual classification module and the individual map generation module;
the thalamus ROI acquisition module is configured to acquire a cluster thalamus probability map under a first threshold value and carry out binarization to obtain a mask of the cluster thalamus probability map, and the mask is used as a first mask; registering the first mask to a diffusion magnetic resonance space of the brain of the individual subject to obtain a thalamic ROI of the brain of the individual subject;
the group prior map acquisition module is configured to acquire a group thalamus probability map under a second threshold value, and register the group thalamus probability map to a diffusion magnetic resonance space of the brain of the individual subject to obtain a group prior map of the brain of the individual subject;
the elimination processing module is configured to eliminate the group prior map in the thalamus ROI in the diffusion magnetic resonance space of the brain of the individual subject to obtain a mask of an undefined thalamus region as a second mask;
the individualized classification module is configured to calculate a coefficient of a 45-dimensional spherical harmonic function and a position coordinate of a 3-dimensional diffusion magnetic resonance space of each voxel in the group prior map by combining the second mask, and combine the coefficient and the position coordinate into a 48-dimensional feature vector as an individual feature; inputting the individual characteristics into a pre-constructed individual classification model to obtain a prediction probability value vector of a voxel in each undefined region, and taking a maximum subregion label corresponding to the prediction probability value vector as a final subregion label of the voxel to further generate a map of the undefined region; the individualized classification model is constructed on the basis of a deep learning neural network;
the individualized map generation module is configured to combine the group prior map and the map of the undefined region to generate a thalamic map which is individualized by the subject;
the construction method of the cluster thalamus probability map comprises the following steps:
a100, acquiring structural nuclear magnetic resonance images and diffusion tensor nuclear magnetic resonance images of brains of N subjects as input images; n is a positive integer;
a200, sequentially carrying out HCP minimum preprocessing, ROI registration and ROI post-processing on the input image to obtain a final thalamus ROI of the individual brain of the subject, and obtaining the local diffusion characteristic of each voxel in the final thalamus ROI of the individual brain of the subject through an ODF estimation algorithm;
a300, calculating the similarity between voxels by combining the local diffusion characteristics obtained in the A200, and clustering to obtain the final clustering result of the voxels in the thalamus ROI of the individual brain of the subject;
a400, registering the clustering result obtained in the step A300 to a standard space, and performing label remapping to obtain a sub-region label corresponding to each voxel in a final thalamus ROI of the individual brains of the subjects, namely obtaining thalamus regions of the individual brains of the N subjects in the standard space;
a500, calculating a corresponding subregion probability value of each voxel in a final thalamus ROI of the individual brain of the subject, removing voxel points with the maximum subregion probability value lower than a first threshold value, and then constructing a thalamus probability map of a cohort level, namely a cohort thalamus probability map, from the remaining voxel points; and the probability value of the subregions is the ratio of the number of the testees in each subregion to the total number of the testees.
In a third aspect of the present invention, an electronic device is provided, including: at least one processor; and a memory communicatively coupled to at least one of the processors; wherein the memory stores instructions executable by the processor for execution by the processor to implement the cohort a priori guided, deep learning based individualized atlas mapping method described above.
In a fourth aspect of the present invention, a computer-readable storage medium is provided, which stores computer instructions for execution by the computer to implement the group a priori guided hypothalamic personalized atlas mapping method based on deep learning described above.
The invention has the beneficial effects that:
according to the invention, the individual thalamic subareas and map drawing are guided by using the high-reliability group in a priori manner, so that the accuracy, robustness and repeatability of the drawn individual thalamic map are improved.
1) The individual thalamic atlas is drawn based on the high-reliability group horizontal thalamic atlas and single tested data, so that the group consistency of the thalamic zoning mode is fused, and the specificity of the individual zoning mode is reflected. When the partition accuracy of the individual thalamic atlas is quantified by using the intra-test partition mode consistency index, the result shows that the method has higher partition accuracy on HCP-3T and HCP-7T data than single tested clustering and cluster atlas registration. When the individualized method is tested by using the Test-retest data for repeatability Test, the result shows that the method has higher consistency of the partition mode among scans on the Test-retest data. When repeated tests were performed using 7 and 12 as numbers of subregions of the thalamus, the results indicated that the method was robust in terms of different numbers of subregions and more accurate in terms of more subtle 12 numbers of subregions.
2) According to the invention, the thalamus is subjected to group level partition based on the high-order diffusion characteristic and the spatial position, and a more refined thalamus partition mode is found; and the higher consistency of the regions to be tested is shown on the finer thalamic map, so that the drawing of the individual thalamic map with finer granularity can be supported; the method has robustness under different magnetic field scanning strengths, and higher magnetic field scanning strength can provide higher consistency gain of the tested inner partitions; repeatability is obtained under different scanning batches of the same test subject.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings.
FIG. 1 is a schematic flow chart of a group prior guided deep learning based individual thalamus mapping method according to an embodiment of the present invention;
FIG. 2 is a block diagram of a cohort prior guided deep learning based individual atlas mapping system of the thalamus according to an embodiment of the present invention;
FIG. 3 is a schematic flow chart of a procedure for obtaining a final thalamic ROI of a brain of an individual subject according to an embodiment of the present invention;
FIG. 4 is a flow chart illustrating ODF estimation according to one embodiment of the present invention;
FIG. 5 is a schematic flow chart of the construction of a probability atlas of the cohort thalamus according to an embodiment of the present invention;
FIG. 6 is a schematic diagram of a process for constructing a personalized thalamic map, in accordance with an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without making any creative effort based on the embodiments in the present invention, belong to the protection scope of the present invention.
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.
It should be noted that, in the present application, the embodiments and features of the embodiments may be combined with each other without conflict.
The invention discloses a group prior guided thalamus individuation atlas drawing method based on deep learning, which comprises the following steps as shown in figure 1:
s100, acquiring a cluster thalamus probability map under a first threshold value, and carrying out binarization to obtain a mask of the cluster thalamus probability map, wherein the mask is used as a first mask; registering the first mask to a diffusion magnetic resonance space of the brain of the subject individual to obtain a thalamic ROI of the brain of the subject individual;
s200, acquiring a cluster thalamus probability map under a second threshold value, and registering the cluster thalamus probability map to a diffusion magnetic resonance space of the individual brain of the subject to obtain a cluster prior map of the individual brain of the subject;
s300, in a diffusion magnetic resonance space of individual brains of a subject, eliminating a group prior map in the thalamus ROI to obtain a mask of an undefined thalamus region as a second mask;
s400, calculating the coefficient of a 45-dimensional spherical harmonic function and the position coordinate of a 3-dimensional diffusion magnetic resonance space of each voxel in the group prior map by combining the second mask, and combining the coefficient and the position coordinate into a 48-dimensional characteristic vector serving as an individual characteristic; inputting the individual characteristics into a pre-constructed individual classification model to obtain a prediction probability value vector of a voxel in each undefined region, and taking a maximum subregion label corresponding to the prediction probability value vector as a final subregion label of the voxel to further generate a map of the undefined region; the individualized classification model is constructed on the basis of a deep learning neural network;
s500, combining the group prior map and the undefined region map to generate a thalamic map of individual subjects;
the construction method of the cohort thalamus probability map comprises the following steps:
a100, acquiring structural nuclear magnetic resonance images and diffusion tensor nuclear magnetic resonance images of brains of N subjects as input images; n is a positive integer;
a200, sequentially carrying out HCP minimum preprocessing, ROI registration and ROI post-processing on the input image to obtain a final thalamic ROI of the individual brain of the subject, and obtaining the local diffusion characteristic of each voxel in the final thalamic ROI of the individual brain of the subject through an ODF estimation algorithm;
a300, calculating the similarity between voxels by combining the local diffusion characteristics obtained in the A200, and clustering to obtain the final clustering result of the voxels in the thalamus ROI of the individual brain of the subject;
a400, registering the clustering result obtained in the step A300 to a standard space, and performing label remapping to obtain a sub-region label corresponding to each voxel in a final thalamus ROI of the individual brains of the subjects, namely obtaining thalamus regions of the individual brains of the N subjects in the standard space;
a500, calculating a corresponding subregion probability value of each voxel in a final thalamus ROI of the individual brain of the subject, removing voxel points with the maximum subregion probability value lower than a first threshold value, and then constructing a thalamus probability map of a cohort level, namely a cohort thalamus probability map, from the remaining voxel points; and the probability value of the subregions is the ratio of the number of the testees in each subregion to the total number of the testees.
In order to more clearly describe the group prior guided individual atlas plotting method of thalamus based on deep learning, the following describes the steps of an embodiment of the method in detail with reference to the attached drawings.
The drawing method of the individual thalamus map guided by group prior based on deep learning is divided into two main steps: constructing a thalamus group probability map and constructing a thalamus individuation map. In the thalamic cohort probability map construction stage, firstly, based on a group of high-quality data, a direction distribution function of each voxel in a thalamic Region of Interest (ROI) and an individual diffusion magnetic resonance space coordinate are extracted in diffusion magnetic resonance imaging, the direction distribution function and the individual diffusion magnetic resonance space coordinate are used as features to construct a similarity matrix between the voxels, an initial partition of an individual is preliminarily drawn by using a spectral clustering algorithm, and the initial partition is registered to an MNI standard space to establish a cohort level thalamic probability map and a maximum probability map. In the individual atlas drawing stage, atlas areas with high group level probability values of thalamus are regarded as group priors, the atlas areas are registered to an individual diffusion magnetic resonance space to serve as high-reliability group priors of individual atlases, and meanwhile, a template of a maximum probability atlas is registered to the individual diffusion magnetic resonance space to serve as an ROI of the individual thalamus atlas. Next, 45-dimensional coefficients of 8-order Spherical harmonic functions in the ROIs are extracted as ODF features on the individual diffusion magnetic resonance data through Multi-Shell Multi-Tissue Constrained Spherical Deconvolution (MSMT-CSD), and 3-dimensional individual diffusion magnetic resonance spatial coordinates are calculated as spatial position features, thereby constructing 48-dimensional individual features for voxels in each ROI. And (3) constructing an individualized classification model based on a deep learning method by taking the subregion class of the voxel in the ROI as a label, taking the group prior map as training data and taking an unmarked region in the ROI as test data. Finally, the high confidence panel prior atlas and the predicted unlabeled atlas are combined to generate an individualized thalamic atlas.
In the following embodiments, the construction process of the cohort thalamus probability map is detailed, and then the process of generating the individual map by using the thalamus individual map drawing method based on a cohort prior guidance and based on deep learning is detailed.
1. Construction process of probability map of cluster thalamus
A100, acquiring structural nuclear magnetic resonance images and diffusion tensor nuclear magnetic resonance images of brains of N subjects as input images; n is a positive integer;
in this embodiment, a set (i.e., N) of structural and diffusion tensor magnetic resonance images of the brains of individual subjects are acquired.
A200, sequentially carrying out HCP minimum preprocessing, ROI registration and ROI post-processing on the input image to obtain a final thalamus ROI of the individual brain of the subject, and obtaining the local diffusion characteristic of each voxel in the final thalamus ROI of the individual brain of the subject through an ODF estimation algorithm;
in this embodiment, the minimum preprocessing is performed on the structural nuclear magnetic resonance image (T1) and the Diffusion Weighted Imaging (DWI) data using the minimum preprocessing of the HCP data (i.e., the HCP minimum preprocessing), respectively, to obtain a preprocessed structural nuclear magnetic resonance image and a preprocessed Diffusion tensor nuclear magnetic resonance image.
Based on the preprocessed structural nuclear magnetic resonance image and the preprocessed diffusion tensor nuclear magnetic resonance image, in FMRIB's Software Library (FSL), a diffusion magnetic resonance space (B0, which is called as a diffusion space for short, and is shown in figure 3) of the individual brain of the subject and a structural magnetic resonance space (T1, which is called as a structural space for short, and is shown in figure 3) are linearly registered, the structural magnetic resonance space (T1) and a standard space (MNI) are nonlinearly registered, then the registration matrixes of the diffusion magnetic resonance space (B0) of the individual brain of the subject to the standard space (MNI) are combined to generate a registration matrix of the diffusion magnetic resonance space (B0) of the individual brain of the subject to the individual diffusion space, and transposition is carried out to obtain the registration matrix of the standard space to the individual diffusion space. Based on the registration matrix, a classical Morel thalamus atlas is registered to the individual diffusion magnetic resonance space as the individual thalamus ROI. As shown in fig. 3 (a).
Then, ROI post-processing is performed as shown in (b) of fig. 3: the method specifically comprises the following steps: calculating an FA map (i.e. fractional anisotropy values for each voxel point) using FSL in the diffusion magnetic resonance space of the subject's individual brain; calculating a CSF probability map (i.e., cerebrospinal fluid probability values corresponding to each voxel point) using SPM in the structural magnetic resonance space of the brain of the subject individual; voxel points with a Fractional Anisotropy (FA) value greater than a set FA threshold (preferably set to 0.6 herein) or a Cerebrospinal Fluid (CSF) probability value greater than a set CSF probability threshold (preferably set to 0.05 herein) are removed from the individual's thalamic ROI, with the remaining voxel points being the final individual thalamic ROI (i.e., the ROI in individual diffusion space in fig. 3).
Finally, ODF estimation was performed as shown in fig. 4: the method comprises the following specific steps: in MRtrix3, a dhollander algorithm is used to calculate response functions of brain tissues under different b-value parameters of diffusion weighting factors on diffusion magnetic resonance data, and then, in combination with the response functions of the brain tissues, 45-dimensional coefficients (i.e., a spherical harmonic coefficient matrix in fig. 4) of an 8-order spherical harmonic function are calculated by using a multi-tissue multi-spherical-shell restrictive spherical deconvolution method (i.e., a multi-tissue multi-spherical-shell restrictive deconvolution briefly shown in fig. 4) to represent local diffusion characteristics of each voxel in the thalamus (i.e., the 8-order spherical harmonic function is taken as a sampling base, and the 45-dimensional spherical harmonic function coefficient of each voxel in the thalamus ROI is calculated to construct an ODF model thereof, so that a spherical harmonic function coefficient matrix in the thalamus ROI is obtained).
A300, calculating the similarity between voxels by combining the local diffusion characteristics obtained in the A200 and clustering to obtain the final clustering result of the voxels in the thalamus ROI of the individual brain of the subject;
in this embodiment, the similarity between voxels is calculated first by combining the diffusion feature and the spatial location feature of the voxels, and a similarity matrix is constructed. The specific calculation method of the similarity between voxels is shown in formula (1):
Figure BDA0003742857540000151
where S (i, j) denotes the similarity between two voxels, E pos (i, j) denotes the Euclidean distance between 3-dimensional coordinates of two voxels in diffusion magnetic resonance space, E odf (i, j) represents the Euclidean distance, w, between the sampling coefficients of the 45-dimensional spherical harmonic functions of two voxels pos And w odf Respectively represent E pos (i,j)、E odf (i, j) corresponding weighting coefficients in calculating the similarity. In the present invention, w pos And w odf Preferably set to 0.5. In addition, in order to balance the distance deviation in the clustering process, the euclidean distance between ODFs is calculated after multiplying the coefficients of the 45-dimensional spherical harmonic function by scaling factors of 89 (3T data) and 98 (7T data). Among them, FIG. 5 shows (Indi 1 \8230; indi100, thalamic ROI corresponding to 100 subjects preferred in the present invention)
Then, clustering is carried out, specifically: reducing the local diffusion characteristics of each voxel in a final thalamus ROI of the individual brains of the subjects and the similarity among the voxels by a spectral clustering method; based on the local diffusion characteristics after dimensionality reduction and the similarity between voxels, clustering each voxel into K classes by using K-means clustering, and taking the K classes as the clustering result of the voxels in the final thalamus ROI of the individual brain of the subject. In the present invention, K is a value of 2 to 28 in order to test the optimal number of thalamic partitions.
A400, registering the clustering result obtained in the step A300 to a standard space, and performing label remapping to obtain a sub-region label corresponding to each voxel in a final thalamus ROI of the individual brains of the subjects, namely obtaining thalamus regions of the individual brains of the N subjects in the standard space;
in this embodiment, label remapping is performed first, specifically:
after the clustering results in the individual space of the subject are registered to the MNI space, the subregion labels between the initial subregions of the individual subjects can not correspond to each other. Label remapping is performed by first calculating an N-dimensional label vector for each pixel point on N (preferably set to N =100 in the present invention) subjects (i.e., subjects) (i.e., extracting the corresponding sub-region label for each subject on the pixel point, and extracting one sub-region label for each of the N subjects, respectively, to form an N-dimensional label vector). Next, all voxel points in the final thalamic ROI of the subject's individual brain are clustered according to the similarity of their label vectors, with the clustering result being a group-level partition label. Based on the partition label, each tested clustering result is re-labeled according to a spatial maximum overlapping method, so that a sub-region label corresponding to each voxel in a final thalamus ROI of the individual brain of the subject is obtained, namely thalamus partitions of the individual brains of the N subjects in a standard space are obtained, and accordingly labels which are consistent among individuals are obtained.
A500, calculating a corresponding subregion probability value of each voxel in a final thalamus ROI of the individual brain of the subject, removing voxel points with the maximum subregion probability value lower than a first threshold value, and then constructing a thalamus probability map of a cohort level, namely a cohort thalamus probability map, from the remaining voxel points; and the probability value of the subregions is the ratio of the number of the testees in each subregion to the total number of the testees.
In this embodiment, based on the thalamic regions in the MNI space of N individual subjects, the probability value of each sub-region of each voxel in the final thalamic ROI of the brain of the individual subject, i.e. the ratio of the number of subjects belonging to a certain sub-region to the total number of subjects, is calculated. After calculating the subregion probability values of all voxels, taking a certain threshold (0.25), removing the voxel point with the maximum subregion probability value lower than the threshold (namely, obtaining the maximum subregion probability corresponding to each voxel point, and removing the voxel point if the maximum subregion probability is lower than a set first threshold), and constructing a thalamus probability map of a cohort level, namely a cohort thalamus probability map, namely the cohort map in fig. 5 based on the remaining voxel points. And (4) binarizing the probability map to obtain the ROI (mask) of the group probability map. And setting the sub-region label of each voxel in the probability map as the label corresponding to the maximum probability value according to the principle of the winner eating all, thereby constructing the maximum probability map.
In addition, after a500, verifying the optimal partition number: based on the maximum probability atlas generated, the partition consistency between individuals and groups and the topological consistency between cerebral hemispheres are calculated to determine the optimal number of thalamic subregions. The partition consistency between the individual and the group is verified by using the overlapping rate Dice values of the individual partitions and the group partitions. The closer the Dice value is to 1, the higher the partition consistency of the individual and the population, and the more consistent the number of thalamic partitions is in accordance with the partition pattern inside the thalamus. And dividing the individual partition test set and the group partition test set based on a leave-one-out method, and calculating the Dice average value on the consistency of 100 times of individual partitions and group partitions. The inter-hemispheric Topological consistency index was verified on cohort maps using the Topological Distance TpD (TpD). The closer the TpD value is to 0, the higher the degree of homology between the left and right hemispheres of the brain, where the number of thalamic divisions corresponds to the division pattern inside the thalamus. The topological distance firstly calculates the number of adjacent voxels between each subregion and other subregions in a single hemisphere, and generates a K x K dimensional matrix. After the matrix is expanded, the similarity of the matrix between two cerebral hemispheres, namely the topological distance inside the partition, is calculated.
When the maximum probability atlas of the group is verified, local maxima appear when the clustering numbers are 7 and 12 in the Dice index, and the TpD value under the two clustering numbers is basically 0, which indicates that the partition numbers are 7 and 12 and accord with the partition mode in the thalamus.
When the thalamus is divided into 7 subregions, the number of the subregions is equal to that of the anterior thalamus, the anterior ventral nucleus, the dorsal medial nucleus, the dorsolateral-posterolateral nucleus, the lateral occipital nucleus, the ventrolateral-ventrolateral nucleus and the medial occipital nucleus. Strong homology was shown between the left and right hemispheres of the thalamus. The thalamus subregions show extremely strong structural homogeneity.
When the thalamus is divided into 12 subregions, the number of the subregions is equal to that of the anterior thalamus, the anterior ventral nucleus, the dorsal medial nucleus, the posterior medial nucleus, the lateral ventral nucleus, the posterior ventral nucleus, the anterior occipital nucleus, the lateral occipital nucleus, the medial occipital nucleus, the anterior dorsal medial nucleus, the posterior dorsal medial nucleus and the dorsal medial nucleus. Very strong homology is shown between the left and right hemispheres of the thalamus. The thalamus exhibits strong structural homogeneity in each subregion. The thalamic spectrum of 12 subregions is more refined than the zonular pattern of 7 subregions.
2. The group prior guided individual atlas drawing method based on deep learning is shown in figure 6
S100, acquiring a cluster thalamus probability map under a first threshold value, and carrying out binarization to obtain a mask of the cluster thalamus probability map, wherein the mask is used as a first mask; registering the first mask to a diffusion magnetic resonance space of the brain of the subject individual to obtain a thalamic ROI of the brain of the subject individual;
in this embodiment, the above constructed cohort thalamus probability map (i.e., the cohort maps in fig. 6 (a) and (b)) is set to a first threshold (preferably set to 0.25 in the present invention), and a mask for generating the cohort thalamus probability map, i.e., the maximum probability mask in fig. 6 (a), is binarized. The most probable mask is registered to the diffusion magnetic resonance space of the subject's individual brain, resulting in a thalamic ROI of the subject's individual brain (i.e., the individual mask in (a) of fig. 6).
S200, acquiring a cluster thalamus probability map under a second threshold, and registering the cluster thalamus probability map to a diffusion magnetic resonance space of the individual brain of the subject to obtain a cluster prior map of the individual brain of the subject;
in this embodiment, the above-constructed cohort thalamic probability map is taken as the second threshold (preferably set to >0.75 in the present invention) to generate a high probability thalamic map (i.e. a high confidence cohort map in (b) of fig. 6) and to register it to the individual diffusion magnetic resonance space as a cohort prior map (i.e. an individual prior map) of the individual brains of the subject, as shown in (b) of fig. 6. The first threshold and the second threshold are used to remove the voxel points with the maximum subregion probability value lower than the threshold (i.e. the first threshold or the second threshold), so as to construct a cohort thalamic probability map (specifically, as shown in a 500).
S300, in a diffusion magnetic resonance space of the individual brain of the subject, removing the group prior map in the thalamus ROI to obtain a mask of an undefined thalamus region as a second mask;
in this embodiment, the cohort prior map in the thalamic ROI is culled, resulting in a mask of undefined thalamic regions as the second mask, as shown in fig. 6 (c).
S400, calculating the coefficient of a 45-dimensional spherical harmonic function and the position coordinate of a 3-dimensional diffusion magnetic resonance space of each voxel in the group prior map by combining the second mask, and combining the coefficient and the position coordinate into a 48-dimensional characteristic vector serving as an individual characteristic; inputting the individual characteristics into a pre-constructed individual classification model to obtain a prediction probability value vector of a voxel in each undefined region, and taking a maximum subregion label corresponding to the prediction probability value vector as a final subregion label of the voxel to further generate a map of the undefined region; the individualized classification model is constructed based on a deep learning neural network;
in this embodiment, with reference to the second mask (when training an individualized classification model, a region inside the second mask is used, and when verifying, a region outside the second mask is used), the coefficients of the 45-dimensional spherical harmonic function and the position coordinates of the 3-dimensional diffusion magnetic resonance space of each voxel in the cohort prior map are first calculated, and the two are combined to obtain the individual features. Inputting the individual features (namely 48 dimensions) into a pre-constructed individual classification model to obtain a prediction probability value vector of a voxel in each undefined region, taking a subregion label with the maximum prediction probability value vector as a final subregion label of the voxel, and generating a map of the undefined region; the individualized classification model is constructed based on a deep learning neural network (i.e., the deep learning model in fig. 6 (e)).
Wherein, the individualized classification model is constructed based on the deep learning neural network, and the training process is as follows, as shown in (d) in fig. 6:
obtaining individual characteristics through the methods of S100-S400, inputting the individual characteristics into a pre-constructed individual classification model, and obtaining a K (K number of subregions) dimensional prediction probability value vector of voxels in each undefined region;
based on the prediction probability value vector, obtaining a loss value through a mean square error loss function by combining with the sub-region category corresponding to the group prior map, and updating the model parameters of the individualized classification model; and (4) training the individualized classification model in a circulating manner until the trained individualized classification model is obtained.
And S500, combining the group prior map and the map of the undefined region to generate a thalamic map individualized by the subject. As shown in (f) of fig. 6.
Finally, the individualized thalamic map spectrum is verified by using the improved contour coefficient Silhouette value, which is specifically as follows:
the euclidean distance of the 45 dimensional spherical harmonic function coefficients is used as the distance measure. And quantitatively comparing three individualized methods of group prior guidance, group registration and single-test clustering by using the lifting ratio of Silhouette as a gain index. The gain index takes the Silhouette value of the individual thalamic atlas of a single tested cluster as a denominator, and takes the difference value of the Silhouette value of the individual thalamic atlas guided by group prior or registered by groups as a numerator. The gain is positive number, which means that the accuracy of a certain individualized atlas drawing method is higher than that of a single tested cluster, the accuracy is improved more when the gain is larger, and the gain is negative number, which means that the individual atlas drawing method is lower than that of the single tested cluster. The robustness and repeatability of the individual thalamus mapping scheme at different magnetic field strengths and different scanning batches were verified respectively using four data sets, HCP 3t, HCP 7t, HCP Test and HCP Retest. The robustness of the mapping scheme of the individualized thalamic map at different thalamic subregion numbers was verified using different partition numbers 7 and 12.
The intra-assay regional pattern consistency indicators for individualized thalamic maps on the HCP four datasets are validation indicators on the HCP 3T (N = 100), HCP 7T (N = 100), HCP Test (N = 30), and HCP Retest (N = 30) datasets, respectively. On the four data sets, the validation indexes at the partition numbers of 7 and 12 were calculated, respectively. The results show that three individual methods of group prior guidance, group registration and single-subject clustering on four data sets show similar intra-subject region pattern consistency, but the individual thalamic map spectrum guided by the group prior has higher intra-subject consistency compared with the other two methods.
Gain indexes of the tested intra-region consistency of the individualized maps, namely gain comparison of the tested intra-region consistency on HCP 3T data and HCP 7T data, and gain comparison of the tested intra-region consistency on HCP Test data and HCP Retest data. The intra-test region consistency of the individual thalamus atlas drawn by the group registration method is poorer than that of single-test clustering. And the individual thalamic atlas guided by the group priori has stronger regional consistency in the test than the tested thalamic atlas clustered by a single test. When the number of partitions is 7, the gains of the 3T and 7T data are identical. The gain of 7T data is higher than 3T data at the partition number of 12. The gains of Test and Retest are substantially the same for the number of partitions 7 and 12. On the four datasets, the cohort a priori guided individualized thalamic atlas had a higher gain at the number of partitions 12 than at the number of partitions 7.
When the cohort a priori guided individualized thalamic maps were subjected to consistency verification on HCP Test and HCP Retest, the cohort a priori guided individualized thalamic maps showed stable and reliable scan batch-to-batch consistency at partition numbers of 7 and 12.
A second embodiment of the present invention relates to a group prior guided deep learning based thalamus personalized atlas system, as shown in fig. 2, including: a thalamus ROI acquisition module 100, a group prior map acquisition module 200, a rejection processing module 300, an individualized classification module 400 and an individualized map generation module 500;
the thalamic ROI acquisition module 100 is configured to acquire a cluster thalamic probability map under a first threshold value and perform binarization to obtain a mask of the cluster thalamic probability map, wherein the mask is used as a first mask; registering the first mask to a diffusion magnetic resonance space of the brain of the subject individual to obtain a thalamic ROI of the brain of the subject individual;
the group prior map acquisition module 200 is configured to acquire a group thalamus probability map under a second threshold, and register the group thalamus probability map to a diffusion magnetic resonance space of the individual brain of the subject to obtain a group prior map of the individual brain of the subject;
the culling processing module 300 is configured to cull the cohort prior map in the thalamic ROI in the diffusion magnetic resonance space of the subject's individual brain to obtain a mask of an undefined thalamic region as a second mask;
the individualized classification module 400 is configured to calculate a coefficient of a 45-dimensional spherical harmonic function and a position coordinate of a 3-dimensional diffusion magnetic resonance space of each voxel in the group prior map, and combine the coefficients and the position coordinate to obtain individual features; inputting the individual characteristics into a pre-constructed individual classification model to obtain a prediction probability value vector of a voxel in each undefined region, taking a subregion label with the maximum prediction probability value vector as a final subregion label of the voxel, and generating a map of the undefined region; the individualized classification model is constructed based on a deep learning neural network;
the individualized atlas generation module 500 is configured to combine the cohort prior atlas and the atlas of the undefined region to generate a thalamic atlas individualized for the subject;
the construction method of the cluster thalamus probability map comprises the following steps:
a100, acquiring structural nuclear magnetic resonance images and diffusion tensor nuclear magnetic resonance images of brains of N subjects as input images; n is a positive integer;
a200, sequentially carrying out HCP minimum preprocessing, ROI registration and ROI post-processing on the input image to obtain a final thalamus ROI of the individual brain of the subject, and obtaining the local diffusion characteristic of each voxel in the final thalamus ROI of the individual brain of the subject through an ODF estimation algorithm;
a300, calculating the similarity between voxels by combining the local diffusion characteristics obtained in the A200 and clustering to obtain the final clustering result of the voxels in the thalamus ROI of the individual brain of the subject;
a400, registering the clustering result obtained in the step A300 to a standard space, and performing label remapping to obtain a sub-region label corresponding to each voxel in a final thalamus ROI of the individual brains of the subjects, namely obtaining thalamus regions of the individual brains of the N subjects in the standard space;
a500, calculating a corresponding subregion probability value of each voxel in a final thalamus ROI of the individual brain of the subject, removing voxel points of which the maximum subregion probability value is lower than a first threshold value, and then constructing a thalamus probability map of a group level, namely a cohort thalamus probability map, by using the remaining voxel points; and the probability value of the subregions is the ratio of the number of the testees in each subregion to the total number of the testees.
It should be noted that, the group prior guided individual thalamus atlas system based on deep learning provided in the above embodiment is only exemplified by the division of the above functional modules, and in practical applications, the above function allocation may be completed by different functional modules according to needs, that is, the modules or steps in the embodiment of the present invention are further decomposed or combined, for example, the modules in the above embodiment may be combined into one module, or may be further split into multiple sub-modules, so as to complete all or part of the above described functions. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the modules or steps, and are not to be construed as unduly limiting the present invention.
An electronic apparatus according to a third embodiment of the present invention includes: at least one processor; and a memory communicatively coupled to at least one of the processors; wherein the memory stores instructions executable by the processor for execution by the processor to implement the cohort a priori guided, deep learning based individualized atlas mapping method described above.
A computer readable storage medium of a fourth embodiment of the present invention stores computer instructions for being executed by the computer to implement the above-mentioned cohort a-priori guided deep learning-based thalamus individualized mapping method.
A fifth embodiment of the present invention provides a group prior guided deep learning-based thalamus personalized atlas drawing apparatus, including: a magnetic resonance image acquisition device, a central processing device,
the magnetic resonance image acquisition equipment comprises magnetic resonance imaging equipment and a superconducting magnetic resonance instrument, and is configured to acquire a structural nuclear magnetic resonance image and a diffusion tensor nuclear magnetic resonance image of the brain of an individual subject;
the central processing equipment comprises a GPU (graphics processing unit), wherein the GPU is configured to acquire a cluster thalamus probability map under a first threshold value and carry out binarization to obtain a mask of the cluster thalamus probability map, and the mask is used as a first mask; registering the first mask to a diffusion magnetic resonance space of the brain of the subject individual to obtain a thalamic ROI of the brain of the subject individual;
acquiring a cluster thalamus probability map under a second threshold value, and registering the cluster thalamus probability map to a diffusion magnetic resonance space of the brain of the individual subject to obtain a cluster prior map of the brain of the individual subject; in a diffusion magnetic resonance space of the individual brain of the subject, eliminating a group prior map in the thalamic ROI to obtain a mask of an undefined thalamic region as a second mask;
combining the second mask, calculating the coefficient of a 45-dimensional spherical harmonic function and the position coordinate of a 3-dimensional diffusion magnetic resonance space of each voxel in the group prior map, and combining the coefficient and the position coordinate into a 48-dimensional characteristic vector as an individual characteristic; inputting the individual characteristics into a pre-constructed individual classification model to obtain a prediction probability value vector of a voxel in each undefined region, and taking a maximum subregion label corresponding to the prediction probability value vector as a final subregion label of the voxel to further generate a map of the undefined region; the individualized classification model is constructed based on a deep learning neural network; merging the cohort prior atlas and the atlas of the undefined region to generate a thalamic atlas individualized for the subject;
the construction method of the cluster thalamus probability map comprises the following steps:
a100, acquiring structural nuclear magnetic resonance images and diffusion tensor nuclear magnetic resonance images of brains of N subjects as input images; n is a positive integer;
a200, sequentially carrying out HCP minimum preprocessing, ROI registration and ROI post-processing on the input image to obtain a final thalamic ROI of the individual brain of the subject, and obtaining the local diffusion characteristic of each voxel in the final thalamic ROI of the individual brain of the subject through an ODF estimation algorithm;
a300, calculating the similarity between voxels by combining the local diffusion characteristics obtained in the A200, and clustering to obtain the final clustering result of the voxels in the thalamus ROI of the individual brain of the subject;
a400, registering the clustering result obtained in the step A300 to a standard space, and performing label remapping to obtain a sub-region label corresponding to each voxel in a final thalamus ROI of the individual brains of the subjects, namely obtaining thalamus regions of the individual brains of the N subjects in the standard space;
a500, calculating a corresponding subregion probability value of each voxel in a final thalamus ROI of the individual brain of the subject, removing voxel points with the maximum subregion probability value lower than a first threshold value, and then constructing a thalamus probability map of a cohort level, namely a cohort thalamus probability map, from the remaining voxel points; and the probability value of the subregions is the ratio of the number of the testees in each subregion to the total number of the testees.
As will be clear to those skilled in the art, for convenience and brevity of description, the specific working processes and related descriptions of the electronic device, the computer-readable storage medium, and the thalamus individualized atlas drawing apparatus based on deep learning guided by group apriori described above may refer to the corresponding processes in the foregoing method examples, and are not described herein again.
Those of skill in the art will appreciate that the various illustrative modules, method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that programs corresponding to the software modules, method steps may be located in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. To clearly illustrate this interchangeability of electronic hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether these functions are performed in electronic hardware or software depends upon the particular application and design constraints imposed on the solution. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
The terms "first," "second," "third," and the like are used for distinguishing between similar elements and not necessarily for describing or implying a particular order or sequence.
So far, the technical solutions of the present invention have been described in connection with the preferred embodiments shown in the drawings, but it is apparent to those skilled in the art that the scope of the present invention is not limited to these specific embodiments. Equivalent changes or substitutions of related technical features can be made by those skilled in the art without departing from the principle of the invention, and the technical scheme after the changes or substitutions can be within the protection scope of the invention.

Claims (10)

1. A group prior guided thalamus individualized mapping method based on deep learning is characterized by comprising the following steps:
s100, acquiring a cluster thalamus probability map under a first threshold value, and carrying out binarization to obtain a mask of the cluster thalamus probability map, wherein the mask is used as a first mask; registering the first mask to a diffusion magnetic resonance space of the brain of the individual subject to obtain a thalamic ROI of the brain of the individual subject;
s200, acquiring a cluster thalamus probability map under a second threshold, and registering the cluster thalamus probability map to a diffusion magnetic resonance space of the individual brain of the subject to obtain a cluster prior map of the individual brain of the subject;
s300, in a diffusion magnetic resonance space of individual brains of a subject, eliminating a group prior map in the thalamus ROI to obtain a mask of an undefined thalamus region as a second mask;
s400, combining the second mask, calculating the coefficient of a 45-dimensional spherical harmonic function and the position coordinate of a 3-dimensional diffusion magnetic resonance space of each voxel in the group prior map in the thalamus ROI, and combining the coefficient and the position coordinate into a 48-dimensional feature vector serving as an individual feature; inputting the individual characteristics into a pre-constructed individual classification model to obtain a prediction probability value vector of a voxel in each undefined region, and taking a maximum subregion label corresponding to the prediction probability value vector as a final subregion label of the voxel to further generate a map of the undefined region; the individualized classification model is constructed based on a deep learning neural network;
s500, combining the group prior map in the thalamic ROI and the map of the undefined region to generate a thalamic map of individual subjects;
the construction method of the cohort thalamus probability map comprises the following steps:
a100, acquiring structural nuclear magnetic resonance images and diffusion tensor nuclear magnetic resonance images of brains of N subjects as input images; n is a positive integer;
a200, sequentially carrying out HCP minimum preprocessing, ROI registration and ROI post-processing on the input image to obtain a final thalamic ROI of the individual brain of the subject, and obtaining the local diffusion characteristic of each voxel in the final thalamic ROI of the individual brain of the subject through an ODF estimation algorithm; the local dispersion characteristics are expressed by using a diffusion direction distribution function ODF, and the ODF estimation algorithm is restrictive spherical deconvolution or multi-spherical-shell multi-organization restrictive spherical deconvolution;
wherein, the ROI postprocessing is carried out on the input image, and the method comprises the following steps:
calculating fractional anisotropy values corresponding to each voxel point using FSL in a diffusion magnetic resonance space of the brain of the individual subject; FSL is FMRIB's Software Library for brain imaging data analysis;
calculating a cerebrospinal fluid probability value corresponding to each voxel point in a structural magnetic resonance space of the brain of the individual subject by using SPM; the SPM is Statistical Parametric Mapping and is used for analyzing a brain imaging data sequence;
removing voxel points with the anisotropy fraction value larger than a set anisotropy fraction threshold value or the cerebrospinal fluid probability value larger than a set cerebrospinal fluid probability threshold value from the individual thalamus ROI, and taking the remaining voxel points as the final individual thalamus ROI; wherein, the thalamus ROI of the individual is the thalamus ROI obtained after the input image is sequentially subjected to the HCP minimum preprocessing and the ROI registration; HCP minimal pre-processing is a minimal pre-processing pipeline for deconstruction, functional and diffusion MRI, including spatial artifact removal, spatial distortion removal, surface generation, cross-modality registration, and alignment to a standard space;
a300, calculating the similarity between voxels by combining the local diffusion characteristics obtained in the A200, and clustering to obtain the final clustering result of the voxels in the thalamus ROI of the individual brain of the subject;
a400, registering the clustering result obtained in the step A300 to a standard space, and performing label remapping to obtain a sub-region label corresponding to each voxel in a final thalamus ROI of the individual brains of the subjects, namely obtaining thalamus regions of the individual brains of the N subjects in the standard space;
a500, calculating a corresponding subregion probability value of each voxel in a final thalamus ROI of the individual brain of the subject, removing voxel points with the maximum subregion probability value lower than a first threshold value, and then constructing a thalamus probability map of a cohort level, namely a cohort thalamus probability map, from the remaining voxel points; and the probability value of the subregions is the ratio of the number of the testees in each subregion to the total number of the testees.
2. The cohort a priori guided deep learning-based thalamus individualized atlas rendering method of claim 1, wherein the input images are subjected to HCP minimum preprocessing, ROI registration, in sequence by:
carrying out HCP minimum preprocessing on a structural nuclear magnetic resonance image and a diffusion tensor nuclear magnetic resonance image of the brain of an individual subject to obtain a preprocessed structural nuclear magnetic resonance image and a preprocessed diffusion tensor nuclear magnetic resonance image;
and registering the diffusion magnetic resonance space with the structure magnetic resonance space, the structure magnetic resonance space and the standard space by using an ROI (region of interest) registration method based on the preprocessed structure magnetic resonance image and the preprocessed diffusion tensor magnetic resonance image to obtain an individual thalamus ROI.
3. The group a priori guided deep learning based thalamus individualized mapping method according to claim 1, wherein the subject individual brain's final thalamus ROI's local diffusion feature of each voxel is obtained by:
in a diffusion magnetic resonance space in a final thalamus ROI of the brain of the individual subject, calculating response functions of brain tissues under different diffusion weighting factor b value parameters on diffusion magnetic resonance data by using a dhollander algorithm; combining the response function of the brain tissue, calculating 45-dimensional coefficients of 8-order spherical harmonic functions by using a multi-tissue multi-spherical-shell restrictive spherical deconvolution method, and quantifying the local diffusion characteristic of each voxel; the dhollander algorithm is an estimated response function and is used for calculating response functions corresponding to different objects.
4. The cohort a priori guided, deep learning-based thalamus individualized mapping method according to claim 1, wherein the similarity between voxels is calculated by:
Figure FDA0004114497920000031
where S (i, j) denotes the similarity between two voxels, E pos (i, j) denotes the Euclidean distance between 3-dimensional coordinates of two voxels in the diffusion magnetic resonance space, E odf (i, j) represents the Euclidean distance, w, between the sampling coefficients of the 45-dimensional spherical harmonic functions of the two voxels pos And w odf Respectively represent E pos (i,j)、E odf (i, j) corresponding weighting coefficients in calculating the similarity.
5. The group a priori guided deep learning-based thalamus individualized mapping method according to claim 4, wherein the clustering result of voxels in the final thalamus ROI of the subject individual brain is obtained by:
reducing the local diffusion characteristics of each voxel in a final thalamus ROI of the individual brains of the subjects and the similarity among the voxels by a spectral clustering method;
based on the local diffusion characteristics after dimensionality reduction and the similarity between voxels, clustering each voxel into K classes by using K-means clustering, and taking the K classes as the clustering result of the voxels in the final thalamus ROI of the individual brain of the subject.
6. The group a priori guided deep learning-based thalamic individualized mapping method according to claim 1, wherein the sub-region labels corresponding to each voxel in the final thalamic ROI of the subject's individual brain are obtained by:
calculating N-dimensional label vectors of each voxel point on N subjects, then clustering all voxel points in the final thalamus ROI of the individual brains of the subjects according to the similarity of the label vectors, and taking the clustering result as a partition label;
based on the subarea labels, the clustering result of each voxel in the final thalamic ROI of the individual brain of the subject is re-marked according to a space maximum overlapping method, so that a subarea label corresponding to each voxel in the final thalamic ROI of the individual brain of the subject is obtained;
wherein, calculating N dimension label vector of each subject on N subjects, namely extracting corresponding sub-region label of each subject on the subject, and respectively extracting a sub-region label for N subjects to form the N dimension label vector.
7. The cohort a priori guided deep learning-based thalamus individualized mapping method according to claim 6, wherein the individualized classification model is trained by:
obtaining individual characteristics through the methods of S100-S400, and inputting the individual characteristics into a pre-constructed individualized classification model to obtain a prediction probability value vector of a voxel in each undefined region;
based on the prediction probability value vector, combining with the sub-region categories corresponding to the group prior map, obtaining a loss value through a mean square error loss function, and updating model parameters of the individualized classification model;
and circulating the steps until the trained individual classification model is obtained.
8. A population group prior guided deep learning based thalamus individualized mapping system, the system comprising: the thalamus ROI acquisition module, the group prior map acquisition module, the elimination processing module, the individual classification module and the individual map generation module;
the thalamus ROI acquisition module is configured to acquire a cluster thalamus probability map under a first threshold value and carry out binarization to obtain a mask of the cluster thalamus probability map, and the mask is used as a first mask; registering the first mask to a diffusion magnetic resonance space of the brain of the individual subject to obtain a thalamic ROI of the brain of the individual subject;
the group prior map acquisition module is configured to acquire a group thalamus probability map under a second threshold value, and register the group thalamus probability map to a diffusion magnetic resonance space of the brain of the individual subject to obtain a group prior map of the brain of the individual subject;
the elimination processing module is configured to eliminate the group prior map in the thalamus ROI in the diffusion magnetic resonance space of the individual brain of the subject to obtain a mask of an undefined thalamus region as a second mask which is used as a training set of an individual model;
the individualized classification module is configured to calculate the coefficient of a 45-dimensional spherical harmonic function and the position coordinate of a 3-dimensional diffusion magnetic resonance space of each voxel in the group prior map in the thalamus ROI by combining the coefficient and the position coordinate into a 48-dimensional feature vector as an individual feature; inputting the individual characteristics into a pre-constructed individual classification model to obtain a prediction probability value vector of a voxel in each undefined region, and taking a maximum subregion label corresponding to the prediction probability value vector as a final subregion label of the voxel to further generate a map of the undefined region; the individualized classification model is constructed on the basis of a deep learning neural network;
the individualized atlas generation module is configured to combine the cohort prior atlas in the thalamic ROI and the atlas of the undefined region to generate a thalamic atlas individualized for the subject;
the construction method of the cohort thalamus probability map comprises the following steps:
a100, acquiring structural nuclear magnetic resonance images and diffusion tensor nuclear magnetic resonance images of brains of N subjects as input images; n is a positive integer;
a200, sequentially carrying out HCP minimum preprocessing, ROI registration and ROI post-processing on the input image to obtain a final thalamus ROI of the individual brain of the subject, and obtaining the local diffusion characteristic of each voxel in the final thalamus ROI of the individual brain of the subject through an ODF estimation algorithm; the local dispersion characteristic is expressed by using a diffusion direction distribution function ODF, and the ODF estimation algorithm is restrictive spherical deconvolution or multi-spherical-shell multi-tissue restrictive spherical deconvolution;
wherein, the ROI postprocessing is carried out on the input image, and the method comprises the following steps:
calculating a fractional anisotropy value corresponding to each voxel point using FSL in a diffusion magnetic resonance space of the brain of an individual subject; FSL is FMRIB's Software Library for brain imaging data analysis;
calculating a cerebrospinal fluid probability value corresponding to each voxel point in a structural magnetic resonance space of the brain of the individual subject by using SPM; SPM is Statistical Parametric Mapping, used for analyzing brain imaging data sequence;
removing voxel points with the anisotropy fraction value larger than a set anisotropy fraction threshold value or the cerebrospinal fluid probability value larger than a set cerebrospinal fluid probability threshold value from the individual thalamus ROI, and taking the remaining voxel points as the final individual thalamus ROI; wherein, the thalamus ROI of the individual is the thalamus ROI obtained after the input image is sequentially subjected to the HCP minimum preprocessing and the ROI registration; HCP minimum preprocessing is the minimum preprocessing pipeline for deconstruction, functional and diffusion MRI, including spatial artifact removal, spatial distortion removal, surface generation, cross-modality registration, and alignment to standard space;
a300, calculating the similarity between voxels by combining the local diffusion characteristics obtained in the A200, and clustering to obtain the final clustering result of the voxels in the thalamus ROI of the individual brain of the subject;
a400, registering the clustering result obtained in the step A300 to a standard space, and performing label remapping to obtain a sub-region label corresponding to each voxel in a final thalamus ROI of the individual brains of the subjects, namely obtaining thalamus regions of the individual brains of the N subjects in the standard space;
a500, calculating a corresponding subregion probability value of each voxel in a final thalamus ROI of the individual brain of the subject, removing voxel points with the maximum subregion probability value lower than a first threshold value, and then constructing a thalamus probability map of a cohort level, namely a cohort thalamus probability map, from the remaining voxel points; and the probability value of the subregions is the ratio of the number of the testees in each subregion to the total number of the testees.
9. An electronic device, comprising:
at least one processor; and a memory communicatively coupled to at least one of the processors;
wherein the memory stores instructions executable by the processor for execution by the processor to implement the cohort a priori guided deep learning based individualized atlas mapping method of any of claims 1-7.
10. A computer-readable storage medium storing computer instructions for execution by the computer to implement the cohort a priori guided deep learning-based individualized atlas mapping method of any one of claims 1-7.
CN202210822674.4A 2022-07-12 2022-07-12 Group prior guided thalamus individualized atlas drawing method based on deep learning Active CN115082586B (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
CN202210822674.4A CN115082586B (en) 2022-07-12 2022-07-12 Group prior guided thalamus individualized atlas drawing method based on deep learning
PCT/CN2022/106161 WO2024011647A1 (en) 2022-07-12 2022-07-18 Deep learning-based thalamus individualized map drawing method guided by group prior knowledge

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210822674.4A CN115082586B (en) 2022-07-12 2022-07-12 Group prior guided thalamus individualized atlas drawing method based on deep learning

Publications (2)

Publication Number Publication Date
CN115082586A CN115082586A (en) 2022-09-20
CN115082586B true CN115082586B (en) 2023-04-04

Family

ID=83259199

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210822674.4A Active CN115082586B (en) 2022-07-12 2022-07-12 Group prior guided thalamus individualized atlas drawing method based on deep learning

Country Status (2)

Country Link
CN (1) CN115082586B (en)
WO (1) WO2024011647A1 (en)

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107330267A (en) * 2017-06-28 2017-11-07 首都医科大学宣武医院 Utilize the white matter fiber brain map construction method of diffusion tensor medical image
CN109528197B (en) * 2018-11-20 2022-07-08 中国科学院脑科学与智能技术卓越创新中心 Individual prediction method and system for mental diseases based on brain function map
CN110599461B (en) * 2019-08-21 2023-04-07 东南大学 Thalamus function partitioning method based on subspace feature learning
CN111081351B (en) * 2019-12-02 2024-02-20 北京优脑银河科技有限公司 Brain functional map drawing method and system
EP3940637A1 (en) * 2020-07-15 2022-01-19 Siemens Healthcare GmbH Method and system for characterizing an impact of brain lesions on brain connectivity using mri
CN112614126B (en) * 2020-12-31 2023-07-25 中国科学院自动化研究所 Magnetic resonance image brain region dividing method, system and device based on machine learning
CN113592836B (en) * 2021-08-05 2022-11-18 东南大学 Deep multi-modal graph convolution brain graph classification method
CN113610808B (en) * 2021-08-09 2023-11-03 中国科学院自动化研究所 Group brain map individuation method, system and equipment based on individual brain connection diagram
CN114376558B (en) * 2022-03-24 2022-07-19 之江实验室 Brain atlas individuation method and system based on magnetic resonance and twin map neural network

Also Published As

Publication number Publication date
CN115082586A (en) 2022-09-20
WO2024011647A1 (en) 2024-01-18

Similar Documents

Publication Publication Date Title
Garyfallidis et al. Recognition of white matter bundles using local and global streamline-based registration and clustering
O'Donnell et al. Fiber clustering versus the parcellation-based connectome
Shen et al. MRI fuzzy segmentation of brain tissue using neighborhood attraction with neural-network optimization
Wismüller et al. Cluster analysis of biomedical image time-series
Cover et al. Computational methods for corpus callosum segmentation on MRI: A systematic literature review
CN110840468B (en) Autism risk assessment method and device, terminal device and storage medium
CN108171697B (en) WMH automatic extraction system based on cluster
WO2007079207A2 (en) An integrated segmentation and classification approach applied to medical applications analysis
Kruggel et al. Alignment of magnetic-resonance brain datasets with the stereotactical coordinate system
Mostapha et al. A statistical framework for the classification of infant DT images
Kong et al. Automatic tissue image segmentation based on image processing and deep learning
CN112614126A (en) Magnetic resonance image brain region dividing method, system and device based on machine learning
CN116468655A (en) Brain development atlas and image processing system based on fetal magnetic resonance imaging
Ziyan et al. Consistency clustering: a robust algorithm for group-wise registration, segmentation and automatic atlas construction in diffusion MRI
CN115082586B (en) Group prior guided thalamus individualized atlas drawing method based on deep learning
CN114862799B (en) Full-automatic brain volume segmentation method for FLAIR-MRI sequence
CN115496743A (en) Cerebrovascular lesion segmentation method, cerebrovascular lesion segmentation device, storage medium and electronic device
CN115337000A (en) Machine learning method for evaluating brain aging caused by diseases based on brain structure images
Caan et al. Nonrigid point set matching of white matter tracts for diffusion tensor image analysis
Mihaylova et al. A brief survey of spleen segmentation in MRI and CT images
WO2004051568A1 (en) Brain connectivity mapping
Logiraj et al. TractNet: a deep learning approach on 3D curves for segmenting white matter fibre bundles
Ugurlu et al. Neighborhood resolved fiber orientation distributions (NRFOD) in automatic labeling of white matter fiber pathways
Noorizadeh et al. Multi-atlas based neonatal brain extraction using atlas library clustering and local label fusion
Thompson et al. Structural Magnetic Resonance Imaging

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant