CN116894800A

CN116894800A - Multi-level data fusion method and system based on cerebrovascular knowledge

Info

Publication number: CN116894800A
Application number: CN202310876159.9A
Authority: CN
Inventors: 戴亚康; 耿辰; 戴斌; 周志勇; 刘燕
Original assignee: Jinan Guoke Medical Engineering Technology Development Co ltd; Suzhou Institute of Biomedical Engineering and Technology of CAS
Current assignee: Jinan Guoke Medical Engineering Technology Development Co ltd; Suzhou Institute of Biomedical Engineering and Technology of CAS
Priority date: 2023-07-17
Filing date: 2023-07-17
Publication date: 2023-10-17

Abstract

The invention relates to the technical field of data fusion, and discloses a multi-level data fusion method and system based on cerebrovascular knowledge, wherein the method comprises the following steps: dividing the brain image data to obtain brain blood vessel division data, and extracting the central line of the brain blood vessel division data to obtain brain blood vessel tree central line data; after the cerebrovascular segmentation and the cerebrovascular knowledge data layering processing, carrying out characteristic quantization by adopting a vascular pretreatment algorithm set to obtain corresponding layering quantization characteristics and setting weights of the layering quantization characteristics; combining central line data of a cerebrovascular tree, carrying out feature map fusion according to layered quantization features to obtain a data fusion feature map, carrying out subgraph construction based on positions of nodes of the data fusion feature map, obtaining node subgraphs, carrying out node weight updating on the node subgraphs to obtain corresponding weighted subgraphs, and fusing the weighted subgraphs to obtain a node weight adjustment feature map, forming multi-level vascular knowledge fusion data, combining performance of a preset model to obtain optimal multi-level vascular knowledge fusion data, and defining an optimal processing path of the multi-level vascular knowledge fusion data.

Description

Multi-level data fusion method and system based on cerebrovascular knowledge

Technical Field

The invention relates to the technical field of data fusion, in particular to a multi-level data fusion method and system based on cerebrovascular knowledge.

Background

The existing research has more research on the utilization of three-dimensional morphological characteristics of blood vessels and the spatial distribution of aneurysms, and related research proves that the surface profile is important for aneurysms detection, but the surface three-dimensional point set depends on accurate surface reconstruction, other characteristics except the profile can be lost, and the performance is difficult to further improve.

For the task of improving the aneurysm detection performance, related scholars propose various methods, and although the detection performance of the various methods is improved, the theoretical basis of the optimal processing method for the input data is not clear. For example, assuming that the multi-channel data improves the input data volume of the convolutional network, so as to improve the performance, but meanwhile, taking the contour and the high-density area as parallel channels to input three channels, no corresponding improvement is obtained, which proves that the assumption is incorrect. The second channel of the continuity structure is assumed to enhance the study of the continuity characteristic of the network to the blood vessel, and similar effects of hard attention are achieved, but the gradient profile information is replaced by the profile of the binarization result or the center line of the blood vessel, so that the training of the model is not obviously improved, and the assumption is not proved to be true.

In summary, in the stage of inputting data into the aneurysm detection model, the processing of the supervision data formed by the input data and the label data is strongly related to the detection performance of the model focus, and related scholars have explored a plurality of supervision data generation methods capable of improving the focus detection sensitivity. However, the existing researches have the problems of one-sided description, and the best processing path of the supervision data is not clear yet depending on manual observation and experience, and how to find the best supervision data construction method is still an important problem which plagues researchers at present.

Disclosure of Invention

In view of the above, the invention provides a multi-level data fusion method and system based on brain vascular knowledge, which performs knowledge structuring layering study on cerebral artery and cerebral aneurysm based on brain vascular related knowledge, and forms a set of brain artery related knowledge multi-level data fusion method by studying the implementation path of the structuring layering manner on brain image data, thereby solving the problem of optimizing the knowledge characterization method in knowledge-guided focus detection, improving the focus detection effect, and solving the technical problem proposed in the background.

In a first aspect, the present invention provides a method for multi-level data fusion based on cerebrovascular knowledge, the method comprising:

Acquiring brain image data and brain vessel knowledge data;

dividing the brain image data to obtain brain blood vessel dividing data, and extracting the central line of the brain blood vessel dividing data to obtain brain blood vessel tree central line data;

respectively carrying out layering treatment on the cerebrovascular segmentation data and the cerebrovascular knowledge data;

carrying out feature quantization on the brain blood vessel segmentation data subjected to layering processing by adopting a blood vessel preprocessing algorithm set according to the brain blood vessel knowledge data subjected to layering processing to obtain corresponding layering quantization features and setting corresponding weights;

combining the central line data of the cerebrovascular tree, and carrying out feature map fusion according to the layered quantization features to obtain a data fusion feature map;

carrying out subgraph construction based on the positions of all nodes in the data fusion feature graph to obtain subgraphs of all nodes;

updating the node weights of the node subgraphs to obtain corresponding weighted subgraphs, and fusing the weighted subgraphs to obtain a node weight adjustment feature map to form multi-level vascular knowledge fusion data; and obtaining optimal multi-level vascular knowledge fusion data according to the multi-level vascular knowledge fusion data and the performance of the preset model.

The invention can carry out knowledge structuring layering study on cerebral artery and cerebral aneurysm based on the relevant knowledge of cerebral blood vessels, and define the optimal processing path of multi-level blood vessel knowledge fusion data by studying the realization path of the structuring layering manner in brain image data, solves the problem of optimizing the knowledge characterization method in knowledge-guided focus detection, and improves the focus detection effect to a certain extent.

In an alternative embodiment, the cerebrovascular knowledge data is text data, comprising: clinical guidelines, medical-related books, clinical care records, and medical records; the brain image data includes: CT images, magnetic resonance angiography images, and time-fly magnetic resonance angiography images.

According to the invention, the data fusion is carried out on the text data of the brain vascular knowledge and the image data of the brain image, and the fused data are used for detecting and researching related focuses in medicine. By means of the brain image data assisted by the brain blood vessel related knowledge representation, high quality and diversification of input data in a relevant focus detection model in medicine can be guaranteed, and detection accuracy of the model is improved to a certain extent.

In an alternative embodiment, the process of segmenting brain image data to obtain brain blood vessel segmented data and extracting the central line of the brain blood vessel segmented data to obtain brain blood vessel tree central line data includes:

the brain image data are sequentially subjected to blood vessel enhancement, automatic deboning, seed point extraction, seed point area density distribution statistics and area growth treatment, so that brain blood vessel segmentation data are obtained;

performing binarization processing and corrosion processing on the cerebral vessel segmentation data to obtain cerebral vessel tree central line data, wherein the directed graph comprises nodes, connection relations and node types of the blood vessels, and the node types comprise: vessel bifurcation, vessel superior and vessel end points.

The invention performs segmentation and central line extraction processing on brain image data, can obtain high-precision segmentation data and central line data of a brain blood vessel tree, and ensures the quality of multi-level blood vessel knowledge fusion data to a certain extent.

In an alternative embodiment, the process of layering the cerebrovascular segmentation data and the cerebrovascular knowledge data respectively includes:

global to local vessel partitioning is performed on the brain vessel segmentation data and the brain vessel knowledge data according to the vessel structure, and the method comprises the following steps: whole brain, vascular tree, arterial region, arterial segment, focal part; wherein the arterial region is divided into anterior cerebral artery, middle cerebral artery, posterior cerebral artery, internal carotid artery, vertebral artery, basilar artery, anterior cerebral circulation, posterior cerebral circulation and anterior choroidal artery;

vessel segmentation is performed again in each of the arterial regions.

According to the invention, layering processing comprising blood vessel segmentation and blood vessel segmentation is respectively carried out on the cerebral vessel segmentation data and the cerebral vessel knowledge data, so that the level of knowledge characterization can be thinned, and the level fine granularity and the result accuracy of knowledge characterization are improved. The knowledge points of the cerebral vessels of each level are further obtained based on a knowledge layering quantitative representation mode to represent the mapping corresponding to the brain image data, and the multi-level vessel knowledge fusion data are optimally solved by combining a preset model to obtain the optimal multi-level vessel knowledge fusion data, so that the problem that the existing multi-level vessel knowledge fusion data are seriously dependent on manual observation and experience can be solved.

In an alternative embodiment, according to the hierarchical brain blood vessel knowledge data, a process of performing feature quantization on the hierarchical brain blood vessel segmentation data by using a blood vessel preprocessing algorithm set to obtain corresponding hierarchical quantization features and setting corresponding weights thereof includes:

for the image characteristics of the corresponding cerebral vascular segmentation data of all cerebral vascular knowledge data after layering processing, screening an algorithm intensively adapted to a vascular preprocessing algorithm to quantize the corresponding cerebral vascular segmentation data, and extracting to obtain corresponding layering quantization characteristics; the hierarchy of hierarchical quantization features includes: vessel segmentation, image feature data and measurement feature data; the blood vessel preprocessing algorithm set is a set of algorithms for processing blood vessel images and extracting blood vessel characteristics, and comprises the following steps: edge operators, texture feature extraction algorithms, density distribution, threshold segmentation, geometry measurement and filters;

and setting the weight of the layering quantitative characteristic according to the blood vessel partition of the cerebral blood vessel knowledge data and the layering processing result of the blood vessel partition.

The blood vessel preprocessing algorithm set adopted by the invention can acquire different hierarchical features and image features, improves the diversity of the features, and improves the quality of multi-level blood vessel knowledge fusion data to a certain extent.

In an alternative embodiment, the process of constructing the subgraph based on the positions of the nodes in the data fusion feature graph to obtain the subgraph of each node includes:

separating a plurality of nodes based on the positions of the bifurcation points of the blood vessels to obtain node subgraphs, and obtaining the position information of the corresponding nodes, wherein the position information comprises: the length of the vascular region, the vascular segment and the bifurcation point of the blood vessel to which the coordinates belong;

and performing position coding on each separated node, wherein the position coding is used for representing the mapping relation between each node and the data fusion feature map, and the two-dimensional position coordinates of the point on the central line data of the cerebrovascular tree are the corresponding position coding.

According to the invention, based on the prior knowledge of blood vessel partition and segmentation, the characteristics of each level are subjected to graph data construction, and a plurality of subgraphs containing accurate position information are used as input data of the preset model, so that the input requirement of the preset model can be met, and the training process of the model is accelerated to a certain extent.

In an alternative embodiment, the process of obtaining optimal multi-level vascular knowledge fusion data according to the multi-level vascular knowledge fusion data and the performance of the preset model includes:

The multi-level vascular knowledge fusion data is used as input data of a preset model, a weighted set of sensitivity and specificity is used as a loss function of the model, the loss function is minimized by training the model, so that the model obtains optimal performance, and optimal multi-level vascular knowledge fusion data is obtained.

According to the invention, the optimal multi-level vascular knowledge fusion data is determined through the performance of the preset model, the optimal processing path of the multi-level vascular knowledge fusion data is defined, the problem of optimizing the knowledge characterization method in knowledge-guided focus detection is solved, and the focus detection effect is improved.

In a second aspect, the present invention provides a multi-level data fusion system based on cerebrovascular knowledge, the system comprising:

the data acquisition module is used for acquiring brain image data and brain blood vessel knowledge data;

the data processing module is used for carrying out segmentation processing on the brain image data to obtain brain blood vessel segmentation data, and extracting the central line of the brain blood vessel segmentation data to obtain brain blood vessel tree central line data;

the data layering module is used for respectively layering the cerebrovascular segmentation data and the cerebrovascular knowledge data;

the feature quantization module is used for carrying out feature quantization on the brain blood vessel segmentation data subjected to the layering processing by adopting a blood vessel pretreatment algorithm set according to the brain blood vessel knowledge data subjected to the layering processing, so as to obtain corresponding layering quantization features and set corresponding weights;

The feature fusion module is used for combining the central line data of the cerebrovascular tree, and carrying out feature map fusion according to the layered quantization features to obtain a data fusion feature map;

the sub-graph construction module is used for carrying out sub-graph construction based on the positions of all nodes in the data fusion feature graph to obtain sub-graphs of all nodes;

the weight optimization module is used for updating the node weights of the node subgraphs to obtain corresponding weighted subgraphs, and fusing the weighted subgraphs to obtain a node weight adjustment feature map to form multi-level vascular knowledge fusion data; and obtaining node weight information according to the multi-level blood vessel knowledge fusion data and the performance of the preset model to obtain an optimal node weight adjustment characteristic diagram, so as to form optimal multi-level blood vessel knowledge fusion data.

The multi-level data fusion system based on the cerebral vascular knowledge can carry out knowledge structuring layering study on cerebral artery and cerebral aneurysm based on the cerebral vascular related knowledge, and define the optimal processing path of the multi-level vascular knowledge fusion data by studying the implementation path of the structuring layering manner on brain image data, solve the problem of optimizing a knowledge characterization method in knowledge-guided focus detection, and promote the focus detection effect.

In a third aspect, the present invention provides a computer device comprising: the memory and the processor are in communication connection, computer instructions are stored in the memory, and the processor executes the computer instructions, so that the multi-level data fusion method based on the brain blood vessel knowledge in the first aspect or any corresponding implementation mode is executed.

In a fourth aspect, the present invention provides a computer readable storage medium, where computer instructions are stored on the computer readable storage medium, where the computer instructions are configured to cause a computer to perform a multi-level data fusion method based on cerebrovascular knowledge according to the first aspect or any one of its corresponding embodiments.

Drawings

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings that are needed in the description of the embodiments or the prior art will be briefly described, and it is obvious that the drawings in the description below are some embodiments of the present invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a multi-level data fusion method based on cerebrovascular knowledge according to an embodiment of the invention;

FIG. 2 is a schematic structural diagram of a multi-level data fusion method based on cerebrovascular knowledge according to an embodiment of the present invention;

FIG. 3 is a flow chart of a method for multi-level data fusion based on brain vascular knowledge according to an embodiment of the present invention;

FIG. 4 is a block diagram of a multi-level data fusion system based on cerebrovascular knowledge in accordance with an embodiment of the present invention;

fig. 5 is a schematic diagram of a hardware structure of a computer device according to an embodiment of the present invention.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments of the present invention. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.

Intracranial aneurysms are saccular deformities formed by abnormal expansion of the local lumen due to damage to the wall of the intracranial artery or local congenital defects, with a prevalence of about 3% -5% in the population. Aneurysms, once ruptured, can cause subarachnoid hemorrhage with a higher mortality rate and permanent disability rate. Thus, it is of great clinical importance to discover aneurysms early and to predict the risk of rupture of the aneurysms.

In the existing research, the detection of cerebral aneurysm often uses the original image data to directly perform model training for detecting and identifying cerebral aneurysm. However, cerebral aneurysm detection is a typical small target detection task, and due to the complex and changeable cerebral artery structure, it is difficult to obtain higher model training efficiency and detection accuracy only by using the original brain image data. The related research improves the uniformity of data by carrying out vascular knowledge fusion on cerebral arterial data, thereby improving the detection precision and efficiency of cerebral aneurysms.

The inventor finds that priori knowledge such as aneurysms, cerebral vessels and the like can play a role in improving the performance of a focus detection model in earlier-stage researches, and the form of the priori knowledge representation is quite diversified. Through early experiments, the detection sensitivity of the model can be improved by taking the blood vessel gradient profile information and the original blood vessel data as parallel channels to be input into the detection model of the full convolution network; if the contour information is replaced by the binarization result of the threshold segmentation of the central high-density region of the blood vessel, the detection sensitivity can be further improved. In addition, when the optimal processing method of the input data is not clear, the inventor starts from the labeling data, the texture features of the region after the aneurysm is expanded are input instead of the labeling, the performance improvement of about 5% is realized in sensitivity, and the knowledge representation form of the label data has the influence on the detection performance. Thus, in the data input stage of the aneurysm detection model, the input data and the label data are processed based on knowledge representation, so that the model detection performance can be improved.

However, for the problems that the optimal processing path of the supervision data input by the model is not clear and the knowledge representation forms are diversified, the prior art cannot obtain the optimal aneurysm knowledge representation mode and the data fusion method to construct the supervision data. The invention provides a multi-level data fusion method and a system based on cerebral vascular knowledge, which are used for carrying out knowledge structuring layering study on cerebral arteries and cerebral aneurysms based on cerebral vascular related knowledge, and forming a set of multi-level data fusion method of cerebral arterial related knowledge by studying the implementation path of a structuring layering manner on brain image data, so that the optimal processing path of monitoring data input by taking the multi-level vascular knowledge fusion data as a model can be clarified, the problem of optimizing a knowledge characterization method in knowledge-guided focus detection is solved, and the detection effect of aneurysms is improved to a certain extent.

The embodiment of the invention provides a multi-level data fusion method based on cerebrovascular knowledge, as shown in fig. 1, comprising the following steps:

step S101, acquiring brain image data and brain blood vessel knowledge data.

In this embodiment, brain image data and brain blood vessel knowledge data are acquired based on clinical data and an existing public database, and the brain image data are image data obtained by a clinical common imaging technology. For example, clinically usual imaging techniques include: digital subtraction angiography (digital subtraction angiography, DSA), computed tomography angiography (computed tomographyangiography, CTA), magnetic resonance angiography (magnetic resonance angiography, MRA) and time-of-flight magnetic resonance angiography (time of flightmagnetic resonance angiography, TOF MRA). It should be noted that, the clinical data, the existing public database and the image data of the specific imaging technology adopted in the embodiment are not particularly limited, and are determined according to the actual application requirements.

Step S102, segmenting the brain image data to obtain brain blood vessel segmentation data, and extracting the central line of the brain blood vessel segmentation data to obtain brain blood vessel tree central line data.

It should be noted that, as a research difficulty in the field of medical image processing, accurate brain blood vessel segmentation can provide important basis for analysis of brain aneurysm images, and is used for matching, three-dimensional reconstruction, and the like of brain blood vessels.

Step S103, layering processing is respectively carried out on the cerebral vessel segmentation data and the cerebral vessel knowledge data.

It should be noted that, the layering process in this embodiment includes blood vessel segmentation and blood vessel segmentation, which are all divided according to the specific structure of the blood vessel.

Step S104, carrying out feature quantization on the brain blood vessel segmentation data subjected to layering processing by adopting a blood vessel preprocessing algorithm set according to the brain blood vessel knowledge data subjected to layering processing, obtaining corresponding layering quantization features and setting corresponding weights.

Step S105, combining the central line data of the cerebrovascular tree, and carrying out feature map fusion according to the layered quantization features to obtain a data fusion feature map.

And S106, carrying out subgraph construction based on the positions of all nodes in the data fusion feature graph to obtain subgraphs of all nodes.

It should be noted that, in the process of constructing the subgraph to obtain multiple subgraphs, one node may include one or more hierarchical quantization features, and the specific number of the hierarchical quantization features is determined according to practical applications.

Step S107, updating node weights of the node subgraphs to obtain corresponding weighted subgraphs, and fusing the weighted subgraphs to obtain a node weight adjustment feature map to form multi-level vascular knowledge fusion data; and obtaining optimal multi-level vascular knowledge fusion data according to the multi-level vascular knowledge fusion data and the performance of the preset model.

It should be noted that, in the process of updating weights of the node sub-sets of one or more hierarchical quantization features included in each node and performing performance optimization based on a preset model, the weights of different sub-nodes are analyzed, and the optimal performance of the preset model is combined, so that a corresponding node weight adjustment feature map, namely optimal multi-level vascular knowledge fusion data, is obtained. In this embodiment, the preset model is determined based on the actual application requirement, which is not specifically limited herein.

According to the method, the mapping relation between each level of knowledge and the corresponding brain image data is obtained by using the hierarchical representation form of knowledge quantification, the optimal multi-level vascular knowledge fusion data is obtained by using the performance of the preset model, the optimal processing path of the multi-level vascular knowledge fusion data can be clarified, the problem of optimizing the knowledge representation method in knowledge-guided focus detection is solved, and the detection effect of cerebral aneurysms is improved to a certain extent.

In this embodiment, the cerebrovascular knowledge data is text data, including: clinical guidelines, medical-related books, clinical care records, and medical records; the brain image data includes: CT images, magnetic resonance angiography images, and time-fly magnetic resonance angiography images.

In one embodiment, the aneurysm text obtained from a medical-related book is described as "aneurysm" meaning that three layers of the arterial wall are all distended out to an outside limitation, often at an arterial branch. About 85% is in the anterior circulation, 30% to 40% is from the anterior cerebral artery or anterior transport artery, 30% is in the posterior transport artery, 20% to 30% is in the middle cerebral artery branch, 5% to 10% is in the internal carotid artery. About 15% originate from the posterior circulation, including the basal tip, the upper cerebellum artery, and the lower cerebellum artery. Aneurysms on CT represent well-defined circular lesions of slightly higher density, sometimes with surrounding calcifications ", and a time-of-flight magnetic resonance angiography image TOF MRA is obtained by a hospital.

It should be noted that, manual labeling and paragraph entity extraction may be performed on the text data, where the bolded entity is the labeled entity, and the text data is specifically as follows:

Aneurysms are defined as three layers of the arterial wall that bulge out in an outward localized manner, often at the arterial bifurcation. About 85% is in the anterior circulation, 30% to 40% is from the anterior cerebral artery or anterior transport artery, 30% is in the posterior transport artery, 20% to 30% is in the middle cerebral artery branch, 5% to 10% is in the internal carotid artery. About 15% originate from the posterior circulation, including the basal tip, the upper cerebellum artery, and the lower cerebellum artery. Aneurysms on CT represent a well-defined, somewhat higher-density lesion, sometimes with surrounding calcification. By way of example only, and not limitation.

In this embodiment, the text data of the brain vascular knowledge and the image data of the brain image are subjected to data fusion, and the fused data are used for relevant focus detection research in medicine. By means of the brain image data assisted by the brain blood vessel related knowledge representation, high quality and diversification of input data in an aneurysm detection model can be guaranteed, and detection accuracy of the model is improved to a certain extent.

Specifically, the step S102 includes:

step S1021, sequentially performing blood vessel enhancement, automatic deboning, seed point extraction, seed point region density distribution statistics and region growth treatment on the brain image data to obtain brain blood vessel segmentation data.

It should be noted that, the method for segmenting brain image data in this embodiment is not limited, and can be adaptively adjusted according to the actual application requirement and the data segmentation accuracy requirement.

Step S1022, performing binarization processing and corrosion processing on the cerebral vessel segmentation data to obtain cerebral vessel tree central line data, wherein the directed graph comprises nodes, connection relations and node types of the blood vessels, and the node types comprise: vessel bifurcation, vessel superior and vessel end points.

In this embodiment, by performing segmentation and centerline extraction on the brain image data, high-precision segmentation data and brain vessel tree centerline data can be obtained, and the quality of multi-level vessel knowledge fusion data is ensured to a certain extent.

Specifically, the step S103 includes:

step S1031, performing global-to-local blood vessel partitioning on the brain blood vessel segmentation data and the brain blood vessel knowledge data according to the blood vessel structure, including: whole brain, vascular tree, arterial region, arterial segment, focal part; wherein the arterial region is divided into anterior cerebral artery, middle cerebral artery, posterior cerebral artery, internal carotid artery, vertebral artery, basilar artery, anterior cerebral circulation, posterior cerebral circulation and anterior choroidal artery.

In step S1032, blood vessel segmentation is performed again in each of the arterial regions.

In a specific embodiment, performing global-to-local vascular partitioning on image data of brain vascular segmentation and text data of brain vascular knowledge according to a vascular structure to obtain a whole brain, a vascular tree, an arterial region, arterial segmentation and focus part; wherein the arterial region is divided into anterior cerebral artery ACA, middle cerebral artery MCA, posterior cerebral artery PCA, internal carotid artery ICA, vertebral artery VA, basilar artery BA, anterior cerebral circulation ACOM, posterior cerebral circulation PCOM and anterior choroidal artery AchA. The artery is segmented again, i.e. a refined blood vessel is segmented in each of the above-mentioned artery regions. For example, the anterior cerebral artery ACA can be further divided into vessel segments A1, A2, A3 and A4, and the vessel segmentation conditions in other regions of the artery are determined based on practice by way of illustration only.

In this embodiment, the layering processing manner of the global layer is not limited to the layering processing manner of layering the blood vessels according to the anatomical structure, but may layering the blood vessels according to the region in the image, that is, the non-vascular anatomical region, for example, the layering manner of classifying four quadrant blocks around the center block of the image, which is only used as an example.

In this embodiment, layering processing including blood vessel segmentation and blood vessel segmentation is performed on the cerebral blood vessel segmentation data and the cerebral blood vessel knowledge data, so that the level of knowledge characterization can be thinned, and the level fine granularity and result accuracy of knowledge characterization can be improved. The knowledge points of the cerebral vessels of each level are further obtained based on a knowledge layering quantitative representation mode to represent the mapping corresponding to the brain image data, and the multi-level vessel knowledge fusion data are optimally solved by combining a preset model to obtain the optimal multi-level vessel knowledge fusion data, so that the problem that the existing multi-level vessel knowledge fusion data are seriously dependent on manual observation and experience can be solved.

Specifically, the step S104 includes:

step S1041, for the image features of the corresponding cerebrovascular segmentation data of all the cerebrovascular knowledge data after layering processing, screening the algorithm intensively adapted by the vascular preprocessing algorithm to quantize the corresponding cerebrovascular segmentation data, and extracting to obtain the corresponding layering quantization features; the hierarchy of hierarchical quantization features includes: vessel segmentation, image feature data and measurement feature data; the blood vessel preprocessing algorithm set is a set of algorithms for processing blood vessel images and extracting blood vessel characteristics, and comprises the following steps: edge operators, texture feature extraction algorithms, density distribution, threshold segmentation, geometry measurements and filters.

In this embodiment, the specific configuration of the vascular preprocessing algorithm set includes:

1. an edge operator comprising: canny operator, sobel operator and Sift operator;

2. the texture feature extracted by the texture feature extraction algorithm comprises the following steps: a Hessian matrix and haar wavelet features;

3. the density distribution is histogram statistics;

4. threshold segmentation, comprising: global threshold segmentation, otsu threshold segmentation, and iterative threshold segmentation;

5. an element of geometric measurement, comprising: surface curvature, projected curvature, length, and angle;

6. types of filters, including: gaussian filter and laplacian filter.

It should be noted that, the specific configuration of the blood vessel pretreatment algorithm set is only used as an illustration, and is not limited thereto, and is adaptively adjusted according to the actual application requirements. The adopted vascular preprocessing algorithm set can acquire different hierarchical features and image features, so that the diversity of the features is improved, and the quality of multi-level vascular knowledge fusion data is improved to a certain extent.

Step S1042 sets the weight of the hierarchical quantization feature according to the blood vessel partition of the brain blood vessel knowledge data and the hierarchical processing result of the blood vessel partition.

It should be noted that, in this embodiment, the weight setting form of the hierarchical quantization feature is not specifically limited, and is determined according to the actual application requirement. For example, in the detection of cerebral aneurysms, the primary requirement is the detection of the specific location of the aneurysm lesion in the relevant brain image data. The probability of the aneurysm appearing at a specific position can be counted after layering treatment or weight setting can be carried out according to the experience of doctors. Firstly, obtaining input data of a cerebral aneurysm detection model based on the method, namely performing layering treatment of blood vessel segmentation and blood vessel segmentation on obtained related text data and image data; according to the priori knowledge of the aneurysm, the layering processing result with large influence on the detection precision of the aneurysm focus part is given a larger weight, and the layering processing result with small influence on the detection precision is given a smaller weight. Specifically, the weight of each site is set according to the probability of occurrence of the aneurysm. For example, the probability of cerebral aneurysm at anterior cerebral artery ACA is 40%, middle cerebral artery MCA is 34%, post-cerebral circulation PCOM is 20%, post-cerebral artery PCA is 4%, and other regions are 2%, and the ratio is normalized, and then the weight assignment is performed for each region, which is merely illustrative and not limiting.

Specifically, the step S106 includes:

step S1061, performing separation of a plurality of nodes on the data fusion feature map based on the positions of the bifurcation points of the blood vessels, to obtain a node subgraph, and obtaining position information of the corresponding nodes, where the position information includes: the vascular region, the vascular segment, and the length from the bifurcation point of the blood vessel to which the coordinates belong.

Step S1062, performing position coding on each separated node, where the position coding is used to represent a mapping relationship between each node and the data fusion feature map, and two-dimensional position coordinates of points on the central line data of the cerebrovascular tree are the corresponding position codes.

According to the embodiment, based on the prior knowledge of blood vessel partition and segmentation, the characteristics of each level are subjected to graph data construction, and a plurality of subgraphs containing accurate position information are used as input data of a preset model, so that the input requirement of the preset model can be met, and the training process of the model is accelerated to a certain extent.

Specifically, the process of obtaining optimal multi-level vascular knowledge fusion data according to the multi-level vascular knowledge fusion data and the performance of the preset model in the step S107 includes: the multi-level vascular knowledge fusion data is used as input data of a preset model, a weighted set of sensitivity and specificity is used as a loss function of the model, the loss function is minimized by training the model, so that the model obtains optimal performance, and optimal multi-level vascular knowledge fusion data is obtained.

It should be noted that, the preset model in this embodiment may be a detection model or a classification model related to medicine. The models which can be adopted by the preset model include: the full convolution network model, the dual branch network model, and the U-Net model are merely illustrative and not limiting.

In this embodiment, the optimal multi-level vascular knowledge fusion data is determined by the performance of the preset model, so that the optimal processing path of the multi-level vascular knowledge fusion data is defined, the problem of optimizing the knowledge characterization method in the knowledge-guided focus detection is solved, and the focus detection effect is improved.

In a specific embodiment, referring to fig. 2, based on a cerebral aneurysm detection model, the data preprocessing adopts the multi-level data fusion method based on cerebral vascular knowledge provided in this embodiment to perform data fusion processing. As can be seen from the figure, the brain image data and the brain blood vessel knowledge data obtained in this embodiment are brain artery text knowledge data, brain aneurysm knowledge data and TOF MRA image data, respectively.

The preset model of the embodiment adopts a U-Net model, and the specific flow of the multi-level data fusion method based on the brain blood vessel knowledge is shown in figure 3. As can be seen from the figure, in this embodiment, the brain artery segmentation data is processed by using brain artery segmentation data, brain aneurysm knowledge data and hierarchical data thereof, combining brain artery vessel tree central line data and a vessel preprocessing algorithm set, and then, sub-nodes are generated by combining a vessel skeleton feature map and related data compact data cross fusion processing, so as to obtain a corresponding data fusion feature map. And separating each node in the data fusion feature map to construct each node subgraph. Updating the node weights of the node subgraphs to obtain corresponding weighted subgraphs, and fusing the weighted subgraphs to obtain a node weight adjustment feature map to form multi-level vascular knowledge fusion data; and obtaining optimal multi-level vascular knowledge fusion data according to the multi-level vascular knowledge fusion data and the performance of the U-Net model. Wherein the method comprises the steps of

In a specific embodiment, a multi-level data fusion method based on cerebrovascular knowledge includes:

1. the input data is TOF-MRA brain image data.

2. The input data is a text description of cerebral aneurysm: aneurysms are defined as three layers of the arterial wall that bulge out in an outward localized manner, often at the arterial bifurcation. About 85% is in the anterior circulation, 30% to 40% is from the anterior cerebral artery or anterior transport artery, 30% is in the posterior transport artery, 20% to 30% is in the middle cerebral artery branch, 5% to 10% is in the internal carotid artery. About 15% originate from the posterior circulation, including the basal tip, the upper cerebellum artery, and the lower cerebellum artery. Aneurysms on CT represent a well-defined, somewhat higher-density lesion, sometimes with surrounding calcification.

3. And performing cerebral artery segmentation on TOF-MRA brain image data, namely sequentially performing blood vessel enhancement, automatic bone removal, seed point extraction, seed point region density statistics and region growth treatment, and finally obtaining a cerebral blood vessel segmentation result.

4. Based on the result of the brain blood vessel segmentation, the central line extraction is carried out to obtain the central line of the brain blood vessel tree, and the central line comprises points on the central line and the directional connection relation between the points.

5. A preprocessing algorithm set of cerebral arteries is collected.

6. Hierarchical quantization features comprising the following hierarchy:

(1) Vascular zoning (e.g., anterior cerebral artery, middle cerebral artery, basilar artery, etc.);

(2) Vessel segmentation (e.g., anterior artery A1, A2 segments, middle artery M1, M2 segments);

(3) Image feature data (e.g., contour, density);

(4) Characteristic data (e.g., length, angle, curvature, etc.) are measured.

Layering the data according to the above layers, specifically including: firstly, classifying the knowledge corresponding to different layers in the text knowledge, and quantifying the knowledge by using a preprocessing algorithm according to each specific knowledge entity, for example:

(1) The three layers of the artery wall are outwards limited and bulge, and Canny profile data are added to corresponding image features; surface curvature data is added to the measured features.

(2) Often at arterial branches, corresponding to measured features, surface curvature and projection curvature data are increased.

(3) About 85% in the anterior circulation, 30% to 40% from the anterior cerebral artery or anterior transport artery, 30% in the posterior transport artery, 20% to 30% in the middle cerebral artery branch, 5% to 10% in the internal carotid artery, about 15% from the posterior circulation, including basal cusp, superior cerebellar artery, inferior cerebellar artery; and corresponding to different characteristics of the subareas and the segmentations, and giving different weights to the subarea according to the prior knowledge of the subarea segmentation.

(4) The circle density showing clear boundary is slightly higher, corresponding to the image feature, and Sobel profile data is increased.

7. And according to the quantization relation, taking a point on a central line as a node, taking the central line as an edge between the nodes, taking the quantization characteristic as node information, and constructing a corresponding data fusion characteristic diagram.

8. And carrying out subgraph construction on the data fusion characteristic diagram based on the blood vessel bifurcation point.

9. In the separation process of the subgraph, a piece of position related information, namely the vascular region, the vascular segment and the length of the distance bifurcation point, to which the coordinates belong, is added.

10. Position coding, which is to perform position coding based on the mapping relation between the node characteristics and the original image; the node is derived from a point on the central line data of the cerebrovascular tree, and the two-dimensional position coordinate of the point is the position code of the node.

11. After the sub-image data is subjected to position coding, the characteristic and position coding can be spliced into a characteristic image, the characteristic image is sent to a U-net model for training, a final classification layer is removed, and a probability image generated by a Softmax layer is output as a result.

12. Probability weights of a plurality of positions in the probability map corresponding to different nodes are accumulated to obtain weighted scores of the corresponding nodes, so that a new node weighted map is formed; and determining an optimal node weighted graph through the sensitivity and specificity performance indexes of the U-net model.

13. And multiplying the optimal node weighted graph with the atomic graph to obtain final fusion data. The method solves the problem of optimizing the knowledge characterization method in the knowledge-guided focus detection, and improves the focus detection effect to a certain extent.

The embodiment also provides a multi-level data fusion system based on the brain blood vessel knowledge, which is used for realizing the embodiment and the preferred implementation mode, and the description is omitted. The term "module" as used below may be a combination of software and/or hardware that implements a predetermined function. While the system described in the following embodiments is preferably implemented in software, implementation in hardware, or a combination of software and hardware, is also possible and contemplated.

The invention provides a multi-level data fusion system based on cerebrovascular knowledge, as shown in fig. 4, the system comprises:

the data acquisition module 401 is configured to acquire brain image data and brain blood vessel knowledge data.

The data processing module 402 is configured to perform segmentation processing on the brain image data to obtain brain blood vessel segmentation data, and perform centerline extraction on the brain blood vessel segmentation data to obtain brain blood vessel tree centerline data.

The data layering module 403 is configured to perform layering processing on the cerebrovascular segmentation data and the cerebrovascular knowledge data respectively.

And the feature quantization module 404 is configured to perform feature quantization on the segmented cerebrovascular segmentation data by using a vascular preprocessing algorithm set according to the segmented cerebrovascular knowledge data, obtain corresponding hierarchical quantization features, and set corresponding weights.

And the feature fusion module 405 is configured to combine the central line data of the cerebrovascular tree, and perform feature map fusion according to the hierarchical quantization features, so as to obtain a data fusion feature map.

And the subgraph construction module 406 is configured to perform subgraph construction based on the positions of the nodes in the data fusion feature map, so as to obtain subgraphs of the nodes.

The weight optimization module 407 is configured to update node weights of the node subgraphs to obtain corresponding weighted subgraphs, and fuse the weighted subgraphs to obtain a node weight adjustment feature map, so as to form multi-level vascular knowledge fusion data; and obtaining node weight information according to the multi-level blood vessel knowledge fusion data and the performance of the preset model to obtain an optimal node weight adjustment characteristic diagram, so as to form optimal multi-level blood vessel knowledge fusion data.

Further functional descriptions of the above respective modules are the same as those of the above corresponding embodiments, and are not repeated here. The multi-level data fusion system based on the cerebral vascular knowledge can carry out knowledge structuring layering study on cerebral artery and cerebral aneurysm based on the cerebral vascular related knowledge, and define the optimal processing path of the multi-level vascular knowledge fusion data by studying the realization path of the structuring layering manner in brain image data, solve the problem of optimizing a knowledge characterization method in knowledge-guided focus detection, and improve the focus detection effect to a certain extent.

An embodiment of the present invention further provides a computer device, referring to fig. 5, fig. 5 is a schematic structural diagram of the above-mentioned controller provided in an alternative embodiment of the present invention, as shown in fig. 5, where the controller includes: one or more processors 10, memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are communicatively coupled to each other using different buses and may be mounted on a common motherboard or in other manners as desired. The processor may process instructions executing within the computer device, including instructions stored in or on memory to display graphical information of the GUI on an external input/output device, such as a display device coupled to the interface. In some alternative embodiments, multiple processors and/or multiple buses may be used, if desired, along with multiple memories and multiple memories. Also, multiple computer devices may be connected, each providing a portion of the necessary operations (e.g., as a server array, a set of blade servers, or a multiprocessor system). One processor 10 is illustrated in fig. 5.

The processor 10 may be a central processor, a network processor, or a combination thereof. The processor 10 may further include a hardware chip, among others. The hardware chip may be an application specific integrated circuit, a programmable logic device, or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general-purpose array logic, or any combination thereof.

Wherein the memory 20 stores instructions executable by the at least one processor 10 to cause the at least one processor 10 to perform a method for implementing the embodiments described above.

The memory 20 may include a storage program area that may store an operating system, at least one application program required for functions, and a storage data area; the storage data area may store data created according to the use of the computer device, etc. In addition, the memory 20 may include high-speed random access memory, and may also include non-transitory memory, such as at least one magnetic disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, memory 20 may optionally include memory located remotely from processor 10, which may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

Memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk, or solid state disk; the memory 20 may also comprise a combination of the above types of memories.

The controller also includes a communication interface 30 for the master control chip to communicate with other devices or communication networks.

The embodiments of the present invention also provide a computer readable storage medium, and the method according to the embodiments of the present invention described above may be implemented in hardware, firmware, or as a computer code which may be recorded on a storage medium, or as original stored in a remote storage medium or a non-transitory machine readable storage medium downloaded through a network and to be stored in a local storage medium, so that the method described herein may be stored on such software process on a storage medium using a general purpose computer, a special purpose processor, or programmable or special purpose hardware. The storage medium can be a magnetic disk, an optical disk, a read-only memory, a random access memory, a flash memory, a hard disk, a solid state disk or the like; further, the storage medium may also comprise a combination of memories of the kind described above. It will be appreciated that a computer, processor, microprocessor master chip or programmable hardware includes a storage component that can store or receive software or computer code that, when accessed and executed by the computer, processor or hardware, implements the methods illustrated by the embodiments described above.

Although embodiments of the present invention have been described in connection with the accompanying drawings, various modifications and variations may be made by those skilled in the art without departing from the spirit and scope of the invention, and such modifications and variations fall within the scope of the invention as defined by the appended claims.

Claims

1. A multi-level data fusion method based on cerebrovascular knowledge, the method comprising:

acquiring brain image data and brain vessel knowledge data;

dividing the brain image data to obtain brain blood vessel division data, and extracting the central line of the brain blood vessel division data to obtain brain blood vessel tree central line data;

respectively layering the cerebrovascular segmentation data and the cerebrovascular knowledge data;

2. The multi-level data fusion method based on cerebrovascular knowledge according to claim 1, wherein the cerebrovascular knowledge data is text data, comprising: clinical guidelines, medical-related books, clinical care records, and medical records; the brain image data includes: CT images, magnetic resonance angiography images, and time-fly magnetic resonance angiography images.

3. The multi-level data fusion method based on cerebrovascular knowledge as claimed in claim 2, wherein the process of dividing said brain image data to obtain brain vessel divided data and extracting the central line thereof to obtain the central line data of the brain vessel tree comprises:

4. The multi-level data fusion method based on cerebrovascular knowledge as claimed in claim 2, wherein the process of layering the cerebrovascular segmentation data and the cerebrovascular knowledge data respectively comprises:

vessel segmentation is performed again in each of the arterial regions.

5. The method for multi-level data fusion based on cerebrovascular knowledge according to claim 4, wherein the process of performing feature quantization on the segmented cerebrovascular segmentation data by using a vascular preprocessing algorithm set according to the segmented cerebrovascular knowledge data to obtain corresponding hierarchical quantization features and setting corresponding weights thereof comprises the following steps:

For the image characteristics of the corresponding cerebral vascular segmentation data of all cerebral vascular knowledge data after layering processing, screening an algorithm intensively adapted to a vascular preprocessing algorithm to quantize the corresponding cerebral vascular segmentation data, and extracting to obtain corresponding layering quantization characteristics; the hierarchy of hierarchical quantization features comprises: vessel segmentation, image feature data and measurement feature data; the blood vessel preprocessing algorithm set is an algorithm set containing a plurality of blood vessel images and extracting blood vessel characteristics, and comprises the following steps: edge operators, texture feature extraction algorithms, density distribution, threshold segmentation, geometry measurement and filters;

6. The multi-level data fusion method based on cerebrovascular knowledge according to claim 1, wherein the process of constructing sub-graphs based on the positions of nodes in the data fusion feature graph to obtain sub-graphs of the nodes comprises the following steps:

And carrying out position coding on each separated node, wherein the position coding is used for representing the mapping relation between each node and the data fusion characteristic diagram, and the two-dimensional position coordinates of the point on the central line data of the cerebrovascular tree are the corresponding position coding.

7. The method for multi-level data fusion based on cerebrovascular knowledge according to claim 1, wherein the process for obtaining optimal multi-level vessel knowledge fusion data according to the performance of the multi-level vessel knowledge fusion data and a preset model comprises the following steps:

8. A multi-level data fusion system based on cerebrovascular knowledge, the system comprising:

the data processing module is used for carrying out segmentation processing on the brain image data to obtain brain blood vessel segmentation data, and carrying out central line extraction on the brain blood vessel segmentation data to obtain brain blood vessel tree central line data;

The data layering module is used for respectively layering the cerebral vascular segmentation data and the cerebral vascular knowledge data;

the sub-graph construction module is used for carrying out sub-graph construction based on the positions of all nodes in the data fusion characteristic graph to obtain all node sub-graphs;

9. A computer device, comprising: a memory and a processor, the memory and the processor are in communication connection, the memory stores computer instructions, and the processor executes the computer instructions, thereby executing the multi-level data fusion method based on cerebrovascular knowledge according to any one of claims 1 to 7.

10. A computer-readable storage medium having stored thereon computer instructions for causing a computer to perform the multi-level data fusion method based on cerebrovascular knowledge according to any one of claims 1 to 7.