CN116664938A

CN116664938A - Coronary plaque type determining method and device, electronic equipment and storage medium

Info

Publication number: CN116664938A
Application number: CN202310667293.8A
Authority: CN
Inventors: 张瑜; 马骏; 郑凌霄; 兰宏志
Original assignee: Shenzhen Raysight Intelligent Medical Technology Co Ltd
Current assignee: Shenzhen Raysight Intelligent Medical Technology Co Ltd
Priority date: 2023-06-06
Filing date: 2023-06-06
Publication date: 2023-08-29

Abstract

The application provides a method and a device for determining coronary plaque type, electronic equipment and a storage medium, wherein the method for determining coronary plaque type comprises the following steps: inputting any one blood vessel section and corresponding unlabeled multiple intracavity image sections in the obtained unlabeled coronary artery sample image into an image registration training module to obtain a target intracavity image section which is most matched with each blood vessel section in the coronary artery sample image; inputting any one blood vessel section in the coronary artery sample image and a target intracavity image section corresponding to the blood vessel section into a plaque identification training module for training the plaque identification training module; and determining the trained plaque recognition training module as a coronary plaque recognition model, and inputting the coronary artery image into the coronary plaque recognition model to obtain a coronary plaque type result. By adopting the technical scheme provided by the application, the accuracy of CTA (CTA) identification of coronary plaque types can be improved.

Description

Coronary plaque type determining method and device, electronic equipment and storage medium

Technical Field

The present application relates to the field of medical image processing, and in particular, to a method and apparatus for determining a coronary plaque type, an electronic device, and a storage medium.

Background

Coronary plaque (hereinafter referred to as "coronary plaque") is a hardened plaque existing in the coronary artery, and is mostly caused by increased blood fat, blood impurity accumulation, blood sugar rise and the like of a human body, and the coronary plaque can generate secondary hypertension, influence local blood supply of the human body, further increase the burden of the heart and threaten the physical health of the patient; coronary plaque is of various types, so that it is necessary to determine a specific type of coronary plaque in a patient, and a targeted treatment regimen may be prescribed to the patient.

Currently, there are two ways to identify the type of coronary plaque, one is by invasive examination of the coronary cavity image, such as intravascular ultrasound (intravascular ultrasound, IVUS) and optical coherence tomography (optical coherence tomography, OCT), which is highly accurate but costly and at a certain risk, and is not suitable as a routine examination. The other is the noninvasive examination of CT angiography (Computed Tomography Angiography, CTA), which has the characteristics of wide applicability, low price, no wound, simple operation and the like, and is widely applied to diagnosis of various diseases, but CTA can only identify three plaque types of calcified plaque, non-calcified plaque and mixed plaque, so that CTA has lower resolution for coronary plaque identification. Therefore, how to improve accuracy of CTA in identifying coronary plaque types becomes a urgent problem to be solved.

Disclosure of Invention

In view of the above, the present application is directed to a method, an apparatus, an electronic device, and a storage medium for determining a coronary plaque type, which can improve generalization capability and effect of a model by matching unlabeled coronary artery sample images (CTA) with intra-lumen sample images (e.g., OCT), using large-scale unlabeled data, and directly using the matching result for plaque recognition at the next stage, and in a plaque recognition training stage, using features (capable of recognizing more and finer types of plaque types) of the intra-lumen sample images with high resolution to guide the model to recognize more features in the coronary artery sample images, thereby improving accuracy of CTA recognition of the coronary plaque type.

The application mainly comprises the following aspects:

in a first aspect, an embodiment of the present application provides a method for determining a coronary plaque type, where the determining method includes:

the determining method comprises the following steps:

acquiring a coronary artery image;

inputting the coronary artery image into a coronary artery plaque identification model, and outputting a coronary artery plaque type result corresponding to the coronary artery image;

the coronary plaque recognition model is obtained through training the following steps:

Acquiring a coronary artery sample image and an intra-cavity sample image corresponding to the coronary artery sample image from a pre-created sample data set; wherein the coronary artery sample image and the intra-cavity sample image are unlabeled images; the coronary artery sample image comprises a plurality of blood vessel sections along the blood vessel center line; the intra-cavity sample image comprises a plurality of intra-cavity image sections along the shooting direction;

inputting any one blood vessel section in the coronary artery sample image and a plurality of intra-cavity image sections in the intra-cavity sample image corresponding to the coronary artery sample image into an image registration training module to train the image registration training module, so as to obtain a trained image registration training module;

obtaining a target intra-cavity image section which is most matched with each blood vessel section in the coronary artery sample image from a plurality of intra-cavity image sections corresponding to the coronary artery sample image through the trained image registration training module;

inputting any one blood vessel section in the coronary artery sample image and a target intracavity image section corresponding to the blood vessel section into a plaque identification training module to train the plaque identification training module, so as to obtain a trained plaque identification training module;

And determining the trained plaque recognition training module as a coronary plaque recognition model.

Further, the step of inputting any one blood vessel section in the coronary artery sample image and a plurality of intra-cavity image sections in the intra-cavity sample image corresponding to the coronary artery sample image into an image registration training module to train the image registration training module to obtain a trained image registration training module includes:

inputting any blood vessel section in the coronary artery sample image into a first feature extraction layer of an image registration training module to obtain the coronary artery feature of the blood vessel section;

inputting a plurality of intra-cavity image sections in the intra-cavity sample images corresponding to the coronary artery sample images into a second feature extraction layer of an image registration training module to obtain intra-cavity image features of each intra-cavity image section;

determining a target intra-lumen image section which is most matched with the blood vessel section in the plurality of intra-lumen image sections based on the coronary artery characteristics of the blood vessel section and the intra-lumen image characteristics of each intra-lumen image section;

determining a label of each intra-cavity image section corresponding to the blood vessel section based on the target intra-cavity image section;

Obtaining a first loss function based on the labels of each intra-cavity image section;

determining whether the first loss function converges;

if not, updating parameters of the image registration training module, and acquiring a next coronary artery sample image and an intra-cavity sample image corresponding to the next coronary artery sample image to continuously train the image registration training module until the first loss function converges;

if yes, a trained image registration training module is obtained.

Further, the step of inputting any one blood vessel section in the coronary artery sample image and the target intracavity image section corresponding to the blood vessel section into the plaque identification training module to train the plaque identification training module to obtain a trained plaque identification training module includes:

inputting any one blood vessel section and a target intracavity image section corresponding to the blood vessel section in the coronary artery sample image into a plaque identification training module, and extracting intracavity image characteristics of the coronary artery characteristics in the blood vessel section and the target intracavity image section corresponding to the blood vessel section;

acquiring a current influence coefficient, and fusing the coronary artery characteristics in the section of the blood vessel with the intra-cavity image characteristics of the target intra-cavity image section corresponding to the section of the blood vessel through the current influence coefficient to obtain fusion characteristics;

After the fusion characteristics are subjected to full-connection layer and normalization treatment, the probability that coronary plaque represented by the fusion characteristics belongs to each preset classification is obtained;

obtaining a second loss function based on the probability that the coronary plaque represented by the fusion characteristic belongs to each preset classification;

determining whether the second loss function converges;

if not, updating parameters and current influence coefficients of the plaque identification training module, and acquiring a next coronary artery sample image and a target intracavity sample image corresponding to the next coronary artery sample image to continue training the plaque identification training module until the second loss function converges;

if yes, a trained plaque identification training module is obtained.

Further, the step of inputting the coronary artery image into the coronary artery plaque identification model and outputting the coronary artery plaque type result corresponding to the coronary artery image includes:

inputting each blood vessel section in the coronary artery image into a coronary plaque identification model for feature extraction to obtain the coronary artery feature of the blood vessel section in the coronary artery image;

After the coronary artery characteristics of the blood vessel section in the coronary artery image are subjected to full-connection layer and normalization treatment, obtaining the probability that coronary artery plaques represented by the coronary artery characteristics of the blood vessel section in the coronary artery image belong to each preset classification;

and in the probability, determining a preset classification corresponding to the probability with the largest numerical value as a coronary plaque type result corresponding to the blood vessel section in the coronary artery image.

Further, a sample dataset is created by:

acquiring a plurality of initial images of a patient for atraumatic coronary scanning and intra-cavity sample images of the patient for atraumatic intra-cavity scanning;

performing data processing on a plurality of initial images of the patient to obtain a coronary artery sample image of the patient;

for each patient, storing a coronary artery sample image of the patient and an intra-cavity sample image of the patient as a data pair for the patient;

after storing the data pairs for each patient, a sample data set is obtained.

Further, the step of performing data processing on the plurality of initial images of the patient to obtain a coronary artery sample image of the patient includes:

Preprocessing a plurality of initial images of the patient to obtain a three-dimensional blood vessel image of the patient;

dividing the three-dimensional blood vessel image to obtain a coronary blood vessel image;

extracting a blood vessel center line in the coronary blood vessel image, and determining a blood vessel section tangential to the blood vessel center line along the blood vessel center line;

and acquiring an image interpolation result of each blood vessel section, and splicing the image interpolation results of each blood vessel section to obtain a coronary artery sample image of the patient.

Further, the plaque recognition training module comprises a third feature extraction layer and a fourth feature extraction layer; the third feature extraction layer is used for extracting coronary artery features in the blood vessel section; the fourth feature extraction layer is used for extracting intra-cavity image features of the target intra-cavity image section; the parameters of the plaque identification training module comprise parameters of a third feature extraction layer and parameters of a fourth feature extraction layer; the third feature extraction layer and the fourth feature extraction layer are obtained by:

taking parameters in a first feature extraction layer in the trained image registration training module as parameters of a network used for extracting coronary artery features in a blood vessel section in the plaque recognition training module, and determining the network used for extracting the coronary artery features in the blood vessel section with the parameters as a third feature extraction layer;

And taking the parameters in the second feature extraction layer in the trained image registration training module as parameters of a network for extracting the intra-cavity image features of the target intra-cavity image section in the plaque recognition training module, and determining the network with the parameters for extracting the intra-cavity image features of the target intra-cavity image section as a fourth feature extraction layer.

In a second aspect, an embodiment of the present application further provides a device for determining a coronary plaque type, where the determining device includes:

the acquisition module is used for acquiring coronary artery images;

the determining module is used for inputting the coronary artery image into the coronary artery plaque identification model and outputting a coronary artery plaque type result corresponding to the coronary artery image;

the training module is used for training the coronary plaque recognition model; the training module comprises an acquisition unit, a first training unit, a matching unit, a second training unit and a determining unit;

the acquisition unit is used for acquiring a coronary artery sample image and an intra-cavity sample image corresponding to the coronary artery sample image from a pre-created sample data set; wherein the coronary artery sample image and the intra-cavity sample image are unlabeled images; the coronary artery sample image comprises a plurality of blood vessel sections along the blood vessel center line; the intra-cavity sample image comprises a plurality of intra-cavity image sections along the shooting direction;

The first training unit is used for inputting any one blood vessel section in the coronary artery sample image and a plurality of intra-cavity image sections in the intra-cavity sample images corresponding to the coronary artery sample image into the image registration training module to train the image registration training module, so as to obtain a trained image registration training module;

the matching unit is used for obtaining a target intra-cavity image section which is matched with each blood vessel section in the coronary artery sample image in a plurality of intra-cavity image sections corresponding to the coronary artery sample image through the trained image registration training module;

the second training unit is used for inputting any one blood vessel section in the coronary artery sample image and a target cavity image section corresponding to the blood vessel section into the plaque identification training module to train the plaque identification training module, so as to obtain a trained plaque identification training module;

and the determining unit is used for determining the trained plaque recognition training module as a coronary plaque recognition model.

In a third aspect, an embodiment of the present application further provides an electronic device, including: the computer-readable medium comprises a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, the processor and the memory communicate through the bus when the electronic device is running, and the machine-readable instructions, when executed by the processor, perform the steps of the method for determining coronary plaque type as described above.

In a fourth aspect, embodiments of the present application also provide a computer readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the method for determining a type of coronary plaque as described above.

The embodiment of the application provides a method, a device, electronic equipment and a storage medium for determining coronary plaque types, wherein the method comprises the following steps: acquiring a coronary artery image; inputting the coronary artery image into a coronary artery plaque identification model, and outputting a coronary artery plaque type result corresponding to the coronary artery image; the coronary plaque recognition model is obtained through training the following steps: acquiring a coronary artery sample image and an intra-cavity sample image corresponding to the coronary artery sample image from a pre-created sample data set; wherein the coronary artery sample image and the intra-cavity sample image are unlabeled images; the coronary artery sample image comprises a plurality of blood vessel sections along the blood vessel center line; the intra-cavity sample image comprises a plurality of intra-cavity image sections along the shooting direction; inputting any one blood vessel section in the coronary artery sample image and a plurality of intra-cavity image sections in the intra-cavity sample image corresponding to the coronary artery sample image into an image registration training module to train the image registration training module, so as to obtain a trained image registration training module; obtaining a target intra-cavity image section which is most matched with each blood vessel section in the coronary artery sample image from a plurality of intra-cavity image sections corresponding to the coronary artery sample image through the trained image registration training module; inputting any one blood vessel section in the coronary artery sample image and a target intracavity image section corresponding to the blood vessel section into a plaque identification training module to train the plaque identification training module, so as to obtain a trained plaque identification training module; and determining the trained plaque recognition training module as a coronary plaque recognition model.

In this way, the technical scheme provided by the application can be used for matching unlabeled coronary artery sample images (CTA) with intra-cavity sample images (e.g. OCT), improving the generalization capability and effect of the model by using large-scale unlabeled data, directly using the matching result for plaque identification in the next stage, and guiding the model to identify more features in the coronary artery sample images by using features (capable of identifying more and finer types of plaque types) with high resolution of the intra-cavity sample images in the plaque identification training stage, thereby improving the accuracy of CTA for identifying coronary plaque types.

In order to make the above objects, features and advantages of the present application more comprehensible, preferred embodiments accompanied with figures are described in detail below.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the embodiments will be briefly described below, it being understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and other related drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flowchart of a method for determining a type of coronary plaque according to an embodiment of the present application;

FIG. 2 is a flow chart of another method for determining coronary plaque type according to an embodiment of the present application;

FIG. 3 is a schematic diagram of an image registration module workflow provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of a workflow of a plaque recognition training module according to an embodiment of the present application;

FIG. 5 is a block diagram showing a device for determining a type of coronary plaque according to an embodiment of the present application;

FIG. 6 is a diagram showing a second configuration of a coronary plaque type determination apparatus according to an embodiment of the present application;

fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present application more apparent, the technical solutions of the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application, and it should be understood that the drawings in the present application are for the purpose of illustration and description only and are not intended to limit the scope of the present application. In addition, it should be understood that the schematic drawings are not drawn to scale. A flowchart, as used in this disclosure, illustrates operations implemented according to some embodiments of the present application. It should be appreciated that the operations of the flow diagrams may be implemented out of order and that steps without logical context may be performed in reverse order or concurrently. Moreover, one or more other operations may be added to or removed from the flow diagrams by those skilled in the art under the direction of the present disclosure.

In addition, the described embodiments are only some, but not all, embodiments of the application. The components of the embodiments of the present application generally described and illustrated in the figures herein may be arranged and designed in a wide variety of different configurations. Thus, the following detailed description of the embodiments of the application, as presented in the figures, is not intended to limit the scope of the application, as claimed, but is merely representative of selected embodiments of the application. All other embodiments, which can be made by a person skilled in the art based on embodiments of the application without making any inventive effort, fall within the scope of the application.

In order to enable those skilled in the art to make and use the present disclosure, the following embodiments are provided in connection with a particular application scenario "determination of coronary plaque type", and it will be apparent to those skilled in the art that the general principles defined herein may be applied to other embodiments and application scenarios without departing from the spirit and scope of the present disclosure.

The method, the device, the electronic equipment or the computer readable storage medium can be applied to any scene requiring determination of the coronary plaque type, the embodiment of the application is not limited to specific application scenes, and any scheme using the method, the device, the electronic equipment and the storage medium for determining the coronary plaque type provided by the embodiment of the application is within the protection scope of the application.

It is noted that coronary plaque (hereinafter referred to as "coronary plaque") is a hardened plaque existing in the coronary artery, and is mostly caused by increased blood fat, accumulation of blood impurities, and increase of blood sugar, etc., which can generate secondary hypertension, affect local blood supply of human body, and further burden heart, and threaten physical health of patient; coronary plaque is of various types, so that it is necessary to determine a specific type of coronary plaque in a patient, and a targeted treatment regimen may be prescribed to the patient.

Based on the above, the application provides a method and a device for determining coronary plaque type, an electronic device and a storage medium, wherein the method for determining coronary plaque type comprises the following steps: acquiring a coronary artery image; inputting the coronary artery image into a coronary artery plaque identification model, and outputting a coronary artery plaque type result corresponding to the coronary artery image; the coronary plaque recognition model is obtained through training the following steps: acquiring a coronary artery sample image and an intra-cavity sample image corresponding to the coronary artery sample image from a pre-created sample data set; wherein the coronary artery sample image and the intra-cavity sample image are unlabeled images; the coronary artery sample image comprises a plurality of blood vessel sections along the blood vessel center line; the intra-cavity sample image comprises a plurality of intra-cavity image sections along the shooting direction; inputting any one blood vessel section in the coronary artery sample image and a plurality of intra-cavity image sections in the intra-cavity sample image corresponding to the coronary artery sample image into an image registration training module to train the image registration training module, so as to obtain a trained image registration training module; obtaining a target intra-cavity image section which is most matched with each blood vessel section in the coronary artery sample image from a plurality of intra-cavity image sections corresponding to the coronary artery sample image through the trained image registration training module; inputting any one blood vessel section in the coronary artery sample image and a target intracavity image section corresponding to the blood vessel section into a plaque identification training module to train the plaque identification training module, so as to obtain a trained plaque identification training module; and determining the trained plaque recognition training module as a coronary plaque recognition model.

In order to facilitate understanding of the present application, the technical solutions provided by the present application will be described in detail below with reference to specific embodiments.

Referring to fig. 1, fig. 1 is a flowchart of a method for determining a coronary plaque type according to an embodiment of the present application, where the method includes:

s101, acquiring a coronary artery image;

in this step, the coronary artery image is coronary artery CTA, a medical imaging technique, and a Computer Tomography (CT) is used to generate an image of the three-dimensional coronary artery. This technique can show the anatomy, stenosis or occlusion of the coronary arteries for assessment of coronary heart disease, heart valve disease and other cardiovascular diseases. Coronary CTAs can provide high resolution vascular imaging, typically by intravenous injection of contrast media, followed by image reconstruction using a CT scanner. CTA can generally identify only three plaque types, calcified plaque, non-calcified plaque and mixed plaque.

S102, inputting the coronary artery image into a coronary artery plaque identification model, and outputting a coronary artery plaque type result corresponding to the coronary artery image;

in the step, the coronary plaque recognition model comprises an image registration training module and a plaque recognition training module in the training process, and the coronary plaque recognition model comprises the plaque recognition training module in the application process.

It should be noted that, before step S102 is performed, a coronary plaque recognition module needs to be obtained, please refer to fig. 2, fig. 2 is a flowchart of another method for determining a coronary plaque type according to an embodiment of the present application, and as shown in fig. 2, the coronary plaque recognition module is obtained through training the following steps:

s201, acquiring a coronary artery sample image and an intra-cavity sample image corresponding to the coronary artery sample image from a pre-created sample data set;

in the step, the coronary artery sample image and the intra-cavity sample image are unlabeled images; the coronary artery sample image comprises a plurality of blood vessel sections along the blood vessel center line; the intra-cavity sample image comprises a plurality of intra-cavity image sections along the shooting direction.

Here, the intra-cavity sample image is a coronary intra-cavity image, and is mainly divided into IVUS and OCT. The IVUS is used for sending the miniature ultrasonic probe into a blood vessel cavity through a catheter technology, scanning 360 degrees in the blood vessel, clearly displaying the structure and lesions of the blood vessel of the heart through a display screen, and displaying the cross-sectional image of the blood vessel, so as to provide an image in the body blood vessel cavity, unlike the coronary artery imaging which displays the coronary artery through the outline of the cavity filled with contrast agent. The IVUS can accurately measure the diameters of a lumen and a blood vessel and judge the severity and the nature of the lesion, and plays a very important role in improving the understanding of coronary lesions and guiding interventional therapy. OCT is a high resolution cardiovascular imaging technique that is based on the principle of placing an imaging catheter inside a blood vessel, converting internal structural information into a high resolution image by analyzing the time delay of reflection from an internal light source to the vessel wall tissue, and rendering the image on a display. OCT uses optical coherence tomography to generate high-definition cardiovascular structure images, can provide high-resolution images, is used for evaluating coronary intima thickness, plaque characteristics, plaque stability and the like, and has higher sensitivity and specificity for diagnosing and evaluating cardiovascular diseases such as coronary heart disease, atherosclerosis and the like. OCT is typically guided by a guidewire to introduce an optical probe into the coronary artery and then use optical coherence tomography to generate high resolution images, where calcified plaque, lipid plaque, fibrous plaque, mixed plaque, thrombus, etc. are typically seen by the intracavity image.

It should be noted that the sample data set is created by:

1) Acquiring a plurality of initial images of a patient subjected to atraumatic coronary scanning and intra-cavity sample images of the patient subjected to atraumatic intra-cavity scanning;

in the step, a patient performs coronary artery CTA scanning, which is a non-invasive examination method, by a Computer Tomography (CT), and in the scanning process, the patient needs to lie on a scanning bed, and the CT machine rotates around the body to generate a series of X-ray images, namely a plurality of initial images; the coronary artery sample image of the patient is obtained through a plurality of initial images, and corresponding intra-cavity sample images are found according to the patient and the coronary artery sample images collected in the CTA, for example, the blood vessel LAD of the patient A exists in CTA data, and the blood vessel is also subjected to intra-cavity image examination, so that an effective data pair can be formed.

Here, the general patient is first subjected to CTA examination, so that the cost is low and the injury is low; when CTA confirms that a lesion exists, the CTA needs to further observe the lesion, and then the CTA does not need to be used for making an intracavity image, so that the cost is high, and the CTA needs to be detected on an operating table, so that the CTA has a certain damage to a human body. When creating the sample dataset, patient data can be acquired that is both CTA exam and intra-luminal imaging.

2) Performing data processing on a plurality of initial images of the patient to obtain a coronary artery sample image of the patient;

the step of performing data processing on a plurality of initial images of a patient to obtain a coronary artery sample image of the patient includes:

(1) Preprocessing a plurality of initial images of the patient to obtain a three-dimensional blood vessel image of the patient;

in this step, preprocessing is a process in which a computer generates three-dimensional blood vessel images from these initial images.

(2) Dividing the three-dimensional blood vessel image to obtain a coronary blood vessel image;

in the step, in the three-dimensional blood vessel image, the coronary blood vessel image can be extracted by a method such as threshold segmentation.

(3) Extracting a blood vessel center line in the coronary blood vessel image, and determining a blood vessel section perpendicular to the blood vessel center line along the blood vessel center line;

in the step, based on the segmentation result of the step (2), the central line of the blood vessel is obtained through algorithms such as minimum loss distance and the like, and after the central line of the blood vessel is obtained, some post-treatments such as branch removal, smoothing, gray level transformation and the like can be performed, so that a better visual effect is obtained; then along the centerline, a section perpendicular to the centerline tangential direction (i.e., a vessel section) is acquired.

(4) And acquiring an image interpolation result of each blood vessel section, and splicing the image interpolation results of each blood vessel section to obtain a coronary artery sample image of the patient.

In this step, the image interpolation result of each section obtained in the step (3) is obtained, and the sections are spliced to obtain a final blood vessel straightened image (i.e., a coronary artery sample image), where the intra-cavity sample image itself is already a straightened image along the shooting direction, so that no straightening process is required.

3) Storing, for each patient, a coronary artery sample image of the patient and an intra-cavity sample image of the patient as a data pair for the patient;

4) After storing the data pairs of each patient, a sample data set is obtained.

S202, inputting any blood vessel section in the coronary artery sample image and a plurality of intra-cavity image sections in an intra-cavity sample image corresponding to the coronary artery sample image into an image registration training module to train the image registration training module, and obtaining a trained image registration training module;

in this step, an image registration module needs to be built, and an intra-cavity image feature extraction network (i.e. a second feature extraction layer) is first built, and because the resolution of the intra-cavity image is higher (512×512 is common), a deeper network, such as res net, is adopted, which is not limited in particular. Whereas CTA is small in cross-section (typically 64 x 64), the coronary feature extraction network (i.e., the first feature extraction layer) may employ a shallower neural network, such as VGG or the like. The final output layer feature numbers of the two are unified, for example, the feature vectors with the length of 1024 are set.

The method includes the steps of inputting any one blood vessel section in the coronary artery sample image and a plurality of intra-cavity image sections in intra-cavity sample images corresponding to the coronary artery sample image into an image registration training module to train the image registration training module, and obtaining a trained image registration training module, wherein the method includes the steps of:

s2021, inputting any one blood vessel section in the coronary artery sample image into a first feature extraction layer of an image registration training module to obtain the coronary artery feature of the blood vessel section;

in this step, the first feature extraction layer is input as a single CTA section (vessel section), while the second feature extraction layer is input as a plurality of intra-luminal image sections from random several sections in the unified vessel of the same patient, a single CTA section is selected by Patch _CTA The method comprises the following steps:

Patch _CTA ＝f _CTA (n)|n＝random(N，1)；

wherein f _CTA (N) represents the nth cross section (blood vessel cross section) selected from the CTA blood vessel straightening image (coronary artery sample image), and random (N, 1) represents the total number of cross sections of the CTA blood vessel straightening image selected randomly from 1 to N by 1 integer.

S2022, inputting a plurality of intra-cavity image sections in the intra-cavity sample images corresponding to the coronary artery sample images into a second feature extraction layer of the image registration training module to obtain intra-cavity image features of each intra-cavity image section;

In the step, the selection modes of the image sections in the cavities are as follows:

Patch _Intra ＝f _Intra ({m})|{m}＝random(M，m)；

wherein { M } is a set of M integers, and random (M, M) represents a random selection of M integers from 1 to M, M being the total number of intra-cavity image sections in the intra-cavity sample image.

S2023, determining a target intra-cavity image section which is most matched with the blood vessel section in the plurality of intra-cavity image sections based on the coronary artery characteristics of the blood vessel section and the intra-cavity image characteristics of each intra-cavity image section;

the output of the first feature extraction layer is illustratively a feature vector F of dimension 1024×1 _CTA ∈R ^1024×1 While the output of the second feature extraction layer is the feature vector F of m x 1024 _Intra ∈R ^m×1024 Then, the cosine similarity r (F _CTA ，F _Intra )：

Wherein F is _Intra Intra-cavity image features representing intra-cavity image cross sections, F _CTA Coronary artery characteristics representing vessel cross-section, F _Intra ×F _CTA The matrix multiplication of the intra-lumen image features representing the intra-lumen image cross section and the coronary artery features of the vessel cross section results in a vector of m x 1, i.e. the similarity of the vessel cross section with m intra-lumen image cross sections, respectively, and the intra-lumen image cross section with the largest similarity is determined as the target intra-lumen image cross section which is the closest match with the vessel cross section.

S2024, determining a label of each intra-cavity image section corresponding to the blood vessel section based on the target intra-cavity image section;

in this step, based on the similarity calculation result in step S2023, the position (target intra-cavity image section) where the largest 1 similarity is located is selected, and set to be the tag 1, and the tag of the remaining items (other intra-cavity image sections except the target intra-cavity image section) is 0, with the following specific formula:

x＝argmax(r(F _CTA ，F _Intra ))；

where l (i) is the label of the ith intra-cavity image section, and x is the position of the target intra-cavity image section in the intra-cavity sample image.

S2025, obtaining a first loss function based on labels of each intra-cavity image section;

in this step, with the similarity result and the automatic labeling, the calculation of the first Loss function Loss1 function can be performed, where a commonly used two-class Loss function cross Entropy can be used:

wherein p (i) represents the predicted value of the ith element in the result vector, i.e. the similarity of the vessel cross section and the ith intra-cavity image cross section.

S2026, determining whether the first loss function converges;

s2027, if not, updating parameters of the image registration training module, and acquiring a next coronary artery sample image and an intra-cavity sample image corresponding to the next coronary artery sample image to continue training the image registration training module until the first loss function converges;

And S2028, if yes, obtaining a trained image registration training module.

In the above steps S2026 to S2028, the random sampling data pairs (the coronary artery sample image and the intra-cavity sample image corresponding to the coronary artery sample image) are continuously input into the image registration training module, the Loss1 is calculated for the output, and then the Loss1 is continuously reduced in the training iteration process by using the gradient descent method, so that the Loss is not reduced (i.e. converged) finally, thereby completing the training of the image registration training module.

S203, obtaining a target intra-cavity image section which is most matched with each blood vessel section in the coronary artery sample image from a plurality of intra-cavity image sections corresponding to the coronary artery sample image through the trained image registration training module;

in this step, please refer to fig. 3, fig. 3 is a schematic diagram of a workflow of an image registration module provided by the embodiment of the present application, as shown in fig. 3, a CTA cross section (i.e. a blood vessel cross section) and a plurality of corresponding intra-cavity image cross sections are obtained, the CTA cross section is input into a first feature extraction layer to extract CTA features (i.e. coronary artery features) in the CTA cross section, a plurality of intra-cavity image cross sections are input into a second feature extraction layer to extract intra-cavity image features in each intra-cavity image cross section, the intra-cavity image features in each intra-cavity image cross section and the CTA features are compared in similarity, for example, the similarity of the CTA features to the intra-cavity image features in the first intra-cavity image cross section is 0.9, the similarity of the CTA features to the intra-cavity image features in the second intra-cavity image cross section is 0.7 …, and so on, the intra-cavity image cross section is the most similar to the intra-cavity image label 1, and the intra-cavity label is set to be the most similar to the intra-cavity label 1.

After the trained image registration module is obtained, the data of the plaque recognition training module is required to be prepared before the plaque recognition training module is trained, all CTA sections are sequentially input into the trained image registration module, an intracavity image section with the largest similarity can be found for each CTA section to serve as a target intracavity image section, each CTA section and the corresponding target intracavity image section are stored, and the process is repeated until all CTA sections find the corresponding target intracavity image sections. Then, only the matched image sections in the target cavity need to be marked (the number is small), for example, the marked classification labels can be 6 types (classification labels are respectively marked as 0-5) of plaque, calcified plaque, lipid plaque, fibrous plaque, mixed plaque and thrombus, for example, if the calcified plaque exists on a certain image section in the target cavity, the image section in the target cavity is marked as 1.

S204, inputting any one blood vessel section in the coronary artery sample image and a target intracavity image section corresponding to the blood vessel section into a plaque identification training module to train the plaque identification training module, and obtaining a trained plaque identification training module;

In the step, the network architecture of the plaque identification training module is consistent with the network architecture of the image registration module, and a plurality of fusion feature layers are arranged.

The step of inputting any one blood vessel section in the coronary artery sample image and the target intracavity image section corresponding to the blood vessel section into the plaque identification training module to train the plaque identification training module to obtain the trained plaque identification training module comprises the following steps:

s2041, inputting any one blood vessel section in the coronary artery sample image and a target intracavity image section corresponding to the blood vessel section into a plaque identification training module, and extracting intracavity image characteristics of the coronary artery characteristics in the blood vessel section and the target intracavity image section corresponding to the blood vessel section;

it should be noted that the plaque recognition training module includes a third feature extraction layer and a fourth feature extraction layer; the third feature extraction layer is used for extracting coronary artery features in the blood vessel section; the fourth feature extraction layer is used for extracting intra-cavity image features of the target intra-cavity image section; the third feature extraction layer and the fourth feature extraction layer are obtained by:

1. taking parameters in a first feature extraction layer in the trained image registration training module as parameters of a network used for extracting coronary artery features in a blood vessel section in the plaque recognition training module, and determining the network used for extracting the coronary artery features in the blood vessel section with the parameters as a third feature extraction layer;

2. And taking the parameters in the second feature extraction layer in the trained image registration training module as parameters of a network for extracting the intra-cavity image features of the target intra-cavity image section in the plaque recognition training module, and determining the network with the parameters for extracting the intra-cavity image features of the target intra-cavity image section as a fourth feature extraction layer.

In the first to second steps, the plaque recognition module is trained, since the intra-cavity image feature extraction network is consistent with the CTA feature extraction network structure and the image registration module, the big data training advantage of the previous step can be fully utilized, and the parameters of the extraction network (the first feature extraction layer and the second feature extraction layer) of the image registration module trained in the previous step are used as the initialization of the parameters of the current network (the third feature extraction layer and the fourth feature extraction layer), so that the model can be converged rapidly, and a better training effect is achieved. So, in the first training, the initial parameters of the third feature extraction layer may use the parameters of the first feature extraction layer, the initial parameters of the fourth feature extraction layer may use the parameters of the second feature extraction layer, and then in the subsequent training process, updating the parameters of the plaque recognition training module includes updating the parameters of the third feature extraction layer and the parameters of the fourth feature extraction layer to train the third feature extraction layer and the fourth feature extraction layer.

S2042, acquiring a current influence coefficient, and fusing the coronary artery characteristics in the section of the blood vessel with the intra-cavity image characteristics of the target intra-cavity image section corresponding to the section of the blood vessel through the current influence coefficient to obtain fusion characteristics;

in this step, the plaque recognition training module further includes a fusion feature layer, and inputs the current influence coefficient, the coronary artery feature in the blood vessel section and the intra-cavity image feature of the target intra-cavity image section corresponding to the blood vessel section to the fusion feature layer, so as to obtain a fusion feature, which can be specifically expressed as:

F _fuse ＝F _Intra ×τ+F _CTA ；

wherein F is _fuse ∈R ^1024×1 Representing the fused feature (i.e., fused feature), τ represents the current impact coefficient, which is a dynamically changing value that may be initially set to 1.

Here, τ may be continuously reduced until 0 in the training process, and the current influence coefficient may be determined by the current training round of the plaque recognition training module, the step length of τ decreasing during each training, and the preset initial value of τ, for example, the current training round of the plaque recognition training module is the epoch-th round, and each round is reduced by 0.01 (step length) until 0, and the specific formula is as follows:

τ _epoch ＝1-0.01×epoch；

wherein τ _epoch When the training round is the epoch, the current influence coefficient is valued; 1 is the initial value of tau preset, and 0.01 is the step size of tau decrease at each training time preset.

S2043, after the fusion features are subjected to full-connection layer and normalization treatment, obtaining the probability that coronary plaque represented by the fusion features belongs to each preset classification;

in the step, after feature fusion, a final result is obtained through a full connection layer and a normalized softmax layer:

F _out ＝softmax(Linear(F _fuse ))；

wherein, linear is a full connection layer, if the number of the preset classifications is 6, the parameter dimension is R ^6×1024 Thus Linear (F) _fuse ) The resulting dimension of (2) is R ^6×1 Probability F corresponding to each preset classification _out 。

S2044, obtaining a second loss function based on the probability that coronary plaque represented by the fusion characteristics belongs to each preset classification;

in this step, the second Loss function Loss2 is calculated, where a commonly used Multi-class Loss function Multi-cross entropy may be used, and if the preset class is 6, loss2 is expressed as follows:

where y (i) represents the result of the ith channel of the classification label of the current sample (the image section in the target cavity), for example, the classification label (the preset classification) marked on the image section in the target cavity in the previous step includes 6 types (respectively marked by 0 to 5) of no plaque, calcified plaque, lipid plaque, fibrous plaque, mixed plaque and thrombus, if the current sample (the image section in the target cavity) is the calcified plaque, y= {0,1,0,0,0,0}, y (1) =1, y (0) =y (2) =y (3) =y (4) =y (5) =0. p (i) represents a predicted channel result (i.e., a probability that a plaque in a predicted vessel section belongs to an ith preset classification) corresponding to the coronary plaque represented by the fusion feature predicted by the plaque identification training module.

S2045, determining whether the second loss function converges;

s2046, if not, updating parameters and current influence coefficients of the plaque identification training module, and acquiring a next coronary artery sample image and a target intra-cavity sample image corresponding to the next coronary artery sample image to continue training the plaque identification training module until the second loss function is converged;

and S2047, if yes, obtaining a trained plaque identification training module.

In the above steps S2045 to S2047, the training plaque recognition training module needs to continuously randomly sample the matched data pairs, respectively input the data pairs into the third feature extraction layer and the fourth feature extraction layer, calculate the second loss function after fusing the output features, and then continuously reduce the second loss function by using a gradient descent method in the training iteration process, so that the training is completed until the second loss function is not descended finally. Here, the parameters of the plaque recognition training module include parameters of the third feature extraction layer, parameters of the fourth feature extraction layer, parameters of the full connection layer, parameters of the fusion feature layer, and the like. Each time the parameters of the plaque identification training module are updated by training, all the parameters included in the plaque identification training module need to be updated.

S205, determining the trained plaque recognition training module as a coronary plaque recognition model.

In the step, in the actual application process, the coronary plaque identification module is a model for classifying coronary plaque by CTA, so that an intracavity image is not needed, and the CTA section is only needed to be input into the coronary plaque identification module to obtain the type of the coronary plaque in the CTA section; therefore, the plaque recognition module with the fourth feature extraction layer and the fused feature layer removed is determined as the coronary plaque recognition model in the trained plaque recognition module.

Referring to fig. 4, fig. 4 is a schematic diagram of a workflow of a plaque recognition training module provided by the embodiment of the present application, as shown in fig. 4, a CTA cross section is input to a third feature extraction layer to obtain CTA features, a target intra-cavity influence cross section is input to a fourth feature extraction layer to obtain intra-cavity image features, a product of the intra-cavity image features and a current influence coefficient τ is determined as a first feature, a sum of the first feature and the CTA features is determined as a fusion feature, the fusion feature is subjected to a full connection layer and normalization processing to obtain a predicted result predicted by the plaque recognition training module, i.e., a probability that coronary plaque in the predicted CTA cross section belongs to each preset classification, for example, the probability that coronary plaque in the predicted CTA cross section belongs to each preset classification is respectively 0.5, 0.1, 0.2, 0.1 and 0, a label of the preset classification corresponding to the largest-value probability is set as 1, the label of the rest preset classification is set as 0, and the preset classification label of the plaque in the CTA cross section predicted by the training module is determined as the predicted result of the coronary plaque in the CTA cross section predicted by the plaque recognition training module.

In step S102, a step of inputting a coronary artery image into a coronary artery plaque identification model and outputting a coronary artery plaque type result corresponding to the coronary artery image includes:

s1021, inputting each blood vessel section in the coronary artery image into a coronary plaque identification model for feature extraction to obtain the coronary artery feature of the blood vessel section in the coronary artery image;

s1022, after the coronary artery characteristics of the blood vessel section in the coronary artery image are subjected to full-connection layer and normalization treatment, obtaining the probability that coronary artery plaques represented by the coronary artery characteristics of the blood vessel section in the coronary artery image belong to each preset classification;

s1023, in the probability, determining a preset classification corresponding to the probability with the largest numerical value as a coronary plaque type result corresponding to the blood vessel section in the coronary artery image.

In steps S1021 to S1023, after the plaque recognition training module is trained, the current influence coefficient τ is reduced to 0, and as can be seen from fig. 4, the fourth feature extraction layer and the fusion feature layer do not contribute, so that in the practical application process, by only shooting the patient with CTA, the more abundant plaque types can be predicted without matching the intra-cavity images of the patient, the setting of the influence coefficient is aimed at guiding the CTA features to learn the plaque features hidden in the data (invisible to the naked human eye) by using the intra-cavity image features in the initial stage of training, and in the training process, the influence of the intra-cavity image network (fourth feature extraction layer) on the CTA network (third feature extraction layer) is gradually reduced, so that the plaque classification can be independently and autonomously performed finally. Therefore, in application, only the CTA section is input into the CTA feature extraction network (third feature extraction layer), then the output CTA features are directly input into the full-connection layer Linear layer, and then the final result is obtained by using the Softmax layer:

F _out ＝softmax(Linear(F _CTA ))；

Here, it can be seen that F in training _fuse Directly become F _CTA F is to F _out The preset classification corresponding to the label 1 (i.e. the probability of the largest value) is determined as the coronary plaque type result in the CTA section.

The embodiment of the application provides a method for determining coronary plaque type, which comprises the following steps: acquiring a coronary artery image; inputting the coronary artery image into a coronary artery plaque identification model, and outputting a coronary artery plaque type result corresponding to the coronary artery image; the coronary plaque recognition model is obtained through training the following steps: acquiring a coronary artery sample image and an intra-cavity sample image corresponding to the coronary artery sample image from a pre-created sample data set; wherein the coronary artery sample image and the intra-cavity sample image are unlabeled images; the coronary artery sample image comprises a plurality of blood vessel sections along the blood vessel center line; the intra-cavity sample image comprises a plurality of intra-cavity image sections along the shooting direction; inputting any one blood vessel section in the coronary artery sample image and a plurality of intra-cavity image sections in the intra-cavity sample image corresponding to the coronary artery sample image into an image registration training module to train the image registration training module, so as to obtain a trained image registration training module; obtaining a target intra-cavity image section which is most matched with each blood vessel section in the coronary artery sample image from a plurality of intra-cavity image sections corresponding to the coronary artery sample image through the trained image registration training module; inputting any one blood vessel section in the coronary artery sample image and a target intracavity image section corresponding to the blood vessel section into a plaque identification training module to train the plaque identification training module, so as to obtain a trained plaque identification training module; and determining the trained plaque recognition training module as a coronary plaque recognition model.

Based on the same application conception, the embodiment of the present application further provides a coronary plaque type determining device corresponding to the method for determining a coronary plaque type according to the foregoing embodiment, and since the principle of solving the problem by the device in the embodiment of the present application is similar to that of the method for determining a coronary plaque type according to the foregoing embodiment of the present application, implementation of the device may refer to implementation of the method, and repeated descriptions are omitted.

Referring to fig. 5 and 6, fig. 5 is a first block diagram of a coronary plaque type determining apparatus according to an embodiment of the present application, and fig. 6 is a second block diagram of a coronary plaque type determining apparatus according to an embodiment of the present application. As shown in fig. 5, the determining means 510 includes:

An acquisition module 511 for acquiring a coronary artery image;

a determining module 512, configured to input the coronary artery image into a coronary plaque identification model, and output a coronary plaque type result corresponding to the coronary artery image;

a training module 513 for training a coronary plaque recognition model; the training module 513 includes an acquisition unit 5131, a first training unit 5132, a matching unit 5133, a second training unit 5134, and a determination unit 5135;

the acquiring unit 5131 is configured to acquire a coronary artery sample image and an intra-cavity sample image corresponding to the coronary artery sample image from a sample dataset created in advance; wherein the coronary artery sample image and the intra-cavity sample image are unlabeled images; the coronary artery sample image comprises a plurality of blood vessel sections along the blood vessel center line; the intra-cavity sample image comprises a plurality of intra-cavity image sections along the shooting direction;

a first training unit 5132, configured to input any one vessel section in the coronary artery sample image and a plurality of intra-cavity image sections in an intra-cavity sample image corresponding to the coronary artery sample image into an image registration training module to train the image registration training module, so as to obtain a trained image registration training module;

A matching unit 5133, configured to obtain, by using the trained image registration training module, a target intra-cavity image section that is most matched with each vessel section in the coronary artery sample image from a plurality of intra-cavity image sections corresponding to the coronary artery sample image;

the second training unit 5134 is configured to input any one blood vessel section in the coronary artery sample image and a target intracavity image section corresponding to the blood vessel section into the plaque identification training module to train the plaque identification training module, so as to obtain a trained plaque identification training module;

a determining unit 5135, configured to determine the trained plaque recognition training module as a coronary plaque recognition model.

Optionally, the first training unit 5132 is specifically configured to:

determining whether the first loss function converges;

if yes, a trained image registration training module is obtained.

Optionally, the second training unit 5134 is specifically configured to:

determining whether the second loss function converges;

if yes, a trained plaque identification training module is obtained.

Optionally, the determining module 512 is specifically configured to:

Optionally, as shown in fig. 6, the determining apparatus 510 further includes a creating module 514, where the creating module 514 is configured to:

after storing the data pairs for each patient, a sample data set is obtained.

Optionally, when the creating module 514 is configured to perform data processing on a plurality of initial images of the patient to obtain a coronary artery sample image of the patient, the creating module 514 is specifically configured to:

Optionally, the plaque recognition training module includes a third feature extraction layer and a fourth feature extraction layer; the third feature extraction layer is used for extracting coronary artery features in the blood vessel section; the fourth feature extraction layer is used for extracting intra-cavity image features of the target intra-cavity image section; the parameters of the plaque identification training module comprise parameters of a third feature extraction layer and parameters of a fourth feature extraction layer; the second training unit 5134 is further configured to:

The embodiment of the application provides a device for determining coronary plaque type, which comprises the following steps: the acquisition module is used for acquiring coronary artery images; the determining module is used for inputting the coronary artery image into the coronary artery plaque identification model and outputting a coronary artery plaque type result corresponding to the coronary artery image; the training module is used for training the coronary plaque recognition model; the training module comprises an acquisition unit, a first training unit, a matching unit, a second training unit and a determining unit; the acquisition unit is used for acquiring a coronary artery sample image and an intra-cavity sample image corresponding to the coronary artery sample image from a pre-created sample data set; wherein the coronary artery sample image and the intra-cavity sample image are unlabeled images; the coronary artery sample image comprises a plurality of blood vessel sections along the blood vessel center line; the intra-cavity sample image comprises a plurality of intra-cavity image sections along the shooting direction; the first training unit is used for inputting any one blood vessel section in the coronary artery sample image and a plurality of intra-cavity image sections in the intra-cavity sample images corresponding to the coronary artery sample image into the image registration training module to train the image registration training module, so as to obtain a trained image registration training module; the matching unit is used for obtaining a target intra-cavity image section which is matched with each blood vessel section in the coronary artery sample image in a plurality of intra-cavity image sections corresponding to the coronary artery sample image through the trained image registration training module; the second training unit is used for inputting any one blood vessel section in the coronary artery sample image and a target cavity image section corresponding to the blood vessel section into the plaque identification training module to train the plaque identification training module, so as to obtain a trained plaque identification training module; and the determining unit is used for determining the trained plaque recognition training module as a coronary plaque recognition model.

Referring to fig. 7, fig. 7 is a schematic structural diagram of an electronic device according to an embodiment of the application. As shown in fig. 7, the electronic device 700 includes a processor 710, a memory 720, and a bus 730.

The memory 720 stores machine-readable instructions executable by the processor 710, when the electronic device 700 is running, the processor 710 communicates with the memory 720 through the bus 730, and when the machine-readable instructions are executed by the processor 710, the steps of the method for determining the type of coronary plaque in the method embodiments shown in fig. 1 and fig. 2 can be executed, and detailed description thereof will be omitted.

The embodiment of the present application further provides a computer readable storage medium, where a computer program is stored, where the computer program may execute the steps of the method for determining a coronary plaque type in the method embodiments shown in fig. 1 and fig. 2 when the computer program is executed by a processor, and a specific implementation manner may refer to the method embodiments and will not be described herein.

It will be clear to those skilled in the art that, for convenience and brevity of description, specific working procedures of the above-described systems, apparatuses and units may refer to corresponding procedures in the foregoing method embodiments, and are not repeated herein.

In the several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods may be implemented in other manners. The above-described apparatus embodiments are merely illustrative, for example, the division of the units is merely a logical function division, and there may be other manners of division in actual implementation, and for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not performed. Alternatively, the coupling or direct coupling or communication connection shown or discussed with each other may be through some communication interface, device or unit indirect coupling or communication connection, which may be in electrical, mechanical or other form.

The units described as separate units may or may not be physically separate, and units shown as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

In addition, each functional unit in the embodiments of the present application may be integrated in one processing unit, or each unit may exist alone physically, or two or more units may be integrated in one unit.

The functions, if implemented in the form of software functional units and sold or used as a stand-alone product, may be stored in a non-volatile computer readable storage medium executable by a processor. Based on this understanding, the technical solution of the present application may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present application. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (Random Access Memory, RAM), a magnetic disk, or an optical disk, or other various media capable of storing program codes.

Finally, it should be noted that: the above examples are only specific embodiments of the present application, and are not intended to limit the scope of the present application, but it should be understood by those skilled in the art that the present application is not limited thereto, and that the present application is described in detail with reference to the foregoing examples: any person skilled in the art may modify or easily conceive of the technical solution described in the foregoing embodiments, or perform equivalent substitution of some of the technical features, while remaining within the technical scope of the present disclosure; such modifications, changes or substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present application, and are intended to be included in the scope of the present application. Therefore, the protection scope of the application is subject to the protection scope of the claims.

Claims

1. A method for determining a type of coronary plaque, the method comprising:

acquiring a coronary artery image;

2. The method according to claim 1, wherein the step of inputting any one vessel section in the coronary artery sample image and a plurality of intra-cavity image sections in an intra-cavity sample image corresponding to the coronary artery sample image into an image registration training module to train the image registration training module to obtain a trained image registration training module includes:

determining whether the first loss function converges;

if yes, a trained image registration training module is obtained.

3. The method according to claim 1, wherein the step of inputting any one of the blood vessel sections in the coronary artery sample image and the target intra-lumen image section corresponding to the blood vessel section into the plaque identification training module to train the plaque identification training module, and obtaining the trained plaque identification training module includes:

determining whether the second loss function converges;

if yes, a trained plaque identification training module is obtained.

4. The method according to claim 1, wherein the step of inputting the coronary artery image into a coronary plaque recognition model and outputting a type result of a coronary plaque corresponding to the coronary artery image includes:

5. The method of determining of claim 1, wherein the sample dataset is created by:

After storing the data pairs for each patient, a sample data set is obtained.

6. The method of determining according to claim 5, wherein the step of data processing the plurality of initial images of the patient to obtain the coronary artery sample image of the patient comprises:

7. The determination method of claim 3, wherein the plaque recognition training module comprises a third feature extraction layer and a fourth feature extraction layer; the third feature extraction layer is used for extracting coronary artery features in the blood vessel section; the fourth feature extraction layer is used for extracting intra-cavity image features of the target intra-cavity image section; the parameters of the plaque identification training module comprise parameters of a third feature extraction layer and parameters of a fourth feature extraction layer; the third feature extraction layer and the fourth feature extraction layer are obtained by:

8. A determination apparatus of a coronary plaque type, characterized in that the determination apparatus comprises:

the acquisition module is used for acquiring coronary artery images;

9. An electronic device, comprising: a processor, a memory and a bus, said memory storing machine readable instructions executable by said processor, said processor and said memory communicating via said bus when the electronic device is running, said machine readable instructions when executed by said processor performing the steps of the method of determining a type of coronary plaque as claimed in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that the computer-readable storage medium has stored thereon a computer program which, when executed by a processor, performs the steps of the method of determining a type of coronary plaque as claimed in any one of claims 1 to 7.