CN114926460A

CN114926460A - Training method of fundus image classification model, and fundus image classification method and system

Info

Publication number: CN114926460A
Application number: CN202210845237.4A
Authority: CN
Inventors: 谷宗运; 赵士博; 韩啸; 李传富
Original assignee: Institute of Artificial Intelligence of Hefei Comprehensive National Science Center
Current assignee: Institute of Artificial Intelligence of Hefei Comprehensive National Science Center
Priority date: 2022-07-19
Filing date: 2022-07-19
Publication date: 2022-08-19
Anticipated expiration: 2042-07-19
Also published as: CN114926460B

Abstract

The invention belongs to the technical field of intelligent medical diagnosis, and particularly relates to a training method of an eyeground image classification model, a pathological change image classification method and a system.

Description

Training method of fundus image classification model, and fundus image classification method and system

Technical Field

The invention belongs to the technical field of intelligent medical diagnosis, and particularly relates to a training method of an eye fundus image classification model, an eye fundus image classification method and an eye fundus image classification system.

Background

Diabetic retinopathy, which is an ocular complication caused by diabetes mellitus, is a leading cause of impaired vision and even blindness, and has become a major medical problem worldwide. Diabetic retinopathy classification refers to classifying a set of retinal fundus images according to the severity of a patient with diabetic retinopathy.

The high brightness area of the fundus image at the blood vessel convergence part is the optic disc, and the deep dark part at the other side is the fovea centralis. Mild diabetic retinopathy appears as deep red spots, i.e., small bleeding spots or very small red-spot microaneurysms. Moderate diabetic retinopathy is characterized by a few yellow lesions added on top of small red spots, and a few yellowish-white punctate hard exudates. Severe diabetic retinopathy adds a white, soft exudate like a cotton ball, in various shapes, from small to large plaques, on top of the red and yellow foci. The greater the number of such lesions, the more severe the diabetic retinopathy condition. Proliferative diabetic retinopathy implies that new retinal blood vessels have formed in or around the optic disc. It can cause vitreous hemorrhage, retinal hemorrhage, and severe retinal detachment. These variations are reflected on the fundus image set.

DR is classified into five grades according to the international classification standard for diabetic retinopathy. Respectively, grade 0 (non-diabetic retinopathy), grade 1 (mild non-proliferative diabetic retinopathy), grade 2 (moderate non-proliferative diabetic retinopathy), grade 3 (severe non-proliferative diabetic retinopathy) and grade 4 (proliferative diabetic retinopathy).

To date, there is no effective treatment to completely cure the disease. Research shows that early diagnosis and timely treatment of diabetic retinopathy help to prevent blindness. This goal may be achieved by periodically screening the project. Therefore, many national health agencies are promoting diabetic retinopathy screening, which has been proven to be effective in reducing the blindness rate caused by diabetic retinopathy.

The digital color fundus image set is the most widely used imaging modality for ophthalmologists to screen and identify the severity of diabetic retinopathy, and it can show the severity of the pathology. However, for many underdeveloped countries, diabetic retinopathy screening is a heavy burden due to the very shortage of ophthalmologists. The intelligent medical image-assisted diagnosis can greatly improve the screening and identifying efficiency of fundus diseases, becomes an effective way for solving the problem, and is a trend for intelligently diagnosing the severity of diabetic retinopathy.

The defects and shortcomings of the prior art are as follows:

the proportion difference of the sample cases of the diabetic retinopathy of different degrees is large, so that the distribution of samples of different levels in the diabetic retinopathy image data set is uneven, and in addition, the characteristic is difficult to extract due to uneven illumination of the shooting environment;

the resolution of the retina image is very high, the problems of gradient disappearance, explosion and the like can occur when the retina image is directly trained by a neural network, in addition, the calculation cost is high, and the model training is difficult;

the classification precision of the fundus lesion images is finer than other image categories, and due to the excessively tiny lesion points, the differences among the categories are very fine and difficult to distinguish, so that the classification accuracy of the model is low.

Disclosure of Invention

In view of the above drawbacks of the prior art, an object of the present invention is to provide a method for training a fundus image classification model, a method and a system for classifying fundus images, so as to solve the problem of low prediction accuracy of the existing classification model.

In order to achieve the above objects and other related objects, the present invention provides a method for training a fundus image classification model, comprising the steps of:

acquiring a fundus image sample set, wherein the fundus image sample set comprises a fundus image set and a classification label set corresponding to the fundus image set;

performing feature extraction on the fundus image set by using a Transformer encoder to obtain a feature matrix;

classifying the feature matrix by using a residual attention mechanism to obtain a classification result;

calculating a loss value according to the classification result and the classification label set;

and adjusting parameters of the fundus image classification model until the loss value is smaller than a preset threshold value, and finishing training of the fundus image classification model.

In an optional embodiment of the present invention, the step of performing feature extraction on the fundus image set by using a Transformer encoder to obtain a feature matrix includes:

assuming that the dimension of an input fundus image is H × W × C, where H denotes the height of the image, W denotes the width of the image, and C denotes the number of channels of the image; dividing the fundus image into N image blocks, wherein the dimension of each image block is M = P multiplied by C, and the number of N is

；

Rearranging the image block sequences into N multiplied by M matrixes, and linearly mapping each matrix to a specified dimension D; adding a special character cls to the matrix, wherein the dimension of the matrix is (N + 1) xM;

adding the position information of each image block to the matrix, keeping the dimension of the matrix unchanged, and obtaining an input item;

and inputting the input items into a Transformer encoder to obtain the feature matrix.

In an optional embodiment of the present invention, the step of classifying the feature matrix by using a residual attention mechanism to obtain a classification result includes:

and performing linear classification processing on the extracted feature matrix by using a residual attention mechanism and a Softmax classifier to obtain a classification result.

In an optional embodiment of the present invention, the step of performing linear classification processing on the extracted feature matrix using a residual attention mechanism and a Softmax classifier includes:

calculating class specific residual attention scores corresponding to each classification result according to the feature matrix;

calculating class specific residual attention corresponding to each classification result according to the class specific residual attention scores;

calculating single-head attention logic output corresponding to each classification result according to the class specific residual attention;

and calculating multi-head attention logic output corresponding to each classification result according to the single-head attention logic output, and taking the multi-head attention logic output as the prediction probability of the fundus image set corresponding to each classification result.

In an optional embodiment of the invention, the step of calculating a loss value according to the classification result and the classification label set comprises:

and bringing the classification result and the classification mark into a cross entropy function, and calculating the loss value.

In an optional embodiment of the present invention, the fundus image sample set is a diabetic retinopathy fundus image sample set, and the classification label set is a diabetic retinopathy grade.

To achieve the above and other related objects, the present invention further provides a fundus image classifying method, comprising the steps of:

acquiring a fundus image to be diagnosed;

and inputting the fundus image to be diagnosed into the fundus image classification model, obtaining a predicted value of the probability of the fundus image to be diagnosed under each classification label, and taking the classification label corresponding to the predicted value as the lesion grade of the fundus image to be diagnosed.

To achieve the above and other related objects, the present invention further provides a system for training a classification model of a fundus image, comprising:

the system comprises a sample acquisition module, a classification module and a classification module, wherein the sample acquisition module is used for acquiring a fundus image sample set, and the fundus image sample set comprises a fundus image set and a classification label set corresponding to the fundus image set;

the characteristic extraction module is used for extracting characteristics of the fundus image set by using a Transformer encoder to obtain a characteristic matrix;

the classification module is used for classifying the characteristic matrix by utilizing a residual error attention mechanism to obtain a classification result;

the loss calculation module is used for calculating a loss value according to the classification result and the classification label set;

and the control training module is used for adjusting the parameters of the fundus image classification model until the loss value is smaller than a preset threshold value so as to finish the training of the fundus image classification model.

To achieve the above and other related objects, the present invention further provides an electronic device, which includes a memory, a processor and a computer program stored in the memory and running on the processor, wherein the processor implements the steps of the method when executing the computer program.

To achieve the above and other related objects, the present invention also provides a computer-readable storage medium on which a computer program is stored, the computer program implementing the steps of the method when executed by a processor.

The invention has the technical effects that:

the method uses the transform coder to extract the image characteristics, the transform coder has a multi-head attention mechanism, the weight of key information in the operation process can be strengthened, meanwhile, the extracted characteristics are classified by using a residual attention mechanism, the key information is further strengthened, and therefore the grading effect of the diabetic retinopathy fundus image is improved.

Drawings

Fig. 1 is a block diagram of a structure of a fundus image classification model provided by an embodiment of the present invention;

FIG. 2 is a flow chart of a method for training a classification model of fundus images according to an embodiment of the present invention;

FIG. 3 is a flow chart of a feature extraction method provided by an embodiment of the invention;

FIG. 4 is a flow chart of a classification method provided by an embodiment of the present invention;

FIG. 5 is a block diagram of a fundus image classification model training system provided by an embodiment of the present invention;

FIG. 6 is a functional block diagram of a transform encoder provided by an embodiment of the present invention;

FIG. 7 is a schematic flow chart of a residual attention mechanism provided by an embodiment of the present invention;

FIG. 8 is a graph of loss value variation for a model training process provided by an embodiment of the present invention;

fig. 9 is a block diagram of an electronic device provided in an embodiment of the present invention.

Detailed Description

The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention.

Please refer to fig. 1-9. It should be noted that the drawings provided in the present embodiment are only for illustrating the basic idea of the present invention, and the components related to the present invention are only shown in the drawings rather than drawn according to the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.

The embodiment of the application can acquire and process related data based on an artificial intelligence technology. Among them, Artificial Intelligence (AI) is a theory, method, technique and application system that simulates, extends and expands human Intelligence using a digital computer or a machine controlled by a digital computer, senses the environment, acquires knowledge and uses the knowledge to obtain the best result.

Fig. 1 shows a flowchart of a preferred embodiment of the fundus image classification model training method of the present invention.

The fundus image classification model training method is applied to one or more electronic devices, wherein the electronic devices are devices capable of automatically performing numerical calculation and/or information processing according to preset or stored instructions, and the hardware of the electronic devices comprises but is not limited to a microprocessor, an Application Specific Integrated Circuit (ASIC), a Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), an embedded device and the like.

The electronic device may be any electronic product capable of performing human-computer interaction with a user, for example, a personal computer, a tablet computer, a smart phone, and the like.

The electronic device may also include a network device and/or a user device. The network device includes, but is not limited to, a single network server, a server group consisting of a plurality of network servers, or a Cloud Computing (Cloud Computing) based Cloud consisting of a large number of hosts or network servers.

The Network where the electronic device is located includes, but is not limited to, the internet, a wide area Network, a metropolitan area Network, a local area Network, a Virtual Private Network (VPN), and the like.

The fundus image classification model training method of the present invention, which can be applied to training a classification model of a diabetic retinopathy fundus image set, for example, will be described in detail below with reference to fig. 1.

The fundus image classification model can be used for disease multi-label classification of fundus image sets such as diabetic retinopathy, glaucoma, cataract, age-related macular degeneration, hypertension, myopia and the like; screening a diabetic retinopathy fundus image set; grading the disease condition of the diabetic retinopathy fundus image set and the like.

Referring to fig. 1 and 2, a training method of a fundus image classification model includes first acquiring a data set of a diabetic retinopathy fundus image set, and then dividing the data set into a training set, a test set and a verification set. Training the training set, performing feature extraction on the fundus image set by using a feature extractor, and putting the extracted feature vectors into a classifier to predict the classification probability. And finally, judging the disease level according to the classification probability. As will be described in detail below with respect to the method implementation, the feature extractor uses a pre-trained Transformer encoder model, and the classifier uses residual attention to determine the optimal threshold obtained from the york index for the lesion threshold.

The method specifically comprises the following steps:

s1: acquiring a fundus image sample set, wherein the fundus image sample set comprises a fundus image set and a classification label set corresponding to the fundus image set; in this embodiment, the fundus image sample set is a diabetic retinopathy fundus image sample set, and the classification label set is a diabetic retinopathy grade.

In one embodiment, the hierarchical labels are represented by five

numbers

0,1,2,3, 4. The international classification criteria for diabetic retinopathy are shown in table 1:

TABLE 1 diabetic retinopathy grading Standard

Grading	Retinopathy of disease
		No obvious diabetic retinopathy Variable (0 grade)	No abnormality
Mild non-proliferative diabetic retinopathy Omentum pathological change (grade 1)	Only micro-aneurysm
		Moderate nonproliferative phase diabetic retinopathy Omentum pathological change (2 grade)	Microaneurysms, between mild and severe
Severe nonproliferative stage diabetic retinopathy Omentum pathological change (grade 3)	Either of the following changes occurs, but the nonproliferative diabetic retinal manifestations: there are more than 20 intraretinal hemorrhages in any of the 4 quadrants; over two elephants Venous beading changes are limited; ③ significant intraretinal microvascular abnormalities in more than one quadrant
		Proliferative diabetic retinopathy Variable (4 grade)	One or more of the following changes occur: neovascularization, vitreous hemorrhage, pre-retinal hemorrhage

In this embodiment, the fundus image sample set is randomly divided into a training set, a verification set, and a test set, which account for 60%, 20%, and 20%, respectively.

S2: and performing feature extraction on the fundus image set by using a Transformer encoder to obtain a feature matrix. The structure of the transform encoder is shown in fig. 6.

Referring to fig. 3, in an embodiment, the step S2 includes:

s21: assuming that the dimension of an input fundus image is H × W × C, where H denotes the height of the image, W denotes the width of the image, and C denotes the number of channels of the image; dividing the fundus image into N image blocks, wherein the dimension of each image block is M = P multiplied by C, and the number of N is

(ii) a In a specific embodiment, N = 9.

S22: rearranging the image block sequences into N multiplied by M matrixes, and linearly mapping each matrix to a specified dimension D; adding a special character cls to the matrix, wherein the dimension of the matrix is (N + 1) xM;

s23: adding the position information of each image block to the matrix, keeping the dimension of the matrix unchanged, and obtaining an input item;

s24: and inputting the input items into a Transformer encoder to obtain the feature matrix.

S3: and classifying the characteristic matrix by using a residual attention mechanism of the fundus image classification model to obtain a classification result.

Referring to fig. 4 and 7, in an embodiment of the present invention, a residual attention mechanism and a Softmax classifier are used to perform a linear classification process on an extracted feature matrix to obtain a classification result, which includes the following steps:

s31: and calculating the class-specific residual attention score corresponding to each classification result according to the position feature matrix.

Specifically, characteristic matrix is extracted from fundus image set

Wherein, in the process,

、

and

respectively representing the dimension, height and width of the feature matrix, and in this embodiment, for example, it can be assumed

In the range of 2048 (parts by weight),

is 7, and

is 7; feature matrix

Decoupling as a set of location feature matrices

Using a full-link layer (1 x 1 convolution) as a classifier, each class having its own full-link layer classifier, th

Classifier corresponding to classes

Has the parameters of

Definition of

Class and item II

Class-specific residual attention score for location:

wherein,

is a temperature control factor, can control the sharpness of attention score, and has

. Will be provided with

Is regarded as the first

Class is present in

Probability at location.

S32: and calculating the class-specific residual attention corresponding to each classification result according to the class-specific residual attention score.

In particular, for example for

Class, location feature vector

The corresponding weight is

. And carrying out weighted summation on all the position feature matrix groups and the corresponding weights thereof to obtain a feature vector with a specific class:

since average pooling is widely used in practice and yields superior results, we are dealing with vectors

And

performing fusion, finally, for

Class-specific residual attention of classes

Comprises the following steps:

wherein,

is a hyper-parameter (setting)0.3), g is a global class independent feature, which constitutes the whole class specific residual attention module.

S33: and calculating the single-head attention logic output corresponding to each classification result according to the class specific residual attention.

Specifically, will be

Class-specific residual attention of classes

Classifiers corresponding to the classes

Taking the dot product to obtain the final logic output:

；

wherein,

is the number of classification categories.

S34: and calculating multi-head attention logic output corresponding to each classification result according to the single-head attention logic output, and taking the multi-head attention logic output as the prediction probability of the fundus image set corresponding to each classification result.

In particular, each head of the multi-head attention mechanism uses different temperature hyper-parameters

. Let us consider the number of attention heads as

The following rules are defined:

，

。

after introducing the multi-head attention mechanism, the logic output of each head can be obtained, which is respectively:

. Note that each header here is a class-specific residual attention module. Each head corresponds to a temperature hyper-parameter which is set by us

. We sum the logic outputs of each head directly to get the final multi-head attention logic output:

。

s4: calculating a loss value according to the classification result and the classification label set; in a specific embodiment, the classification result and the classification index are brought into a cross entropy function, and the loss value is calculated.

In particular, loss value

；

Wherein N is the lumped sample number of fundus images in the training set,

is a first

A disease label for opening the fundus oculi image,

predict for the model

Probability of opening fundus image disease label.

S5: adjusting the fundus image classification modelUntil the loss value is smaller than a preset threshold value, finishing the training of the fundus image classification model. Specifically, will

And (4) performing back propagation for updating the model parameters, repeating the steps S2-S4, continuously performing iterative training, taking the corresponding verification set as an evaluation data set according to the training result until the loss value reaches a specified threshold value, and storing the corresponding model parameters of the final loss value.

To evaluate the performance of the model, the present invention trains and tests the model on both the DDR and IDRiD data sets. FIG. 8 is a loss curve of the inventive model training process on a DDR data set. As can be seen from FIG. 8, the Loss value decreases with the increase of the training times, the Loss values on the test set and the verification set decrease significantly at the first 6 epochs, the Loss values on the test set do not decrease any more at the 7 th epoch and after the 9 th epoch, and the model training tends to be saturated. From the variation of the loss function, it can be seen that the training loss and the testing loss are not very different, which indicates that no overfitting phenomenon occurs in our model.

DDR data sets are provided by ophthalmic disease intelligent recognition (ODIR-2019) for lesion segmentation and lesion detection. The data set included 13673 fundus image sets from 147 hospitals in 23 provinces in china. For the classification task, there has been provided on DDR a partition of training, validation and test sets, of which 6835 are used for training, 2733 are used for validation and the rest 4105 are used for testing.

IDRiD is an Indian diabetic retinopathy image dataset. It is a data set consisting of typical DR and normal retinal structures, containing 413 training images and 103 test images. The test set is divided into five classes according to the rules, and the test set is used as a verification set to evaluate the experimental result. The data set provides information for each image of disease severity of DR and diabetic macular edema. This makes it very suitable for developing and evaluating image analysis algorithms for early detection of diabetic retinopathy.

In order to verify the performance of the model, the invention also uses sensitivity, specificity and accuracy as evaluation indexes to evaluate the prediction result of the model. The sensitivity is the probability that the DR image of the lesion is not missed to be judged as negative, and the higher the value of the sensitivity is, the stronger the abnormality discovering capability of the model is. The sensitivity calculation is shown in equation 1.

Specificity is the probability that a normal DR image is not erroneously judged to be positive. Higher values represent a greater ability of the model to recognize normality. The calculation formula of the specificity is shown in formula 2.

Accuracy represents the correct proportion of the model classification, with higher values representing more accurate model classifications. The accuracy calculation is shown in equation 3.

Wherein tp (true positive) represents the positive samples predicted as positive class by the model, tn (true negative) represents the negative samples predicted as negative class by the model, fp (false positive) represents the negative samples predicted as positive class by the model, and fn (false negative) represents the positive samples predicted as negative class by the model.

Results if shown in table 2:

TABLE 2 Performance parameters of DDR and IDRiD datasets

Based on the fundus image classification model, the invention also provides a diabetic retinopathy fundus image classification method, which comprises the following steps: acquiring a fundus image to be diagnosed; and inputting the fundus image to be diagnosed into the fundus image classification model, obtaining a predicted value of the probability of the fundus image to be diagnosed under each classification label, and taking the classification label corresponding to the predicted value as the lesion grade of the fundus image to be diagnosed.

According to the method, the transform encoder is used for extracting image features, the transform encoder is provided with a multi-head attention mechanism, the weight of key information in the operation process can be strengthened, meanwhile, the extracted features are classified by using a residual attention mechanism, the key information is further strengthened, and therefore the grading effect of the diabetic retinopathy fundus image set is improved.

It should be noted that, the steps of the above methods are divided for clarity, and the implementation may be combined into one step or split some steps, and the steps are divided into multiple steps, and as long as the steps contain the same logical relationship, the steps are within the scope of the present patent; it is within the scope of the patent to add insignificant modifications to the algorithms or processes or to introduce insignificant design changes to the core design without changing the algorithms or processes.

Fig. 5 is a functional block diagram of a fundus image classification model training system according to a preferred embodiment of the present invention. The fundus image classification model training system comprises: a sample acquisition module 10, a feature extraction module 20, a classification module 30, a loss calculation module 40, and a control training module 50. The module referred to in the present invention refers to a series of computer program segments that can be executed by the processor 100 and can perform a fixed function, and is stored in the memory 200.

The sample acquisition module 10 is configured to acquire a fundus image sample set, where the fundus image sample set includes a fundus image set and a classification label set corresponding to the fundus image set; the feature extraction module 20 is configured to perform feature extraction on the fundus image set by using a Transformer encoder to obtain a feature matrix; the classification module 30 is configured to classify the feature matrix by using a residual attention mechanism to obtain a classification result; the loss calculating module 40 is configured to calculate a loss value according to the classification result and the classification label set; the control training module 50 updates the model parameters according to the loss value until the loss value is smaller than a preset threshold value.

It should be noted that the fundus image classification model training system of this embodiment is a device corresponding to the above-mentioned fundus image classification model training method, and the functional modules in the fundus image classification model training system or the functional modules in the fundus image classification model training system respectively correspond to the corresponding steps in the fundus image classification model training method. The fundus image classification model training system of the embodiment can be implemented by being matched with the fundus image classification model training method. Accordingly, the related-art details mentioned in the fundus image classification model training system of the present embodiment can also be applied to the above-described fundus image classification model training method.

It should be noted that, when the above functional modules are actually implemented, all or part of the functional modules may be integrated into one physical entity, or may be physically separated. And these modules can be realized in the form of software called by processing element; or can be implemented in the form of hardware; and part of the modules can be realized in the form of calling software by the processing element, and part of the modules can be realized in the form of hardware. In addition, all or part of the modules can be integrated together or can be independently realized. The processing element described herein may be an integrated circuit having signal processing capabilities. In the implementation process, part or all of the steps of the method or each functional module above may be implemented by an integrated logic circuit of hardware in a processor element or instructions in the form of software.

Fig. 9 is a schematic structural diagram of an electronic device according to a preferred embodiment of the present invention for implementing a method for training a fundus image classification model.

The electronic device may include a memory 200, a processor 100, and a bus, and may further include a computer program, such as a fundus image classification model training program, stored in the memory and executable on the processor.

Wherein the memory includes at least one type of readable storage medium including flash memory, removable hard disks, multimedia cards, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disks, optical disks, etc. The memory may in some embodiments be an internal storage unit of the electronic device, for example a removable hard disk of the electronic device. The memory may also be an external storage device of the electronic device in other embodiments, such as a plug-in removable hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like provided on the electronic device. Further, the memory may also include both an internal storage unit and an external storage device of the electronic device. The memory may be used not only to store application software installed in the electronic device and various types of data such as a code of a fundus image classification model training program, but also to temporarily store data that has been output or is to be output.

A processor may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same or different functions, including one or more Central Processing Units (CPUs), microprocessors, digital Processing chips, graphics processors, and combinations of various control chips. The processor is a Control Unit (Control Unit) of the electronic device, connects various components of the entire electronic device by using various interfaces and lines, executes various functions of the electronic device and processes data by running or executing programs or modules stored in the memory (for example, executing a fundus image classification model training program and the like), and calls data stored in the memory.

The processor executes an operating system of the electronic device and various installed application programs. The processor executes the application program to implement the steps in the above-mentioned embodiments of the fundus image classification model training method, for example, the steps shown in the figure.

Illustratively, the computer program may be partitioned into one or more modules that are stored in the memory and executed by the processor to implement the invention. The one or more modules may be a series of computer program instruction segments capable of performing certain functions, which are used to describe the execution of the computer program in the electronic device. For example, the computer program may be segmented into a sample acquisition module 10, a feature extraction module 20, a classification module 30, a loss calculation module 40, and a control training module 50.

The integrated unit implemented in the form of a software functional module may be stored in a computer-readable storage medium. The software functional module is stored in a storage medium and includes several instructions to enable a computer device (which may be a personal computer, a computer device, or a network device, etc.) or a Processor (Processor) to execute partial functions of the fundus image classification model training method according to various embodiments of the present invention.

The bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one arrow is shown in FIG. 5, but this does not indicate only one bus or one type of bus. The bus is arranged to enable connected communication between the memory and at least one processor or the like.

In summary, the invention uses the transform encoder to extract image features, the transform encoder has a multi-head attention mechanism, so that the weight of key information in the operation process can be strengthened, and meanwhile, the extracted features are classified by using a residual attention mechanism, so that the key information is further strengthened, and the classification effect of the diabetic retinopathy fundus image set is improved.

It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.

Although the present invention has been described in detail with reference to the preferred embodiments, it will be understood by those skilled in the art that various changes may be made and equivalents may be substituted for elements thereof without departing from the spirit and scope of the present invention.

Claims

1. A training method of a fundus image classification model is characterized by comprising the following steps:

2. A method for training a fundus image classification model according to claim 1, wherein said step of performing feature extraction on said fundus image set by using a fransformer encoder, and obtaining a feature matrix comprises:

assuming that the dimension of an input fundus image is H × W × C, where H denotes the height of the image, W denotes the width of the image, and C denotes the number of channels of the image; dividing the fundus image into N image blocks, each image block having dimensions M = P × P × C, where N is the number

；

Rearranging the image block sequences into N multiplied by M matrixes, and linearly mapping each matrix to a specified dimension D; adding a special character to the matrix, wherein the dimension of the matrix is (N + 1) multiplied by M;

3. The training method of the fundus image classification model according to claim 1, wherein said step of classifying said feature matrix using a residual attention mechanism to obtain a classification result comprises:

4. A training method of a fundus image classification model according to claim 3, wherein said step of linear classification processing of the extracted feature matrix using a residual attention mechanism and a Softmax classifier comprises:

calculating the class specific residual attention score corresponding to each classification result according to the feature matrix;

calculating the class specific residual attention corresponding to each classification result according to the class specific residual attention score;

and calculating the multi-head attention logic output corresponding to each classification result according to the single-head attention logic output, and taking the multi-head attention logic output as the prediction probability of each classification result corresponding to the fundus image set.

5. A training method for a fundus image classification model according to claim 1, wherein said step of calculating a loss value based on said classification result and said classification label set comprises:

6. A training method for a fundus image classification model according to claim 1, wherein said fundus image sample set is a diabetic retinopathy fundus image sample set, and said classification label is a diabetic retinopathy grade.

7. A fundus image classification method is characterized by comprising the following steps:

acquiring a fundus image to be diagnosed;

inputting the fundus image to be diagnosed into the fundus image classification model according to any one of claims 1 to 6, obtaining a predicted value of the probability of the fundus image to be diagnosed under each classification label, and taking the classification label corresponding to the predicted value as the lesion grade of the fundus image to be diagnosed.

8. A system for training a fundus image classification model, comprising:

the classification module is used for classifying the characteristic matrix by using a residual attention mechanism to obtain a classification result;

9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor implements the steps of the method of any one of claims 1 to 6 when executing the computer program or the processor implements the steps of the method of claim 7 when executing the computer program.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of one of claims 1 to 6, or which, when being executed by a processor, carries out the steps of the method of claim 7.