CN117392124B - Medical ultrasonic image grading method, system, server, medium and device - Google Patents

Abstract

The invention belongs to the field of artificial intelligence deep learning, and provides a medical ultrasonic image grading method, a system, a server, a medium and equipment, wherein the grading method comprises the following steps: acquiring a medical ultrasound image and an RGB image dataset; pre-training the constructed grading model based on the RGB image data set to obtain a first grading model, and transferring the first grading model to a medical ultrasonic image for fine tuning training to obtain a second grading model; and obtaining a grading result based on the medical ultrasonic image to be graded and the grading model. The invention can adaptively weight different channels and spatial positions through the use of an attention mechanism, solves the problem of unbalanced class, improves the classification accuracy, finally outputs the classification grade of the image by the system, can provide auxiliary reference opinion for doctors, and improves the accuracy and the efficiency of clinical diagnosis.

Description

A medical ultrasound image classification method, system, server, media and equipment

Technical field

The invention belongs to the field of deep learning of artificial intelligence, and in particular relates to a medical ultrasound image classification method, system, server, medium and equipment.

Background technique

The statements in this section merely provide background technical information related to the present invention and do not necessarily constitute prior art.

Artificial intelligence classification based on medical images refers to the method of analyzing and diagnosing medical images using computer vision technology and artificial intelligence algorithms. Medical images include X-ray, CT, MRI, ultrasound and other types. The analysis and diagnosis of medical images play an important role in the early detection and treatment of diseases. In medical image classification, artificial intelligence technology can greatly improve the accuracy and efficiency of diagnosis, shorten diagnosis time, and help doctors make more accurate diagnosis and treatment decisions. At the same time, medical image classification technology also faces some problems and challenges, such as insufficient data volume, data imbalance, lack of annotation information, etc., and needs further improvement and improvement.

The inventor found that current pathological examination requires operations such as surgery or puncture sampling. For example, fatty liver is a common metabolic disease. Currently, the gold standard for grading fatty liver is liver histopathological examination. Pathological examination is to obtain liver tissue specimens and determine the fat content, fibrosis degree, inflammatory response and other aspects of the liver through observation and evaluation under a microscope. This invasive diagnostic method is invasive and risky for patients. , and it is expensive and not suitable for large-scale screening or dynamic monitoring. Therefore, the development of non-invasive diagnostic technology based on medical imaging has become a hot and difficult topic in current research.

The main technologies used for clinical pathology examination are medical imaging technologies such as ultrasound imaging, CT and MRI. However, these technologies suffer from strong operator subjectivity, cumbersome operating procedures, high diagnostic costs, and inaccurate and non-standard grading results. question.

Contents of the invention

In order to solve at least one technical problem existing in the above background technology, the present invention provides a medical ultrasound image grading method, system, server, medium and equipment, which uses a deep convolutional neural network model to realize automatic grading of medical ultrasound images. Improve doctors’ diagnostic efficiency.

According to a first aspect of the present disclosure, a medical ultrasound image classification method is provided, which method includes:

Obtain medical ultrasound image and RGB image data sets;

Pre-train the constructed grading model based on the RGB image data set to obtain the first grading model, and migrate the first grading model to medical ultrasound images for fine-tuning training to obtain the second grading model;

Among them, the construction process of the hierarchical model includes: based on the ResNeAt network structure, introducing the residual block and attention mechanism, adding the channel attention module and space before the first residual block and after the last residual block. The attention module obtains the first feature map through the channel attention module. The first feature map passes through the spatial attention module and extracts horizontal and vertical features to obtain the second feature map. The first feature map and the second feature map are After weighting processing, the classification results are output through the classification layer;

The classification result is obtained based on the medical ultrasound image to be classified and the two-classification model.

In some embodiments, after the medical ultrasound image is acquired, the preprocessing of the medical ultrasound image includes black frame removal, image grayscale processing, local histogram adaptive equalization, image enhancement processing, and data partitioning processing.

In some embodiments, the medical ultrasound images are divided into a training set, a verification set and a test set, and the saved first hierarchical model is migrated to the training set and the verification set for retraining and verification to obtain a second hierarchical model based on the test The second graded model is tested together until it meets the usage standards.

In some embodiments, when pre-training the hierarchical model, a cosine annealing learning rate adjustment algorithm is used to adjust the learning rate.

In some embodiments, after obtaining the second hierarchical model, cross-validation is performed on the second hierarchical model, and the average and variance of the accuracy obtained by cross-validation are calculated.

In some embodiments, the ResNeAt network includes four different numbers of residual blocks, three downsampling layers, attention modules, and classification layers. Each residual block consists of one depthwise separable convolutional layer, two ordinary The convolution consists of a LayerNormalization (Layer Norm) layer, the downsampling layer consists of a Layer Norm layer and a normal convolution layer, and the classification layer consists of a global average pooling layer, a Layer Norm layer and a fully connected layer.

According to a second aspect of the present invention, a server is provided. The server includes an acquisition module, a training module and a grading module. The acquisition module is configured to acquire medical ultrasound images and RGB image data sets; the training module is configured to acquire a data set based on RGB images. The data set pre-trains the constructed classification model to obtain the first classification model, and migrates the first classification model to medical ultrasound images for fine-tuning training to obtain the second classification model; the classification module is configured to be based on the medical ultrasound images to be classified. and two-level models to obtain hierarchical results.

According to a third aspect of the present invention, a medical ultrasound image grading system based on a depth model is provided, including an image acquisition device and an image processing device; the image acquisition device is used to collect medical ultrasound images and RGB image data sets, and provide the The medical processing device sends the medical ultrasound image and the RGB image data set; the image processing device is used to perform the method in any one of the above aspects.

According to a fourth aspect of the present invention, there is provided a non-transitory computer-readable medium of computer program instructions, which when executed by a processor cause the processor to perform a method according to any one of the above aspects.

According to a fifth aspect of the present invention, there is provided a computer device, comprising a processor and a memory storing a computer program thereon, the computer program being configured to execute any one of the above aspects on the processor. method.

Compared with the prior art, the beneficial effects of the present invention are:

1. The present invention provides an intelligent grading method and system for ultrasonic images based on a deep model. The automatic grading of ultrasonic images is realized based on the use of a deep convolutional neural network model. The constructed ResNeAt network contains residual blocks and convolutional block attention. module (CBAM) attention mechanism. The CBAM attention module is added before the first residual block and after the last residual block. The features of each channel are multiplied by the corresponding weight, and the feature map of the channel attention module is input to The spatial attention module performs two processes, namely the horizontal GAP and the vertical GAP, as well as the corresponding weight coefficients. The feature maps processed by the channel attention mechanism and the spatial attention mechanism are multiplied to obtain the final The feature map realizes adaptive weighting of different channels and spatial positions, allowing the network to focus on the most important features in channels and space, thereby increasing the network's attention to important features in the image and improving the model's performance. Performance; effectively solves the problems of network gradient explosion and data imbalance. Ultimately, efficient, accurate, low-cost, and highly reproducible hierarchical diagnosis of diseases such as fatty liver based on ultrasound images will be achieved, which has broad application prospects.

2. The hierarchical model of the present invention adopts Layer Normalization (Layer Norm). Compared with BatchNormalization, Layer Norm performs more stably on small samples, and Layer Norm normalizes each feature of each sample instead of the entire batches, which makes it very effective for suppressing the vanishing gradient problem when training deep networks; at the same time, the hierarchical model uses the Cosine Annealing LR adjustment algorithm to reduce the learning rate, and uses improved Focal Loss as the loss function to reduce data imbalance impact.

3. The present invention uses images collected by a variety of devices, the images are very different, and establishes a universal and efficient classification model.

4. The hierarchical model of the present invention uses depth-separable convolution. First, depth spatial convolution is applied, and then point-wise convolution is applied. This can significantly reduce the number of parameters and improve the speed of the model.

These and other advantages of the present disclosure will be apparent from and elucidated with reference to the embodiments described hereinafter.

Description of the drawings

Embodiments of the present disclosure will now be described in greater detail and with reference to the accompanying drawings, in which:

Figure 1 is a flow chart of a deep model-based medical ultrasound image classification method provided by an embodiment of the present invention;

Figure 2 is a schematic structural diagram of the ResNeAt network provided by an embodiment of the present invention;

Figure 3 is a schematic diagram of a server for grading medical ultrasound images provided by an embodiment of the present invention;

Figure 4 is a schematic diagram of a deep model-based medical ultrasound image classification system provided by an embodiment of the present invention;

Figure 5 is an example system that includes an example computing device representative of one or more systems and/or devices that may implement various techniques described herein.

Detailed ways

The following description provides specific details for fully understanding and practicing various embodiments of the disclosure. It will be understood by those skilled in the art that the technical solutions of the present disclosure may be practiced without some of these details. In some instances, well-known structures and functions are not shown or described in detail to avoid unnecessarily obscuring the description of the embodiments of the present disclosure. The terms used in this disclosure are to be understood in their broadest reasonable manner, even if used in connection with specific embodiments of the disclosure.

The present invention uses the data enhancement RandAugment method to increase the diversity and quantity of data for small sample problems; uses the transfer learning method to improve classification performance and obtains more reliable model evaluation results through five-fold cross-validation. Through the use of the attention mechanism, the present invention can Adaptively weight different channels and spatial positions to solve the class imbalance problem and improve the classification accuracy. The final system outputs the lesion level, which can provide auxiliary reference opinions for doctors and improve the accuracy and efficiency of clinical diagnosis.

Figure 1 schematically shows a depth model-based intelligent classification method 100 for ultrasound images, which includes the following steps:

Step 101: Data acquisition, obtain ultrasound images;

Step 102: Data preprocessing, preprocess the ultrasound images and divide them into three parts: training set, verification set and test set;

Step 103: Hierarchical model construction, based on the ResNeAt network model, adding the residual block (residual block) and convolutional block attention module (CBAM) attention mechanism, using the Cosine Annealing Learning Rate (CosineAnnealingLR) adjustment algorithm to reduce the learning rate, and using Improved Focal Loss is used as the loss function to reduce the impact of data imbalance;

Step 104: Transfer learning, use the ImageNet data set to pre-train the built ResNeAt network, and migrate the resulting model to the pre-processed data set for fine-tuning training to obtain the final ultrasound image classification model;

Step 105: Divide the images of the training set and the verification set into five equal parts, each of which is used as a verification set, and the remaining four parts are used as a training set. Perform five cross-validations through the hierarchical model obtained in step 104 to prove the model. generalization ability;

Step 106: Input the test set images into the trained ResNeAt network to obtain the grading results, and test whether the network meets the standards for clinical use.

In this embodiment, the fatty liver classification of liver histopathological examination is taken as an example;

In the data acquisition module, the ultrasound image is an abdominal ultrasound image, acquired through four different devices: GE Healthcare, SAMSUNG, TOSHIBA and HITACHI;

The acquired image resolutions range from 720ⅹ576 to 1920ⅹ1080. The imaging doctor annotates them, and finally obtains 2000 images each of normal, mild fatty liver, moderate fatty liver, and 500 images of severe fatty liver.

The data preprocessing process includes the following steps:

Step 201: Remove poor quality images with a large number of black areas in the collected ultrasound images;

Step 202: Crop the black border in each ultrasound image, leaving only the middle fan-shaped area;

Step 203: Grayscale all images;

Step 204: Use local histogram adaptive equalization on all images to enhance contrast;

Step 205: Use the open source data enhancement algorithm RandAugment to obtain 2,000 enhanced images of severe fatty liver, specifically: rotate the ultrasound image of severe fatty liver 90°/180°/270°, mirror in vertical and horizontal directions, etc.;

Step 206: Divide all the processed images into three parts: training set, verification set and test set according to the ratio of 8:1:1.

In step 103, the ResNeAt network structure constructed in step 103 is shown in Figure 2 and consists of four different numbers of residual blocks, three downsampling layers, an attention module and a final classification layer. Each residual block consists of a depth-separable convolution layer, two ordinary convolutions and a Layer Norm layer. The convolution kernel sizes are respectively ,/> ,/> , the step size is all 1, and the numbers of the four residual blocks are 3, 3, 9, and 3 respectively; the downsampling layer consists of a Layer Norm layer and an ordinary convolution layer, and the convolution kernel size is 2/> 2. The step size is 2; the classification layer consists of a global average pooling layer, a Layer Norm layer and a fully connected layer; ConvolutionalBlock Attention Module (CBAM) is added before the first residual block and after the last residual block. The attention module, including a channel attention module and a spatial attention module, uses the Cosine Annealing LR adjustment algorithm to reduce the learning rate, and uses improved Focal Loss as the loss function to reduce the impact of data imbalance. .

Specifically, the ultrasound image data is processed through this model as follows:

First, input the image obtained in step 102 (size HⅹWⅹC, H is height, W is width, C is the number of channels) into the channel attention module of the first CBAM attention module, and through global average pooling (Global Average Pooling, GAP) performs a pooling operation on each channel to obtain a vector of size C (each element represents the average activation value of a channel), and then maps the obtained vector into two through a fully connected (FC) layer. Vectors, respectively representing the maximum activation value and average activation value of the channel, then perform Softmax activation on the two vectors respectively to obtain the corresponding weight coefficients, and then apply these two weight coefficients to the original feature map respectively, that is, these two weights The coefficients are multiplied respectively by the features of the corresponding channels.

The feature map after the channel attention module is input to the spatial attention module for two processings, namely the horizontal GAP and the vertical GAP. Two vectors of size Hⅹ1ⅹ1 and 1ⅹWⅹ1 are obtained. The two vectors obtained are Mapping is performed through two fully connected layers respectively. Then perform Softmax activation on the two vectors to obtain the corresponding weight coefficients, which represent the importance in the horizontal and vertical directions. Apply these two weight coefficients to the original feature map respectively, that is, multiply these two weight coefficients with the characteristics of the corresponding pixels. Finally, the feature maps processed by the channel attention mechanism and the spatial attention mechanism are multiplied to obtain the final feature map, allowing the network to focus on the most important features in channel and space.

The feature map obtained through the first CBAM attention module is input to the residual block 1. The residual block 1 consists of a depth-separable convolution layer, two ordinary convolutions and a Layer Norm layer. The convolution kernel sizes are respectively ,/> ,/> , the step size is all 1, and the residual block 1 is repeated 3 times. Then it is input to the downsampling layer. The downsampling layer consists of a Layer Norm layer and an ordinary convolution layer. The convolution kernel size is 2/> 2, the step size is 2. Then it is input to residual block 2. Residual block 2 has the same structure as residual block 1. It is repeated three times and then input to residual block 3 through the downsampling layer. Residual block 3 has the same structure as residual block 1. Repeat After 9 times, it is input to residual block 4 through the downsampling layer. Residual block 4 has the same structure as residual block 1. After repeated 3 times, it is input to the second CBAM attention module, and the process is the same as the first CBAM attention module. operation, and finally input into the classification layer. The classification layer consists of a global average pooling layer, a Layer Norm layer and a fully connected layer, and outputs the final classification probability. The use of the Cosine Annealing LR adjustment algorithm to reduce the learning rate specifically includes:

An algorithm that starts the learning rate from a larger value and then gradually decreases in the form of a cosine function can automatically adjust the learning rate during training and improve the model's generalization ability. The principle is as follows: , where i represents the number of runs, /> and/> Represents the maximum and minimum values of the learning rate respectively, defining the range of the learning rate, /> It indicates how many epochs are currently executed, but/> It will be updated after each batch is run, and at this time an epoch has not been executed yet, so/> The value of can be a decimal,/> Indicates how many epochs are needed for the i-th restart. In this way, within a cycle, the learning rate will follow the cosine decay trend from/> Reduce to/> , and then enter the next cycle.

The ImageNet data set is used to pre-train the constructed ResNeAt network, including:

Step 401: First, use the ImageNet data set to pre-train the built ResNeAt network, and save the model after n training iterations; in this embodiment, n is preferably 10,000.

Step 402: Migrate the model saved in step 401 to the training set and verification set preprocessed in step 2 for retraining and verification, thereby obtaining the final ultrasound image classification model.

In the step 5, the images of the training set and the verification set are divided into five parts equally, each part is used as the verification set, and the remaining four parts are used as the training set, and five cross-validations are performed through the hierarchical model obtained in step 4; Calculate the average of the accuracy and variance five times to prove the generalization ability of the model.

In step 106, the test set image preprocessed in step 2 is used as input to obtain the ResNeAt network classification result;

Calculate each graded evaluation index, including Accuracy, Precision, Recall, Specificity, and F1-score, to test whether the network meets the standards for clinical use.

The grading evaluation index in step 106 is specifically, the evaluation formula is Among them, TP means that the predicted positive example is actually a positive example; TN means that the predicted negative example is actually a negative example; FP means that the predicted positive example is actually a negative example; FN means that the predicted positive example is actually a negative example.

FIG. 3 schematically illustrates a server 500 for classifying medical ultrasound images according to one embodiment of the present disclosure. The server 500 includes an acquisition module 501, a training module 502 and a grading module 503. The acquisition module 501 is configured to acquire medical ultrasound images and RGB image data sets. The training module 502 is configured to pre-train the constructed hierarchical model based on the RGB image data set to obtain a first hierarchical model, and migrate the first hierarchical model to medical ultrasound images for fine-tuning training to obtain a second hierarchical model. The classification module 503 is configured to obtain a classification result based on the medical ultrasound image to be classified and the two-classification model. FIG. 4 schematically illustrates a schematic diagram of a depth model-based medical ultrasound image classification system 600 according to an embodiment of the present disclosure.

Figure 5 illustrates an example system 700 that includes an example computing device 710 representative of one or more systems and/or devices that may implement various techniques described herein. Computing device 710 may be, for example, a service provider's server, a device associated with a client (eg, a client device), a system on a chip, and/or any other suitable computing device or computing system. The server 500 described above with respect to FIG. 4 for classifying medical images may take the form of a computing device 710. Alternatively, the server 500 for classifying medical images may be implemented as a computer program in the form of a medical ultrasound image classification application 716 .

The example computing device 710 as illustrated includes a processing system 711 , one or more computer-readable media 712 , and one or more I/O interfaces 713 communicatively coupled with each other. Although not shown, computing device 710 may also include a system bus or other data and command transfer system that couples various components to one another. The system bus may include any one or combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and/or a processor or processor utilizing any of the various bus architectures. local bus. Various other examples are also contemplated, such as control and data lines.

Processing system 711 represents functionality that uses hardware to perform one or more operations. Accordingly, processing system 711 is illustrated as including hardware elements 714 that may be configured as processors, functional blocks, and the like. This may include implementation in hardware as an application specific integrated circuit or other logic device formed using one or more semiconductors. Hardware element 714 is not limited by the materials from which it is formed or the processing mechanisms employed therein. For example, a processor may be composed of semiconductor(s) and/or transistors (eg, electronic integrated circuits (ICs)). In such context, processor-executable instructions may be electronically executable instructions.

Computer-readable medium 712 is illustrated as including memory/storage device 715 . Memory/storage 715 represents the memory/storage capacity associated with one or more computer-readable media. Memory/storage 315 may include volatile media (such as random access memory (RAM)) and/or non-volatile media (such as read only memory (ROM), flash memory, optical disks, magnetic disks, etc.). Memory/storage 715 may include fixed media (eg, RAM, ROM, fixed hard drive, etc.) as well as removable media (eg, flash memory, removable hard drive, optical disk, etc.). Computer-readable medium 712 may be configured in various other ways as described further below.

One or more I/O interfaces 713 represent functionality that allows a user to enter commands and information into computing device 710 and optionally also allows information to be presented to the user and/or other components or devices using various input/output devices. Examples of input devices include keyboards, cursor control devices (e.g., mice), microphones (e.g., for voice input), scanners, touch capabilities (e.g., capacitive or other sensors configured to detect physical touch), cameras ( For example, motion that does not involve touch can be detected as gestures using visible or invisible wavelengths (such as infrared frequencies), and so on. Examples of output devices include display devices (e.g., monitors or projectors), speakers, printers, network cards, haptic-responsive devices, and so on. Therefore, computing devices

710 may be configured in various ways to support user interaction as described further below.

Computing device 710 also includes medical ultrasound image classification 716 . Medical ultrasound image classification 716 may, for example, be a software instance of server 500 for classifying medical images described with respect to FIG. 5 , and in combination with other elements in computing device 710 implement the techniques described herein.

Various techniques may be described herein in the general context of software, hardware elements, or program modules. Generally, these modules include routines, programs, objects, elements, components, data structures, etc. that perform specific tasks or implement specific abstract data types. As used herein, the terms "module," "function," and "component" generally mean software, firmware, hardware, or a combination thereof. The techniques described in this article are characterized by being platform-independent, meaning that they can be implemented on a variety of computing platforms with a variety of processors.

Implementations of the described modules and techniques may be stored on or transmitted across some form of computer-readable media. Computer-readable media may include a variety of media that can be accessed by computing device 710 . By way of example, and not limitation, computer-readable media may include "computer-readable storage media" and "computer-readable signal media." As opposed to a mere transmission of a signal, a carrier wave, or the signal itself, "computer-readable storage medium" refers to a medium and/or device capable of persistent storage of information, and/or a tangible storage device. Therefore, computer-readable storage media refers to non-signal bearing media. Computer-readable storage media includes volatile and nonvolatile, removable and non-removable media and/or media suitable for storage of information such as computer-readable instructions, data structures, program modules, logic elements/circuits or other data.

Hardware such as storage devices implemented according to) methods or technologies. Examples of computer-readable storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical storage devices, hard drives, cassettes, magnetic tape, magnetic disk storage A device or other magnetic storage device, or other storage device, tangible medium, or article of manufacture suitable for storing the desired information and accessible by a computer.

"Computer-readable signal medium" refers to signal bearing medium configured as hardware to transmit instructions to computing device 710, such as via a network. Signal media typically may embody computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave, data signal, or other transport mechanism. Signal media also includes any information delivery media. The term "modulated data signal" refers to a signal that encodes information in the signal in such a way that it sets or changes one or more of its characteristics. By way of example, and not limitation, communication media include wired media, such as a wired network or direct wire, and wireless media, such as acoustic, RF, infrared, and other wireless media.

As previously described, hardware elements 714 and computer-readable media 712 represent instructions, modules, programmable device logic, and/or fixed device logic implemented in hardware that, in some embodiments, may be used to implement the techniques described herein. At least some aspects. Hardware elements may include integrated circuits or systems on a chip, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), complex programmable logic devices (CPLDs), and other implementations in silicon or components of other hardware devices.

In this context, a hardware element may serve as a processing device for performing program tasks defined by the instructions, modules, and/or logic embodied by the hardware element, as well as a hardware device for storing instructions for execution, e.g., previously described computer-readable storage media.

Combinations of the foregoing may also be used to implement the various technologies and modules described herein. Thus, software, hardware, or program modules and other program modules may be implemented as one or more instructions and/or logic embodied on some form of computer-readable storage medium and/or embodied by one or more hardware elements 714 . Computing device 710 may be configured to implement specific instructions and/or functions corresponding to software and/or hardware modules. Thus, modules may be implemented, at least in part, in hardware, as modules executable by computing device 710 as software, such as by using computer-readable storage media and/or hardware elements 714 of the processing system. Instructions and/or functions may be executable/operable by one or more articles of manufacture (eg, one or more computing devices 710 and/or processing systems 711 ) to implement the techniques, modules, and examples described herein.

In various implementations, computing device 710 may employ a variety of different configurations. For example, computing device 710 may be implemented as a computer-type device including a personal computer, a desktop computer, a multi-screen computer, a laptop computer, a netbook, and the like. Computing device 710 may also be implemented as a mobile device-type device including mobile devices such as mobile phones, portable music players, portable gaming devices, tablet computers, multi-screen computers, and the like. Computing device 710 may also be implemented as a television-type device, which includes devices having or being connected to generally larger screens in casual viewing environments. These devices include televisions, set-top boxes, game consoles, and more.

The techniques described herein may be supported by these various configurations of computing device 710 and are not limited to the specific examples of the techniques described herein. Functionality may also be implemented in whole or in part on the "cloud" 720 through the use of distributed systems, such as through platform 722 as described below.

Cloud 720 includes and/or represents a platform 722 for resources 724 . Platform 722 abstracts the underlying functionality of cloud 720's hardware (eg, servers) and software resources. Resources 724 may include applications and/or data that may be used while performing computer processing on a server remote from computing device 710 . Resources 724 may also include services provided over the Internet and/or through subscriber networks, such as cellular or Wi-Fi networks.

Platform 722 may abstract resources and functionality to connect computing device 710 with other computing devices. Platform 722 may also be used to abstract the hierarchy of resources to provide a corresponding level of hierarchy of requirements encountered for resources 324 implemented via platform 722 . Thus, in interconnected device embodiments, implementation of the functionality described herein may be distributed throughout system 700.

For example, functionality may be implemented in part on computing device 710 and through platform 722 that abstracts the functionality of cloud 720 .

It should be understood that, for clarity, embodiments of the present disclosure have been described with reference to different functional modules. However, it will be apparent that the functionality of each functional module may be implemented in a single module, in multiple modules, or as part of other functional modules without departing from the present disclosure. For example, functionality described as performed by a single module may be performed by multiple different modules. Therefore, references to specific functional modules are to be considered merely as references to the appropriate module for providing the described functionality and are not intended to indicate a strict logical or physical structure or organization. Thus, the present disclosure may be implemented in a single module, or may be physically and functionally distributed between different modules and circuits.

It will be understood that, although the terms first, second, third, etc. may be used herein to describe various devices, elements, or components, these devices, elements, or components should not be limited by these terms. These terms are only used to distinguish one device, element, or section from another device, element, or section.

Although the present disclosure has been described in connection with some embodiments, it is not intended to be limited to the specific forms set forth herein. Rather, the scope of the disclosure is limited only by the appended claims. Additionally, although individual features may be included in different claims, these may possibly be advantageously combined, and the inclusion in different claims does not imply that a combination of features is not feasible and/or advantageous. The order of features in the claims does not imply any specific order in which the features must be worked. Furthermore, in the claims, the word "comprising" does not exclude other elements, and the indefinite article "a" or "an" does not exclude a plurality. Reference signs in the claims are provided merely as a clear example and shall not be construed as limiting the scope of the claims in any way.

Claims (9)

1. A medical ultrasound image classification method, characterized in that it includes the following steps: Obtain medical ultrasound image and RGB image data sets; Pre-train the constructed grading model based on the RGB image data set to obtain the first grading model, and migrate the first grading model to medical ultrasound images for fine-tuning training to obtain the second grading model; Among them, the construction process of the hierarchical model includes: based on the ResNeAt network structure, introducing the residual block and attention mechanism, adding the channel attention module and space before the first residual block and after the last residual block. The attention module obtains the first feature map through the channel attention module. The first feature map passes through the spatial attention module and extracts horizontal and vertical features to obtain the second feature map. The first feature map and the second feature map are After weighting processing, the classification results are output through the classification layer; Obtain the grading result based on the medical ultrasound image to be classified and the two-classification model; The ResNeAt network includes four different numbers of residual blocks, three downsampling layers, attention modules and classification layers. Each residual block consists of a depth-separable convolution layer, two ordinary convolutions and a Layer Norm. The downsampling layer consists of a Layer Norm layer and an ordinary convolution layer, and the classification layer consists of a global average pooling layer, a LayerNorm layer and a fully connected layer. 2. A medical ultrasound image classification method as claimed in claim 1, characterized in that, after obtaining the medical ultrasound image, the preprocessing of the medical ultrasound image includes removal of black frames, image grayscale processing, and local histogramming. Graph adaptive equalization, image enhancement processing and data partitioning processing. 3. A medical ultrasound image grading method as claimed in claim 1, characterized in that the medical ultrasound image is divided into a training set, a verification set and a test set, and the saved first grading model is migrated to the training set and the verification set. Retrain and verify the model to obtain the second hierarchical model, and test the second hierarchical model based on the test set until it reaches the usage standard. 4. A medical ultrasound image classification method according to claim 1, characterized in that when pre-training the classification model, a cosine annealing learning rate adjustment algorithm is used to adjust the learning rate. 5. A medical ultrasound image grading method as claimed in claim 1, characterized in that, after obtaining the second grading model, cross-validate the second grading model, and calculate the average and variance of the accuracy obtained by the cross-validation. . 6. A server, characterized in that it includes: an acquisition module configured to acquire medical ultrasound images and RGB image data sets; The training module is configured to pre-train the constructed hierarchical model based on the RGB image data set to obtain the first hierarchical model, and migrate the first hierarchical model to medical ultrasound images for fine-tuning training to obtain the second hierarchical model; Among them, the construction process of the hierarchical model includes: based on the ResNeAt network structure, introducing the residual block and attention mechanism, adding the channel attention module and space before the first residual block and after the last residual block. The attention module obtains the first feature map through the channel attention module. The first feature map passes through the spatial attention module and extracts horizontal and vertical features to obtain the second feature map. The first feature map and the second feature map are After weighting processing, the classification results are output through the classification layer; A grading module configured to obtain a grading result based on the medical ultrasound image to be graded and the two-grade model; The ResNeAt network includes four different numbers of residual blocks, three downsampling layers, attention modules and classification layers. Each residual block consists of a depth-separable convolution layer, two ordinary convolutions and a Layer Norm. The downsampling layer consists of a Layer Norm layer and an ordinary convolution layer, and the classification layer consists of a global average pooling layer, a LayerNorm layer and a fully connected layer. 7. A medical ultrasound image classification system, characterized by including an image acquisition device and an image processing device; The image acquisition device is used to collect medical ultrasound images and RGB image data sets, and send the medical ultrasound images and RGB image data sets to the medical processing device; The image processing device is used to perform the following operations: Pre-train the constructed grading model based on the RGB image data set to obtain the first grading model, and migrate the first grading model to medical ultrasound images for fine-tuning training to obtain the second grading model; Among them, the construction process of the hierarchical model includes: based on the ResNeAt network structure, introducing the residual block and attention mechanism, adding the channel attention module and space before the first residual block and after the last residual block. The attention module obtains the first feature map through the channel attention module. The first feature map passes through the spatial attention module and extracts horizontal and vertical features to obtain the second feature map. The first feature map and the second feature map are After weighting processing, the classification results are output through the classification layer; Obtain the grading result based on the medical ultrasound image to be classified and the two-classification model; The ResNeAt network includes four different numbers of residual blocks, three downsampling layers, attention modules and classification layers. Each residual block consists of a depth-separable convolution layer, two ordinary convolutions and a Layer Norm. The downsampling layer consists of a Layer Norm layer and an ordinary convolution layer, and the classification layer consists of a global average pooling layer, a LayerNorm layer and a fully connected layer. 8. A computer-readable storage medium with a computer program stored thereon, characterized in that when the program is executed by a processor, the medical ultrasound image classification method according to any one of claims 1-5 is implemented. A step of. 9. A computer device, including a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that when the processor executes the program, any of claims 1-5 is implemented. A step in a medical ultrasound image classification method.