CN115063637A - Image classification method, storage medium, and program product - Google Patents

Abstract

The present application relates to an image classification method, storage medium and program product. The method includes: determining a first target segmented image and a second target segmented image corresponding to the lesion area in the medical images under different body positions according to the acquired medical images of the part to be tested under different body positions; the first target segmentation The lesion area included in the image is the lesion area of the first shape, and the lesion area included in the second target segmentation image is the lesion area of the second shape; the image and the second target are segmented according to the first target in each of the postures The segmented image and the preset neural network model identify the category of the lesion area, and determine the target category of the lesion area. Using this method can save labor cost and save classification time.

Description

Image classification method, storage medium and program product

technical field

The present application relates to the technical field of image processing, and in particular, to an image classification method, a storage medium and a program product.

Background technique

With the increasing number of breast diseases in women, contemporary women pay more attention to their breast health. At present, many women regularly go to the hospital to check their breasts so that they can intervene early when they know of breast problems.

In the related art, when a patient goes to the hospital to check the breast, most of the patients first take some breast images, and then the doctor outlines the lesions in the breast image taken by the patient based on experience, and compares it with the existing standard breast signs. Repeated comparison and classification are performed to finally obtain which category the lesions in the patient's breast images belong to.

However, the above method of classifying lesions in breast images has the problem of time-consuming and labor-intensive.

SUMMARY OF THE INVENTION

Based on this, it is necessary to provide an image classification method, a storage medium and a program product that can save labor cost and classification time in view of the above technical problems.

In a first aspect, the present application provides an image classification method, the method comprising:

Determine the first target segmented image and the second target segmented image corresponding to the lesion area in the medical image in each body position according to the acquired medical images of the part to be tested in different body positions; the lesion area included in the first target segmented image is the lesion area of the first shape, and the lesion area included in the second target segmentation image is the lesion area of the second shape;

According to the first target segmented image and the second target segmented image in each body position, and the preset neural network model, the category of the lesion area is identified, and the target category of the lesion area is determined.

In one of the embodiments, the above-mentioned neural network model includes a first classification network and a second classification network; the above-mentioned segmentation images according to the first target and the second target segmentation image in each body position, and the preset neural network model for the lesion area to identify the target category of the lesion area, including:

Input the first target segmentation image and the second target segmentation image under each body position into the first classification network for classification, and determine the feature map and initial category corresponding to the lesion area in each target segmentation image;

According to the feature map and the initial category corresponding to the lesion area in each target segmented image, and the second classification network, the target category of the lesion area is determined.

In one embodiment, the above-mentioned feature map and initial category corresponding to the lesion area in each target segmentation image, and the second classification network, determine the target category of the lesion area, including:

According to the first target segmented image and the second target segmented image in each body position, determine the quantitative feature corresponding to the lesion area in each target segmented image; the above-mentioned quantitative feature is used to characterize the distribution of the lesion area;

The target category of the lesion area is determined according to the quantitative feature corresponding to the lesion area in each target segmented image, the feature map and initial category corresponding to the lesion area in each target segmented image, and the second classification network.

In one embodiment, the above-mentioned quantification feature corresponding to the lesion area in each target segmented image, the feature map and initial category corresponding to the lesion area in each target segmented image, and the second classification network, determine the target category of the lesion area ,include:

Obtain the clinical characteristic information of the object to be tested;

The target category of the lesion area is determined according to the clinical feature information, the quantitative features corresponding to the lesion area in each target segmented image, the feature map and initial category corresponding to the lesion area in each target segmented image, and the second classification network.

In one embodiment, the lesion is determined according to the clinical feature information, the quantitative features corresponding to the lesion area in each target segmented image, the feature map and initial category corresponding to the lesion area in each target segmented image, and the second classification network. Target categories for the area, including:

The clinical feature information, the quantitative features corresponding to the lesion area in each target segmentation image, and the feature map corresponding to the lesion area in each target segmentation image and the initial category are fused, and then input into the second classification network to determine the lesion area. target category;

The second classification network is obtained by training according to the sample feature information sets corresponding to multiple sample objects, and the sample feature information of each sample object includes sample clinical feature information, sample quantitative features, sample feature maps, sample initial categories and lesions The label category for the area.

In one embodiment, the above-mentioned determination of the first target segmented image and the second target segmented image corresponding to the lesion area in the medical image under each body position according to the obtained medical images of the part to be tested in different body positions, includes:

According to the preset first segmentation model and the second segmentation model, segment the lesion area in the medical images of the part to be measured in different body positions respectively, and determine the first target segmentation image and the second target segmentation image corresponding to the medical images in each body position. target segmentation image;

The first segmentation model is obtained by training based on multiple first sample medical images, and each first sample medical image is marked with a lesion area of the first shape; the second segmentation model is based on multiple second sample medical images. Images obtained by training, each second sample medical image is marked with a lesion area of the second shape.

In one of the embodiments, the above-mentioned first classification network is a classification network using an attention mechanism.

In one embodiment, the site to be tested is a breast site, and the different body positions include the CC axial position and the MLO internal oblique position.

In a second aspect, the present application also provides an image classification device, the device comprising:

a determination module, configured to determine the first target segmented image and the second target segmented image corresponding to the lesion area in the medical image under each body position according to the obtained medical images of the part to be tested under different body positions; the above-mentioned first target segmented image The lesion area included in the image is a lesion area of the first shape, and the lesion area included in the second target segmentation image is a lesion area of the second shape;

The classification module is used for identifying the category of the lesion area according to the first target segmentation image and the second target segmentation image in each body position and the preset neural network model, and determining the target category of the lesion area.

In a third aspect, the present application also provides a computer device, the computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

Determine the first target segmented image and the second target segmented image corresponding to the lesion area in the medical image in each body position according to the acquired medical images of the part to be tested in different body positions; the lesion area included in the first target segmented image is the lesion area of the first shape, and the lesion area included in the second target segmentation image is the lesion area of the second shape;

According to the first target segmented image and the second target segmented image in each body position, and the preset neural network model, the category of the lesion area is identified, and the target category of the lesion area is determined.

In a fourth aspect, the present application also provides a computer-readable storage medium, the computer-readable storage medium having a computer program stored thereon, and the computer program implements the following steps when executed by a processor:

Determine the first target segmented image and the second target segmented image corresponding to the lesion area in the medical image in each body position according to the acquired medical images of the part to be tested in different body positions; the lesion area included in the first target segmented image is the lesion area of the first shape, and the lesion area included in the second target segmentation image is the lesion area of the second shape;

According to the first target segmented image and the second target segmented image in each body position, and the preset neural network model, the category of the lesion area is identified, and the target category of the lesion area is determined.

In a fifth aspect, the present application also provides a computer program product, the computer program product includes a computer program that implements the following steps when the computer program is executed by the processor:

Determine the first target segmented image and the second target segmented image corresponding to the lesion area in the medical image in each body position according to the acquired medical images of the part to be tested in different body positions; the lesion area included in the first target segmented image is the lesion area of the first shape, and the lesion area included in the second target segmentation image is the lesion area of the second shape;

According to the first target segmented image and the second target segmented image in each body position, and the preset neural network model, the category of the lesion area is identified, and the target category of the lesion area is determined.

The above-mentioned image classification method, storage medium and program product, determine the first target segmented image and the second target segmented image corresponding to the lesion area in the medical image under each body position by using the medical images of the part to be tested in different body positions, according to each body position The first target segmentation target and the second target segmentation image below and the preset neural network model identify the category of the lesion area, and determine the target category of the lesion area; wherein, the lesion area included in the first target segmentation image is the first target area. The lesion area of the shape, the lesion area included in the second target segmentation image is the second lesion area. Here, because the category of the lesion area can be identified through the preset neural network, without manual classification based on experience, labor costs and classification time can be saved, and high errors caused by manual classification can be avoided, and the classification results can be improved. accuracy. In addition, since two different lesion information, the second lesion and the first-shaped lesion, are combined here, and both the second lesion and the first-shaped lesion are lesions in images in multiple postures, the images of different postures are combined with The lesions of different annotated types are used to classify the lesion area, and the combined information is more and richer, so the obtained classification results are more accurate.

Description of drawings

Fig. 1 is the application environment diagram of the image classification method in one embodiment;

2 is a schematic flowchart of an image classification method in one embodiment;

3 is a schematic flowchart of an image classification method in another embodiment;

FIG. 4 is an example diagram of using the first classification network for classification in another embodiment;

5 is a schematic flowchart of an image classification method in another embodiment;

FIG. 6 is an exemplary diagram of obtaining quantitative features of a lesion area in another embodiment;

7 is a schematic flowchart of an image classification method in another embodiment;

Fig. 8 is a detailed structural example diagram of classifying the lesion area in another embodiment;

Fig. 9 is a structural example diagram of segmenting the lesion area in another embodiment;

10 is a structural block diagram of an image classification apparatus in one embodiment;

Figure 11 is a diagram of the internal structure of a computer device in one embodiment.

Detailed ways

In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

At present, when examining the breast images of patients, it is mainly determined by manual interpretation. Generally, doctors will make the following reports with reference to the existing breast image reporting and data system (BI-RADS reporting system). The system involves many The judgment and classification of imaging signs is a huge workload, and there are differences in the diagnosis of doctors; and generally when BI-RADS reaches 4A, 4B, 4C, and 5 levels, it may be necessary to perform a pathological examination of the breast to determine whether it is a malignant lesion. . Based on this, it can be seen that the prior art method for classifying lesions in breast images has the problem of time-consuming and labor-intensive. Therefore, the embodiments of the present application provide an image classification method, a storage medium and a program product, which can solve the above technical problems.

The image classification method provided in this embodiment of the present application can be applied to the application environment shown in FIG. 1 . Among them, the scanning device 102 is connected and communicated with the computer device 104 . The scanning device 102 can transmit the data obtained after scanning the object to be tested to the computer device 104 for processing. The data storage system may store data that the computer device 104 needs to process. The data storage system can be integrated on the computer device 104, or it can be placed on the cloud or other network server. The scanning device 102 may be a scanning device in which the object to be measured is standing during scanning, or a scanning device in which the object to be measured is lying during scanning, and of course other scanning devices. The computer device 104 can be a terminal or a server; when it is a terminal, it can be but not limited to various personal computers, notebook computers, smart phones, tablet computers, etc. When it is a server, an independent server or multiple A server cluster composed of servers is implemented. In addition, the scanning device 102 and the computer device 104 may be integrated into one device, or may be two separate devices.

In one embodiment, as shown in FIG. 2 , an image classification method is provided. Taking the method applied to the computer device in FIG. 1 as an example, the method may include the following steps:

S202 , according to the acquired medical images of the part to be tested under different body positions, determine a first target segmented image and a second target segmented image corresponding to the lesion area in the medical images under each body position.

Wherein, optionally, the above-mentioned part to be tested is a breast part, the breast part here may be any one breast of the subject to be tested, or two breasts of the subject to be tested, and the lesion area may be a tumor, calcified mass, lumps etc. The above different positions include the CC axial position and the MLO internal oblique position, and of course other positions can also be included. The medical images acquired in each body position may be two-dimensional images or three-dimensional images. Here, medical images under different body positions are obtained, that is, medical images under different views are obtained.

Specifically, the to-be-measured part of the object to be measured can be placed in the scanning device according to a body position requirement, and then the scanning device is used to scan the to-be-measured part to obtain a medical image under the body position, and then the to-be-measured part can be placed according to other body positions. The body position is required to be placed in the scanning device and scanned to obtain a medical image under the body position. By doing so, the medical image of the part to be measured under each body position can be obtained.

After that, segmentation models, segmentation algorithms, or manual segmentation can be used to segment the medical images under each posture, and two target segmentation images corresponding to the medical images in each posture can be obtained, and the two target segmentation images are recorded separately. An image is segmented for the first object and an image is segmented for the second object. The lesion area included in the first target segmented image is a lesion area of a first shape, and the lesion area of the first shape refers to a cloud-shaped lesion in the segmented image. The lesion area included in the second target segmented image is the lesion area of the second shape, and the lesion area of the second shape refers to the lesion area in the segmented image in the shape of a star point, that is, each lesion area is point-like, separate Distributed, unconnected lesions.

Usually, the lesion area of the first shape and the lesion area of the second shape can be divided by a threshold, and the threshold can be thresholds such as area and volume. For example, the lesion area with an area greater than the threshold is the lesion area of the first shape, and the area is less than or equal to the threshold. The lesion area of is the second shape lesion area. Generally, the lesion area of the first shape is larger than the lesion area of the second shape, and the size relationship of the lesion area can be measured by area, volume, etc. For example, the area of the lesion area of the first shape is larger than the area of the lesion area of the second shape. . In addition, in some cases, the lesion area of the first shape may also be a lesion area formed by connecting a plurality of lesion areas of the second shape.

It should be noted that the lesion areas targeted in the first target segmented image and the second target segmented image are the same lesions in the medical image, but exist in different forms in each target segmented image.

S204: Identify the category of the lesion area according to the first target segmented image and the second target segmented image in each body position and a preset neural network model, and determine the target category of the lesion area.

In this step, the neural network model may be a model composed of classification networks, which may include one classification network, or multiple classification networks, and may also include other networks, such as segmentation networks, feature extraction networks, and the like.

After obtaining the first target segmented image and the second target segmented image in each posture, the two target segmented images in each posture can be combined and input into the neural network model to classify the lesion area and obtain the target of the lesion area. It is also possible to obtain the relevant feature information of the lesion area through the two target segmentation images in each body position, and then fuse the obtained feature information and input it into the neural network model to classify the lesion area and obtain the target category of the lesion area; of course It can also be input into the neural network model in combination with other data to classify the lesion area to obtain the target category of the lesion area, which is not specifically limited here, in short, the target category of the lesion area can be obtained.

In addition, when determining the target category of the lesion area, a possible implementation may be to output the probability that the lesion area belongs to each category through a neural network model, and select the maximum probability from the probabilities of each category, and the maximum probability The corresponding category is used as the target category of the lesion area.

It can be seen from the above description that it is precisely because the shape and other characteristics formed by the lesion area in the medical image of the part to be tested under different body positions will produce obvious differences, and the characteristics of the lesions that can be represented by lesions of different shapes are different. Therefore, lesions of different postures and shapes are used to identify the types of lesions, so that the combined feature information is richer, and the more relevant information can be referenced during classification, so the classification results obtained are more accurate.

In the above image classification method, the first target segmented image and the second target segmented image corresponding to the lesion area in the medical image under each body position are determined by using the medical images of the part to be tested in different body positions, and according to the first target segment under each body position The segmentation target and the second target segmentation image and the preset neural network model identify the category of the lesion area, and determine the target category of the lesion area; wherein, the lesion area included in the first target segmentation image is the lesion area of the first shape, The lesion area included in the second target segmented image is the second lesion area. Here, because the category of the lesion area can be identified through the preset neural network, without manual classification based on experience, labor costs and classification time can be saved, and high errors caused by manual classification can be avoided, and the classification results can be improved. accuracy. In addition, since two different lesion information, the second lesion and the first-shaped lesion, are combined here, and both the second lesion and the first-shaped lesion are lesions in images in multiple postures, the images of different postures are combined with The lesions of different annotated types are used to classify the lesion area, and the combined information is more and richer, so the obtained classification results are more accurate.

In the above-mentioned embodiment, it is mentioned that the classification of the lesion area is identified by the two target segmentation images under each body position of the neural network model. , the specific process of how to identify the category of lesions is explained.

In another embodiment, as shown in FIG. 3, another image classification method is provided. On the basis of the foregoing embodiment, the foregoing S204 may include the following steps:

S302: Input the first target segmented image and the second target segmented image in each body position into a first classification network for classification, and determine a feature map and an initial category corresponding to the lesion area in each target segmented image.

Specifically, as shown in FIG. 4 , after obtaining two target segmentation images in each body position, each target segmentation image in each body position can be input into the first classification network in turn, and convolution, pooling, and dense volume are performed. Block processing, fully connected layer and other processing, to obtain the feature map corresponding to each target segmentation image and the category of the lesion area under each body position. Since the category of the lesion area obtained here is not the final category, it is recorded as the initial category, and the initial category here may also be determined by selecting the category corresponding to the maximum probability from the obtained multiple classification probabilities.

In addition, the above-mentioned first classification network also belongs to a neural network model. Optionally, the first classification network is a classification network using an attention mechanism. Generally, for the part to be tested is the breast part, the lesions on the breast are more distributed, so here we choose to use target segmentation images in various body positions and a variety of different lesion annotations (for example, the first target segmentation image is the first target segmentation image. The lesions of the shape, that is, the cloud patch annotation view in the figure, and the second target segmentation image is the lesion of the second shape, that is, the star point annotation view in the figure), and the classification network of the attention mechanism is used to learn various postures and multi-positions. In order to improve the stability and accuracy of the classification of the lesion area, the lesion feature information in the case of various lesions is marked.

Further, the above-mentioned first classification network may be a network trained by using mean square error, which can also improve the accuracy of the trained classification network and further improve the accuracy of classifying the lesion area.

S304 , according to the feature map and the initial category corresponding to the lesion area in each target segmented image, and the second classification network, determine the target category of the lesion area.

In this step, after obtaining the feature map corresponding to the lesion area in each target segmentation image under each body position and the initial category of the lesion area, each feature map of the lesion area may be input into the second classification network for classification, At the same time, the target category of the lesion area is comprehensively determined in combination with each initial category of the lesion area; it can also be classified by inputting each target segmentation image under each body position, each feature map and each initial category into the second classification network. Obtain the target category of the lesion area; of course, other situations are also possible, which are not specifically limited here.

In this embodiment, by inputting each target segmented image under each body position into the first classification network for classification, the feature map and initial category of the lesion area are obtained, and the target category of the lesion area is determined in combination with the second classification network. The two cascaded classification networks determine the target category of the lesion area, and the accuracy of the determined category of the lesion area can be improved by means of layer-by-layer progressive classification. Further, the first classification network is a classification network using an attention mechanism, which can further improve the stability and accuracy of classifying the lesion area.

In the above embodiment, it is mentioned that the classification of the lesion area can be identified by combining the first classification network and the second classification network, and the identification process of the first classification network is described in detail. The following embodiments describe the process of how to identify the second classification network. Explain in detail.

In another embodiment, as shown in FIG. 5, another image classification method is provided. On the basis of the foregoing embodiment, the foregoing S304 may include the following steps:

S402 , according to the first target segmented image and the second target segmented image in each body position, determine a quantitative feature corresponding to the lesion area in each target segmented image; the above-mentioned quantitative feature is used to characterize the distribution of the lesion area.

In this step, the first target segmented image is an image including a lesion area with a first shape, and the second target segmented image is an image including a lesion area with a second shape.

Here, after obtaining the segmented images of each target in each body position, as shown in FIG. 6 , methods such as feature extraction model or manual extraction method can be used to perform omics feature analysis on the lesion area in the first target segmented image in each body position. Extracting and obtaining the omics features corresponding to the lesion area in each first target segmentation image, which may also be recorded as the quantitative features corresponding to the lesion area. Here, the quantified features obtained from the lesion area in each first target segmented image are generally multiple quantified features, and each quantified feature is used as a dimension, and each first target segmented image can obtain N-dimensional features, where N is greater than or equal to 1.

For example, there may be more than 100 omics features of each first target segmented image, such as shape features (such as flatness, elongation, etc.), pixel-level statistical features (such as entropy, 10% gray value, etc.) ), texture features (such as gray level co-occurrence matrix, etc.), etc.

Similarly, methods such as feature extraction models or manual extraction methods can also be used to extract statistical features for the lesion areas in the second target segmented images in each body position, and obtain the corresponding data of the lesion areas in each second target segmented image. Statistical features can also be recorded as quantitative features corresponding to the lesion area. Here, the quantitative features obtained from the lesion area in each second target segmentation image are generally multiple quantitative features, and each quantitative feature is used as a dimension, then each second target segmentation image can obtain M-dimensional features, where M is greater than or equal to 1, M and N may or may not be equal in size.

For example, there may be more than 50 omics features of each second target segmented image, such as distribution features (such as minimum distance value, number of outliers, etc.), pixel-level statistical features (such as entropy, 10% gray level, etc.) value, etc.) etc.

S404: Determine the target category of the lesion area according to the quantitative feature corresponding to the lesion area in each target segmented image, the feature map and initial category corresponding to the lesion area in each target segmented image, and the second classification network.

In this step, after obtaining the quantified features of multiple dimensions corresponding to the lesion area in each target segmented image, the above feature maps of each target segmented image and the initial category of the lesion area can be combined, and other information can also be input to the Classify in the second classification network to obtain the target class of the lesion area.

In this embodiment, the quantified features corresponding to the lesion area in each target segmentation image are obtained from the target segmentation images in each body position, and the lesion area is obtained by combining the feature map of each target segmentation image, the initial category of the lesion area, and the second classification network. The target category of the area, the quantitative feature here can represent the distribution of the lesion area, so that the characteristics of the lesion can be characterized in an all-round way, and then the lesion can be comprehensively analyzed, so that the obtained lesion category is more accurate.

When actually classifying the lesion area of the object to be tested, in order to consider the actual perception of the object to be tested and further improve the accuracy, the analysis can also be carried out in combination with relevant clinical information. The information to carry out the classification of the lesion area is described in detail.

In another embodiment, as shown in FIG. 7, another image classification method is provided. On the basis of the above embodiment, the above S404 may include the following steps:

S502: Acquire clinical feature information of the object to be tested.

Wherein, the above-mentioned clinical feature information includes at least the perception information of the to-be-measured part of the to-be-measured object and/or the medical history information of the to-be-measured object. The perception information may be the pain perception of the object to be tested at the site to be tested. Of course, the clinical feature information may also include other information, for example, the age, gender, occupation, and purpose of filming (such as physical examination, medical treatment, etc.) of the object to be tested.

Specifically, the clinical feature information of the object to be tested can be obtained through processes such as interaction with the object to be tested before the object to be tested is examined, and the clinical feature information can be input into a computer device for storage.

S504: Determine the target category of the lesion area according to the clinical feature information, the quantitative feature corresponding to the lesion area in each target segmented image, the feature map and initial category corresponding to the lesion area in each target segmented image, and the second classification network.

In this step, after obtaining the clinical feature information of the object to be tested, the quantitative features corresponding to the lesion area in each target segmentation image, the feature map and initial category corresponding to the lesion area in each target segmentation image, optionally, you can The clinical feature information, the quantitative features corresponding to the lesion area in each target segmentation image, and the feature map corresponding to the lesion area in each target segmentation image and the initial category are fused, and then input into the second classification network to determine the lesion area. target category.

Referring to the example diagram of the overall structure shown in FIG. 8 , after obtaining the above-mentioned various information, for example, two target segmentation images in two body positions, a total of four target segmentation images, then four feature maps (that is, in the figure) can be obtained. Featuremap 1/2/3/4) and four initial classes (i.e. classification probabilities 1/2/3/4 in the figure). Two first target segmentation images can obtain two N-dimensional features, and two second target segmentation images can obtain two M-dimensional features, that is, M*2-dimensional features and N*2-dimensional features are obtained.

After that, four feature maps, four initial categories, M*2-dimensional features, N*2-dimensional features, and clinical feature information of the object to be tested can be fused and input into the second classification network for classification of lesion categories. Identify and obtain the target category of the lesion area. For example, taking binary classification as an example, the probability of belonging to two categories, such as Probability1 and Probability0, may be output through the second classification network, and the category with the highest probability may be selected as the target category.

In addition, the second classification network here may be a network using DenseNet (dense net), and before the second classification network is used here, the second classification network may also be generally trained. The second classification network is obtained by training according to the sample feature information sets corresponding to multiple sample objects, and the sample feature information of each sample object includes sample clinical feature information, sample quantitative features, sample feature maps, sample initial categories and lesions The label category for the area.

That is to say, during training, the target segmentation images of multiple body positions of multiple sample objects can be obtained first, and the corresponding sample quantization features, sample feature maps and sample initial categories can be obtained through the target segmentation images. The labeling category is set for the lesion area, and the sample clinical feature information of each sample object can also be obtained; after that, the sample clinical feature information, sample quantitative feature, sample feature map and sample initial category of each sample object and the labeling category of the lesion area can be used as A sample feature information set is used to train the second classification network to obtain the trained second classification network.

In this embodiment, the lesion is obtained by combining the clinical feature information of the object to be tested, the quantitative features corresponding to the lesion area in each target segmented image, the feature map of each target segmented image, the initial category of the lesion area, and the second classification network. The target category of the area is combined with the clinical feature information of the object to be tested, so that the final determined category of the lesion area is directly related to the individual, and is more in line with the actual situation of the individual, that is, more accurate.

The following embodiments describe in detail the process of how to obtain the corresponding two target segmentation images from the medical images in each body position.

In another embodiment, another image classification method is provided. On the basis of the foregoing embodiment, the foregoing S202 may include the following steps:

According to the preset first segmentation model and the second segmentation model, segment the lesion area in the medical images of the part to be measured in different body positions respectively, and determine the first target segmentation image and the second target segmentation image corresponding to the medical images in each body position. target segmentation image.

Specifically, as shown in FIG. 9 , the detection model can be used to detect the lesion area on the medical image in each body position to obtain the detection result of the lesion area, that is, the lesion area is first located in the medical image in each body position. After that, the first segmentation model can be used to continue to perform lesion segmentation processing on the medical images in each position where the lesion area is located, to obtain a first target segmentation image in each position, and the lesion area included in each first target segmentation image is the first target segmentation image. At the same time, the second segmentation model can also be used to perform lesion segmentation processing on the medical images in each position where the lesion area is located, so as to obtain the second target segmentation image in each position, and in each second target segmentation image The lesion area included is the lesion area of the second shape.

That is to say, here the detection model and the first segmentation model can be in a cascade relationship, that is, the detection model is used to first locate the lesion area in the medical image, that is, perform rough segmentation, and then input the detection result into the first segmentation model. Perform fine segmentation, perform fine segmentation on the edge of the lesion area, and obtain a first target segmentation image. Similarly, the detection model and the second segmentation model can also be in a cascade relationship, that is, the detection model is used to first locate the lesion area in the medical image, that is, perform rough segmentation, and then input the detection results into the second segmentation model for Fine segmentation, the edge of the lesion area is finely segmented to obtain a second target segmented image.

Here, the lesion area is segmented through a cascaded detection network and a segmentation model. Since the lesion area can be quickly located by the detection model, the accuracy and efficiency of subsequent segmentation of the lesion area by the segmentation model can be improved.

In addition, before using the detection model, the first segmentation model and the second segmentation model to segment the lesion area, these models may also be trained first. The detection model here may be obtained by training on pre-collected sample images and their labeling data, where the labeling data includes detection frame information of the lesion area. The first segmentation model may be obtained by training based on a plurality of first sample medical images, and each first sample medical image is marked with a lesion area of a first shape. The second segmentation model may be obtained by training based on a plurality of second sample medical images, and each second sample medical image is marked with a lesion area of the second shape.

In this embodiment, the pre-trained first segmentation model and the second segmentation model are used to perform segmentation processing on the medical images in each body position to obtain the first target segmentation image and the second target segmentation image in each body position, in which the first target segmentation image and the second target segmentation image are obtained. The segmentation model is trained based on the sample images of the lesion area marked with the first shape, and the first segmentation model is trained based on the sample images of the lesion area marked with the second shape. The accuracy of medical image segmentation; in addition, using a segmentation model to segment medical images can also improve the efficiency of segmentation when there are many medical images.

It should be noted that the lines in FIGS. 4 , 6 , 8 , and 9 do not affect the essential content of the embodiments of the present application.

It should be understood that, although the steps in the flowcharts involved in the above embodiments are sequentially displayed according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, the execution of these steps is not strictly limited to the order, and these steps may be performed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but may be performed at different times The execution order of these steps or phases is not necessarily sequential, but may be performed alternately or alternately with other steps or at least a part of the steps or phases in the other steps.

Based on the same inventive concept, an embodiment of the present application also provides an image classification apparatus for implementing the above-mentioned image classification method. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme described in the above method, so the specific limitations in the embodiments of one or more image classification apparatuses provided below can refer to the above limitations on the image classification method, It is not repeated here.

In one embodiment, as shown in FIG. 10, an image classification apparatus is provided, including a determination module 11 and a classification module 12, wherein:

The determination module 11 is configured to determine, according to the obtained medical images of the part to be tested under different body positions, the first target segmented image and the second target segmented image corresponding to the lesion area in the medical images under each body position; the above-mentioned first target segmentation The lesion area included in the image is the lesion area of the first shape, and the lesion area included in the second target segmentation image is the lesion area of the second shape;

The classification module 12 is configured to identify the category of the lesion area according to the first target segmented image and the second target segmented image in each body position and a preset neural network model, and determine the target category of the lesion area.

Optionally, the above-mentioned part to be tested is a breast part, and the above-mentioned different body positions include CC axial position and MLO internal oblique position.

In another embodiment, another image classification apparatus is provided. On the basis of the foregoing embodiment, the foregoing neural network model includes a first classification network and a second classification network; the foregoing classification module 12 may include:

The first classification unit is used to input the first target segmentation image and the second target segmentation image under each body position into the first classification network for classification, and determine the feature map and the initial category corresponding to the lesion area in each target segmentation image;

The second classification unit is configured to determine the target category of the lesion area according to the feature map and the initial category corresponding to the lesion area in each target segmentation image, and the second classification network.

Optionally, the above-mentioned first classification network is a classification network using an attention mechanism.

In another embodiment, another image classification apparatus is provided, and on the basis of the foregoing embodiment, the foregoing second classification unit may include:

The quantitative feature determination subunit is used to determine the quantitative feature corresponding to the lesion area in each target segmented image according to the first target segmented image and the second target segmented image under each body position; the above-mentioned quantitative feature is used to characterize the distribution of the lesion area ;

The classification subunit is used to determine the target category of the lesion area according to the quantitative feature corresponding to the lesion area in each target segmented image, the feature map and initial category corresponding to the lesion area in each target segmented image, and the second classification network.

In another embodiment, another image classification device is provided. On the basis of the above-mentioned embodiment, the above-mentioned classification subunit is specifically used to obtain clinical feature information of the object to be tested; The quantitative feature corresponding to the lesion area in each target segmented image, the feature map and the initial category corresponding to the lesion area in each target segmentation image, and the second classification network to determine the target category of the lesion area.

In another embodiment, another image classification apparatus is provided. On the basis of the foregoing embodiment, the foregoing classification subunit is specifically configured to classify clinical feature information, quantitative features corresponding to lesion areas in each target segmented image, and After the feature map corresponding to the lesion area in each target segmentation image and the initial category are feature fusion, they are input into the second classification network to determine the target category of the lesion area; wherein, the second classification network is based on samples corresponding to multiple sample objects. The sample feature information of each sample object includes sample clinical feature information, sample quantitative features, sample feature map and sample initial category and the labeling category of the lesion area, obtained by training the feature information set.

In another embodiment, another image classification apparatus is provided. On the basis of the foregoing embodiment, the foregoing determining module 11 may include:

The segmentation unit is used for segmenting and processing the lesion areas in the medical images of the parts to be measured in different postures according to the preset first segmentation model and the second segmentation model, and determining the first target corresponding to the medical images in each posture The segmented image and the second target segmented image; wherein, the first segmentation model is obtained by training based on a plurality of first sample medical images, and each first sample medical image is marked with a lesion area of the first shape; the second segmentation The model is obtained by training based on a plurality of second sample medical images, and each second sample medical image is marked with a lesion area of a second shape.

Each module in the above image classification apparatus can be implemented in whole or in part by software, hardware and combinations thereof. The above modules can be embedded in or independent of the processor in the computer device in the form of hardware, or stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

In one embodiment, a computer device is provided. Taking the computer device as a terminal as an example, its internal structure diagram may be as shown in FIG. 11 . The computer equipment includes a processor, memory, a communication interface, a display screen, and an input device connected by a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium, an internal memory. The nonvolatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the execution of the operating system and computer programs in the non-volatile storage medium. The communication interface of the computer equipment is used for wired or wireless communication with an external terminal, and the wireless communication can be realized by WIFI, mobile cellular network, NFC (Near Field Communication) or other technologies. The computer program, when executed by a processor, implements an image classification method. The display screen of the computer equipment may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment may be a touch layer covered on the display screen, or a button, a trackball or a touchpad set on the shell of the computer equipment , or an external keyboard, trackpad, or mouse.

Those skilled in the art can understand that the structure shown in FIG. 11 is only a block diagram of a partial structure related to the solution of the present application, and does not constitute a limitation on the computer equipment to which the solution of the present application is applied. Include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.

In one embodiment, a computer device is provided, including a memory and a processor, a computer program is stored in the memory, and the processor implements the following steps when executing the computer program:

Determine the first target segmented image and the second target segmented image corresponding to the lesion area in the medical image in each body position according to the acquired medical images of the part to be tested in different body positions; the lesion area included in the first target segmented image is the lesion area of the first shape, and the lesion area included in the second target segmentation image is the lesion area of the second shape; according to the first target segmentation image and the second target segmentation image under each body position, and the preset neural network The model identifies the category of the lesion area and determines the target category of the lesion area.

In one embodiment, the processor further implements the following steps when executing the computer program:

Input the first target segmentation image and the second target segmentation image under each body position into the first classification network for classification, and determine the feature map and initial category corresponding to the lesion area in each target segmentation image; The feature map and the initial category corresponding to the lesion area, and the second classification network, determine the target category of the lesion area.

In one embodiment, the processor further implements the following steps when executing the computer program:

According to the first target segmented image and the second target segmented image in each body position, determine the quantitative feature corresponding to the lesion area in each target segmented image; the above-mentioned quantitative feature is used to characterize the distribution of the lesion area; The quantitative feature corresponding to the lesion area, the feature map and the initial category corresponding to the lesion area in each target segmented image, and the second classification network determine the target category of the lesion area.

In one embodiment, the processor further implements the following steps when executing the computer program:

Obtain the clinical feature information of the object to be tested; according to the clinical feature information, the quantitative feature corresponding to the lesion area in each target segmentation image, the feature map and initial category corresponding to the lesion area in each target segmentation image, and the second classification network, determine The target category of the lesion area.

In one embodiment, the processor further implements the following steps when executing the computer program:

The clinical feature information, the quantitative features corresponding to the lesion area in each target segmentation image, and the feature map corresponding to the lesion area in each target segmentation image and the initial category are fused, and then input into the second classification network to determine the lesion area. Target category; wherein, the second classification network is obtained by training according to the sample feature information set corresponding to multiple sample objects, and the sample feature information of each sample object includes sample clinical feature information, sample quantitative feature, sample feature map and sample initial category and the labeling category of the lesion area.

In one embodiment, the processor further implements the following steps when executing the computer program:

According to the preset first segmentation model and the second segmentation model, segment the lesion area in the medical images of the part to be measured in different body positions respectively, and determine the first target segmentation image and the second target segmentation image corresponding to the medical images in each body position. target segmentation image; wherein, the first segmentation model is obtained by training based on multiple first sample medical images, and each first sample medical image is marked with a lesion area of the first shape; the second segmentation model is based on multiple first sample medical images. The second sample medical images are obtained by training, and each second sample medical image is marked with a lesion area of the second shape.

In one embodiment, the above-mentioned first classification network is a classification network using an attention mechanism.

In one embodiment, the site to be tested is a breast site, and the different body positions include a CC axial position and an MLO internal oblique position.

In one embodiment, a computer-readable storage medium is provided on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented:

Determine the first target segmented image and the second target segmented image corresponding to the lesion area in the medical image in each body position according to the acquired medical images of the part to be tested in different body positions; the lesion area included in the first target segmented image is the lesion area of the first shape, and the lesion area included in the second target segmentation image is the lesion area of the second shape; according to the first target segmentation image and the second target segmentation image under each body position, and the preset neural network The model identifies the category of the lesion area and determines the target category of the lesion area.

In one embodiment, the computer program further implements the following steps when executed by the processor:

Input the first target segmentation image and the second target segmentation image under each body position into the first classification network for classification, and determine the feature map and initial category corresponding to the lesion area in each target segmentation image; The feature map and the initial category corresponding to the lesion area, and the second classification network, determine the target category of the lesion area.

In one embodiment, the computer program further implements the following steps when executed by the processor:

According to the first target segmented image and the second target segmented image in each body position, determine the quantitative feature corresponding to the lesion area in each target segmented image; the above-mentioned quantitative feature is used to characterize the distribution of the lesion area; The quantitative feature corresponding to the lesion area, the feature map and the initial category corresponding to the lesion area in each target segmented image, and the second classification network determine the target category of the lesion area.

In one embodiment, the computer program further implements the following steps when executed by the processor:

Obtain the clinical feature information of the object to be tested; according to the clinical feature information, the quantitative feature corresponding to the lesion area in each target segmentation image, the feature map and initial category corresponding to the lesion area in each target segmentation image, and the second classification network, determine The target category of the lesion area.

In one embodiment, the computer program further implements the following steps when executed by the processor:

The clinical feature information, the quantitative features corresponding to the lesion area in each target segmentation image, and the feature map corresponding to the lesion area in each target segmentation image and the initial category are fused, and then input into the second classification network to determine the lesion area. Target category; wherein, the second classification network is obtained by training according to the sample feature information set corresponding to multiple sample objects, and the sample feature information of each sample object includes sample clinical feature information, sample quantitative feature, sample feature map and sample initial category and the labeling category of the lesion area.

In one embodiment, the computer program further implements the following steps when executed by the processor:

According to the preset first segmentation model and the second segmentation model, segment the lesion area in the medical images of the part to be measured in different body positions respectively, and determine the first target segmentation image and the second target segmentation image corresponding to the medical images in each body position. target segmentation image; wherein, the first segmentation model is obtained by training based on multiple first sample medical images, and each first sample medical image is marked with a lesion area of the first shape; the second segmentation model is based on multiple first sample medical images. The second sample medical images are obtained by training, and each second sample medical image is marked with a lesion area of the second shape.

In one embodiment, the above-mentioned first classification network is a classification network using an attention mechanism.

In one embodiment, the site to be tested is a breast site, and the different body positions include a CC axial position and an MLO internal oblique position.

In one embodiment, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the following steps:

Determine the first target segmented image and the second target segmented image corresponding to the lesion area in the medical image in each body position according to the acquired medical images of the part to be tested in different body positions; the lesion area included in the first target segmented image is the lesion area of the first shape, and the lesion area included in the second target segmentation image is the lesion area of the second shape; according to the first target segmentation image and the second target segmentation image under each body position, and the preset neural network The model identifies the category of the lesion area and determines the target category of the lesion area.

In one embodiment, the computer program further implements the following steps when executed by the processor:

Input the first target segmentation image and the second target segmentation image under each body position into the first classification network for classification, and determine the feature map and initial category corresponding to the lesion area in each target segmentation image; The feature map and the initial category corresponding to the lesion area, and the second classification network, determine the target category of the lesion area.

In one embodiment, the computer program further implements the following steps when executed by the processor:

According to the first target segmented image and the second target segmented image in each body position, determine the quantitative feature corresponding to the lesion area in each target segmented image; the above-mentioned quantitative feature is used to characterize the distribution of the lesion area; The quantitative feature corresponding to the lesion area, the feature map and the initial category corresponding to the lesion area in each target segmented image, and the second classification network determine the target category of the lesion area.

In one embodiment, the computer program further implements the following steps when executed by the processor:

Obtain the clinical feature information of the object to be tested; according to the clinical feature information, the quantitative feature corresponding to the lesion area in each target segmentation image, the feature map and initial category corresponding to the lesion area in each target segmentation image, and the second classification network, determine The target category of the lesion area.

In one embodiment, the computer program further implements the following steps when executed by the processor:

The clinical feature information, the quantitative features corresponding to the lesion area in each target segmentation image, and the feature map corresponding to the lesion area in each target segmentation image and the initial category are fused, and then input into the second classification network to determine the lesion area. Target category; wherein, the second classification network is obtained by training according to the sample feature information set corresponding to multiple sample objects, and the sample feature information of each sample object includes sample clinical feature information, sample quantitative feature, sample feature map and sample initial category and the labeling category of the lesion area.

In one embodiment, the computer program further implements the following steps when executed by the processor:

According to the preset first segmentation model and the second segmentation model, segment the lesion area in the medical images of the part to be measured in different body positions respectively, and determine the first target segmentation image and the second target segmentation image corresponding to the medical images in each body position. target segmentation image; wherein, the first segmentation model is obtained by training based on multiple first sample medical images, and each first sample medical image is marked with a lesion area of the first shape; the second segmentation model is based on multiple first sample medical images. The second sample medical images are obtained by training, and each second sample medical image is marked with a lesion area of the second shape.

In one embodiment, the above-mentioned first classification network is a classification network using an attention mechanism.

In one embodiment, the site to be tested is a breast site, and the different body positions include a CC axial position and an MLO internal oblique position.

It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data for analysis, stored data, displayed data, etc.) involved in this application are all Information and data authorized by the user or fully authorized by the parties.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage In the medium, when the computer program is executed, it may include the processes of the above-mentioned method embodiments. Wherein, any reference to a memory, a database or other media used in the various embodiments provided in this application may include at least one of a non-volatile memory and a volatile memory. Non-volatile memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetic variable memory (Magnetoresistive Random Memory) Access Memory, MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (Phase Change Memory, PCM), graphene memory, etc. Volatile memory may include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration and not limitation, the RAM may be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM). The database involved in the various embodiments provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a blockchain-based distributed database, etc., but is not limited thereto. The processors involved in the various embodiments provided in this application may be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, data processing logic devices based on quantum computing, etc., and are not limited to this.

The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features It is considered to be the range described in this specification.

The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are relatively specific and detailed, but should not be construed as a limitation on the scope of the patent of the present application. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the present application should be determined by the appended claims.

Claims (10)

1. A method of image classification, the method comprising:

determining a first target segmentation image and a second target segmentation image corresponding to a focus area in the medical image of each body position according to the acquired medical images of the part to be detected in different body positions; a lesion region included in the first target segmentation image is a lesion region of a first shape, and a lesion region included in the second target segmentation image is a lesion region of a second shape;

and identifying the category of the focus area according to the first target segmentation image and the second target segmentation image in each body position and a preset neural network model, and determining the target category of the focus area.

2. The method of claim 1, wherein the neural network model comprises a first classification network and a second classification network; the identifying the type of the focus area according to the first target segmentation image and the second target segmentation image in each body position and a preset neural network model, and determining the target type of the focus area comprise:

inputting the first target segmentation image and the second target segmentation image in each body position into the first classification network for classification, and determining a feature map and an initial class corresponding to the focus area in each target segmentation image;

and determining the target class of the focus area according to the feature map and the initial class corresponding to the focus area in each target segmentation image and the second classification network.

3. The method according to claim 2, wherein the determining the target classification of the lesion area according to the feature map and the initial classification corresponding to the lesion area in each target segmentation image and the second classification network comprises:

determining a quantitative feature corresponding to the focus region in each target segmentation image according to the first target segmentation image and the second target segmentation image in each body position; the quantitative characteristics are used for characterizing the distribution of the lesion area;

and determining the target class of the focus region according to the quantitative feature corresponding to the focus region in each target segmentation image, the feature map and the initial class corresponding to the focus region in each target segmentation image and the second classification network.

4. The method according to claim 3, wherein the determining the target classification of the lesion area according to the quantified feature corresponding to the lesion area in each target segmentation image, the feature map and the initial classification corresponding to the lesion area in each target segmentation image, and the second classification network comprises:

acquiring clinical characteristic information of a to-be-detected object;

and determining the category of the lesion area according to the clinical feature information, the quantitative feature corresponding to the lesion area in each target segmentation image, the feature map and the initial category corresponding to the lesion area in each target segmentation image, and the second classification network.

5. The method of claim 4, wherein determining the category of the lesion area according to the clinical feature information, the quantified feature corresponding to the lesion area in each target segmented image, the feature map and the initial category corresponding to the lesion area in each target segmented image, and the second classification network comprises:

after feature fusion is carried out on the clinical feature information, the quantitative features corresponding to the focus areas in the target segmentation images, and the feature maps and the initial categories corresponding to the focus areas in the target segmentation images, the feature fusion is input into the second classification network, and the target categories of the focus areas are determined;

the second classification network is obtained by training according to a sample feature information set corresponding to a plurality of sample objects, and the sample feature information of each sample object comprises sample clinical feature information, sample quantitative features, a sample feature map, a sample initial category and a labeling category of a lesion region.

6. The method according to any one of claims 1 to 5, wherein the determining, according to the acquired medical images of the part to be measured in different body positions, a first target segmentation image and a second target segmentation image corresponding to a lesion area in the medical images in the body positions comprises:

according to a preset first segmentation model and a preset second segmentation model, respectively segmenting focus regions of the medical image of the part to be detected in different body positions, and determining a first target segmentation image and a second target segmentation image corresponding to the medical image in each body position;

the first segmentation model is obtained by training based on a plurality of first sample medical images, and each first sample medical image is marked with a focus area of a first shape; the second segmentation model is obtained by training based on a plurality of second sample medical images, and each second sample medical image is marked with a focus region of a second shape.

7. The method according to any of claims 2-5, wherein the first classification network is a classification network employing an attention mechanism.

8. The method of any one of claims 1-5, wherein the site to be measured is a breast site and the different body positions include CC axial position and MLO oblique position.

9. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the method of any one of claims 1 to 8.

10. A computer program product comprising a computer program, characterized in that the computer program realizes the steps of the method of any one of claims 1 to 8 when executed by a processor.

Images