CN110852367B - Image classification method, computer device, and storage medium - Google Patents
Image classification method, computer device, and storage medium Download PDFInfo
- Publication number
- CN110852367B CN110852367B CN201911068736.1A CN201911068736A CN110852367B CN 110852367 B CN110852367 B CN 110852367B CN 201911068736 A CN201911068736 A CN 201911068736A CN 110852367 B CN110852367 B CN 110852367B
- Authority
- CN
- China
- Prior art keywords
- image
- training sample
- classified
- images
- network
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000012549 training Methods 0.000 claims abstract description 254
- 239000011159 matrix material Substances 0.000 claims abstract description 78
- 238000000605 extraction Methods 0.000 claims description 20
- 230000006870 function Effects 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 15
- 238000005096 rolling process Methods 0.000 claims description 13
- 230000008569 process Effects 0.000 claims description 11
- 238000010276 construction Methods 0.000 claims description 5
- 238000012360 testing method Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000000694 effects Effects 0.000 description 7
- 238000013527 convolutional neural network Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 4
- 230000003925 brain function Effects 0.000 description 3
- 239000013598 vector Substances 0.000 description 3
- 238000013528 artificial neural network Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000002591 computed tomography Methods 0.000 description 2
- 238000002599 functional magnetic resonance imaging Methods 0.000 description 2
- 238000002610 neuroimaging Methods 0.000 description 2
- 230000001360 synchronised effect Effects 0.000 description 2
- 208000024827 Alzheimer disease Diseases 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000002059 diagnostic imaging Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/03—Recognition of patterns in medical or anatomical images
Abstract
The present invention relates to an image classification method, a computer device and a storage medium, the method comprising: acquiring an image to be classified and a plurality of training sample images; the images to be classified and the plurality of training sample images are images of the same part of different patients; respectively combining the images to be classified with a plurality of training sample images, inputting a relational network model to obtain the distance between the images to be classified and each training sample image, and constructing a graph characteristic matrix of the images to be classified and the plurality of training sample images according to the distance between the images to be classified and each training sample image; and inputting the graph characteristic matrix into a classification network model to obtain a classification result of the image to be classified. The method improves the accuracy of the obtained classification result of the image to be classified.
Description
Technical Field
The present invention relates to the field of images, and in particular, to an image classification method, a computer device, and a storage medium.
Background
Along with the development of the deep learning network, the convolutional neural network and the variant thereof are increasingly applied to medical images, and the convolutional neural network is used for identifying medical images, so that the method is one of the wider applications. However, the identification of medical images by the existing convolutional neural network is limited to independent individuals, namely, similarity and relevance among patients are not considered in the training and testing process, so that the performance effect of the convolutional neural network on small samples is limited.
In the conventional technology, for a learning mode of a small sample, matrix learning is performed, by measuring the distance between a test sample and a training sample, classification is completed by means of the nearest neighbor idea, for example, a twinning network uses two networks with identical structures in a supervised training mode, and the distance between the test sample and the training sample is measured by adopting a preset linear measure, such as a vector inner product, a Euclidean distance and the like, so that input sample pairs are classified.
However, the conventional technique has a problem in that the sample pairs cannot be accurately classified.
Disclosure of Invention
Based on this, it is necessary to provide an image classification method, a computer device, and a storage medium for solving the problem that the conventional technology cannot accurately classify the sample pairs.
In a first aspect, an embodiment of the present invention provides an image classification method, including:
acquiring an image to be classified and a plurality of training sample images; the image to be classified and the plurality of training sample images are images of the same part of different patients;
respectively combining the images to be classified with the plurality of training sample images, inputting a relational network model to obtain the distance between the images to be classified and each training sample image, and constructing a graph characteristic matrix of the images to be classified and the plurality of training sample images according to the distance between the images to be classified and each training sample image, wherein the relational network model is a model obtained by inputting a sample image pair formed by two pairs of images in a training sample image set into a preset relational network;
And inputting the graph characteristic matrix into a classification network model to obtain a classification result of the image to be classified, wherein the classification network model is a model obtained by inputting the training sample image set into the relation network model to obtain a sample graph characteristic matrix corresponding to the training sample image set and inputting the sample graph characteristic matrix into a graph network for training.
In one embodiment, the relational network model includes a feature extraction network and a comparison network, the image to be classified is combined with the plurality of training sample images respectively, and the distance between the image to be classified and each training sample image is obtained by inputting the relational network model, including:
combining the images to be classified with the training sample images to obtain a plurality of image pairs;
inputting each image pair into the feature extraction network to obtain a high-order feature of each image pair;
and inputting the high-order features of each image pair into the comparison network to obtain the distance between the image to be classified and each training sample image.
In one embodiment, the inputting the higher-order features of each image pair into the comparison network to obtain the distance between the image to be classified and each training sample image includes:
Inputting the high-order features of each image pair into the comparison network to obtain probability values of the images to be classified and the training sample images which are the same type of image;
and taking each probability value as the distance between the image to be classified and each training sample image.
In one embodiment, the feature extraction network includes a convolution layer for extracting higher-order features of each of the image pairs; the comparison network comprises a convolution layer and a full connection layer and is used for acquiring the distance between the image to be classified and each training sample image.
In one embodiment, the graph network comprises a graph roll-up network.
In one embodiment, the classification network model includes at least one graph roll network layer and a classification function layer; the graph rolling network layer is used for extracting a characteristic graph of the graph characteristic matrix; the classification function layer is used for obtaining a classification result of the image to be classified.
In one embodiment, the training process of the relational network model includes:
acquiring the training sample image set;
the images in the training sample image set are paired to obtain a plurality of sample image pairs;
and training the plurality of sample images input into the preset relation network to obtain the relation network model.
In one embodiment, the method further comprises:
inputting a preset sample image set into the relation network model, and training parameters in the relation network model to obtain an adjusted relation network model, wherein the number of images in the preset sample image set is smaller than that in the training sample image set.
In a second aspect, an embodiment of the present invention provides an image classification apparatus, including:
the first acquisition module is used for acquiring an image to be classified and a plurality of training sample images; the image to be classified and the plurality of training sample images are images of the same part of different patients;
the construction module is used for respectively combining the images to be classified with the plurality of training sample images and inputting a relation network model to obtain the distance between the images to be classified and each training sample image, and constructing a graph characteristic matrix of the images to be classified and the plurality of training sample images according to the distance between the images to be classified and each training sample image, wherein the relation network model is a model obtained by inputting sample image pairs formed by two pairs of images in a training sample image set into a preset relation network;
The second obtaining module is configured to input the graph characteristic matrix into a classification network model to obtain a classification result of the image to be classified, where the classification network model is a model obtained by inputting the training sample image set into the relational network model to obtain a sample graph characteristic matrix corresponding to the training sample image set, and inputting the sample graph characteristic matrix into a graph network for training.
In a third aspect, an embodiment of the present invention provides a computer device, including a memory and a processor, the memory storing a computer program, the processor implementing the following steps when executing the computer program:
acquiring an image to be classified and a plurality of training sample images; the image to be classified and the plurality of training sample images are images of the same part of different patients;
respectively combining the images to be classified with the plurality of training sample images, inputting a relational network model to obtain the distance between the images to be classified and each training sample image, and constructing a graph characteristic matrix of the images to be classified and the plurality of training sample images according to the distance between the images to be classified and each training sample image, wherein the relational network model is a model obtained by inputting a sample image pair formed by two pairs of images in a training sample image set into a preset relational network;
And inputting the graph characteristic matrix into a classification network model to obtain a classification result of the image to be classified, wherein the classification network model is a model obtained by inputting the training sample image set into the relation network model to obtain a sample graph characteristic matrix corresponding to the training sample image set and inputting the sample graph characteristic matrix into a graph network for training.
In a fourth aspect, embodiments of the present invention provide a computer readable storage medium having stored thereon a computer program which when executed by a processor performs the steps of:
acquiring an image to be classified and a plurality of training sample images; the image to be classified and the plurality of training sample images are images of the same part of different patients;
respectively combining the images to be classified with the plurality of training sample images, inputting a relational network model to obtain the distance between the images to be classified and each training sample image, and constructing a graph characteristic matrix of the images to be classified and the plurality of training sample images according to the distance between the images to be classified and each training sample image, wherein the relational network model is a model obtained by inputting a sample image pair formed by two pairs of images in a training sample image set into a preset relational network;
And inputting the graph characteristic matrix into a classification network model to obtain a classification result of the image to be classified, wherein the classification network model is a model obtained by inputting the training sample image set into the relation network model to obtain a sample graph characteristic matrix corresponding to the training sample image set and inputting the sample graph characteristic matrix into a graph network for training.
In the image classification method, the device, the computer equipment and the readable storage medium provided in the above embodiments, the computer equipment acquires an image to be classified and a plurality of training sample images; the images to be classified and the plurality of training sample images are images of the same part of different patients; respectively combining the images to be classified with a plurality of training sample images, inputting a relational network model to obtain the distance between the images to be classified and each training sample image, and constructing a graph characteristic matrix of the images to be classified and the plurality of training sample images according to the distance between the images to be classified and each training sample image; and inputting the graph characteristic matrix into a classification network model to obtain a classification result of the image to be classified. In the method, the image characteristic matrixes of the images to be classified and the training sample images constructed by the computer equipment are formed by respectively combining the images to be classified with the training sample images and inputting a relation network model to obtain the distance between the images to be classified and each training sample image, and the image characteristic matrixes of the images to be classified and the training sample images are constructed according to the distance between the images to be classified and each training sample image.
Drawings
FIG. 1 is a schematic diagram of an internal structure of a computer device according to one embodiment;
FIG. 2 is a flow chart of an image classification method according to an embodiment;
FIG. 3 is a flow chart of an image classification method according to another embodiment;
FIG. 4 is a schematic diagram of a relational network model according to one embodiment;
FIG. 5 is a schematic diagram of a graph network architecture according to one embodiment;
FIG. 6 is a flowchart of an image classification method according to another embodiment;
fig. 7 is a schematic structural diagram of an image classification apparatus according to an embodiment.
Detailed Description
The present application will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present application more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the application.
The image classification method provided by the embodiment of the application can be applied to the computer equipment shown in the figure 1. The computer device comprises a processor, a memory, and a computer program stored in the memory, wherein the processor is connected through a system bus, and when executing the computer program, the processor can execute the steps of the method embodiments described below. Optionally, the computer device may further comprise a network interface, a display screen and an input means. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium, which stores an operating system and a computer program, an internal memory. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The network interface of the computer device is used for communicating with an external terminal through a network connection. Optionally, the computer device may be a server, a personal computer, a personal digital assistant, other terminal devices, such as a tablet computer, a mobile phone, etc., or a cloud or remote server, and the embodiment of the present application does not limit a specific form of the computer device.
When the existing neural network is used for identifying the medical image, the neural network is limited to independent individuals, only the characteristics of each individual are considered, and when the number of training samples is small, only the characteristics on each sample are considered, the network is extremely easy to cause over fitting of the network, so that a network is required to measure the relationship or similarity between each individual besides the characteristics of each individual, and the medical image is classified. In the traditional technology, two main learning modes are adopted for learning small samples or very small samples with a small number of training samples, one learning mode is meta-learning, a data set is decomposed into different meta-tasks in a training stage, the generalization capability of a model under the change of a learning class is removed, and in a testing stage, classification can be completed in the face of a brand new task without changing the existing model; the other is matrix learning, and classification is completed by the nearest neighbor idea by measuring the distance between the test sample and the training sample. However, meta-learning is generally applicable to the case of very small samples (3-10 samples), while the sample set of medical images is larger, and meta-learning is no longer applicable to classification of medical images, requiring a more accurate classification model; the matrix learning measures the distance between the test sample and the training sample through a twin network, a matching network or a prototype network, etc., but the networks have some problems, when the distance between the samples is measured, the networks can adopt linear, predefined metrics such as vector inner products, euclidean distances, etc., however, for different tasks, the different metrics have different performances, the predefined metrics cannot achieve the best performance, besides, the linear metrics are too simple, the complex relationship between the vectors is simplified, in addition, the nearest neighbor classification mode used in the matrix learning defines the labels of the samples according to the nearest points around the samples, and the number of the points is a preset value, so that the matrix learning method cannot classify the input samples more accurately. To this end, embodiments of the present invention provide an image classification method, a computer device, and a storage medium, which aim to solve the above technical problems of the conventional technology.
The following describes the technical scheme of the present invention and how the technical scheme of the present invention solves the above technical problems in detail with specific embodiments. The following embodiments may be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.
Fig. 2 is a flow chart of an image classification method according to an embodiment. The embodiment relates to a specific implementation process of obtaining a classification result of an image to be classified by computer equipment according to the image to be classified and a plurality of training sample images. As shown in fig. 2, the method may include:
s201, obtaining an image to be classified and a plurality of training sample images; the image to be classified and the plurality of training sample images are images of the same part of different patients.
Specifically, a computer device acquires an image to be classified and a plurality of training sample images; the images to be classified and the plurality of training sample images are images of the same part of different patients. It should be noted that the image to be classified and the plurality of training sample images are the same type of medical image, and optionally, the image to be classified and the plurality of training sample images may be computed tomography (Computed Tomography, CT) images, functional magnetic resonance (Functional Magnetic Resonance Imaging, FMRI) images, or other medical image images. Optionally, the computer device may obtain the image to be classified and the plurality of training sample images from a PACS (Picture Archiving and Communication Systems, image archiving and communication system) server, or may obtain the image to be classified and the plurality of training sample images from the medical imaging device in real time.
S202, respectively combining an image to be classified with a plurality of training sample images, inputting a relational network model to obtain the distance between the image to be classified and each training sample image, and constructing a graph characteristic matrix of the image to be classified and the plurality of training sample images according to the distance between the image to be classified and each training sample image, wherein the relational network model is a model obtained by inputting a sample image pair formed by two pairs of images in a training sample image set into a preset relational network.
Specifically, the computer equipment respectively combines the image to be classified with the plurality of training sample images and inputs the image to be classified and the training sample images into a relational network model to obtain the distance between the image to be classified and each training sample image, and constructs a graph characteristic matrix of the image to be classified and the plurality of training sample images according to the obtained distance between the image to be classified and each training sample image. The relation network model is a model obtained by inputting a sample image pair formed by two pairs of images in a training sample image set into a preset relation network. It should be noted that, the process of constructing the graph characteristic matrix of the image to be classified and the plurality of training sample images by the computer device according to the obtained distance between the image to be classified and each training sample image is as follows: taking each training sample image and a plurality of training sample images as nodes of a graph characteristic matrix, and taking the distance between the image to be classified and each training sample image as the connection between the nodes to construct the graph characteristic matrix of the image to be classified and the plurality of training sample images; in addition, it should be noted that, the distance between the image to be classified and each training sample image obtained through the relational network model is not symmetrical, that is, the distance between the image to be classified and each training sample image is different from the distance between each training sample image and each training sample image, so in this embodiment, the average value of the distance between the image to be classified and each training sample image and the distance between each training sample image and each training sample image is taken as the distance between the image to be classified and each training sample image. For example, if the graph characteristic matrix of the constructed image to be classified and the plurality of training sample images is X, the elements of the ith row and the jth column of the graph characteristic matrix X are:
S203, inputting the graph characteristic matrix into a classification network model to obtain a classification result of the image to be classified, wherein the classification network model is a model obtained by inputting a training sample image set into a relation network model to obtain a sample graph characteristic matrix corresponding to the training sample image set and inputting the sample graph characteristic matrix into a graph network for training.
Specifically, the computer equipment inputs the obtained graph characteristic matrix into a classification network model to obtain a classification result of the image to be classified. The classification network model is a model obtained by inputting a training sample image set into the relational network model to obtain a sample graph characteristic matrix corresponding to the training sample image set and inputting the sample graph characteristic matrix into a graph network for training. The training mode of the graph network by the computer equipment is a semi-supervised learning training mode, namely, a sample graph characteristic matrix corresponding to the training sample image set and a test graph characteristic matrix corresponding to the test sample image set are input into the graph network together for training, but the value of the loss function of the test sample is not returned.
In this embodiment, the image characteristic matrix of the to-be-classified image and the plurality of training sample images constructed by the computer device is obtained by combining the to-be-classified image with the plurality of training sample images respectively, inputting a relational network model to obtain the distance between the to-be-classified image and each training sample image, and constructing the image characteristic matrix of the to-be-classified image and the plurality of training sample images according to the distance between the to-be-classified image and each training sample image.
Fig. 3 is a flowchart of an image classification method according to another embodiment. Fig. 4 is a schematic structural diagram of a relational network model according to an embodiment. The relational network model includes a feature extraction network and a comparison network. The embodiment relates to a specific implementation process of obtaining the distance between an image to be classified and each training sample image by computer equipment. As an optional implementation manner, as shown in fig. 3, based on the foregoing embodiment, in S202, the image to be classified is combined with a plurality of training sample images, and a relational network model is input to obtain a distance between the image to be classified and each training sample image, where the method includes:
s301, respectively combining the images to be classified with a plurality of training sample images to obtain a plurality of image pairs.
Specifically, the computer equipment respectively combines the images to be classified with a plurality of training sample images to obtain a plurality of image pairs. Illustratively, the image to be classified is denoted by 0, and the plurality of training sample images are denoted by 1,2,3, and 4, and then the image to be classified is respectively combined with the plurality of training sample images, so as to obtain a result of 01,02,03,04 of the plurality of image pairs.
S302, inputting each image pair into a feature extraction network to obtain the high-order features of each image pair.
Specifically, the computer device inputs the obtained plurality of image pairs into a feature extraction network of the relational network model, and obtains high-order features of each image pair. The feature extraction network comprises a convolution layer for extracting high-order features of each image pair, and the convolution layer of the feature extraction network is a nonlinear convolution layer, and optionally, the convolution layer of the feature extraction network can be three layers or other layers. As shown in fig. 4, the parameters of the feature extraction network are shared for the input image pairs, i.e. the parameters employed for the different image feature extraction networks are the same. In this embodiment, the obtained high-order features of each image pair are obtained after the computer device performs the stitching operation on the obtained high-order features of each image pair.
S303, inputting the high-order features of each image pair into a comparison network to obtain the distance between the image to be classified and each training sample image.
Specifically, as shown in fig. 4, the computer device inputs the obtained higher-order features of each image pair into a comparison network of the relational network model to obtain the distance between the image to be classified and each training sample image. The comparison network comprises a convolution layer and a full connection layer and is used for classifying the high-order features of each input image pair, judging whether each input image pair belongs to the same type of image or not, obtaining probability values of images to be classified and training sample images in the same type of image, and taking the probability values of the images to be classified and the training sample images in the same type of image as the distance between the images to be classified and the training sample images. It should be noted that the convolutional layer included in the comparison network is a nonlinear convolutional layer.
In this embodiment, the relational network model includes a feature extraction network and a comparison network, where the feature extraction network can extract high-order features of each input image pair, and the comparison network can classify the image to be classified and each training sample image according to the high-order features of each input image pair extracted by the feature extraction network, so as to obtain distances between the image to be classified and each training sample image, so that the relationship or similarity between each input image pair can be simulated more accurately, the accuracy of classifying the image to be classified and each training sample image is improved, and the accuracy of the distances between the obtained image to be classified and each training sample image is further improved.
Fig. 5 is a schematic diagram of a graph network structure according to an embodiment. Based on the above embodiments, as an alternative implementation, the graph network includes a graph roll-up network.
Specifically, the graph network for inputting the obtained sample graph characteristic matrix into the graph network for training includes a graph rolling network, as shown in fig. 5, where the graph rolling network includes at least one graph rolling network layer and a classification function layer, the graph rolling network layer is used to extract a feature graph of the graph characteristic matrix, and the classification function layer is used to obtain a classification result of the image to be classified according to the feature graph of the graph characteristic matrix extracted by the graph rolling network layer. Alternatively, the activation function of the connection graph rolling network layer may be a ReLU function, and the classification function of the classification function layer may be a Softmax function. It can be understood that the classification network model is a model obtained by inputting a training sample image set into a relational network model to obtain a sample image characteristic matrix corresponding to the training sample image set, inputting the sample image characteristic matrix into a graph network to be trained, wherein the graph rolling network comprises at least one graph rolling network layer and a classification function layer, namely the classification network model comprises at least one graph rolling network layer and a classification function layer, the graph rolling network layer is used for extracting a characteristic image of the image characteristic matrix, and the classification function layer is used for obtaining a classification result of an image to be classified. Optionally, the graph network that inputs the obtained sample graph characteristic matrix into the graph network for training may further include other graph convolution network variants, such as a graph attention network, and other proximity-based classifiers, such as kNN (k-nearest neighbor) classifiers, may also be used.
In this embodiment, the graph network in which the obtained sample graph characteristic matrix is input into the graph network for training includes a graph convolution network, and because the graph convolution network can analyze and process the sample graph characteristic matrix more accurately, the classification network model is a model obtained by training the graph network, so that the classification network model can analyze and process the graph characteristic matrix better, and the accuracy of the classification result of the obtained image to be classified is improved.
Fig. 6 is a flowchart of an image classification method according to another embodiment. The embodiment relates to a specific implementation process of training a relational network model by computer equipment. As shown in fig. 6, the training process of the relational network model may include:
s601, acquiring a training sample image set.
Optionally, the computer device may acquire a plurality of brain function images from the fMRI scanning device, record labels corresponding to the brain function images, and use the acquired plurality of brain function images as a training sample image set. Alternatively, the computer device may also obtain a training sample image set from an Alzheimer's neuroimaging (Alzheimer's Disease Neuroimaging Initiative, ADNI) database. The obtained training sample image set includes a normal sample image and a diseased sample image.
S602, two images in the training sample image set are paired to obtain a plurality of sample image pairs.
Specifically, the computer device pairs images in the training sample image set two by two to obtain a plurality of sample image pairs. For example, the training sample image set includes four training sample images, denoted by a, b, c, and d, and the images in the training sample image set are paired two by two, so that the obtained plurality of sample image pairs are ab, ac, ad, bc, bd, and cd.
S603, training the plurality of sample image pairs into a preset relation network to obtain a relation network model.
Specifically, the computer device inputs the obtained plurality of sample image pairs into a preset relationship network, trains the preset relationship network, and obtains the relationship network model. It can be understood that the input of the preset relationship network is a plurality of sample image pairs, the output is the distance between the sample images in the plurality of sample image pairs, and the computer device trains the preset relationship network by inputting a large number of sample image pairs into the preset relationship network, so as to obtain the relationship network model.
In this embodiment, the computer device acquires pairs of images in the acquired training sample image set, acquires a plurality of sample image pairs, inputs the acquired plurality of sample image pairs into a preset relationship network for training, and can train the preset relationship network more accurately through a large number of sample image pairs, thereby improving the accuracy of the acquired relationship network model.
On the basis of the above embodiment, as an optional implementation manner, the above method further includes: inputting a preset sample image set into a relational network model, and training parameters in the relational network model to obtain an adjusted relational network model, wherein the number of images in the preset sample image set is smaller than that in the training sample image set.
Specifically, after the computer equipment obtains the relationship network model, a preset sample image set is input into the relationship network model, parameters in the relationship network model are trained, and an adjusted relationship network model is obtained, wherein the number of images in the preset sample image set is smaller than that in the training sample image set.
In this embodiment, parameters in the relational network model are further trained through a preset sample image set, so as to obtain an adjusted relational network model, and compared with the obtained relational network model, the adjusted relational network model is more accurate, so that the accuracy of the obtained relational network model is improved, and further, the accuracy of obtaining the distance between the image to be classified and each training sample image according to the relational network model is improved.
It should be understood that, although the steps in the flowcharts of fig. 2-6 are shown in order as indicated by the arrows, these steps are not necessarily performed in order as indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in fig. 2-6 may include multiple sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, nor does the order in which the sub-steps or stages are performed necessarily occur in sequence, but may be performed alternately or alternately with at least a portion of the sub-steps or stages of other steps or other steps.
Fig. 7 is a schematic structural diagram of an image classification apparatus according to an embodiment. As shown in fig. 7, the apparatus may include: a first acquisition module 10, a construction module 11 and a second acquisition module 12.
Specifically, the first acquiring module 10 is configured to acquire an image to be classified and a plurality of training sample images; the images to be classified and the plurality of training sample images are images of the same part of different patients;
The construction module 11 is configured to combine the image to be classified with a plurality of training sample images, input a relational network model to obtain distances between the image to be classified and each training sample image, and construct a graph characteristic matrix of the image to be classified and the plurality of training sample images according to the distances between the image to be classified and each training sample image, where the relational network model is a model obtained by inputting a pair of sample image pairs formed by two pairs of images in a training sample image set into a preset relational network;
the second obtaining module 12 is configured to input the graph characteristic matrix into a classification network model to obtain a classification result of the image to be classified, where the classification network model is a model obtained by inputting a training sample image set into a relational network model to obtain a sample graph characteristic matrix corresponding to the training sample image set, and inputting the sample graph characteristic matrix into the graph network for training.
Optionally, the graph network comprises a graph roll-up network.
Optionally, the classification network model includes at least one graph roll-up network layer and a classification function layer; the graph rolling network layer is used for extracting a characteristic graph of the graph characteristic matrix; the classification function layer is used for obtaining a classification result of the image to be classified.
The image classification device provided in this embodiment may execute the above method embodiment, and its implementation principle and technical effects are similar, and will not be described herein.
On the basis of the above embodiment, the relational network model includes a feature extraction network and a comparison network, and optionally, the above building module 11 includes: the device comprises a combination unit, a feature acquisition unit and a distance acquisition unit.
Specifically, the combination unit is used for respectively combining the images to be classified with a plurality of training sample images to obtain a plurality of image pairs;
the feature acquisition unit is used for inputting each image pair into the feature extraction network to acquire the high-order feature of each image pair;
the distance acquisition unit is used for inputting the high-order features of each image pair into the comparison network to obtain the distance between the image to be classified and each training sample image.
Optionally, the feature extraction network includes a convolution layer for extracting higher-order features of each image pair; the comparison network comprises a convolution layer and a full connection layer and is used for acquiring the distance between the image to be classified and each training sample image.
The image classification device provided in this embodiment may execute the above method embodiment, and its implementation principle and technical effects are similar, and will not be described herein.
On the basis of the above embodiment, optionally, the distance acquiring unit is specifically configured to input higher-order features of each image pair into the comparison network to obtain a probability value that the image to be classified and each training sample image are the same type of image; and taking each probability value as the distance between the image to be classified and each training sample image.
The image classification device provided in this embodiment may execute the above method embodiment, and its implementation principle and technical effects are similar, and will not be described herein.
On the basis of the above embodiment, optionally, the above apparatus further includes: the system comprises a third acquisition module, a fourth acquisition module and a training module.
Specifically, the third acquisition module is used for acquiring a training sample image set;
the fourth acquisition module is used for pairwise pairing images in the training sample image set to obtain a plurality of sample image pairs;
and the training module is used for training the plurality of sample image pairs input into a preset relationship network to obtain a relationship network model.
The image classification device provided in this embodiment may execute the above method embodiment, and its implementation principle and technical effects are similar, and will not be described herein.
For specific limitations of the image classification apparatus, reference may be made to the above limitations of the image classification method, and no further description is given here. The respective modules in the above-described image classification apparatus may be implemented in whole or in part by software, hardware, and combinations thereof. The above modules may be embedded in hardware or may be independent of a processor in the computer device, or may be stored in software in a memory in the computer device, so that the processor may call and execute operations corresponding to the above modules.
In one embodiment, a computer device is provided comprising a memory and a processor, the memory having stored therein a computer program, the processor when executing the computer program performing the steps of:
acquiring an image to be classified and a plurality of training sample images; the images to be classified and the plurality of training sample images are images of the same part of different patients;
respectively combining an image to be classified with a plurality of training sample images, inputting a relational network model to obtain the distance between the image to be classified and each training sample image, and constructing a graph characteristic matrix of the image to be classified and the plurality of training sample images according to the distance between the image to be classified and each training sample image, wherein the relational network model is a model obtained by inputting sample image pairs formed by pairwise pairs of images in a training sample image set into a preset relational network;
and inputting the graph characteristic matrix into a classification network model to obtain a classification result of the image to be classified, wherein the classification network model is a model obtained by inputting a training sample image set into a relation network model to obtain a sample graph characteristic matrix corresponding to the training sample image set and inputting the sample graph characteristic matrix into a graph network for training.
The computer device provided in the foregoing embodiments has similar implementation principles and technical effects to those of the foregoing method embodiments, and will not be described herein in detail.
In one embodiment, a computer readable storage medium is provided having a computer program stored thereon, which when executed by a processor, performs the steps of:
acquiring an image to be classified and a plurality of training sample images; the images to be classified and the plurality of training sample images are images of the same part of different patients;
respectively combining an image to be classified with a plurality of training sample images, inputting a relational network model to obtain the distance between the image to be classified and each training sample image, and constructing a graph characteristic matrix of the image to be classified and the plurality of training sample images according to the distance between the image to be classified and each training sample image, wherein the relational network model is a model obtained by inputting sample image pairs formed by pairwise pairs of images in a training sample image set into a preset relational network;
and inputting the graph characteristic matrix into a classification network model to obtain a classification result of the image to be classified, wherein the classification network model is a model obtained by inputting a training sample image set into a relation network model to obtain a sample graph characteristic matrix corresponding to the training sample image set and inputting the sample graph characteristic matrix into a graph network for training.
The computer readable storage medium provided in the above embodiment has similar principle and technical effects to those of the above method embodiment, and will not be described herein.
Those skilled in the art will appreciate that implementing all or part of the above described methods may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed, may comprise the steps of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in embodiments provided herein may include non-volatile and/or volatile memory. The nonvolatile memory can include Read Only Memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), memory bus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), among others.
The technical features of the above-described embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above-described embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.
The above examples illustrate only a few embodiments of the invention, which are described in detail and are not to be construed as limiting the scope of the invention. It should be noted that it will be apparent to those skilled in the art that several variations and modifications can be made without departing from the spirit of the invention, which are all within the scope of the invention. Accordingly, the scope of protection of the present invention is to be determined by the appended claims.
Claims (10)
1. A method of classifying images, the method comprising:
acquiring an image to be classified and a plurality of training sample images; the image to be classified and the plurality of training sample images are images of the same part of different patients;
respectively combining the images to be classified with the plurality of training sample images, inputting a relational network model to obtain the distance between the images to be classified and each training sample image, and constructing a graph characteristic matrix of the images to be classified and the plurality of training sample images according to the distance between the images to be classified and each training sample image, wherein the relational network model is a model obtained by inputting a sample image pair formed by two pairs of images in a training sample image set into a preset relational network;
Inputting the graph characteristic matrix into a classification network model to obtain a classification result of the image to be classified, wherein the classification network model is a model obtained by inputting the training sample image set into the relation network model to obtain a sample graph characteristic matrix corresponding to the training sample image set, inputting the sample graph characteristic matrix into a graph network, and the graph network comprises a graph convolution network;
the constructing a graph characteristic matrix of the image to be classified and the plurality of training sample images according to the distance between the image to be classified and each training sample image includes:
acquiring the distance between each training sample image and each image to be classified, and correcting the distance between each image to be classified and each training sample image according to the distance between each training sample image and each image to be classified;
and constructing a graph characteristic matrix of the image to be classified and the training sample images according to the corrected distance between the image to be classified and each training sample image.
2. The method of claim 1, wherein the relational network model includes a feature extraction network and a comparison network, wherein combining the image to be classified with the plurality of training sample images, respectively, and inputting the relational network model to obtain a distance between the image to be classified and each of the training sample images, comprises:
Combining the images to be classified with the training sample images to obtain a plurality of image pairs;
inputting each image pair into the feature extraction network to obtain a high-order feature of each image pair;
and inputting the high-order features of each image pair into the comparison network to obtain the distance between the image to be classified and each training sample image.
3. The method of claim 2, wherein inputting the higher order features of each of the image pairs into the comparison network to obtain the distance between the image to be classified and each of the training sample images comprises:
inputting the high-order features of each image pair into the comparison network to obtain probability values of the images to be classified and the training sample images which are the same type of image;
and taking each probability value as the distance between the image to be classified and each training sample image.
4. The method of claim 2, wherein the feature extraction network comprises a convolution layer for extracting higher order features of each of the image pairs; the comparison network comprises a convolution layer and a full connection layer and is used for acquiring the distance between the image to be classified and each training sample image.
5. The method of claim 1, wherein the classification network model comprises at least one graph roll-up network layer and a classification function layer; the graph rolling network layer is used for extracting a characteristic graph of the graph characteristic matrix; the classification function layer is used for obtaining a classification result of the image to be classified.
6. The method of claim 1, wherein the training process of the relational network model comprises:
acquiring the training sample image set;
the images in the training sample image set are paired to obtain a plurality of sample image pairs;
and training the plurality of sample images input into the preset relation network to obtain the relation network model.
7. The method of claim 5, wherein the method further comprises:
inputting a preset sample image set into the relation network model, and training parameters in the relation network model to obtain an adjusted relation network model, wherein the number of images in the preset sample image set is smaller than that in the training sample image set.
8. An image classification apparatus, the apparatus comprising:
The first acquisition module is used for acquiring an image to be classified and a plurality of training sample images; the image to be classified and the plurality of training sample images are images of the same part of different patients;
the construction module is used for respectively combining the images to be classified with the plurality of training sample images and inputting a relation network model to obtain the distance between the images to be classified and each training sample image, and constructing a graph characteristic matrix of the images to be classified and the plurality of training sample images according to the distance between the images to be classified and each training sample image, wherein the relation network model is a model obtained by inputting sample image pairs formed by two pairs of images in a training sample image set into a preset relation network;
the second obtaining module is used for inputting the graph characteristic matrix into a classification network model to obtain a classification result of the image to be classified, wherein the classification network model is a model obtained by inputting the training sample image set into the relation network model to obtain a sample graph characteristic matrix corresponding to the training sample image set and inputting the sample graph characteristic matrix into a graph network, and the graph network comprises a graph convolution network;
Wherein the construction module is further configured to:
acquiring the distance between each training sample image and each image to be classified, and correcting the distance between each image to be classified and each training sample image according to the distance between each training sample image and each image to be classified;
and constructing a graph characteristic matrix of the image to be classified and the training sample images according to the corrected distance between the image to be classified and each training sample image.
9. A computer device comprising a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1-7 when the computer program is executed.
10. A computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911068736.1A CN110852367B (en) | 2019-11-05 | 2019-11-05 | Image classification method, computer device, and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911068736.1A CN110852367B (en) | 2019-11-05 | 2019-11-05 | Image classification method, computer device, and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110852367A CN110852367A (en) | 2020-02-28 |
| CN110852367B true CN110852367B (en) | 2023-10-31 |
Family
ID=69598427
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911068736.1A Active CN110852367B (en) | 2019-11-05 | 2019-11-05 | Image classification method, computer device, and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110852367B (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111488933B (en) * | 2020-04-13 | 2024-02-27 | 上海联影智能医疗科技有限公司 | Image classification method, network, computer device, and storage medium |
| CN111949812A (en) * | 2020-07-10 | 2020-11-17 | 上海联影智能医疗科技有限公司 | Brain image classification method and storage medium |
| CN112016601B (en) * | 2020-08-17 | 2022-08-05 | 华东师范大学 | Network model construction method based on knowledge graph enhanced small sample visual classification |
| CN112949740B (en) * | 2021-03-17 | 2022-11-25 | 重庆邮电大学 | Small sample image classification method based on multilevel measurement |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105023024A (en) * | 2015-07-23 | 2015-11-04 | 湖北大学 | Remote sensing image classification method and system based on regularization set metric learning |
| CN109800789A (en) * | 2018-12-18 | 2019-05-24 | 中国科学院深圳先进技术研究院 | Diabetic retinopathy classification method and device based on figure network |
| CN109816009A (en) * | 2019-01-18 | 2019-05-28 | 南京旷云科技有限公司 | Multi-tag image classification method, device and equipment based on picture scroll product |
| WO2019128367A1 (en) * | 2017-12-26 | 2019-07-04 | 广州广电运通金融电子股份有限公司 | Face verification method and apparatus based on triplet loss, and computer device and storage medium |
| CN110020682A (en) * | 2019-03-29 | 2019-07-16 | 北京工商大学 | A kind of attention mechanism relationship comparison net model methodology based on small-sample learning |
| CN110189302A (en) * | 2019-05-07 | 2019-08-30 | 上海联影智能医疗科技有限公司 | Brain image analysis method, computer equipment and readable storage medium storing program for executing |
| CN110222771A (en) * | 2019-06-10 | 2019-09-10 | 成都澳海川科技有限公司 | A kind of classification recognition methods of zero samples pictures |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10565708B2 (en) * | 2017-09-06 | 2020-02-18 | International Business Machines Corporation | Disease detection algorithms trainable with small number of positive samples |
-
2019
- 2019-11-05 CN CN201911068736.1A patent/CN110852367B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105023024A (en) * | 2015-07-23 | 2015-11-04 | 湖北大学 | Remote sensing image classification method and system based on regularization set metric learning |
| WO2019128367A1 (en) * | 2017-12-26 | 2019-07-04 | 广州广电运通金融电子股份有限公司 | Face verification method and apparatus based on triplet loss, and computer device and storage medium |
| CN109800789A (en) * | 2018-12-18 | 2019-05-24 | 中国科学院深圳先进技术研究院 | Diabetic retinopathy classification method and device based on figure network |
| CN109816009A (en) * | 2019-01-18 | 2019-05-28 | 南京旷云科技有限公司 | Multi-tag image classification method, device and equipment based on picture scroll product |
| CN110020682A (en) * | 2019-03-29 | 2019-07-16 | 北京工商大学 | A kind of attention mechanism relationship comparison net model methodology based on small-sample learning |
| CN110189302A (en) * | 2019-05-07 | 2019-08-30 | 上海联影智能医疗科技有限公司 | Brain image analysis method, computer equipment and readable storage medium storing program for executing |
| CN110222771A (en) * | 2019-06-10 | 2019-09-10 | 成都澳海川科技有限公司 | A kind of classification recognition methods of zero samples pictures |
Non-Patent Citations (1)
| Title |
|---|
| Learing to Compare: Relation Network for Few-Shot Learning;Flood Sung,Yongxin Yang, Li Zhang, Tao Xiang;《2018 IEEE/CVF Conference on Computer Vision and Pattern Rcognition》;20181217;1199-1208 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110852367A (en) | 2020-02-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110852367B (en) | Image classification method, computer device, and storage medium | |
| US11210503B2 (en) | Systems and methods for facial representation | |
| CN110444263B (en) | Disease data processing method, device, equipment and medium based on federal learning | |
| WO2021036471A1 (en) | Sample generation method and apparatus, and computer device and storage medium | |
| Kulin et al. | Data-driven design of intelligent wireless networks: An overview and tutorial | |
| EP2869239A2 (en) | Systems and methods for facial representation | |
| Chen et al. | A novel distribution regression approach for data loss compensation in structural health monitoring | |
| CN111738244A (en) | Image detection method, image detection device, computer equipment and storage medium | |
| US20200226451A1 (en) | Method and apparatus with neural network layer contraction | |
| US20150116597A1 (en) | Trajectory Features and Distance Metrics for Hierarchical Video Segmentation | |
| US9436893B2 (en) | Distributed similarity learning for high-dimensional image features | |
| Tong et al. | Nonlinear graph fusion for multi-modal classification of Alzheimer’s disease | |
| KR102334338B1 (en) | Action recognition method and device | |
| JP7423998B2 (en) | Visual analysis frameworks, methods, programs, devices, and systems for understanding missing links in bipartite networks | |
| CN113780584B (en) | Label prediction method, label prediction device, and storage medium | |
| Ekanayake et al. | Crowd density level estimation and anomaly detection using multicolumn multistage bilinear convolution attention network (MCMS-BCNN-Attention) | |
| EP3745321A1 (en) | An operating envelope recommendation system with guaranteed probabilistic coverage | |
| Li et al. | Transformer-enabled generative adversarial imputation network with selective generation (SGT-GAIN) for missing region imputation | |
| CN112183303A (en) | Transformer equipment image classification method and device, computer equipment and medium | |
| Arshad et al. | Hybrid machine learning techniques to detect real time human activity using UCI dataset | |
| Niu et al. | Restoration of missing structural health monitoring data using spatiotemporal graph attention networks | |
| CN115761371A (en) | Medical image classification method and device, storage medium and electronic equipment | |
| Jiang et al. | Dynamic graph learning: A structure-driven approach | |
| US20210097402A1 (en) | Storage medium, optimum solution acquisition method and information processing apparatus | |
| Pan et al. | Towards a Controllable and Reversible Privacy Protection System for Facial Images through Enhanced Multi-Factor Modifier Networks |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |