CN113177602B - Image classification method, device, electronic equipment and storage medium - Google Patents
Image classification method, device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113177602B CN113177602B CN202110510967.4A CN202110510967A CN113177602B CN 113177602 B CN113177602 B CN 113177602B CN 202110510967 A CN202110510967 A CN 202110510967A CN 113177602 B CN113177602 B CN 113177602B
- Authority
- CN
- China
- Prior art keywords
- image
- sub
- sample
- images
- image classification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/29—Graphical models, e.g. Bayesian networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
Landscapes
- Engineering & Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
- Image Analysis (AREA)
Abstract
The invention provides an image classification method, an image classification device, electronic equipment and a storage medium, wherein the method comprises the following steps: inputting the image to be classified into an image classification model to obtain an image classification result output by the image classification model; the image classification model is obtained by performing unsupervised learning based on clustering results of all sample sub-images of the sample image; the clustering result is determined based on morphological similarity of the sub-images of each sample. According to the invention, the image classification model is obtained by performing the non-supervision learning based on the clustering result of each sample sub-image in the sample image, a large number of sample images are not marked manually, the training efficiency of the model is improved, and the clustering result of each sample sub-image is determined based on the morphological similarity of each sample sub-image, so that the spatial information contained in the sample sub-image is fully utilized for clustering, the problem of reduced recognition degree caused by deformation generated during the transformation operation of the image during the traditional non-supervision learning is avoided, and the accuracy of image classification is improved.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to an image classification method, an image classification device, an electronic device, and a storage medium.
Background
The image classification is to distinguish images of different categories according to semantic information of the images, quantitatively analyze the images by a computer, and classify each pixel or area in the images into one of a plurality of categories to replace visual interpretation of people.
At present, image classification is carried out by taking a large number of sample images as an image classification model of a training set, but for high-throughput large-scale images (such as pathological images, satellite images and the like), in order to ensure the training effect of the model, a large number of high-quality labels are required to be manually carried out in the sample images, which is time-consuming and easy to make mistakes, and the training effect of the model is influenced.
Disclosure of Invention
The invention provides an image classification method, an image classification device, electronic equipment and a storage medium, which are used for solving the defect that the model training effect is affected by manually marking a large number of training sets in the prior art.
The invention provides an image classification method, which comprises the following steps:
acquiring an image to be classified;
inputting the image to be classified into an image classification model to obtain an image classification result output by the image classification model;
the image classification model is obtained by performing unsupervised learning based on clustering results of all sample sub-images of the sample image; the clustering result is determined based on morphological similarity of the sample sub-images.
According to the image classification method provided by the invention, the clustering result is determined based on the following steps:
dividing the sample image into a plurality of sample sub-images based on a preset rule;
shifting each sample sub-image to obtain a sub-image set corresponding to each sample sub-image;
calculating the morphological similarity of each sub-image set and the corresponding sample sub-image based on the conditional random field probability model;
and if the morphological similarity meets a preset condition, the category of the corresponding sub-image set is consistent with the category of the corresponding sample sub-image.
According to the image classification method provided by the invention, the shifting is carried out on each sample sub-image to obtain a sub-image set corresponding to each sample sub-image, and the method comprises the following steps:
and based on a preset offset set, respectively offsetting each sample sub-image according to each offset in the preset offset set to obtain a sub-image set corresponding to each sample sub-image.
According to the image classification method provided by the invention, the conditional random field probability model is as follows:
wherein P represents the morphological similarity,
represents a normalization constant, exp represents an exponential function, E represents an energy function, ++>
Representing a random vector describing the annotation of an image, +.>
Representing the original image set, ++>
Represents the ith 1 Random variable of individual sub-image annotation, +.>
Represents the ith k The random variables of the individual sub-image labels,
i 1 represents the ith 1 Sub-images i after increasing the offset K Represents the ith k Sub-images after increasing the offset +.>
Representing adjacent->
Labeling vector of>
Representing adjacent->
Image set, k 1 Representation->
Specific value of k r×c Representing Z r×c Specific value of K 1 Representing adjacent to random variable->
Rearranging whether the first sub-image after the subscript is used for random variables for unsupervised learning, K r×c Representing adjacent to random variable->
Whether the last sub-image after rearranging the subscript is used for random variables for unsupervised learning.
According to the image classification method provided by the invention, the image to be classified is input into an image classification model to obtain an image classification result output by the image classification model, and the image classification method comprises the following steps:
inputting the image to be classified into an image dividing layer of the image classifying model to obtain a plurality of sub-images output by the image dividing layer;
and inputting each sub-image to an image clustering layer of the image classification model to obtain the image classification result output by the image clustering layer.
According to the image classification method provided by the invention, the image classification model is used for performing unsupervised learning by taking the maximized mutual information as an objective function.
According to the image classification method provided by the invention, the image to be classified is a medical image or a satellite data image.
The invention also provides an image classification device, comprising:
the acquisition unit is used for acquiring the images to be classified;
the classification unit is used for inputting the images to be classified into an image classification model to obtain an image classification result output by the image classification model;
the image classification model is obtained by performing unsupervised learning based on clustering results of all sample sub-images of the sample image; the clustering result is determined based on morphological similarity of the sample sub-images.
The invention also provides an electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, the processor implementing the steps of any one of the image classification methods described above when executing the computer program.
The present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the steps of the image classification method as described in any of the above.
According to the image classification method, the device, the electronic equipment and the storage medium, the image classification model is obtained by performing unsupervised learning based on the clustering result of each sample sub-image in the sample image, a large number of sample images are not marked manually, and the training efficiency of the model is improved. Meanwhile, the clustering result of each sample sub-image is determined based on the morphological similarity of each sample sub-image, so that the spatial information contained in the sample sub-images is fully utilized for clustering, the problem that the recognition degree is reduced due to deformation generated when the images are subjected to transformation operation in the traditional unsupervised learning is avoided, and the accuracy of image classification is improved.
Drawings
In order to more clearly illustrate the invention or the technical solutions of the prior art, the following description will briefly explain the drawings used in the embodiments or the description of the prior art, and it is obvious that the drawings in the following description are some embodiments of the invention, and other drawings can be obtained according to the drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of an image classification method provided by the invention;
FIG. 2 is a schematic diagram of a clustering result provided by the invention;
FIG. 3 is a schematic flow diagram of an image classification algorithm framework provided by the present invention;
FIG. 4 is a second flow chart of the image classification algorithm framework provided by the invention;
FIG. 5 is a schematic view of an image classification apparatus according to the present invention;
fig. 6 is a schematic structural diagram of an electronic device provided by the present invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The image classification is to distinguish images of different categories according to semantic information of the images, quantitatively analyze the images by a computer and classify each pixel or area in the images into one of a plurality of categories.
At present, image classification is carried out by taking a large number of sample images as an image classification model of a training set, but for high-throughput large-scale images (such as pathological images, satellite images and the like), in order to ensure the training effect of the model, a large number of high-quality labels are required to be manually carried out in the sample images, which is time-consuming and easy to make mistakes, and the training effect of the model is influenced. For example, in the medical field, full-field digital sectioning under a microscope is the gold standard for cancer diagnosis, but the size of full-field microscope images is large (about 80000×80000 pixels), and labeling thereof depends on an experienced pathologist or doctor, whereas a high-precision deep learning-based image classification model often depends on a large number of high-quality labels, so it is difficult to obtain a large number of labeled full-field images for training a robust deep learning model.
In this regard, the present invention provides an image classification method. Fig. 1 is a schematic flow chart of an image classification method provided by the invention, as shown in fig. 1, the method comprises the following steps:
the image classification model is obtained by performing unsupervised learning based on clustering results of all sample sub-images of the sample image; the clustering result is determined based on morphological similarity of the sub-images of each sample.
In the embodiment of the invention, before training the image classification model, a large number of sample images need to be obtained as training samples of the image classification model, but in the prior art, a large number of areas in the samples need to be marked manually, so that a large amount of labor cost and time cost are consumed, and error marking is caused by manual errors. As a training mode without labels, the unsupervised learning utilizes the characteristics of data rather than relying on manual labeling training models, and has considerable application prospect on high-flux large-scale images. However, the mutual information of the existing unsupervised learning mainly comes from: firstly, matrix transformation operations such as overturning and twisting of a single image, or other brightness contrast adjustment and the like are performed to generate a matched image. And secondly, the dimension reduction processing is carried out on the large image, so that the analysis space is reduced. Thirdly, generating mutual information based on the GAN generating matrix. However, when the original image is transformed, the generated paired image may be deformed relative to the original image, so that the recognition degree is reduced, and the classification accuracy of the model is further reduced.
Therefore, the embodiment of the invention utilizes adjacent images to generate mutual information for unsupervised learning of the image classification model based on the spatial information contained in the sample images, not only can greatly reduce the manual labeling quantity and improve the training efficiency of the model, but also fully utilizes the spatial information contained between the images compared with the existing unsupervised learning method, and avoids the reduction of the recognition degree caused by deformation in the image transformation operation and the influence on the training effect of the model.
Specifically, the embodiment of the invention inputs the image to be classified into the image classification model to obtain the image classification result output by the image classification model. For example, after the image a is input to the image classification model, the image classification result output by the image classification model is that the region 1, the region 2, and the region 3 in the image a belong to the same category, and the region 4 and the region 5 belong to the same category. The image to be classified refers to an image to be classified, and can be a high-flux large-scale image with large size and high fraction ratio (such as a pathological image, a satellite image and the like) or an image with common pixel size.
The image classification model is obtained by performing unsupervised learning based on the clustering result of each sample sub-image of the sample image, and the clustering result is determined based on the morphological similarity of each sample sub-image. The sub-images of each sample may be obtained by dividing the sub-images of each sample in the sample image according to a preset fixed size, or may be obtained by dividing the sub-images of each sample in the sample image according to different sizes. The clustering result of each sample sub-image is determined based on the morphological similarity of each sample sub-image, for example, if the morphological similarity of the sample sub-image a and the sample sub-image B is greater than a preset value, the sample sub-image a and the sample sub-image B may be classified into the same class; or the sample sub-image A is shifted to obtain a plurality of sub-images in different areas, a sub-image set is formed, the morphological similarity between the sub-image set and the sample sub-image A is calculated, and if the morphological similarity meets the preset condition, each image category in the sub-image set is considered to be consistent with the category of the sample sub-image A. The morphological similarity of each sample sub-image may be determined based on euclidean distance, cosine distance, or conditional random field model, which is not particularly limited in the embodiment of the present invention.
The image classification model may also be trained in advance before executing step 120, specifically, the following steps may be performed: firstly, a large number of sample images are collected, a clustering result is determined based on the morphological similarity of each sample sub-image in the sample images, and unsupervised learning is performed based on the clustering result, so that an image classification model is obtained.
As shown in fig. 2, by adopting the image classification method provided by the embodiment of the present invention, the obtained classification result is that two images of k=2, i=0.89 are grouped into one type, two images of k=2, i=0.12 are grouped into one type, three images of k=3, i=0.91 are grouped into one type, and four images of k=4, i=0.02 are grouped into one type. As shown in fig. 3, after the image to be classified is input into the image classification model, the image classification model generates corresponding clusters 1-9, and the classification result is visualized. Compared with the existing unsupervised learning method, the preliminary experiment shows that at least 10% -20% of precision improvement can be obtained on the pathology image of the large microscope, and compared with the existing unsupervised learning accuracy (60% -70%), the accuracy is greatly improved.
Therefore, according to the image classification method provided by the embodiment of the invention, the image classification model is obtained by performing unsupervised learning based on the clustering result of each sample sub-image in the sample image, a large number of sample images are not marked manually, and the training efficiency of the model is improved. Meanwhile, the clustering result of each sample sub-image is determined based on the morphological similarity of each sample sub-image, so that the spatial information contained in the sample sub-images is fully utilized for clustering, the problem that the recognition degree is reduced due to deformation generated when the images are subjected to transformation operation in the traditional unsupervised learning is avoided, and the accuracy of image classification is improved.
Based on the above embodiment, the clustering result is determined based on the following steps:
dividing a sample image into a plurality of sample sub-images based on a preset rule;
shifting each sample sub-image to obtain a sub-image set corresponding to each sample sub-image;
calculating the morphological similarity of each sub-image set and the corresponding sample sub-image based on the conditional random field probability model;
if the morphological similarity meets the preset condition, the category of the corresponding sub-image set is consistent with the category of the corresponding sample sub-image.
Specifically, when dividing the sample image into a plurality of sample sub-images, the sample image may be divided into a plurality of sample sub-images according to a preset fixed size set, or the sample image may be divided into a plurality of sample sub-images according to different size sets, which is not particularly limited in the embodiment of the present invention. For example, if the preset fixed size set is size a, size B and size C, the sample image may be divided into a plurality of sample sub-images of size a according to size a, a plurality of sample sub-images of size B according to size B, and a plurality of sample sub-images of size C according to size C; for example, the center point 1 in the sample image may be divided into a size a, the center point 2 into a size B, the center point 3 into a size C, and the like for the first time, i.e., the sample image may be divided into a plurality of sample sub-images with different sizes and different areas according to different rules.
After the sub-images of the samples are obtained through division, the sub-images of the samples are shifted, and a sub-image set corresponding to the sub-images of the samples is obtained. For example, the sample sub-image a is shifted rightward by Δp to obtain a sub-image a1, shifted leftward by Δp to obtain a sub-image a2, shifted upward by Δp to obtain a sub-image a3, shifted downward by Δp to obtain a sub-image a4, shifted rightward by 2Δp to obtain a sub-image b1, shifted leftward by 2Δp to obtain a sub-image b2, shifted upward by 2Δp to obtain a sub-image b3, shifted downward by 2Δp to obtain a sub-image b4, and so on. Wherein a1-a3 can be used as a sub-image set, a1-a4 can be used as a sub-image set, b1-b4 can be used as a sub-image set, and a1-a4 and b1-b4 can be used as sub-image sets, so that according to different offset rules, different sub-image sets can be obtained from the same sub-image.
When sub-image sets corresponding to the sub-images of each sample are obtained, calculating the morphological similarity between each sub-image set and the corresponding sub-image of the sample based on a conditional random field probability model; if the morphological similarity meets the preset condition, if the morphological similarity is larger than the preset value, the category of the corresponding sub-image set is consistent with the category of the corresponding sample sub-image. It should be noted that, since different sub-image sets can be obtained from the same sub-image, after the morphological similarity between different sub-image sets and the sample sub-image is calculated, the higher the morphological similarity is, the higher the morphological correlation degree between the image in the corresponding sub-image set and the sample image is, and when the morphological similarity satisfies the preset condition, the image in the sub-image set can be used as the paired image of the sample image for unsupervised learning.
As shown in fig. 4, in the embodiment of the present invention, a plurality of matching images with the highest morphological relevance are dynamically selected in a certain Region (ROI), r×c patch sets are set initially, the offset Δp of each image forms new r×c patch sets, the new r×c patch sets are clustered respectively, and the maximum mutual information (Multiple mutual information) is used as an objective function to perform unsupervised learning.
Based on any of the above embodiments, offsetting each sample sub-image to obtain a small sub-image set corresponding to each sample sub-image, including:
and based on the preset offset set, respectively offsetting each sample sub-image according to each offset in the preset offset set to obtain a sub-image set corresponding to each sample sub-image.
Specifically, after the sub-images of the samples are obtained by dividing, each sub-image of the samples can be shifted according to a preset offset set, so as to obtain a small sub-image set corresponding to each sub-image of the samples. For example, if the preset offset set is the offset Δp and the offset 2Δp, the sample sub-image a shifts rightward by Δp to obtain a sub-image a1, shifts leftward by Δp to obtain a sub-image a2, shifts upward by Δp to obtain a sub-image a3, shifts downward by Δp to obtain a sub-image a4, shifts rightward by 2Δp to obtain a sub-image b1, shifts leftward by 2Δp to obtain a sub-image b2, shifts upward by 2Δp to obtain a sub-image b3, shifts downward by 2Δp to obtain a sub-image b4, and so on. Wherein a1-a3 can be used as a sub-image set, a1-a4 can be used as a sub-image set, b1-b4 can be used as a sub-image set, a1-a4 and b1-b4 can be used as sub-image sets, therefore, for the same sample sub-image, a plurality of sub-image sets can be obtained based on a preset offset set, each sub-image set is different from the morphological similarity of the sample sub-image, and if the morphological similarity meets a preset condition, the corresponding sub-image set is consistent with the category of the sample sub-image and can be used for unsupervised learning.
Based on any of the above embodiments, the conditional random field probability model is:
wherein P represents morphological similarity, k i Is equal to either 0 or 1 and,
representing a normalization constant such that the output value of the probability representation is between 0 and 1, exp represents an exponential function, E represents an energy function determining the magnitude of the probability values of the set of random variables, +.>
Representing a random vector describing the annotation of an image, +.>
Representing the original image set and constituting the condition of the conditional probability, < >>
Represents the ith 1 Random variable of individual sub-image annotation, +.>
Represents the ith k Random variable of individual sub-image annotation, +.>
For all +.>
i 1 Represents the ith 1 Sub-images i after increasing the offset K Represents the ith k Sub-images after increasing the offset +.>
Representing adjacent->
Labeling vector of>
Representing adjacent->
Image set, k 1 Representation->
Specific value of k r×c Representing Z r×c Specific value of K 1 Representing adjacency to random variables
Rearranging whether the first sub-image after the subscript is used for random variables for unsupervised learning, K r×c Representing adjacent to random variable->
Whether the last sub-image after rearranging the subscript is used for random variables for unsupervised learning.
Specifically, K i The method is a discrete random variable, wherein 1 is equal to the random variable, the degree of morphological correlation between the corresponding sub-image set and the sample sub-image is higher, the sub-image set belongs to the same category, the corresponding sub-image set (patch set) can be selected for unsupervised learning, and 0 is equal to the random variable, the degree of morphological correlation between the corresponding sub-image set and the sample sub-image is lower, the sub-image set does not belong to the same category, and the sub-image set is not used for unsupervised learning. Because of K i There is a one-to-one correspondence between the ith sub-image and the gas coordinates, so embodiments of the present invention pass through K i May describe whether all sub-images are used to train the unsupervised model.
Based on any of the above embodiments, inputting the image to be classified into the image classification model to obtain an image classification result output by the image classification model, including:
inputting an image to be classified into an image dividing layer of an image classifying model to obtain a plurality of sub-images output by the image dividing layer;
and inputting each sub-image to an image clustering layer of the image classification model to obtain an image classification result output by the image clustering layer.
Specifically, the image to be classified is input to an image classification layer of the image classification model, and the image classification layer can divide the image to be classified according to a preset fixed size or divide the image to be classified according to different sizes. And inputting each sub-image into an image clustering layer of the image classification model, wherein the image clustering layer clusters each sub-image to obtain an image classification result. Therefore, based on the spatial information contained in the sample image, the embodiment of the invention utilizes the adjacent images to generate mutual information for the unsupervised learning of the image classification model, which not only can greatly reduce the manual labeling quantity and improve the training efficiency of the model, but also fully utilizes the spatial information contained between the images and avoids the reduction of recognition degree caused by deformation in the image transformation operation and the influence on the training effect of the model compared with the existing unsupervised learning method.
Based on any of the above embodiments, the image classification model is unsupervised learned with maximized mutual information as an objective function.
Specifically, in order to enable the image classification model to learn good characterization from the clustering result of the sample sub-images, the characterization of the sample sub-images can be clearly distinguished, namely the characterization containing the most unique information of the sample sub-images. For example, for the sample sub-image a, the image classification model may learn the representation X of the image, and the representation X may reconstruct the image, i.e., based on the representation X, sub-images belonging to the same class as the image may be accurately classified into one class, thereby further improving the training effect of the model.
Based on any of the above embodiments, the image to be classified is a medical image or a satellite data image.
Specifically, for medical images or satellite data images, in order to ensure the training effect of the model, a large number of high-quality labels are required to be manually performed in sample images, which is time-consuming and error-prone, and influences the training effect of the model. For example, the image size is large (about 80000×80000 pixels) for a full field microscope in medical images, and its labeling depends on an experienced pathologist or doctor, while a high-precision deep learning-based image classification model often depends on a large number of high-quality labels. According to the embodiment of the invention, the image classification model is obtained by performing unsupervised learning based on the clustering result of each sample sub-image in the sample image, a large number of sample images are not marked manually, and the training efficiency of the model is improved. Meanwhile, the clustering result of each sample sub-image is determined based on the morphological similarity of each sample sub-image, so that the spatial information contained in the sample sub-images is fully utilized for clustering, the problem that the recognition degree is reduced due to deformation generated when the images are subjected to transformation operation in the traditional unsupervised learning is avoided, and the accuracy of image classification is improved.
The image classification apparatus provided by the present invention will be described below, and the image classification apparatus described below and the image classification method described above may be referred to correspondingly to each other.
Based on any of the above embodiments, as shown in fig. 5, the present invention provides an image classification apparatus, including:
an acquiring unit 510, configured to acquire an image to be classified;
the classifying unit 520 is configured to input the image to be classified into an image classification model, and obtain an image classification result output by the image classification model;
the image classification model is obtained by performing unsupervised learning based on clustering results of all sample sub-images of the sample image; the clustering result is determined based on morphological similarity of the sample sub-images.
Based on any one of the foregoing embodiments, the clustering method further includes a determining unit, configured to determine the clustering result, where the determining unit includes:
a dividing unit for dividing the sample image into a plurality of sample sub-images based on a preset rule;
the offset unit is used for offsetting the sub-images of each sample to obtain a sub-image set corresponding to each sub-image of each sample;
the computing unit is used for computing the morphological similarity of each sub-image set and the corresponding sample sub-image based on the conditional random field probability model;
and the clustering unit is used for enabling the category of the corresponding sub-image set to be consistent with the category of the corresponding sample sub-image if the morphological similarity meets the preset condition.
Based on any of the foregoing embodiments, the offset unit is specifically configured to:
and based on a preset offset set, respectively offsetting each sample sub-image according to each offset in the preset offset set to obtain a sub-image set corresponding to each sample sub-image.
Based on any of the above embodiments, the classification unit 520 includes:
the image dividing unit is used for inputting the image to be classified into an image dividing layer of the image classifying model to obtain a plurality of sub-images output by the image dividing layer;
and the image classification unit is used for inputting each sub-image to an image clustering layer of the image classification model to obtain the image classification result output by the image clustering layer.
Based on any of the above embodiments, the image classification model performs unsupervised learning with maximized mutual information as an objective function.
Based on any of the above embodiments, the image to be classified is a medical image or a satellite data image.
Fig. 6 is a schematic structural diagram of an electronic device according to the present invention, and as shown in fig. 6, the electronic device may include: processor 610, communication interface (Communications Interface) 620, memory 630, and communication bus 640, wherein processor 610, communication interface 620, and memory 630 communicate with each other via communication bus 640. The processor 610 may invoke logic instructions in the memory 630 to perform an image classification method comprising: acquiring an image to be classified; inputting the image to be classified into an image classification model to obtain an image classification result output by the image classification model; the image classification model is obtained by performing unsupervised learning based on clustering results of all sample sub-images of the sample image; the clustering result is determined based on morphological similarity of the sample sub-images.
Further, the logic instructions in the memory 630 may be implemented in the form of software functional units and stored in a computer-readable storage medium when sold or used as a stand-alone product. Based on this understanding, the technical solution of the present invention may be embodied essentially or in a part contributing to the prior art or in a part of the technical solution, in the form of a software product stored in a storage medium, comprising several instructions for causing a computer device (which may be a personal computer, a server, a network device, etc.) to perform all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a random access Memory (RAM, random Access Memory), a magnetic disk, or an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the image classification method provided by the above methods, the method comprising: acquiring an image to be classified; inputting the image to be classified into an image classification model to obtain an image classification result output by the image classification model; the image classification model is obtained by performing unsupervised learning based on clustering results of all sample sub-images of the sample image; the clustering result is determined based on morphological similarity of the sample sub-images.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the above provided image classification methods, the method comprising: acquiring an image to be classified; inputting the image to be classified into an image classification model to obtain an image classification result output by the image classification model; the image classification model is obtained by performing unsupervised learning based on clustering results of all sample sub-images of the sample image; the clustering result is determined based on morphological similarity of the sample sub-images.
The apparatus embodiments described above are merely illustrative, wherein the elements illustrated as separate elements may or may not be physically separate, and the elements shown as elements may or may not be physical elements, may be located in one place, or may be distributed over a plurality of network elements. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art will understand and implement the present invention without undue burden.
From the above description of the embodiments, it will be apparent to those skilled in the art that the embodiments may be implemented by means of software plus necessary general hardware platforms, or of course may be implemented by means of hardware. Based on this understanding, the foregoing technical solution may be embodied essentially or in a part contributing to the prior art in the form of a software product, which may be stored in a computer readable storage medium, such as ROM/RAM, a magnetic disk, an optical disk, etc., including several instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method described in the respective embodiments or some parts of the embodiments.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention, and are not limiting; although the invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not depart from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims (7)
1. An image classification method, comprising:
acquiring an image to be classified;
inputting the image to be classified into an image classification model to obtain an image classification result output by the image classification model;
the image classification model is obtained by performing unsupervised learning based on clustering results of all sample sub-images of the sample image; the clustering result is determined based on the morphological similarity of the sample sub-images;
the clustering result is determined based on the following steps:
dividing the sample image into a plurality of sample sub-images based on a fixed size set or a different size set;
shifting each sample sub-image to obtain a sub-image set corresponding to each sample sub-image;
calculating the morphological similarity of each sub-image set and the corresponding sample sub-image based on the conditional random field probability model;
if the morphological similarity meets a preset condition, the category of the corresponding sub-image set is consistent with the category of the corresponding sample sub-image;
the step of inputting the image to be classified into an image classification model to obtain an image classification result output by the image classification model comprises the following steps:
inputting the image to be classified into an image dividing layer of the image classifying model to obtain a plurality of sub-images output by the image dividing layer;
inputting each sub-image to an image clustering layer of the image classification model to obtain the image classification result output by the image clustering layer;
the image classification model performs unsupervised learning by taking the maximized mutual information as an objective function.
2. The method of image classification according to claim 1, wherein said shifting each sample sub-image to obtain a set of sub-images corresponding to each sample sub-image comprises:
and based on a preset offset set, respectively offsetting each sample sub-image according to each offset in the preset offset set to obtain a sub-image set corresponding to each sample sub-image.
3. The image classification method of claim 1, wherein the conditional random field probability model is:
wherein P represents the morphological similarity,
represents a normalization constant, exp represents an exponential function, E represents an energy function, ++>
Representing a random vector describing the annotation of an image, +.>
Representing the original image set, ++>
Represents the ith 1 Random variable of individual sub-image annotation, +.>
Represents the ith k Random variable of individual sub-image annotation, +.>
i 1 Represents the ith 1 Sub-images i after increasing the offset K Represents the ith k Sub-images after increasing the offset +.>
Representing adjacent->
Labeling vector of>
Representing adjacent->
Image set, k 1 Representation->
Specific value of k r×c Representing Z r×c Specific value of K 1 Representing adjacent to random variable->
Rearranging whether the first sub-image after the subscript is used for random variables for unsupervised learning, K r×c Representing adjacent to random variable->
Whether the last sub-image after rearranging the subscript is used for random variables for unsupervised learning.
4. A method of image classification according to any one of claims 1 to 3, wherein the image to be classified is a medical image or a satellite data image.
5. An image classification apparatus, comprising:
the acquisition unit is used for acquiring the images to be classified;
the classification unit is used for inputting the images to be classified into an image classification model to obtain an image classification result output by the image classification model;
the image classification model is obtained by performing unsupervised learning based on clustering results of all sample sub-images of the sample image; the clustering result is determined based on the morphological similarity of the sample sub-images;
the clustering result is determined based on the following steps:
dividing the sample image into a plurality of sample sub-images based on a fixed size set or a different size set;
shifting each sample sub-image to obtain a sub-image set corresponding to each sample sub-image;
calculating the morphological similarity of each sub-image set and the corresponding sample sub-image based on the conditional random field probability model;
if the morphological similarity meets a preset condition, the category of the corresponding sub-image set is consistent with the category of the corresponding sample sub-image;
the classification unit comprises:
the image dividing unit is used for inputting the image to be classified into an image dividing layer of the image classifying model to obtain a plurality of sub-images output by the image dividing layer;
the image classification unit is used for inputting each sub-image to an image clustering layer of the image classification model to obtain the image classification result output by the image clustering layer;
the image classification model performs unsupervised learning by taking the maximized mutual information as an objective function.
6. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the processor implements the steps of the image classification method according to any of claims 1 to 4 when the program is executed.
7. A non-transitory computer readable storage medium, on which a computer program is stored, characterized in that the computer program, when being executed by a processor, implements the steps of the image classification method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110510967.4A CN113177602B (en) | 2021-05-11 | 2021-05-11 | Image classification method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110510967.4A CN113177602B (en) | 2021-05-11 | 2021-05-11 | Image classification method, device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113177602A CN113177602A (en) | 2021-07-27 |
| CN113177602B true CN113177602B (en) | 2023-05-26 |
Family
ID=76928824
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110510967.4A Active CN113177602B (en) | 2021-05-11 | 2021-05-11 | Image classification method, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113177602B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116524248B (en) * | 2023-04-17 | 2024-02-13 | 首都医科大学附属北京友谊医院 | Medical data processing device, method and classification model training device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110737730A (en) * | 2019-10-21 | 2020-01-31 | 腾讯科技（深圳）有限公司 | Unsupervised learning-based user classification method, unsupervised learning-based user classification device, unsupervised learning-based user classification equipment and storage medium |
| CN111179961A (en) * | 2020-01-02 | 2020-05-19 | 腾讯科技（深圳）有限公司 | Audio signal processing method, audio signal processing device, electronic equipment and storage medium |
| CN111753863A (en) * | 2019-04-12 | 2020-10-09 | 北京京东尚科信息技术有限公司 | Image classification method and device, electronic equipment and storage medium |
| CN112132841A (en) * | 2020-09-22 | 2020-12-25 | 上海交通大学 | Medical image cutting method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US7007001B2 (en) * | 2002-06-26 | 2006-02-28 | Microsoft Corporation | Maximizing mutual information between observations and hidden states to minimize classification errors |
-
2021
- 2021-05-11 CN CN202110510967.4A patent/CN113177602B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111753863A (en) * | 2019-04-12 | 2020-10-09 | 北京京东尚科信息技术有限公司 | Image classification method and device, electronic equipment and storage medium |
| CN110737730A (en) * | 2019-10-21 | 2020-01-31 | 腾讯科技（深圳）有限公司 | Unsupervised learning-based user classification method, unsupervised learning-based user classification device, unsupervised learning-based user classification equipment and storage medium |
| CN111179961A (en) * | 2020-01-02 | 2020-05-19 | 腾讯科技（深圳）有限公司 | Audio signal processing method, audio signal processing device, electronic equipment and storage medium |
| CN112132841A (en) * | 2020-09-22 | 2020-12-25 | 上海交通大学 | Medical image cutting method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113177602A (en) | 2021-07-27 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107506761B (en) | Brain image segmentation method and system based on significance learning convolutional neural network | |
| CA3138959C (en) | Image diagnostic system, and methods of operating thereof | |
| CN112101451B (en) | Breast cancer tissue pathological type classification method based on generation of antagonism network screening image block | |
| CN110245657B (en) | Pathological image similarity detection method and detection device | |
| CN109086654B (en) | Handwriting model training method, text recognition method, device, equipment and medium | |
| CN112508850A (en) | Deep learning-based method for detecting malignant area of thyroid cell pathological section | |
| CN108109140A (en) | Low Grade Gliomas citric dehydrogenase non-destructive prediction method and system based on deep learning | |
| CN111524144B (en) | Intelligent lung nodule diagnosis method based on GAN and Unet network | |
| CN111401156B (en) | Image identification method based on Gabor convolution neural network | |
| Pardo et al. | Semantic segmentation of mFISH images using convolutional networks | |
| CN113947607A (en) | Cancer pathology image survival prognosis model construction method based on deep learning | |
| WO2015146113A1 (en) | Identification dictionary learning system, identification dictionary learning method, and recording medium | |
| CN114155397A (en) | Small sample image classification method and system | |
| Li et al. | HEp-2 specimen classification via deep CNNs and pattern histogram | |
| CN113177602B (en) | Image classification method, device, electronic equipment and storage medium | |
| CN106709490B (en) | Character recognition method and device | |
| CN107590806B (en) | Detection method and system based on brain medical imaging | |
| CN111127407B (en) | Fourier transform-based style migration forged image detection device and method | |
| CN116958221A (en) | Cell data analysis method, device, equipment and storage medium | |
| CN116665039A (en) | Small sample target identification method based on two-stage causal intervention | |
| CN109190645A (en) | A kind of expression and classification method of high-order high dimensional image | |
| Athanasiadis et al. | Segmentation of complementary DNA microarray images by wavelet-based Markov random field model | |
| Li et al. | HEp-2 cells staining patterns classification via wavelet scattering network and random forest | |
| CN109359694B (en) | Image classification method and device based on mixed collaborative representation classifier | |
| CN117392693B (en) | Method and equipment for removing handwriting of pathological image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |