CN113177602A - Image classification method and device, electronic equipment and storage medium - Google Patents
Image classification method and device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113177602A CN113177602A CN202110510967.4A CN202110510967A CN113177602A CN 113177602 A CN113177602 A CN 113177602A CN 202110510967 A CN202110510967 A CN 202110510967A CN 113177602 A CN113177602 A CN 113177602A
- Authority
- CN
- China
- Prior art keywords
- image
- sub
- sample
- image classification
- images
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/29—Graphical models, e.g. Bayesian networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/088—Non-supervised learning, e.g. competitive learning
Abstract
The invention provides an image classification method, an image classification device, an electronic device and a storage medium, wherein the method comprises the following steps: inputting the image to be classified into an image classification model to obtain an image classification result output by the image classification model; the image classification model is obtained by carrying out unsupervised learning based on the clustering result of each sample subimage of the sample image; the clustering result is determined based on the morphological similarity of the sample subimages. The image classification model is obtained by carrying out unsupervised learning based on the clustering result of each sample subimage in the sample image, a large number of sample images are not required to be marked manually, the training efficiency of the model is improved, and the clustering result of each sample subimage is determined based on the morphological similarity of each sample subimage, so that the spatial information contained in the sample subimage is fully utilized for clustering, the problem of reduced recognition degree caused by deformation generated when the image is transformed in the conventional unsupervised learning process is solved, and the accuracy of image classification is improved.
Description
Technical Field
The present invention relates to the field of computer technologies, and in particular, to an image classification method and apparatus, an electronic device, and a storage medium.
Background
The image classification is to distinguish different image types according to the semantic information of the image, and the computer is used for carrying out quantitative analysis on the image, and classifying each pixel or area in the image into one of a plurality of image types to replace the visual interpretation of people.
At present, image classification is mostly performed through an image classification model taking a large number of sample images as a training set, but for high-throughput large-scale images (such as pathological images, satellite images and the like), in order to ensure the training effect of the model, a large number of high-quality labels need to be manually performed in the sample images, which is time-consuming and easy to make mistakes, and the training effect of the model is influenced.
Disclosure of Invention
The invention provides an image classification method, an image classification device, electronic equipment and a storage medium, which are used for solving the defect that the training effect of a model is influenced by marking a large number of training sets manually in the prior art.
The invention provides an image classification method, which comprises the following steps:
acquiring an image to be classified;
inputting the image to be classified into an image classification model to obtain an image classification result output by the image classification model;
the image classification model is obtained by carrying out unsupervised learning based on the clustering result of each sample subimage of the sample image; the clustering result is determined based on morphological similarity of the sample sub-images.
According to the image classification method provided by the invention, the clustering result is determined based on the following steps:
dividing the sample image into a plurality of sample sub-images based on a preset rule;
shifting each sample sub-image to obtain a sub-image set corresponding to each sample sub-image;
calculating the morphological similarity of each sub-image set and the corresponding sample sub-image based on the conditional random field probability model;
and if the morphological similarity meets a preset condition, the category of the corresponding sub-image set is consistent with that of the corresponding sample sub-image.
According to the image classification method provided by the invention, the shifting is performed on each sample sub-image to obtain the sub-image set corresponding to each sample sub-image, and the method comprises the following steps:
and on the basis of a preset offset set, offsetting each sample sub-image according to each offset in the preset offset set to obtain a sub-image set corresponding to each sample sub-image.
According to the image classification method provided by the invention, the conditional random field probability model is as follows:
wherein P represents the morphological similarity,
denotes a normalization constant, exp denotes an exponential function, E denotes an energy function,
a random vector is represented that describes the annotation of the image,
a set of original images is represented as,
denotes the ith1The random variable of the labeling of each sub-image,
denotes the ithkThe random variable of the labeling of each sub-image,
i1denotes the ith1Sub-images with increased offset, iKDenotes the ithkThe sub-images with the offset added to them,
indicates adjacency
Is marked withThe vector of the vector is then calculated,
indicates adjacency
Set of images of, k1To represent
Specific value of (a), kr×cRepresents Zr×cSpecific value of (A), K1Representing proximity to random variables
Whether the first subimage after the rearrangement of the subscripts is used for the random variable of unsupervised learning, Kr×cRepresenting proximity to random variables
Whether the last sub-image after the rearrangement of the subscripts is used for random variables of unsupervised learning.
According to the image classification method provided by the invention, the image to be classified is input into an image classification model to obtain an image classification result output by the image classification model, and the image classification method comprises the following steps:
inputting the image to be classified into an image classification layer of the image classification model to obtain a plurality of sub-images output by the image classification layer;
and inputting each sub-image into an image clustering layer of the image classification model to obtain the image classification result output by the image clustering layer.
According to the image classification method provided by the invention, the image classification model is subjected to unsupervised learning by taking maximized mutual information as an objective function.
According to the image classification method provided by the invention, the image to be classified is a medical image or a satellite data image.
The present invention also provides an image classification apparatus, comprising:
an acquisition unit for acquiring an image to be classified;
the classification unit is used for inputting the image to be classified into an image classification model to obtain an image classification result output by the image classification model;
the image classification model is obtained by carrying out unsupervised learning based on the clustering result of each sample subimage of the sample image; the clustering result is determined based on morphological similarity of the sample sub-images.
The invention further provides an electronic device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, wherein the processor executes the computer program to realize the steps of any one of the image classification methods.
The invention also provides a non-transitory computer-readable storage medium having stored thereon a computer program which, when executed by a processor, performs the steps of the image classification method as any one of the above.
According to the image classification method, the device, the electronic equipment and the storage medium, the image classification model is obtained through unsupervised learning based on the clustering result of each sample subimage in the sample image, a large number of sample images are not required to be labeled manually, and the training efficiency of the model is improved. Meanwhile, the clustering result of each sample subimage is determined based on the morphological similarity of each sample subimage, so that the spatial information contained in the sample subimage is fully utilized for clustering, the problem of reduced recognition degree caused by deformation generated when the image is subjected to transformation operation in the conventional unsupervised learning is solved, and the accuracy of image classification is improved.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of an image classification method provided by the present invention;
FIG. 2 is a schematic diagram of the clustering results provided by the present invention;
FIG. 3 is a flow diagram of one of the image classification algorithm frameworks provided by the present invention;
FIG. 4 is a second schematic flowchart of the image classification algorithm framework provided by the present invention;
FIG. 5 is a schematic structural diagram of an image classification apparatus provided in the present invention;
fig. 6 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The image classification is to distinguish different image types according to the semantic information of the image, and the image is quantitatively analyzed by a computer to classify each pixel or area in the image into one of a plurality of image types.
At present, image classification is mostly performed through an image classification model taking a large number of sample images as a training set, but for high-throughput large-scale images (such as pathological images, satellite images and the like), in order to ensure the training effect of the model, a large number of high-quality labels need to be manually performed in the sample images, which is time-consuming and easy to make mistakes, and the training effect of the model is influenced. For example, in the medical field, a full-field digital section under a microscope is the gold standard for cancer diagnosis, but the full-field microscope image is large in size (about 80000 × 80000 pixels) and its labeling depends on an experienced pathologist or doctor, while a high-precision deep learning based image classification model tends to depend on a large number of high-quality labels, so it is difficult to obtain a large number of labeled full-field images for training a robust deep learning model.
In view of the above, the present invention provides an image classification method. Fig. 1 is a schematic flow chart of an image classification method provided by the present invention, as shown in fig. 1, the method includes the following steps:
the image classification model is obtained by carrying out unsupervised learning based on the clustering result of each sample subimage of the sample image; the clustering result is determined based on the morphological similarity of the sample subimages.
In the embodiment of the present invention, it should be noted that before training an image classification model, a large number of sample images need to be acquired as training samples of the image classification model, and in the prior art, a large number of labels need to be manually performed on each region in a sample, which consumes a large amount of labor cost and time cost, and also causes false labels due to manual errors. Unsupervised learning is used as a label-free training mode, the characteristics of data are utilized, and a training model is not depended on manual labeling, so that the unsupervised learning method has considerable application prospect on high-throughput large-scale images. However, the mutual information of the existing unsupervised learning mainly comes from: firstly, a matched image is generated through matrix transformation operations such as turning and distortion of a single image or other contrast adjustment. And secondly, dimension reduction processing is carried out on the large image, so that the analysis space is reduced. And thirdly, generating mutual information based on the generating matrix of the GAN. However, when the original image is transformed, the generated paired image may be deformed with respect to the original image, so that the degree of recognition may be reduced, and the classification accuracy of the model may be reduced.
Therefore, the embodiment of the invention utilizes the adjacent images to generate mutual information for the unsupervised learning of the image classification model based on the spatial information contained in the sample images, so that the manual marking amount can be greatly reduced, the training efficiency of the model is improved, and compared with the existing unsupervised learning method, the embodiment of the invention fully utilizes the spatial information contained in the images, and avoids the phenomenon that the recognition degree is reduced and the training effect of the model is influenced due to the deformation generated during the image transformation operation.
Specifically, the image to be classified is input to the image classification model, and the image classification result output by the image classification model is obtained. For example, after the image a is input into the image classification model, the image classification model outputs the image classification result that the region 1, the region 2 and the region 3 in the image a belong to the same category, and the region 4 and the region 5 belong to the same category. The image to be classified refers to an image to be classified, and may be a large-size high-resolution high-throughput large-scale image (such as a pathological image, a satellite image, and the like) or an image of a common pixel size.
It should be noted that the image classification model is obtained by performing unsupervised learning based on the clustering result of each sample sub-image of the sample image, and the clustering result is determined based on the morphological similarity of each sample sub-image. Each sample sub-image may be obtained by dividing the sample image according to a preset fixed size, or may be obtained by dividing the sample image according to different sizes. The clustering result of each sample sub-image is determined based on the morphological similarity of each sample sub-image, for example, if the morphological similarity of the sample sub-image a and the sample sub-image B is greater than a preset value, the sample sub-image a and the sample sub-image B can be divided into the same type; or shifting the sample sub-image a to obtain a plurality of sub-images in different areas, forming a sub-image set, and calculating the morphological similarity between the sub-image set and the sample sub-image a, wherein if the morphological similarity satisfies a preset condition, each image category in the sub-image set is considered to be consistent with the category of the sample sub-image a. The morphological similarity of each sample sub-image may be determined based on the euclidean distance, may also be determined based on the cosine distance, and may also be determined based on the conditional random field model, which is not specifically limited in the embodiment of the present invention.
Before the step 120 is executed, the image classification model may also be obtained by training in advance, which may specifically be implemented by executing the following steps: firstly, a large number of sample images are collected, clustering results are determined based on morphological similarity of all sample subimages in the sample images, and unsupervised learning is carried out based on the clustering results, so that an image classification model is obtained.
As shown in fig. 2, the image classification method according to the embodiment of the present invention obtains a classification result in which two images with K equal to 2 and I equal to 0.89 are grouped together, two images with K equal to 2 and I equal to 0.12 are grouped together, three images with K equal to 3 and I equal to 0.91 are grouped together, and four images with K equal to 4 and I equal to 0.02 are grouped together. As shown in fig. 3, after the image to be classified is input to the image classification model, the image classification model generates corresponding clusters 1 to 9, and visualizes the classification result. Compared with the existing unsupervised learning method, the preliminary experiment shows that the accuracy of the method can be improved by at least 10% -20% on a large-scale microscope pathological image, and the accuracy (60% -70%) of the method is greatly improved compared with the accuracy of the existing unsupervised learning.
Therefore, the image classification method provided by the embodiment of the invention obtains the image classification model by carrying out unsupervised learning based on the clustering result of each sample sub-image in the sample image, does not depend on manually labeling a large number of sample images, and improves the training efficiency of the model. Meanwhile, the clustering result of each sample subimage is determined based on the morphological similarity of each sample subimage, so that the spatial information contained in the sample subimage is fully utilized for clustering, the problem of reduced recognition degree caused by deformation generated when the image is subjected to transformation operation in the conventional unsupervised learning is solved, and the accuracy of image classification is improved.
Based on the above embodiment, the clustering result is determined based on the following steps:
dividing the sample image into a plurality of sample sub-images based on a preset rule;
shifting each sample sub-image to obtain a sub-image set corresponding to each sample sub-image;
calculating the morphological similarity of each sub-image set and the corresponding sample sub-image based on the conditional random field probability model;
and if the morphological similarity meets the preset condition, the category of the corresponding sub-image set is consistent with that of the corresponding sample sub-image.
Specifically, when the sample image is divided into a plurality of sample sub-images, the sample image may be divided into the plurality of sample sub-images according to a preset fixed size set, or the sample image may be divided into the plurality of sample sub-images according to different size sets, which is not specifically limited in the embodiment of the present invention. For example, if the fixed size set is preset to size a, size B, and size C, the sample image may be divided into a plurality of sample sub-images of size a according to size a, divided into a plurality of sample sub-images of size B according to size B, and divided into a plurality of sample sub-images of size C according to size C; for another example, the center point 1 in the sample image may be divided by a size a, the center point 2 may be divided by a size B, the center point 3 may be divided by a size C, and the like for the first time, that is, the sample image may be divided into a plurality of sample sub-images with different sizes and different areas according to different rules.
And after the sample sub-images are obtained through division, shifting the sample sub-images to obtain a sub-image set corresponding to the sample sub-images. For example, sample sub-image a is shifted to the right by Δ p to obtain sub-image a1, to the left by Δ p to obtain sub-image a2, to the up by Δ p to obtain sub-image a3, to the down by Δ p to obtain sub-image a4, to the right by 2 Δ p to obtain sub-image b1, to the left by 2 Δ p to obtain sub-image b2, to the up by 2 Δ p to obtain sub-image b3, to the down by 2 Δ p to obtain sub-image b4, and so on. Wherein, a1-a3 can be used as the sub-image set, a1-a4 can be used as the sub-image set, b1-b4 can be used as the sub-image set, and a1-a4 and b1-b4 can be used as the sub-image set, so that different sub-image sets can be obtained by the same sample sub-image according to different offset rules.
When a sub-image set corresponding to each sample sub-image is obtained, calculating the morphological similarity of each sub-image set and the corresponding sample sub-image based on a conditional random field probability model; if the morphological similarity meets a preset condition, if the morphological similarity is greater than a preset value, the category of the corresponding sub-image set is consistent with the category of the corresponding sample sub-image. It should be noted that, since different sub-image sets can be obtained from the same sample sub-image, after the morphological similarity between different sub-image sets and the sample sub-image is calculated, the higher the morphological similarity is, the higher the morphological correlation degree between the image in the corresponding sub-image set and the sample image is, and when the morphological similarity satisfies the preset condition, the image in the sub-image set can be used as the matching image of the sample image for unsupervised learning.
As shown in fig. 4, in the embodiment of the present invention, a plurality of matching images having the most morphological correlation are dynamically selected in a certain Region (ROI), r × c patch sets are initially set, a new r × c patch set is formed by an offset Δ p of each image, the new r × c patch sets are respectively clustered, and unsupervised learning is performed using maximum mutual information (Multiple mutual information) as an objective function.
Based on any of the above embodiments, shifting each sample sub-image to obtain a small block sub-image set corresponding to each sample sub-image, including:
and on the basis of the preset offset set, offsetting each sample sub-image according to each offset in the preset offset set to obtain a sub-image set corresponding to each sample sub-image.
Specifically, after the sample sub-images are obtained by division, the sample sub-images may be shifted according to a preset offset set, so as to obtain a small block sub-image set corresponding to each sample sub-image. For example, if the preset offset set is offset Δ p and offset 2 Δ p, the sample sub-image a is offset to the right by Δ p to obtain sub-image a1, offset to the left by Δ p to obtain sub-image a2, offset upward by Δ p to obtain sub-image a3, offset downward by Δ p to obtain sub-image a4, offset to the right by 2 Δ p to obtain sub-image b1, offset leftward by 2 Δ p to obtain sub-image b2, offset upward by 2 Δ p to obtain sub-image b3, offset downward by 2 Δ p to obtain sub-image b4, and so on. The method includes the steps that a1-a3 can be used as a sub-image set, a1-a4 can be used as a sub-image set, b1-b4 can be used as a sub-image set, and a1-a4 and b1-b4 can be used as sub-image sets, so that for the same sample sub-image, a plurality of sub-image sets can be obtained based on a preset offset set, the morphological similarity of each sub-image set is different from that of the sample sub-image, and if the morphological similarity meets a preset condition, the corresponding sub-image set is consistent with the category of the sample sub-image and can be used for unsupervised learning.
Based on any of the above embodiments, the conditional random field probability model is:
wherein P represents a morphological similarity, kiEqual to 0 or 1, or a combination thereof,
representing a normalization constant such that the output value of said probability representation is between 0 and 1, exp representing an exponential function, E representing an energy function determining the magnitude of the probability values of the set of random variables,
a random vector is represented that describes the annotation of the image,
representing a set of original images, and constituting a condition of the above conditional probability,
denotes the ith1The random variable of the labeling of each sub-image,
denotes the ithkThe random variable of the labeling of each sub-image,
for all
i1Denotes the ith1Sub-images with increased offset, iKDenotes the ithkThe sub-images with the offset added to them,
indicates adjacency
The label vector of (a) is,
indicates adjacency
Set of images of, k1To represent
Specific value of (a), kr×cRepresents Zr×cSpecific value of (A), K1Representing proximity to random variables
Whether the first subimage after the rearrangement of the subscripts is used for the random variable of unsupervised learning, Kr×cRepresenting proximity to random variables
Whether the last sub-image after the rearrangement of the subscripts is used for random variables of unsupervised learning.
In particular, KiThe random variable is a discrete random variable, wherein 1 indicates that the form correlation degree of the corresponding sub-image set and the sample sub-image is high and belongs to the same category, the corresponding sub-image set (patch set) can be selected for unsupervised learning, and 0 indicates that the form correlation degree of the corresponding sub-image set and the sample sub-image is low and does not belong to the same category and is not used for unsupervised learning. Because of KiAnd the ith sub-image, and gas coordinatesIn a one-to-one correspondence relationship, so the embodiment of the present invention passes through KiThe value of (a) may describe whether all sub-images are used to train the unsupervised model.
Based on any of the above embodiments, inputting an image to be classified into an image classification model to obtain an image classification result output by the image classification model, including:
inputting an image to be classified into an image classification layer of an image classification model to obtain a plurality of sub-images output by the image classification layer;
and inputting each sub-image into an image clustering layer of the image classification model to obtain an image classification result output by the image clustering layer.
Specifically, the image to be classified is input to an image classification layer of the image classification model, and the image classification layer may classify the image to be classified according to a preset fixed size, or may classify the image to be classified according to different sizes. And inputting each subimage into an image clustering layer of the image classification model, wherein the image clustering layer can cluster each subimage to obtain an image classification result. Therefore, the embodiment of the invention utilizes the adjacent images to generate mutual information for the unsupervised learning of the image classification model based on the spatial information contained in the sample images, so that the manual marking amount can be greatly reduced, the training efficiency of the model is improved, and compared with the existing unsupervised learning method, the embodiment of the invention fully utilizes the spatial information contained in the images, and avoids the phenomenon that the recognition degree is reduced and the training effect of the model is influenced due to the deformation generated during the image transformation operation.
Based on any of the above embodiments, the image classification model is unsupervised learning with maximized mutual information as an objective function.
Specifically, in order to enable the image classification model to learn good characteristics from the clustering result of the sample sub-images, that is, to clearly distinguish the characteristics of the sample sub-images, namely, the characteristics including the most unique information of the sample sub-images, the embodiment of the invention performs unsupervised learning by using the maximized mutual information as an objective function, so that the image classification model can learn the characteristics of each sample sub-image. For example, for the sample sub-image a, the image classification model may learn the characterization X of the image, and the characterization X may reconstruct the image, that is, the sub-images belonging to the same category as the image may be accurately classified into one category based on the characterization X, so as to further improve the training effect of the model.
Based on any of the above embodiments, the image to be classified is a medical image or a satellite data image.
Specifically, for medical images or satellite data images, in order to ensure the training effect of the model, a large amount of high-quality labels need to be manually marked in the sample images, which is time-consuming and easy to make mistakes, and affects the training effect of the model. For example, for a full-field microscope image in a medical image, the image size is large (about 80000 × 80000 pixels), and its labeling depends on an experienced pathologist or doctor, while a high-precision depth learning-based image classification model tends to depend on a large number of high-quality labels. The embodiment of the invention obtains the image classification model by carrying out unsupervised learning based on the clustering result of each sample subimage in the sample image, does not depend on manually labeling a large number of sample images, and improves the training efficiency of the model. Meanwhile, the clustering result of each sample subimage is determined based on the morphological similarity of each sample subimage, so that the spatial information contained in the sample subimage is fully utilized for clustering, the problem of reduced recognition degree caused by deformation generated when the image is subjected to transformation operation in the conventional unsupervised learning is solved, and the accuracy of image classification is improved.
The following describes the image classification apparatus provided by the present invention, and the image classification apparatus described below and the image classification method described above may be referred to in correspondence with each other.
Based on any of the above embodiments, as shown in fig. 5, the present invention provides an image classification apparatus, including:
an obtaining unit 510, configured to obtain an image to be classified;
the classifying unit 520 is configured to input the image to be classified into an image classification model, so as to obtain an image classification result output by the image classification model;
the image classification model is obtained by carrying out unsupervised learning based on the clustering result of each sample subimage of the sample image; the clustering result is determined based on morphological similarity of the sample sub-images.
Based on any of the above embodiments, the method further includes a determining unit, configured to determine the clustering result, where the determining unit includes:
the dividing unit is used for dividing the sample image into a plurality of sample sub-images based on a preset rule;
the offset unit is used for offsetting each sample sub-image to obtain a sub-image set corresponding to each sample sub-image;
the calculating unit is used for calculating the morphological similarity of each sub-image set and the corresponding sample sub-image based on the conditional random field probability model;
and the clustering unit is used for enabling the category of the corresponding sub-image set to be consistent with the category of the corresponding sample sub-image if the morphological similarity meets the preset condition.
Based on any of the embodiments above, the offset unit is specifically configured to:
and on the basis of a preset offset set, offsetting each sample sub-image according to each offset in the preset offset set to obtain a sub-image set corresponding to each sample sub-image.
Based on any of the above embodiments, the classification unit 520 includes:
the image dividing unit is used for inputting the image to be classified into an image dividing layer of the image classification model to obtain a plurality of sub-images output by the image dividing layer;
and the image classification unit is used for inputting each sub-image into the image clustering layer of the image classification model to obtain the image classification result output by the image clustering layer.
Based on any one of the above embodiments, the image classification model is unsupervised learning by taking the maximized mutual information as an objective function.
Based on any embodiment, the image to be classified is a medical image or a satellite data image.
Fig. 6 is a schematic structural diagram of an electronic device provided in the present invention, and as shown in fig. 6, the electronic device may include: a processor (processor)610, a communication Interface (Communications Interface)620, a memory (memory)630 and a communication bus 640, wherein the processor 610, the communication Interface 620 and the memory 630 communicate with each other via the communication bus 640. The processor 610 may invoke logic instructions in the memory 630 to perform an image classification method comprising: acquiring an image to be classified; inputting the image to be classified into an image classification model to obtain an image classification result output by the image classification model; the image classification model is obtained by carrying out unsupervised learning based on the clustering result of each sample subimage of the sample image; the clustering result is determined based on morphological similarity of the sample sub-images.
In addition, the logic instructions in the memory 630 may be implemented in software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk, and other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions which, when executed by a computer, enable the computer to perform the image classification method provided by the above methods, the method comprising: acquiring an image to be classified; inputting the image to be classified into an image classification model to obtain an image classification result output by the image classification model; the image classification model is obtained by carrying out unsupervised learning based on the clustering result of each sample subimage of the sample image; the clustering result is determined based on morphological similarity of the sample sub-images.
In yet another aspect, the present invention also provides a non-transitory computer readable storage medium having stored thereon a computer program which, when executed by a processor, is implemented to perform the image classification method provided above, the method comprising: acquiring an image to be classified; inputting the image to be classified into an image classification model to obtain an image classification result output by the image classification model; the image classification model is obtained by carrying out unsupervised learning based on the clustering result of each sample subimage of the sample image; the clustering result is determined based on morphological similarity of the sample sub-images.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.
Claims (10)
1. An image classification method, comprising:
acquiring an image to be classified;
inputting the image to be classified into an image classification model to obtain an image classification result output by the image classification model;
the image classification model is obtained by carrying out unsupervised learning based on the clustering result of each sample subimage of the sample image; the clustering result is determined based on morphological similarity of the sample sub-images.
2. The image classification method according to claim 1, characterized in that the clustering result is determined based on the steps of:
dividing the sample image into a plurality of sample sub-images based on a preset rule;
shifting each sample sub-image to obtain a sub-image set corresponding to each sample sub-image;
calculating the morphological similarity of each sub-image set and the corresponding sample sub-image based on the conditional random field probability model;
and if the morphological similarity meets a preset condition, the category of the corresponding sub-image set is consistent with that of the corresponding sample sub-image.
3. The image classification method according to claim 2, wherein the shifting the sample sub-images to obtain the sub-image set corresponding to the sample sub-images comprises:
and on the basis of a preset offset set, offsetting each sample sub-image according to each offset in the preset offset set to obtain a sub-image set corresponding to each sample sub-image.
4. The image classification method of claim 2, wherein the conditional random field probability model is:
wherein P represents the morphological similarity,
denotes a normalization constant, exp denotes an exponential function, E denotes an energy function,
a random vector is represented that describes the annotation of the image,
a set of original images is represented as,
denotes the ith1The random variable of the labeling of each sub-image,
denotes the ithkThe random variable of the labeling of each sub-image,
i1denotes the ith1Sub-images with increased offset, iKDenotes the ithkThe sub-images with the offset added to them,
indicates adjacency
The label vector of (a) is,
indicates adjacency
Set of images of, k1To represent
Specific value of (a), kr×cRepresents Zr×cSpecific value of (A), K1Representing proximity to random variables
Whether the first subimage after the rearrangement of the subscripts is used for the random variable of unsupervised learning, Kr×cRepresenting proximity to random variables
Whether the last sub-image after the rearrangement of the subscripts is used for random variables of unsupervised learning.
5. The image classification method according to claim 1, wherein the inputting the image to be classified into an image classification model to obtain an image classification result output by the image classification model, comprises:
inputting the image to be classified into an image classification layer of the image classification model to obtain a plurality of sub-images output by the image classification layer;
and inputting each sub-image into an image clustering layer of the image classification model to obtain the image classification result output by the image clustering layer.
6. The image classification method according to any one of claims 1 to 5, characterized in that the image classification model is unsupervised learned with maximized mutual information as an objective function.
7. The image classification method according to any one of claims 1 to 5, characterized in that the image to be classified is a medical image or a satellite data image.
8. An image classification apparatus, comprising:
an acquisition unit for acquiring an image to be classified;
the classification unit is used for inputting the image to be classified into an image classification model to obtain an image classification result output by the image classification model;
the image classification model is obtained by carrying out unsupervised learning based on the clustering result of each sample subimage of the sample image; the clustering result is determined based on morphological similarity of the sample sub-images.
9. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, characterized in that the steps of the image classification method according to any of claims 1 to 7 are implemented when the processor executes the program.
10. A non-transitory computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the steps of the image classification method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110510967.4A CN113177602B (en) | 2021-05-11 | 2021-05-11 | Image classification method, device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110510967.4A CN113177602B (en) | 2021-05-11 | 2021-05-11 | Image classification method, device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113177602A true CN113177602A (en) | 2021-07-27 |
| CN113177602B CN113177602B (en) | 2023-05-26 |
Family
ID=76928824
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110510967.4A Active CN113177602B (en) | 2021-05-11 | 2021-05-11 | Image classification method, device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113177602B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116524248A (en) * | 2023-04-17 | 2023-08-01 | 首都医科大学附属北京友谊医院 | Medical data processing device, method and classification model training device |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040002930A1 (en) * | 2002-06-26 | 2004-01-01 | Oliver Nuria M. | Maximizing mutual information between observations and hidden states to minimize classification errors |
| CN110737730A (en) * | 2019-10-21 | 2020-01-31 | 腾讯科技（深圳）有限公司 | Unsupervised learning-based user classification method, unsupervised learning-based user classification device, unsupervised learning-based user classification equipment and storage medium |
| CN111179961A (en) * | 2020-01-02 | 2020-05-19 | 腾讯科技（深圳）有限公司 | Audio signal processing method, audio signal processing device, electronic equipment and storage medium |
| CN111753863A (en) * | 2019-04-12 | 2020-10-09 | 北京京东尚科信息技术有限公司 | Image classification method and device, electronic equipment and storage medium |
| CN112132841A (en) * | 2020-09-22 | 2020-12-25 | 上海交通大学 | Medical image cutting method and device |
-
2021
- 2021-05-11 CN CN202110510967.4A patent/CN113177602B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20040002930A1 (en) * | 2002-06-26 | 2004-01-01 | Oliver Nuria M. | Maximizing mutual information between observations and hidden states to minimize classification errors |
| CN111753863A (en) * | 2019-04-12 | 2020-10-09 | 北京京东尚科信息技术有限公司 | Image classification method and device, electronic equipment and storage medium |
| CN110737730A (en) * | 2019-10-21 | 2020-01-31 | 腾讯科技（深圳）有限公司 | Unsupervised learning-based user classification method, unsupervised learning-based user classification device, unsupervised learning-based user classification equipment and storage medium |
| CN111179961A (en) * | 2020-01-02 | 2020-05-19 | 腾讯科技（深圳）有限公司 | Audio signal processing method, audio signal processing device, electronic equipment and storage medium |
| CN112132841A (en) * | 2020-09-22 | 2020-12-25 | 上海交通大学 | Medical image cutting method and device |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116524248A (en) * | 2023-04-17 | 2023-08-01 | 首都医科大学附属北京友谊医院 | Medical data processing device, method and classification model training device |
| CN116524248B (en) * | 2023-04-17 | 2024-02-13 | 首都医科大学附属北京友谊医院 | Medical data processing device, method and classification model training device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113177602B (en) | 2023-05-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11657602B2 (en) | Font identification from imagery | |
| JP6710135B2 (en) | Cell image automatic analysis method and system | |
| US11334982B2 (en) | Method for defect classification, method for training defect classifier, and apparatus thereof | |
| CN110503054B (en) | Text image processing method and device | |
| CN109145964B (en) | Method and system for realizing image color clustering | |
| Pardo et al. | Semantic segmentation of mFISH images using convolutional networks | |
| CN114862845B (en) | Defect detection method, device and equipment for mobile phone touch screen and storage medium | |
| CN112132166A (en) | Intelligent analysis method, system and device for digital cytopathology image | |
| CN111680690A (en) | Character recognition method and device | |
| CN113888536B (en) | Printed matter double image detection method and system based on computer vision | |
| CN113362277A (en) | Workpiece surface defect detection and segmentation method based on deep learning | |
| CN114445268A (en) | Garment style migration method and system based on deep learning | |
| CN111161213B (en) | Industrial product defect image classification method based on knowledge graph | |
| CN114494739B (en) | Toner mixing effect detection method based on artificial intelligence | |
| CN113421223B (en) | Industrial product surface defect detection method based on deep learning and Gaussian mixture | |
| CN106709490B (en) | Character recognition method and device | |
| CN114444565A (en) | Image tampering detection method, terminal device and storage medium | |
| CN113177602A (en) | Image classification method and device, electronic equipment and storage medium | |
| CN107590806B (en) | Detection method and system based on brain medical imaging | |
| CN112200789A (en) | Image identification method and device, electronic equipment and storage medium | |
| CN109460768B (en) | Text detection and removal method for histopathology microscopic image | |
| CN115908363A (en) | Tumor cell counting method, device, equipment and storage medium | |
| CN115937095A (en) | Printing defect detection method and system integrating image processing algorithm and deep learning | |
| CN111401485A (en) | Practical texture classification method | |
| WO2018131723A1 (en) | Image processing device and image processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |