WO2022227192A1

WO2022227192A1 - Image classification method and apparatus, and electronic device and medium

Info

Publication number: WO2022227192A1
Application number: PCT/CN2021/097079
Authority: WO
Inventors: 刘杰; 王健宗; 瞿晓阳
Original assignee: 平安科技（深圳）有限公司
Priority date: 2021-04-28
Filing date: 2021-05-30
Publication date: 2022-11-03
Also published as: CN112990374B; CN112990374A

Abstract

An image classification method in the technical field of image processing, the method comprising: randomly acquiring, from an image set, an image to be processed, and obtaining two processed images by means of two preset data enhancement methods and two preset image processing networks; constructing a probability constraint parameter of a probability graph model by using the two processed images, and updating the probability constraint parameter according to the image to be processed and a category mark of the image to be processed, so as to obtain the probability graph model; training the probability graph model to obtain an image classification model; and acquiring an image to be classified, and inputting, into the image classification model, the image to be classified and classifying same, so as to obtain a category label of the image to be classified. Further provided are an image classification apparatus, and a device and a storage medium. The solution further relates to blockchain technology, and the image set can be stored in a blockchain node. By means of the solution, the accuracy of image classification can be improved.

Description

Image classification method, device, electronic device and medium

This application claims the priority of the Chinese patent application filed on April 28, 2021 with the application number 202110467270.3 and the title of the invention is "image classification method, device, electronic device and medium", the entire contents of which are incorporated by reference in in this application.

technical field

The present application relates to the technical field of image processing, and in particular, to an image classification method, apparatus, electronic device, and computer-readable storage medium.

Background technique

With the development of artificial intelligence technology, there has also been a great breakthrough in the field of image classification, that is, image classification through deep learning methods. However, the existing image classification technologies are based on the premise of having large-scale annotated images. However, the large-scale annotated images themselves are relatively few, and the computational resources consumed to obtain large-scale annotated images are also very large. , and there is also an image noise problem between the labeled images, so in order to denoise, more labeled images are needed for training, making it more difficult to obtain large-scale labeled images.

The inventor realized that in the existing image classification method, the image classification is performed by the trained model, but there is a lack of labeled data in the model training, and there is less data available for training, resulting in the low accuracy of the model obtained by training, thus affecting the accuracy of the model. The accuracy of image classification results.

SUMMARY OF THE INVENTION

An image classification method provided by this application includes:

Acquire an image to be processed from an image set, and enhance the image to be processed by a preset first data enhancement method and a preset second data enhancement method, respectively, to obtain a first enhanced image and a second enhanced image, wherein the to-be-processed image is The processed images include unclassified images and classified images;

Inputting the first enhanced image into a pre-built first image processing network to obtain a first processed image, and inputting the second enhanced image into a pre-built second image processing network to obtain a second processed image;

Using the first processed image and the second processed image to construct a probability constraint parameter of a probabilistic graphical model;

The pseudo-category label of the category-free annotated image is obtained by calculating the uncategorized label image by a preset semi-supervised learning method, wherein the pseudo-category label is used to label the category of the category-free annotated image;

The probability constraint parameter is updated by using the labeled images with categories, the labeled images without categories, and the pseudo-category labels of the labeled images without categories to obtain the probability graph model;

The probabilistic graphical model is trained by using the to-be-processed image and the class label of the to-be-processed image to obtain an image classification model;

Obtaining an image to be classified, inputting the image to be classified into the image classification model for classification, and obtaining a class label of the image to be classified.

The present application also provides an image classification device, the device comprising:

The image enhancement module is used to obtain the image to be processed from the image set, and enhance the image to be processed by the preset first data enhancement method and the preset second data enhancement method respectively, so as to obtain the first enhanced image and the second enhanced image , wherein the to-be-processed image includes an unclassified image and a class-annotated image;

A parameter building module for inputting the first enhanced image into a pre-built first image processing network to obtain a first processed image, and inputting the second enhanced image into a pre-built second image processing network to obtain a second processed image an image, using the first processed image and the second processed image to construct a probability constraint parameter of a probabilistic graphical model;

The label generation module is used to calculate the unclassified labeled image through a preset semi-supervised learning method to obtain the pseudo-classified label of the unclassified labeled image, wherein the pseudo-classified label is used to label the unclassified labeled image. category;

a loss function building module, used for updating the probability constraint parameters through the labeled images with categories, the labeled images without categories, and the pseudo-category labels of the labeled images without categories, to obtain the probability graph model;

A classification model training module, configured to train the probabilistic graphical model by using the to-be-processed image and the class label of the to-be-processed image to obtain an image classification model;

The image classification module is configured to acquire the images to be classified, input the images to be classified into the image classification model for classification, and obtain the class labels of the images to be classified.

The present application also provides an electronic device, the electronic device comprising:

a memory that stores at least one computer program; and

The processor executes the computer program stored in the memory to realize the image classification method as described below:

The present application also provides a computer-readable storage medium, including a storage data area and a storage program area, the storage data area stores created data, and the storage program area stores a computer program; wherein, the computer program is implemented as follows when executed by a processor The described image classification method:

Description of drawings

1 is a schematic flowchart of an image classification method according to an embodiment of the present application;

FIG. 2 is a schematic block diagram of an image classification apparatus provided by an embodiment of the present application;

3 is a schematic diagram of the internal structure of an electronic device for implementing an image classification method provided by an embodiment of the present application;

The realization, functional characteristics and advantages of the purpose of the present application will be further described with reference to the accompanying drawings in conjunction with the embodiments.

Detailed ways

It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

The embodiment of the present application provides an image classification method. The execution subject of the image classification method includes, but is not limited to, at least one of electronic devices that can be configured to execute the method provided by the embodiments of the present application, such as a server and a terminal. In other words, the image classification method can be executed by software or hardware installed in a terminal device or a server device, and the software can be a blockchain platform. The server includes but is not limited to: a single server, a server cluster, a cloud server or a cloud server cluster, and the like.

Referring to FIG. 1 , it is a schematic flowchart of an image classification method according to an embodiment of the present application. In this embodiment, the image classification method includes:

S1. Acquire an image to be processed from an image set, and enhance the image to be processed by using a preset first data enhancement method and a preset second data enhancement method, respectively, to obtain a first enhanced image and a second enhanced image, wherein the The to-be-processed images include unclassified images and classified images.

In the embodiment of the present application, the category-annotated image is an image that has been category-annotated, and the category-unlabeled image is an image that has not undergone category annotation processing. For example, the category-labeled image is an image that has been manually labeled with categories (eg, by a BasicFinder tool) in advance, and the category-free image is an original image without category-labeling processing.

Specifically, the classified-annotated images have data annotations, and the unlabeled images do not have data annotations.

Specifically, the form of data annotation may include, but is not limited to, annotated picture frame, 3D picture frame, text transcription, image dotting, and contour lines of target objects.

In the embodiment of the present application, the unlabeled images and the classified images may be images in the medical field. For example, the unclassified images or the classified images are computer X-ray images CR, computed tomography CT, and nuclear magnetic resonance images. MR and other images.

Further, the number of the labeled images without categories is greater than a first preset value, the number of labeled images with categories is less than a second preset value, and the first preset value is greater than the second preset value.

Preferably, the first preset value is 10 times, 20 times, even 50 times, or 100 times the second preset value. For example, the first preset value is 5000 and the second preset value is 50.

In this embodiment of the present application, one image may be randomly acquired, or multiple images may be randomly acquired continuously.

Preferably, the first data enhancement method is a geometric transformation data enhancement method, and the second data enhancement method is a color transformation data enhancement method.

Further, the first data enhancement method and the second data enhancement method can make limited data generate value equivalent to more data without substantially increasing the data.

In the embodiments of the present application, the geometric transformation data enhancement methods include methods such as flipping, random rotation, random cropping, deformation scaling, and the like. The color transformation data enhancement method includes adding noise, blurring, color transformation, image filling, and the like.

In this embodiment of the present application, both the first processed image and the second processed image are obtained by randomly acquiring images from the image set through the data enhancement method. Therefore, the first processed image and the second processed image are obtained by the data enhancement method. Images are relevant.

S2. Input the first enhanced image into a pre-built first image processing network to obtain a first processed image, and input the second enhanced image into a pre-built second image processing network to obtain a second processed image.

In the embodiment of the present application, after obtaining the second processed image, the method further includes:

determining whether the first processed image is the same as the second processed image;

If the first processed image is the same as the second processed image, perform the operation of acquiring the image to be processed from the image set again;

If the first processed image is different from the second processed image, the operation of constructing a probability constraint parameter of a probabilistic graphical model by using the first processed image and the second processed image is performed.

In the embodiment of the present application, the first image processing network and the second image processing network are networks for processing images.

Further, the first image processing network and the second image processing network may be one of a Lebet-5 network, an AlexNet network, a VGG16 network, and a ResNet-50 network, wherein when the first image processing network and the first image processing network are When the two image processing networks are both Lebet-5 networks, the first image processing network and the second image processing network are respectively used for image screening and classification.

S3. Use the first processed image and the second processed image to construct a probability constraint parameter of a probabilistic graphical model.

In detail, the probability constraint parameters for constructing a probability graph model by using the first processed image and the second processed image include:

obtaining the original loss function for constructing the probabilistic graph model;

The first processed image and the second processed image are input into the original loss function, and the function parameters in the original loss function are updated to obtain probability constraint parameters.

In this embodiment of the present application, the first processed image and the second processed image may be input into the original loss function, and the first processed image and the second processed image may be used as functions of the original loss function item to update the parameters of the original loss function to obtain the updated probability constraint parameters, and use the updated probability constraint parameters as the probability constraint parameters.

S4. Calculate a pseudo-category label of the category-free annotated image through a preset semi-supervised learning method, wherein the pseudo-category label is used to mark the category of the category-free annotated image.

In the embodiment of the present application, the generation of the pseudo-category label of the uncategorized image by the preset semi-supervised learning method includes:

obtaining a first supervised learning model, and training the first supervised learning model by using the classified image to obtain a first training supervised model;

Using the first training supervision model to predict the unclassified labeled image to obtain the predicted probability of the unclassified labeled image;

using the predicted probability to select a target image from the image set;

Using the labeled image to train the first supervised learning model according to the target image to obtain a second supervised learning model;

Judging whether the second supervised learning model is the same as the first supervised learning model;

If not equal, use the second supervised learning model to replace the first supervised learning model, and perform the operation of training the first supervised learning model through the labeled images again;

If they are equal, it is determined that the training is completed, and the second supervised learning model is determined to be a training completed model;

Inputting the uncategorized image into the training completed model to obtain a pseudo-category label of the uncategorized image.

In detail, the obtaining the first supervised learning model includes:

receiving an initial training set, the initial training set includes a first labeled sample set and a first unlabeled sample set;

Obtain an initial semi-supervised learning model, and use the first labeled sample set and the first unlabeled sample set to train the initial semi-supervised learning model to obtain a first supervised learning network.

In the embodiment of the present application, the first supervised learning model is obtained by training an initial semi-supervised learning model through an initial training set, and the initial training set includes a first labeled sample set and a first unlabeled sample set, The first labeled sample set includes labeled data: the first unlabeled sample set includes unlabeled data. Among them, labeled data and unlabeled data are not limited to image data. The data labels include classification labels of image data and classification labels of digital data, wherein image data can be divided into labeled image data and unlabeled image data, and digital data can be divided into symbol data and text data.

Specifically, the initial semi-supervised learning model is any one of Π-model, VAT, LPDSSL, TNAR, pseudo-label, and DCT.

S5. Update the probability constraint parameter by using the labeled images with categories, the labeled images without categories, and the pseudo-category labels of the labeled images without categories, to obtain the probability graph model.

In the embodiment of the present application, the probability constraint parameter is updated by using the labeled images with categories, the labeled images without categories, and the pseudo-category labels of the labeled images without categories, and obtaining the probability graph model includes:

obtaining the first quantity of the unclassified annotated images and the second quantity of the classified annotated images in the to-be-processed image;

Based on the category annotation of the image, a vector composed of category-annotated images and a vector composed of non-category-annotated images are obtained;

Obtain the original loss function of the probabilistic graphical model and use the first number of the unclassified annotated images, the second number of the classified annotated images, the vector composed of the classified classified images, and the unclassified annotated images The formed vector updates the probability constraint parameters in the original loss function to obtain the probability graph model.

In detail, before obtaining the probabilistic graphical model, the method further includes:

Based on the update of the probability constraint parameter, an update loss function is obtained,

The probabilistic graphical model is obtained according to the update loss function.

Specifically, the loss function is a function that maps random time or the value of its related random variable to a non-negative real number to represent the risk or loss of the random event. In the embodiment of the present application, the update loss function uses for numerical estimation of the parameters of the probabilistic graphical model.

In the embodiment of the present application, the update loss function is:

Among them, f ^L represents the vector composed of the labeled images with class, f ^U represents the vector composed of the labeled images without class, n _U represents the number of labeled images without class, n _L represents the number of labeled images with class Quantity, y represents the vector composed of the category labels of the first processed image and the second processed image, CE represents the cross entropy error function, MSE represents the mean square error function, λ is the coefficient function, used to rule the cross entropy Subnomial bias.

In the embodiments of the present application, the probabilistic graphical model is a theory that uses a graph to represent the probability dependence of variables, and combines the knowledge of probability theory and graph theory to use a graph to represent the joint probability distribution of variables related to the model.

S6. Train the probabilistic graphical model by using the to-be-processed image and the class label of the to-be-processed image to obtain an image classification model.

Specifically, the probabilistic graphical model is trained by using the to-be-processed image and the category label of the to-be-processed image, and the obtained image classification model includes:

A class-unlabeled image set is constructed using the unclassified labeled images in the image set, and a class-labeled image set is constructed using the class-labeled images in the image set.

Passing the unclassified annotated image set through the first data augmentation network to obtain a first processed image set of the unclassified annotated image set, and passing the unclassified annotated image set through the second data augmentation network to obtain the first processed image set. a second processed image set of the class-free annotated image set;

Passing the class-labeled image set through the first data augmentation network to obtain a first processed image set of the class-labeled image set, and passing the class-labeled image set through the second data augmentation network to obtain the first processed image set. a second processed image set describing the class-labeled image set;

Annotation, The labeling of the class-labeled image set trains the probabilistic graphical model including the update loss function to obtain an image classification model.

S7. Acquire an image to be classified, input the image to be classified into the image classification model for classification, and obtain a class label of the image to be classified.

In this embodiment of the present application, the image to be classified may be an image that has not undergone image processing, or the image to be classified may also be an image that has undergone image processing (such as color correction processing) but has not been labeled with categories.

The image to be classified may be a medical image.

For example, the image to be classified is a lung CT image, and the image classification model is used to classify the image for lesions, and it is determined whether the lung CT image contains a lesion.

In this embodiment, after the unlabeled images are acquired, pseudo-labels are generated for the unlabeled images through a semi-supervised learning method, and then the unclassified labeled images are used for image classification model training, which increases the amount of data when training the model. And the diversity of data, while further improving the utilization of data and the accuracy of image classification model training. After the image classification model is obtained, the image to be classified is input into the image classification model to obtain an accurate classification result, thereby achieving the purpose of improving the accuracy of image classification.

As shown in FIG. 2 , it is a schematic block diagram of the image classification apparatus of the present application.

The image classification apparatus 100 described in this application can be installed in an electronic device. According to the implemented functions, the image classification apparatus may include an image enhancement module 101 , a parameter construction module 102 , an annotation generation module 103 , a loss function construction module 104 , a classification model training module 105 and an image classification module 106 . The modules described in this application may also be referred to as units, which refer to a series of computer program segments that can be executed by the processor of an electronic device and can perform fixed functions, and are stored in the memory of the electronic device.

In this embodiment, the functions of each module/unit are as follows:

The image enhancement module 101 is configured to obtain the to-be-processed image from an image set, and to enhance the to-be-processed image through a preset first data enhancement method and a preset second data enhancement method, respectively, to obtain a first enhanced image and a second data enhancement method. 2. Enhanced images, wherein the to-be-processed images include images without category annotations and images with category annotations.

In the embodiments of the present application, the geometric transformation data enhancement methods include methods such as flipping, random rotation, random cropping, and deformation scaling. The color transformation data enhancement method includes adding noise, blurring, color transformation, image filling, and the like.

The parameter construction module 102 is configured to input the first enhanced image into a pre-built first image processing network to obtain a first processed image, and input the second enhanced image into a pre-built second image processing network to obtain The second processing image is used to construct the probability constraint parameter of the probability graph model by using the first processing image and the second processing image.

The label generation module 103 is configured to calculate the unclassified labeled image through a preset semi-supervised learning method to obtain the pseudo-classified label of the unclassified labeled image, wherein the pseudo-classified label is used to label the unclassified image. Annotate the category of the image.

using the predicted probability to select a target image from the image set;

In detail, the obtaining the first supervised learning model includes:

The loss function construction module 104 is configured to update the probability constraint parameter by using the labeled images with categories, the labeled images without categories, and the pseudo-category labels of the labeled images without categories to obtain the probability graph model.

In the embodiment of the present application, the update loss function is:

The classification model training module 105 is configured to train the probabilistic graphical model by using the to-be-processed image and the class label of the to-be-processed image to obtain an image classification model.

In detail, the described probabilistic graphical model is trained by using the image to be processed and the category label of the image to be processed, and the obtained image classification model includes:

The image classification module 106 is configured to acquire an image to be classified, input the image to be classified into the image classification model for classification, and obtain a class label of the image to be classified.

The image to be classified may be a medical image.

As shown in FIG. 3 , it is a schematic structural diagram of an electronic device implementing the image classification method of the present application.

The electronic device 1 may include a processor 10, a memory 11 and a bus, and may also include a computer program stored in the memory 11 and executable on the processor 10, such as an image classification program 12.

Wherein, the memory 11 includes at least one type of readable storage medium, and the readable storage medium may be volatile or non-volatile. Specifically, the readable storage medium includes a flash memory, a mobile hard disk, a multimedia card, a card-type memory (eg, SD or DX memory, etc.), a magnetic memory, a magnetic disk, an optical disk, and the like. The memory 11 may be an internal storage unit of the electronic device 1 in some embodiments, such as a mobile hard disk of the electronic device 1 . In other embodiments, the memory 11 may also be an external storage device of the electronic device 1, such as a pluggable mobile hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital) equipped on the electronic device 1. , SD) card, flash memory card (Flash Card), etc. Further, the memory 11 may also include both an internal storage unit of the electronic device 1 and an external storage device. The memory 11 can not only be used to store application software installed in the electronic device 1 and various types of data, such as the code of the image classification program 12, etc., but also can be used to temporarily store data that has been output or will be output.

In some embodiments, the processor 10 may be composed of integrated circuits, for example, may be composed of a single packaged integrated circuit, or may be composed of multiple integrated circuits packaged with the same function or different functions, including one or more integrated circuits. Central Processing Unit (CPU), microprocessor, digital processing chip, graphics processor and combination of various control chips, etc. The processor 10 is the control core (Control Unit) of the electronic device, and uses various interfaces and lines to connect the various components of the entire electronic device, by running or executing the program or module (for example, executing the program) stored in the memory 11. image classification program, etc.), and call data stored in the memory 11 to perform various functions of the electronic device 1 and process data.

The bus may be a peripheral component interconnect (PCI for short) bus or an extended industry standard architecture (Extended industry standard architecture, EISA for short) bus or the like. The bus can be divided into address bus, data bus, control bus and so on. The bus is configured to implement connection communication between the memory 11 and at least one processor 10 and the like.

FIG. 3 only shows an electronic device with components. Those skilled in the art can understand that the structure shown in FIG. 3 does not constitute a limitation on the electronic device 1, and may include fewer or more components than those shown in the figure. components, or a combination of certain components, or a different arrangement of components.

For example, although not shown, the electronic device 1 may also include a power supply (such as a battery) for powering the various components, preferably, the power supply may be logically connected to the at least one processor 10 through a power management device, so that the power management The device implements functions such as charge management, discharge management, and power consumption management. The power source may also include one or more DC or AC power sources, recharging devices, power failure detection circuits, power converters or inverters, power status indicators, and any other components. The electronic device 1 may further include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be repeated here.

Further, the electronic device 1 may also include a network interface, optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a Bluetooth interface, etc.), which is usually used in the electronic device 1 Establish a communication connection with other electronic devices.

Optionally, the electronic device 1 may further include a user interface, and the user interface may be a display (Display), an input unit (eg, a keyboard (Keyboard)), optionally, the user interface may also be a standard wired interface or a wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode, organic light-emitting diode) touch panel, and the like. The display may also be appropriately called a display screen or a display unit, which is used for displaying information processed in the electronic device 1 and for displaying a visualized user interface.

It should be understood that the embodiments are only used for illustration, and are not limited by this structure in the scope of the patent application.

The image classification program 12 stored in the memory 11 in the electronic device 1 is a combination of multiple computer programs, and when running in the processor 10, it can realize:

In the embodiment of the present application, after the unlabeled image is obtained, the pseudo-label is generated for the unlabeled image through the semi-supervised learning method, and then the unclassified image is used for image classification model training, which improves the data for training the model. At the same time, it can further improve the utilization of data and the accuracy of image classification model training. After the image classification model is obtained, the image to be classified is input into the image classification model to obtain an accurate classification result, thereby achieving the purpose of improving the accuracy of image classification.

Further, if the integrated modules/units of the electronic device 1 are implemented in the form of software functional units and sold or used as independent products, they may be stored in a computer-readable storage medium. The readable storage medium may be volatile or non-volatile. Specifically, the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a U disk, a removable hard disk, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM, Read-only memory) Only Memory).

Further, the computer usable storage medium may mainly include a stored program area and a stored data area, wherein the stored program area may store an operating system, an application program required for at least one function, and the like; using the created data, etc.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus, apparatus and method may be implemented in other manners. For example, the apparatus embodiments described above are only illustrative. For example, the division of the modules is only a logical function division, and there may be other division manners in actual implementation.

The modules described as separate components may or may not be physically separated, and the components shown as modules may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional module in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware, or can be implemented in the form of hardware plus software function modules.

It will be apparent to those skilled in the art that the present application is not limited to the details of the above-described exemplary embodiments, but that the present application can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

Accordingly, the embodiments are to be regarded in all respects as illustrative and not restrictive, and the scope of the application is to be defined by the appended claims rather than the foregoing description, which is therefore intended to fall within the scope of the claims. All changes within the meaning and scope of the equivalents of , are included in this application. Any accompanying reference signs in the claims should not be construed as limiting the involved claims.

The blockchain referred to in this application is a new application mode of computer technologies such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain, essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information to verify its Validity of information (anti-counterfeiting) and generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

Furthermore, it is clear that the word "comprising" does not exclude other units or steps and the singular does not exclude the plural. Several units or means recited in the system claims can also be realized by one unit or means by means of software or hardware. Second-class terms are used to denote names and do not denote any particular order.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application rather than limitations. Although the present application has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present application can be Modifications or equivalent substitutions can be made without departing from the spirit and scope of the technical solutions of the present application.

Claims

An image classification method, wherein the method comprises:

Acquire an image to be processed from an image set, and enhance the image to be processed by a preset first data enhancement method and a preset second data enhancement method, respectively, to obtain a first enhanced image and a second enhanced image, wherein the to-be-processed image is The processed images include unclassified images and classified images;

Inputting the first enhanced image into a pre-built first image processing network to obtain a first processed image, and inputting the second enhanced image into a pre-built second image processing network to obtain a second processed image;

Using the first processed image and the second processed image to construct a probability constraint parameter of a probabilistic graphical model;

The pseudo-category label of the category-free annotated image is obtained by calculating the uncategorized label image by a preset semi-supervised learning method, wherein the pseudo-category label is used to label the category of the category-free annotated image;

The probability constraint parameter is updated by using the labeled images with categories, the labeled images without categories, and the pseudo-category labels of the labeled images without categories to obtain the probability graph model;

The probabilistic graphical model is trained by using the to-be-processed image and the class label of the to-be-processed image to obtain an image classification model;

Obtaining an image to be classified, inputting the image to be classified into the image classification model for classification, and obtaining a class label of the image to be classified.
The image classification method according to claim 1, wherein after obtaining the second processed image, the method further comprises:

determining whether the first processed image is the same as the second processed image;

If the first processed image is the same as the second processed image, perform the operation of randomly acquiring images from the image set again;

If the first processed image is different from the second processed image, the operation of constructing a probability constraint parameter of a probabilistic graphical model by using the first processed image and the second processed image is performed.
The image classification method according to claim 1, wherein generating the pseudo-category label of the uncategorized image by a preset semi-supervised learning method comprises:

obtaining a first supervised learning model, and training the first supervised learning model by using the classified image to obtain a first training supervised model;

Using the first training supervision model to predict the unclassified labeled image to obtain the predicted probability of the unclassified labeled image;

using the predicted probability to select a target image from the image set;

Using the labeled image to train the first supervised learning model according to the target image to obtain a second supervised learning model;

Judging whether the second supervised learning model is the same as the first supervised learning model;

If not equal, use the second supervised learning model to replace the first supervised learning model, and perform the operation of training the first supervised learning model through the labeled images again;

If they are equal, it is determined that the training is completed, and the second supervised learning model is determined to be a training completed model;

Inputting the uncategorized image into the training completed model to obtain a pseudo-category label of the uncategorized image.
The image classification method according to claim 3, wherein said obtaining the first supervised learning model comprises:

receiving an initial training set, the initial training set includes a first labeled sample set and a first unlabeled sample set;

Obtain an initial semi-supervised learning model, and use the first labeled sample set and the first unlabeled sample set to train the initial semi-supervised learning model to obtain a first supervised learning network.
The image classification method as claimed in claim 1, wherein, described using the first processed image and the second processed image to construct a probability constraint parameter of a probabilistic graphical model, comprising:

obtaining the original loss function for constructing the probabilistic graph model;

The first processed image and the second processed image are input into the original loss function, and the function parameters in the original loss function are updated to obtain probability constraint parameters.
The image classification method according to claim 1 , wherein the probability map is obtained by updating the probability constraint parameters by using the labeled images with categories, the labeled images without categories, and the pseudo-category labels of the labeled images without categories. models, including:

obtaining the first quantity of the unclassified annotated images and the second quantity of the classified annotated images in the to-be-processed image;

Based on the category annotation of the to-be-processed image, a vector consisting of an image with category annotation and a vector consisting of an image without category annotation are obtained;

Obtain the original loss function of the probabilistic graphical model and use the first number of the unclassified annotated images, the second number of the classified annotated images, the vector composed of the classified classified images, and the unclassified annotated images The formed vector updates the probability constraint parameters in the original loss function to obtain the probability graph model.
The image classification method according to any one of claims 1 to 3, wherein, before the obtaining the probability graphical model, the method further comprises obtaining an update loss function based on the updating the probability constraint parameter , the update loss function is:

Among them, f L represents the vector composed of the labeled images with class, f U represents the vector composed of the labeled images without class, n U represents the number of labeled images without class, n L represents the number of labeled images with class Quantity, y represents a vector composed of the category labels of the first processed image and the second processed image, CE represents the cross-entropy error function, MSE represents the mean square error function, and λ is the coefficient function;

The probabilistic graphical model is obtained according to the update loss function.
An image classification device, wherein the device comprises:

The image enhancement module is used to obtain the image to be processed from the image set, and enhance the image to be processed by the preset first data enhancement method and the preset second data enhancement method respectively, so as to obtain the first enhanced image and the second enhanced image , wherein the to-be-processed image includes an unclassified image and a class-annotated image;

A parameter building module for inputting the first enhanced image into a pre-built first image processing network to obtain a first processed image, and inputting the second enhanced image into a pre-built second image processing network to obtain a second processed image an image, using the first processed image and the second processed image to construct a probability constraint parameter of a probabilistic graphical model;

The label generation module is used to calculate the unclassified labeled image through a preset semi-supervised learning method to obtain the pseudo-classified label of the unclassified labeled image, wherein the pseudo-classified label is used to label the unclassified labeled image. category;

a loss function building module, used for updating the probability constraint parameters through the labeled images with categories, the labeled images without categories, and the pseudo-category labels of the labeled images without categories, to obtain the probability graph model;

A classification model training module, configured to train the probabilistic graphical model by using the to-be-processed image and the class label of the to-be-processed image to obtain an image classification model;

The image classification module is configured to acquire the images to be classified, input the images to be classified into the image classification model for classification, and obtain the class labels of the images to be classified.
An electronic device, wherein the electronic device comprises:

at least one processor; and,

a memory communicatively coupled to the at least one processor; wherein,

The memory stores computer program instructions executable by the at least one processor, the computer program instructions being executed by the at least one processor to enable the at least one processor to perform the image classification method described below :

Acquire an image to be processed from an image set, and enhance the image to be processed by a preset first data enhancement method and a preset second data enhancement method, respectively, to obtain a first enhanced image and a second enhanced image, wherein the to-be-processed image is The processed images include unclassified images and classified images;

Inputting the first enhanced image into a pre-built first image processing network to obtain a first processed image, and inputting the second enhanced image into a pre-built second image processing network to obtain a second processed image;

Using the first processed image and the second processed image to construct a probability constraint parameter of a probabilistic graphical model;

The pseudo-category label of the category-free annotated image is obtained by calculating the uncategorized label image by a preset semi-supervised learning method, wherein the pseudo-category label is used to label the category of the category-free annotated image;

The probability constraint parameter is updated by using the labeled images with categories, the labeled images without categories, and the pseudo-category labels of the labeled images without categories to obtain the probability graph model;

The probabilistic graphical model is trained by using the to-be-processed image and the class label of the to-be-processed image to obtain an image classification model;

Obtaining an image to be classified, inputting the image to be classified into the image classification model for classification, and obtaining a class label of the image to be classified.
The electronic device according to claim 9, wherein after obtaining the second processed image, the method further comprises:

determining whether the first processed image is the same as the second processed image;

If the first processed image is the same as the second processed image, perform the operation of randomly acquiring images from the image set again;

If the first processed image is different from the second processed image, the operation of constructing a probability constraint parameter of a probabilistic graphical model by using the first processed image and the second processed image is performed.
The electronic device according to claim 9, wherein generating the pseudo-category label of the uncategorized image by a preset semi-supervised learning method comprises:

obtaining a first supervised learning model, and training the first supervised learning model by using the classified image to obtain a first training supervised model;

Using the first training supervision model to predict the unclassified labeled image to obtain the predicted probability of the unclassified labeled image;

using the predicted probability to select a target image from the image set;

Using the labeled image to train the first supervised learning model according to the target image to obtain a second supervised learning model;

Judging whether the second supervised learning model is the same as the first supervised learning model;

If not equal, use the second supervised learning model to replace the first supervised learning model, and perform the operation of training the first supervised learning model through the labeled images again;

If they are equal, it is determined that the training is completed, and the second supervised learning model is determined to be a training completed model;

Inputting the uncategorized image into the training completed model to obtain a pseudo-category label of the uncategorized image.
The electronic device as claimed in claim 9, wherein, the probability constraint parameter of constructing a probability graph model by using the first processed image and the second processed image comprises:

obtaining the original loss function for constructing the probabilistic graph model;

The first processed image and the second processed image are input into the original loss function, and the function parameters in the original loss function are updated to obtain probability constraint parameters.
The electronic device according to claim 9, wherein the probabilistic graphical model is obtained by updating the probability constraint parameters by using the labeled images with categories, the labeled images without categories, and the pseudo-category labels of the labeled images without categories. ,include:

obtaining the first quantity of the unclassified annotated images and the second quantity of the classified images in the to-be-processed image;

Based on the category annotation of the to-be-processed image, a vector consisting of an image with category annotation and a vector consisting of an image without category annotation are obtained;

Obtain the original loss function of the probabilistic graphical model and use the first number of the unclassified annotated images, the second number of the classified annotated images, the vector composed of the classified classified images, and the unclassified annotated images The formed vector updates the probability constraint parameters in the original loss function to obtain the probability graph model.
The electronic device according to any one of claims 9 to 11, wherein, before the obtaining the probability graphical model, the method further comprises obtaining an update loss function based on the updating the probability constraint parameter, The update loss function is:

Among them, f L represents the vector composed of the labeled images with class, f U represents the vector composed of the labeled images without class, n U represents the number of labeled images without class, n L represents the number of labeled images with class Quantity, y represents a vector composed of the category labels of the first processed image and the second processed image, CE represents the cross-entropy error function, MSE represents the mean square error function, and λ is the coefficient function;

The probabilistic graphical model is obtained according to the update loss function.
A computer-readable storage medium, comprising a storage data area and a storage program area, the storage data area stores created data, and the storage program area stores a computer program; wherein, when the computer program is executed by a processor, an image as described below is realized Classification:

Acquire an image to be processed from an image set, and enhance the image to be processed by a preset first data enhancement method and a preset second data enhancement method respectively, to obtain a first enhanced image and a second enhanced image, wherein the to-be-processed image is The processed images include unclassified images and classified images;

Inputting the first enhanced image into a pre-built first image processing network to obtain a first processed image, and inputting the second enhanced image into a pre-built second image processing network to obtain a second processed image;

Using the first processed image and the second processed image to construct a probability constraint parameter of a probabilistic graphical model;

The pseudo-category label of the category-free annotated image is obtained by calculating the uncategorized label image by a preset semi-supervised learning method, wherein the pseudo-category label is used to label the category of the category-free annotated image;

The probability constraint parameter is updated through the labeled images with categories, the labeled images without categories, and the pseudo-category labels of the labeled images without categories to obtain the probability graph model;

The probabilistic graphical model is trained by using the to-be-processed image and the class label of the to-be-processed image to obtain an image classification model;

Obtaining an image to be classified, inputting the image to be classified into the image classification model for classification, and obtaining a class label of the image to be classified.
The computer-readable storage medium of claim 15, wherein after the obtaining the second processed image, the method further comprises:

determining whether the first processed image is the same as the second processed image;

If the first processed image is the same as the second processed image, perform the operation of randomly acquiring images from the image set again;

If the first processed image is different from the second processed image, the operation of constructing a probability constraint parameter of a probabilistic graphical model by using the first processed image and the second processed image is performed.
The computer-readable storage medium of claim 15, wherein the generating the pseudo-category label of the uncategorized image by a preset semi-supervised learning method comprises:

obtaining a first supervised learning model, and training the first supervised learning model by using the classified image to obtain a first training supervised model;

Using the first training supervision model to predict the unclassified labeled image to obtain the predicted probability of the unclassified labeled image;

using the predicted probability to select a target image from the image set;

Using the labeled image to train the first supervised learning model according to the target image to obtain a second supervised learning model;

Judging whether the second supervised learning model is the same as the first supervised learning model;

If not equal, use the second supervised learning model to replace the first supervised learning model, and perform the operation of training the first supervised learning model through the labeled images again;

If they are equal, it is determined that the training is completed, and the second supervised learning model is determined to be a training completed model;

Inputting the uncategorized image into the training completed model to obtain a pseudo-category label of the uncategorized image.
The computer-readable storage medium of claim 15, wherein the probability constraint parameters for constructing a probabilistic graphical model using the first processed image and the second processed image comprise:

obtaining the original loss function for constructing the probabilistic graph model;

The first processed image and the second processed image are input into the original loss function, and the function parameters in the original loss function are updated to obtain probability constraint parameters.
The computer-readable storage medium according to claim 15, wherein the probability constraint parameter is updated by using a class-labeled image, a class-free label image, and a pseudo-class label of the class-free label image to obtain the Probabilistic graphical models, including:

obtaining the first quantity of the unclassified annotated images and the second quantity of the classified annotated images in the to-be-processed image;

Based on the category annotation of the to-be-processed image, a vector consisting of an image with category annotation and a vector consisting of an image without category annotation are obtained;

Obtain the original loss function of the probabilistic graphical model and use the first number of the unclassified annotated images, the second number of the classified classified images, the vector of the classified classified images, and the unclassified labeled images The formed vector updates the probability constraint parameters in the original loss function to obtain the probability graph model.
The computer-readable storage medium according to any one of claims 15 to 17, wherein, before the obtaining the probabilistic graphical model, the method further comprises, based on the updating the probability constraint parameter, obtaining an updated Loss function, the update loss function is:

Among them, f L represents the vector composed of the labeled images with class, f U represents the vector composed of the labeled images without class, n U represents the number of labeled images without class, n L represents the number of labeled images with class Quantity, y represents a vector composed of the category labels of the first processed image and the second processed image, CE represents the cross-entropy error function, MSE represents the mean square error function, and λ is the coefficient function;

The probabilistic graphical model is obtained according to the update loss function.