CN116740452A - Image classification method, system and storage medium based on image restoration - Google Patents
Image classification method, system and storage medium based on image restoration Download PDFInfo
- Publication number
- CN116740452A CN116740452A CN202310728926.1A CN202310728926A CN116740452A CN 116740452 A CN116740452 A CN 116740452A CN 202310728926 A CN202310728926 A CN 202310728926A CN 116740452 A CN116740452 A CN 116740452A
- Authority
- CN
- China
- Prior art keywords
- image
- training
- restoration
- module
- unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000000034 method Methods 0.000 title claims abstract description 36
- 238000013136 deep learning model Methods 0.000 claims abstract description 52
- 238000013145 classification model Methods 0.000 claims abstract description 18
- 238000005457 optimization Methods 0.000 claims description 12
- 238000001514 detection method Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 7
- 238000013135 deep learning Methods 0.000 description 3
- 230000009286 beneficial effect Effects 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 206010035148 Plague Diseases 0.000 description 1
- 241000607479 Yersinia pestis Species 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 239000011159 matrix material Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/34—Smoothing or thinning of the pattern; Morphological operations; Skeletonisation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/94—Hardware or software architectures specially adapted for image or video understanding
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Abstract
The invention discloses an image classification method, an image classification system and a storage medium based on image restoration, which comprise the following steps: training a deep learning model comprising an image feature extractor, an image restoration module and an image classification module based on a plurality of training images to obtain a trained deep learning model; the image feature extractor is respectively connected with the image restoration module and the image classification module; deleting the image restoration module in the trained deep learning model to obtain a target image classification model; and inputting the image to be detected into the target image classification model to obtain an image classification result of the image to be detected. The invention solves the problem of wrong image classification caused by image shielding and improves the accuracy of image classification.
Description
Technical Field
The invention relates to the technical field of deep learning, in particular to an image classification method, an image classification system and a storage medium based on image restoration.
Background
In recent years, the deep learning method has achieved remarkable achievements in the field of image classification, and has been widely used in the industry. However, the problem of image occlusion always plagues the existing method, and when a target is occluded in a large scale, the image classification method based on deep learning often has too few input effective features due to occlusion, causes classification errors, and limits the application of the method in certain scenes.
Accordingly, there is a need to provide a solution to the above-mentioned problems.
Disclosure of Invention
In order to solve the technical problems, the invention provides an image classification method, an image classification system and a storage medium based on image restoration.
The technical scheme of the image classification method based on image restoration is as follows:
s1, training a deep learning model comprising an image feature extractor, an image restoration module and an image classification module based on a plurality of training images to obtain a trained deep learning model; the image feature extractor is respectively connected with the image restoration module and the image classification module;
s2, deleting the image restoration module in the trained deep learning model to obtain a target image classification model;
s3, inputting the image to be detected into the target image classification model to obtain an image classification result of the image to be detected.
The image classification method based on image restoration has the following beneficial effects:
the method solves the problem of image classification errors caused by image shielding, and improves the accuracy of image classification.
Based on the scheme, the image classification method based on image restoration can be improved as follows.
Further, step S1 includes:
s11, removing part of images from each training image respectively to obtain a plurality of incomplete images;
s12, inputting a incomplete image corresponding to any training image to the image feature extractor to obtain a first feature image, inputting the first feature image to the image restoration module to obtain a restoration image of any training image, and obtaining a first loss of any training image according to the any training image and the restoration image of any image;
s13, carrying out parameter optimization on the image feature extractor and the image restoration module based on the first loss of any training image to obtain a first optimized image feature extractor and a first image restoration module;
s14, inputting any training image to the first optimized image feature extractor to obtain a second feature image, inputting the second feature image to the image classification module to obtain a training prediction type of any training image, and obtaining a second loss of any training image according to the training prediction type and the real label type of any training image;
s15, carrying out parameter optimization on the first optimized image feature extractor and the image classification module based on the second loss of any training image to obtain a second optimized image feature extractor and a first image classification module;
s16, taking the second optimized image feature extractor as the optimized image feature extractor, the first image restoration module as the image restoration module and the first image classification module as the image classification module;
and S17, repeatedly executing S12-S16 until all training images finish one iteration training on the deep learning model, returning to execute S11 until the deep learning model reaches the maximum iteration number, and determining the deep learning model as the trained deep learning model.
Further, step S11 includes:
s111, cutting any training image to obtain a plurality of image blocks of any training image;
s112, randomly selecting a plurality of image blocks from all the image blocks of any training image as partial images of the any training image and discarding the partial images to obtain a malformed image corresponding to the any training image;
s113, repeatedly executing S111-S112 until the incomplete image corresponding to each training image is obtained.
Further, the image restoration module includes: the first full-connection layer, the second full-connection layer and the resize layer are sequentially connected; inputting a first feature map of any training image to the image restoration module to obtain a restoration image of any training image, wherein the step comprises the following steps:
and inputting the first feature map of any training image to the first full-connection layer, processing the first full-connection layer and the second full-connection layer to obtain an intermediate feature map of any training image, and inputting the intermediate feature map of any training image to the resize layer for processing to obtain a repair image of any training image.
Further, the image feature extractor is: a transducer network.
The technical scheme of the image classification system based on image restoration is as follows:
comprising the following steps: the device comprises a training unit, a processing unit and a detection unit;
the training unit is used for: training a deep learning model comprising an image feature extractor, an image restoration module and an image classification module based on a plurality of training images to obtain a trained deep learning model; the image feature extractor is respectively connected with the image restoration module and the image classification module;
the processing unit is used for: deleting the image restoration module in the trained deep learning model to obtain a target image classification model;
the detection unit is used for: and inputting the image to be detected into the target image classification model to obtain an image classification result of the image to be detected.
The image classification system based on image restoration has the following beneficial effects:
the system solves the problem of image classification errors caused by image shielding, and improves the accuracy of image classification.
Based on the scheme, the image classification system based on image restoration can be improved as follows.
Further, the training unit includes: the training device comprises a first training unit, a second training unit, a third training unit, a fourth training unit, a fifth training unit, a sixth training unit and an iterative training unit;
the first training unit is used for: removing part of images from each training image respectively to obtain a plurality of incomplete images;
the second training unit is used for: inputting a incomplete image corresponding to any training image to the image feature extractor to obtain a first feature image, inputting the first feature image to the image restoration module to obtain a restoration image of the any training image, and obtaining a first loss of the any training image according to the any training image and the restoration image of the any image;
the third training unit is used for: based on the first loss of any training image, performing parameter optimization on the image feature extractor and the image restoration module to obtain a first optimized image feature extractor and a first image restoration module;
the fourth training unit is used for: inputting any training image to the first optimized image feature extractor to obtain a second feature image, inputting the second feature image to the image classification module to obtain a training prediction category of any training image, and obtaining a second loss of any training image according to the training prediction category and the real label category of any training image;
the fifth training unit is used for: based on the second loss of any training image, performing parameter optimization on the first optimized image feature extractor and the image classification module to obtain a second optimized image feature extractor and a first image classification module;
the sixth training unit is configured to: taking the second optimized image feature extractor as the optimized image feature extractor, the first image restoration module as the image restoration module and the first image classification module as the image classification module;
the seventh training unit is configured to: and repeatedly calling the second training unit to the sixth training unit until all training images finish iterative training on the deep learning model, returning to execute the first training unit until the deep learning model reaches the maximum iterative times, and determining the deep learning model as the trained deep learning model.
Further, the first training unit includes: the device comprises an image cutting unit, an image processing unit and an iteration processing unit;
the image cutting unit is used for: cutting any training image to obtain a plurality of image blocks of any training image;
the image processing unit is used for: randomly selecting a plurality of image blocks from all the image blocks of any training image as partial images of the any training image and discarding the partial images to obtain incomplete images corresponding to the any training image;
the iteration processing unit is used for: and repeatedly calling the image cutting unit and the image processing unit until the incomplete image corresponding to each training image is obtained.
Further, the image restoration module includes: the first full-connection layer, the second full-connection layer and the resize layer are sequentially connected; the second training unit is specifically configured to:
and inputting the first feature map of any training image to the first full-connection layer, processing the first full-connection layer and the second full-connection layer to obtain an intermediate feature map of any training image, and inputting the intermediate feature map of any training image to the resize layer for processing to obtain a repair image of any training image.
The technical scheme of the storage medium is as follows:
the storage medium has instructions stored therein which, when read by a computer, cause the computer to perform the steps of an image classification method based on image restoration as in the present invention.
Drawings
FIG. 1 is a schematic flow chart of an embodiment of an image classification method based on image restoration according to the present invention;
FIG. 2 is a schematic structural diagram of a deep learning model in an embodiment of an image classification method based on image restoration according to the present invention;
FIG. 3 is a schematic structural diagram of a target classification model in an embodiment of an image classification method based on image restoration according to the present invention;
fig. 4 is a schematic flow chart of step S11 in an embodiment of an image classification method based on image restoration provided by the present invention;
fig. 5 is a schematic flow chart of step S111 in an embodiment of an image classification method based on image restoration provided by the present invention;
fig. 6 is a schematic structural diagram of a transducer network in an embodiment of an image classification method based on image restoration according to the present invention;
FIG. 7 is a schematic diagram of an image restoration task in an embodiment of an image classification method based on image restoration according to the present invention;
FIG. 8 is a schematic diagram of an image classification task in an embodiment of an image classification method based on image restoration according to the present invention;
fig. 9 is a schematic structural diagram of an embodiment of an image classification system based on image restoration according to the present invention.
Detailed Description
Fig. 1 is a schematic flow chart of an embodiment of an image classification method based on image restoration. As shown in fig. 1, the method comprises the following steps:
s1, training a deep learning model comprising an image feature extractor, an image restoration module and an image classification module based on a plurality of training images to obtain a trained deep learning model.
Wherein, (1) the training image is: an arbitrarily selected image. (2) As shown in fig. 2, the deep learning model includes: the image feature extractor is respectively connected with the image restoration module and the image classification module.
S2, deleting the image restoration module in the trained deep learning model to obtain a target image classification model.
Wherein, as shown in fig. 3, the target image classification model includes: a trained image feature extractor and an image classification module.
S3, inputting the image to be detected into the target image classification model to obtain an image classification result of the image to be detected.
Wherein, the image to be measured is: arbitrarily selected images to be subjected to image classification.
It should be noted that, since the image restoration module in the trained deep learning model is deleted, the parameter and the operand of the target image classification model are not different from those of the normal image classification model, and no additional overhead is generated during actual use.
Preferably, as shown in fig. 4, step S1 includes:
s11, removing part of images from each training image respectively to obtain a plurality of incomplete images.
Specifically, as shown in fig. 5, step S11 includes:
s111, cutting any training image to obtain a plurality of image blocks of any training image.
Specifically, any training image is subjected to image cutting, and a plurality of image blocks with the size of 16×16 are obtained.
S112, randomly selecting a plurality of image blocks from all the image blocks of any training image as partial images of any training image, and discarding the partial images to obtain a malformed image corresponding to any training image.
Specifically, a plurality of image blocks are randomly selected from all image blocks of any training image according to a preset proportion, and the selected image blocks are used as partial images of the training image and discarded to obtain a malformed image of the training image.
It should be noted that (1) the default ratio is set to 50%, and assuming that the number of image blocks is 20, 10 image blocks are randomly selected as partial images and discarded. (2) And the existing image blocks are subjected to position coding in the incomplete image, so that the training of a subsequent model is facilitated.
S113, repeatedly executing S111-S112 until the incomplete image corresponding to each training image is obtained.
Specifically, for each training image, the steps S111-S112 are performed separately until a corresponding incomplete image for each training image is obtained.
S12, inputting a incomplete image corresponding to any training image to the image feature extractor, obtaining a first feature image, inputting the first feature image to the image restoration module, obtaining a restoration image of any training image, and obtaining a first loss of any training image according to any training image and the restoration image of any image.
Wherein (1) the image feature extractor is: the specific structure of the transducer network is shown in fig. 6. (2) The first feature map is: and inputting the incomplete image into the image feature extractor to perform feature extraction to obtain a feature map. (3) The first loss is: l2 Loss between any training image and the repair image of said any image.
As shown in fig. 7, the image classification module is frozen and does not participate in training when performing the image restoration task.
And S13, carrying out parameter optimization on the image feature extractor and the image restoration module based on the first loss of any training image to obtain a first optimized image feature extractor and a first image restoration module.
The process of optimizing the model parameters according to the loss is the prior art, and is not repeated here.
S14, inputting any training image to the first optimized image feature extractor to obtain a second feature image, inputting the second feature image to the image classification module to obtain a training prediction type of any training image, and obtaining a second loss of any training image according to the training prediction type and the real label type of any training image.
Wherein, (1) the second feature map is: the training image is input into a first optimized image feature extractor for feature extraction to obtain a feature map. (2) The second loss is: CE Loss between training predicted class and real label class of training image.
As shown in fig. 8, the image restoration module is frozen and does not participate in training when the image classification task is performed.
And S15, carrying out parameter optimization on the first optimized image feature extractor and the image classification module based on the second loss of any training image to obtain a second optimized image feature extractor and the first image classification module.
The process of optimizing the model parameters according to the loss is the prior art, and is not repeated here.
S16, taking the second optimized image feature extractor as the optimized image feature extractor, the first image restoration module as the image restoration module and the first image classification module as the image classification module.
Specifically, when one training image is respectively input into the image restoration module and the image classification module to complete one training, the second optimized image feature extractor is used as the optimized image feature extractor, the first image restoration module is used as the image restoration module, and the first image classification module is used as the image classification module, so that the next training image can be trained on the basis of the existing model parameters.
And S17, repeatedly executing S12-S16 until all training images finish one iteration training on the deep learning model, returning to execute S11 until the deep learning model reaches the maximum iteration number, and determining the deep learning model as the trained deep learning model.
The maximum iteration number is set according to the actual requirement, and is not limited herein.
Specifically, when each training image is executed to complete the steps of S12-S16, it is indicated that all training images complete one iteration training for the deep learning model, and at this time, the step S11 is returned to execute the step of generating the incomplete image again and training is performed until the deep learning model reaches the maximum iteration number, and the deep learning model is determined as a trained deep learning model.
Preferably, the image restoration module includes: the first full-connection layer, the second full-connection layer and the resize layer are sequentially connected.
Inputting a first feature map of any training image to the image restoration module to obtain a restoration image of any training image, wherein the step comprises the following steps:
and inputting the first feature map of any training image to the first full-connection layer, processing the first full-connection layer and the second full-connection layer to obtain an intermediate feature map of any training image, and inputting the intermediate feature map of any training image to the resize layer for processing to obtain a repair image of any training image.
It should be noted that the functions and roles of the full connection layer and the resize layer are the prior art. For example, the input size of the training image is 3×384×384, where 3 represents RGB three channels, i.e., the training image contains 442368 values. The image restoration module sequentially passes through the first full-connection layer and the second full-connection layer to obtain 442368 outputs, and inputs the 442368 outputs to the resize layer for processing to obtain a matrix (restoration image) of 3×384×384.
The technical scheme of the embodiment solves the problem of image classification errors caused by image shielding, and improves the accuracy of image classification.
Fig. 9 is a schematic structural diagram of an embodiment of an image classification system based on image restoration according to the present invention. As shown in fig. 9, the system 200 includes: training unit 210, processing unit 220, and detection unit 230.
The training unit 210 is configured to: training a deep learning model comprising an image feature extractor, an image restoration module and an image classification module based on a plurality of training images to obtain a trained deep learning model; the image feature extractor is respectively connected with the image restoration module and the image classification module;
the processing unit 220 is configured to: deleting the image restoration module in the trained deep learning model to obtain a target image classification model;
the detection unit 230 is configured to: and inputting the image to be detected into the target image classification model to obtain an image classification result of the image to be detected.
Preferably, the training unit 210 includes: a first training unit 211, a second training unit 212, a third training unit 213, a fourth training unit 214, a fifth training unit 215, a sixth training unit 216, and an iterative training unit 217;
the first training unit 211 is configured to: removing part of images from each training image respectively to obtain a plurality of incomplete images;
the second training unit 212 is configured to: inputting a incomplete image corresponding to any training image to the image feature extractor to obtain a first feature image, inputting the first feature image to the image restoration module to obtain a restoration image of the any training image, and obtaining a first loss of the any training image according to the any training image and the restoration image of the any image;
the third training unit 213 is configured to: based on the first loss of any training image, performing parameter optimization on the image feature extractor and the image restoration module to obtain a first optimized image feature extractor and a first image restoration module;
the fourth training unit 214 is configured to: inputting any training image to the first optimized image feature extractor to obtain a second feature image, inputting the second feature image to the image classification module to obtain a training prediction category of any training image, and obtaining a second loss of any training image according to the training prediction category and the real label category of any training image;
the fifth training unit 215 is configured to: based on the second loss of any training image, performing parameter optimization on the first optimized image feature extractor and the image classification module to obtain a second optimized image feature extractor and a first image classification module;
the sixth training unit 216 is configured to: taking the second optimized image feature extractor as the optimized image feature extractor, the first image restoration module as the image restoration module and the first image classification module as the image classification module;
the seventh training unit 217 is configured to: and repeatedly calling the second training unit 212 to the sixth training unit 216 until all training images finish iterative training on the deep learning model, returning to execute the first training unit 211 until the deep learning model reaches the maximum iterative number, and determining the deep learning model as the trained deep learning model.
Preferably, the first training unit 211 includes: an image cutting unit 2111, an image processing unit 2112, and an iterative processing unit 2113.
The image cutting unit is used for: cutting any training image to obtain a plurality of image blocks of any training image;
the image processing unit is used for: randomly selecting a plurality of image blocks from all the image blocks of any training image as partial images of the any training image and discarding the partial images to obtain incomplete images corresponding to the any training image;
the iteration processing unit is used for: and repeatedly calling the image cutting unit and the image processing unit until the incomplete image corresponding to each training image is obtained.
Preferably, the image restoration module includes: the first full-connection layer, the second full-connection layer and the resize layer are sequentially connected; the second training unit is specifically configured to:
and inputting the first feature map of any training image to the first full-connection layer, processing the first full-connection layer and the second full-connection layer to obtain an intermediate feature map of any training image, and inputting the intermediate feature map of any training image to the resize layer for processing to obtain a repair image of any training image.
The technical scheme of the embodiment solves the problem of image classification errors caused by image shielding, and improves the accuracy of image classification.
The above steps for implementing corresponding functions by using the parameters and the modules in the embodiment of the image classification system 200 based on image restoration according to the present invention may refer to the parameters and the steps in the embodiment of the image classification method based on image restoration provided above, which are not described herein.
The storage medium provided by the embodiment of the invention comprises: the storage medium stores instructions that, when read by a computer, cause the computer to perform steps of an image classification method based on image restoration, for example, reference may be made to the parameters and steps provided in the embodiments of an image classification method based on image restoration, which are not described herein.
Computer storage media such as: flash disk, mobile hard disk, etc.
Those skilled in the art will appreciate that the present invention may be implemented as a method, system, and storage medium.
Thus, the invention may be embodied in the form of: either entirely hardware, entirely software (including firmware, resident software, micro-code, etc.), or entirely software, or a combination of hardware and software, referred to herein generally as a "circuit," module "or" system. Furthermore, in some embodiments, the invention may also be embodied in the form of a computer program product in one or more computer-readable media, which contain computer-readable program code. Any combination of one or more computer readable media may be employed. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the computer-readable storage medium would include the following: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. While embodiments of the present invention have been shown and described above, it will be understood that the above embodiments are illustrative and not to be construed as limiting the invention, and that variations, modifications, alternatives and variations may be made to the above embodiments by one of ordinary skill in the art within the scope of the invention.
Claims (10)
1. An image classification method based on image restoration, comprising the steps of:
s1, training a deep learning model comprising an image feature extractor, an image restoration module and an image classification module based on a plurality of training images to obtain a trained deep learning model; the image feature extractor is respectively connected with the image restoration module and the image classification module;
s2, deleting the image restoration module in the trained deep learning model to obtain a target image classification model;
s3, inputting the image to be detected into the target image classification model to obtain an image classification result of the image to be detected.
2. The image restoration-based image classification method according to claim 1, wherein step S1 includes:
s11, removing part of images from each training image respectively to obtain a plurality of incomplete images;
s12, inputting a incomplete image corresponding to any training image to the image feature extractor to obtain a first feature image, inputting the first feature image to the image restoration module to obtain a restoration image of any training image, and obtaining a first loss of any training image according to the any training image and the restoration image of any image;
s13, carrying out parameter optimization on the image feature extractor and the image restoration module based on the first loss of any training image to obtain a first optimized image feature extractor and a first image restoration module;
s14, inputting any training image to the first optimized image feature extractor to obtain a second feature image, inputting the second feature image to the image classification module to obtain a training prediction type of any training image, and obtaining a second loss of any training image according to the training prediction type and the real label type of any training image;
s15, carrying out parameter optimization on the first optimized image feature extractor and the image classification module based on the second loss of any training image to obtain a second optimized image feature extractor and a first image classification module;
s16, taking the second optimized image feature extractor as the optimized image feature extractor, the first image restoration module as the image restoration module and the first image classification module as the image classification module;
and S17, repeatedly executing S12-S16 until all training images finish one iteration training on the deep learning model, returning to execute S11 until the deep learning model reaches the maximum iteration number, and determining the deep learning model as the trained deep learning model.
3. The image restoration-based image classification method according to claim 2, wherein step S11 includes:
s111, cutting any training image to obtain a plurality of image blocks of any training image;
s112, randomly selecting a plurality of image blocks from all the image blocks of any training image as partial images of the any training image and discarding the partial images to obtain a malformed image corresponding to the any training image;
s113, repeatedly executing S111-S112 until the incomplete image corresponding to each training image is obtained.
4. The image restoration-based image classification method according to claim 2, wherein the image restoration module includes: the first full-connection layer, the second full-connection layer and the resize layer are sequentially connected; inputting a first feature map of any training image to the image restoration module to obtain a restoration image of any training image, wherein the step comprises the following steps:
and inputting the first feature map of any training image to the first full-connection layer, processing the first full-connection layer and the second full-connection layer to obtain an intermediate feature map of any training image, and inputting the intermediate feature map of any training image to the resize layer for processing to obtain a repair image of any training image.
5. The image restoration based image classification method according to any one of claims 1-4, wherein said image feature extractor is: a transducer network.
6. An image classification system based on image restoration, comprising: the device comprises a training unit, a processing unit and a detection unit;
the training unit is used for: training a deep learning model comprising an image feature extractor, an image restoration module and an image classification module based on a plurality of training images to obtain a trained deep learning model; the image feature extractor is respectively connected with the image restoration module and the image classification module;
the processing unit is used for: deleting the image restoration module in the trained deep learning model to obtain a target image classification model;
the detection unit is used for: and inputting the image to be detected into the target image classification model to obtain an image classification result of the image to be detected.
7. The image restoration based image classification system as recited in claim 6, wherein said training unit comprises: the training device comprises a first training unit, a second training unit, a third training unit, a fourth training unit, a fifth training unit, a sixth training unit and an iterative training unit;
the first training unit is used for: removing part of images from each training image respectively to obtain a plurality of incomplete images;
the second training unit is used for: inputting a incomplete image corresponding to any training image to the image feature extractor to obtain a first feature image, inputting the first feature image to the image restoration module to obtain a restoration image of the any training image, and obtaining a first loss of the any training image according to the any training image and the restoration image of the any image;
the third training unit is used for: based on the first loss of any training image, performing parameter optimization on the image feature extractor and the image restoration module to obtain a first optimized image feature extractor and a first image restoration module;
the fourth training unit is used for: inputting any training image to the first optimized image feature extractor to obtain a second feature image, inputting the second feature image to the image classification module to obtain a training prediction category of any training image, and obtaining a second loss of any training image according to the training prediction category and the real label category of any training image;
the fifth training unit is used for: based on the second loss of any training image, performing parameter optimization on the first optimized image feature extractor and the image classification module to obtain a second optimized image feature extractor and a first image classification module;
the sixth training unit is configured to: taking the second optimized image feature extractor as the optimized image feature extractor, the first image restoration module as the image restoration module and the first image classification module as the image classification module;
the seventh training unit is configured to: and repeatedly calling the second training unit to the sixth training unit until all training images finish iterative training on the deep learning model, returning to execute the first training unit until the deep learning model reaches the maximum iterative times, and determining the deep learning model as the trained deep learning model.
8. The image restoration-based image classification system as recited in claim 7, wherein said first training unit comprises: the device comprises an image cutting unit, an image processing unit and an iteration processing unit;
the image cutting unit is used for: cutting any training image to obtain a plurality of image blocks of any training image;
the image processing unit is used for: randomly selecting a plurality of image blocks from all the image blocks of any training image as partial images of the any training image and discarding the partial images to obtain incomplete images corresponding to the any training image;
the iteration processing unit is used for: and repeatedly calling the image cutting unit and the image processing unit until the incomplete image corresponding to each training image is obtained.
9. The image restoration-based image classification system as recited in claim 7, wherein said image restoration module comprises: the first full-connection layer, the second full-connection layer and the resize layer are sequentially connected; the second training unit is specifically configured to:
and inputting the first feature map of any training image to the first full-connection layer, processing the first full-connection layer and the second full-connection layer to obtain an intermediate feature map of any training image, and inputting the intermediate feature map of any training image to the resize layer for processing to obtain a repair image of any training image.
10. A storage medium having instructions stored therein, which when read by a computer, cause the computer to perform the image restoration-based image classification method of any of claims 1 to 5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310728926.1A CN116740452B (en) | 2023-06-19 | 2023-06-19 | Image classification method, system and storage medium based on image restoration |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310728926.1A CN116740452B (en) | 2023-06-19 | 2023-06-19 | Image classification method, system and storage medium based on image restoration |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116740452A true CN116740452A (en) | 2023-09-12 |
| CN116740452B CN116740452B (en) | 2023-12-22 |
Family
ID=87902553
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310728926.1A Active CN116740452B (en) | 2023-06-19 | 2023-06-19 | Image classification method, system and storage medium based on image restoration |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116740452B (en) |
Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8204297B1 (en) * | 2009-02-27 | 2012-06-19 | Kla-Tencor Corp. | Methods and systems for classifying defects detected on a reticle |
| CN112464893A (en) * | 2020-12-10 | 2021-03-09 | 山东建筑大学 | Congestion degree classification method in complex environment |
| CN112528764A (en) * | 2020-11-25 | 2021-03-19 | 杭州欣禾圣世科技有限公司 | Facial expression recognition method, system and device and readable storage medium |
| CN112861785A (en) * | 2021-03-09 | 2021-05-28 | 中山大学 | Shielded pedestrian re-identification method based on example segmentation and image restoration |
| WO2022073453A1 (en) * | 2020-10-10 | 2022-04-14 | 武汉大学 | Personal identification-oriented face quality perception method and system |
| CN114663678A (en) * | 2022-04-19 | 2022-06-24 | 杭州电子科技大学 | ECO-GAN-based image enhancement system and method |
| CN115035567A (en) * | 2022-04-28 | 2022-09-09 | 合肥的卢深视科技有限公司 | Model training, incomplete face image recognition and reconstruction method, equipment and medium |
| US20220309642A1 (en) * | 2021-03-29 | 2022-09-29 | Beijing Xiaomi Mobile Software Co., Ltd. | Repair method and repair apparatus for flexible panel, device and storage medium |
| CN116152542A (en) * | 2022-10-10 | 2023-05-23 | 马上消费金融股份有限公司 | Training method, device, equipment and storage medium for image classification model |
-
2023
- 2023-06-19 CN CN202310728926.1A patent/CN116740452B/en active Active
Patent Citations (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8204297B1 (en) * | 2009-02-27 | 2012-06-19 | Kla-Tencor Corp. | Methods and systems for classifying defects detected on a reticle |
| WO2022073453A1 (en) * | 2020-10-10 | 2022-04-14 | 武汉大学 | Personal identification-oriented face quality perception method and system |
| CN112528764A (en) * | 2020-11-25 | 2021-03-19 | 杭州欣禾圣世科技有限公司 | Facial expression recognition method, system and device and readable storage medium |
| CN112464893A (en) * | 2020-12-10 | 2021-03-09 | 山东建筑大学 | Congestion degree classification method in complex environment |
| CN112861785A (en) * | 2021-03-09 | 2021-05-28 | 中山大学 | Shielded pedestrian re-identification method based on example segmentation and image restoration |
| US20220309642A1 (en) * | 2021-03-29 | 2022-09-29 | Beijing Xiaomi Mobile Software Co., Ltd. | Repair method and repair apparatus for flexible panel, device and storage medium |
| CN114663678A (en) * | 2022-04-19 | 2022-06-24 | 杭州电子科技大学 | ECO-GAN-based image enhancement system and method |
| CN115035567A (en) * | 2022-04-28 | 2022-09-09 | 合肥的卢深视科技有限公司 | Model training, incomplete face image recognition and reconstruction method, equipment and medium |
| CN116152542A (en) * | 2022-10-10 | 2023-05-23 | 马上消费金融股份有限公司 | Training method, device, equipment and storage medium for image classification model |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116740452B (en) | 2023-12-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108664981B (en) | Salient image extraction method and device | |
| CN110781960B (en) | Training method, classification method, device and equipment of video classification model | |
| CN109685805B (en) | Image segmentation method and device | |
| CN113516529B (en) | Abnormal order determining method and device, storage medium and electronic equipment | |
| CN110969600A (en) | Product defect detection method and device, electronic equipment and storage medium | |
| CN115100739B (en) | Man-machine behavior detection method, system, terminal device and storage medium | |
| CN115098717A (en) | Three-dimensional model retrieval method and device, electronic equipment and storage medium | |
| CN116740452B (en) | Image classification method, system and storage medium based on image restoration | |
| US11360940B2 (en) | Method and apparatus for biological sequence processing fastq files comprising lossless compression and decompression | |
| CN115587545B (en) | Parameter optimization method, device and equipment for photoresist and storage medium | |
| CN114494782A (en) | Image processing method, model training method, related device and electronic equipment | |
| CN114998649A (en) | Training method of image classification model, and image classification method and device | |
| CN109783745B (en) | Method, device and computer equipment for personalized typesetting of pages | |
| CN112288748A (en) | Semantic segmentation network training and image semantic segmentation method and device | |
| CN116501176B (en) | User action recognition method and system based on artificial intelligence | |
| CN116627445B (en) | Method, device and product for identifying progress of server out-of-band installation of operating system | |
| CN109766255B (en) | Equipment state analysis method and device, storage medium and electronic equipment | |
| CN110297777B (en) | Input method evaluation method and device | |
| CN114692816B (en) | Processing method and equipment of neural network model | |
| CN116912927A (en) | Training method of time sequence motion detection model, time sequence motion detection method and device | |
| CN106295026B (en) | Traffic similarity analysis method and device | |
| CN117151197A (en) | Model compression method, device, equipment and medium | |
| CN114757360A (en) | Data feature analysis method and device and electronic equipment | |
| CN115762488A (en) | Voice recognition method, system, storage medium and electronic equipment | |
| CN114241243A (en) | Training method and device of image classification model, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |