CN113205061A - Garment classification method and classification system based on capsule network - Google Patents
Garment classification method and classification system based on capsule network Download PDFInfo
- Publication number
- CN113205061A CN113205061A CN202110541866.3A CN202110541866A CN113205061A CN 113205061 A CN113205061 A CN 113205061A CN 202110541866 A CN202110541866 A CN 202110541866A CN 113205061 A CN113205061 A CN 113205061A
- Authority
- CN
- China
- Prior art keywords
- image
- clothing
- network
- garment
- features
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 239000002775 capsule Substances 0.000 title claims abstract description 21
- 238000012545 processing Methods 0.000 claims abstract description 18
- 238000010606 normalization Methods 0.000 claims abstract description 16
- 239000013598 vector Substances 0.000 claims abstract description 12
- 230000007246 mechanism Effects 0.000 claims abstract description 11
- 230000009466 transformation Effects 0.000 claims abstract description 11
- 230000004927 fusion Effects 0.000 claims abstract description 8
- 230000001131 transforming effect Effects 0.000 claims abstract description 4
- 238000007781 pre-processing Methods 0.000 claims description 8
- 230000005284 excitation Effects 0.000 claims description 7
- 239000000284 extract Substances 0.000 claims description 5
- 230000036544 posture Effects 0.000 claims description 5
- 230000002452 interceptive effect Effects 0.000 claims description 4
- 230000002708 enhancing effect Effects 0.000 claims description 3
- 238000013507 mapping Methods 0.000 claims description 3
- 230000006835 compression Effects 0.000 claims 3
- 238000007906 compression Methods 0.000 claims 3
- 238000012549 training Methods 0.000 abstract description 6
- 238000010586 diagram Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 230000008859 change Effects 0.000 description 4
- 238000013528 artificial neural network Methods 0.000 description 3
- 238000013527 convolutional neural network Methods 0.000 description 3
- 238000001125 extrusion Methods 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000000717 retained effect Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000012360 testing method Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/10—Terrestrial scenes
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/253—Fusion techniques of extracted features
Abstract
The invention relates to a garment classification method based on a capsule network, which comprises the following steps: carrying out image enhancement and normalization processing on the input clothing picture; performing feature downsampling and feature fusion on the clothing image; amplifying the key vectors and weights of the clothing image features by adopting an attention mechanism, performing convolution and normalization processing on the image features, and transforming the receptive field of the image features by utilizing a space transformation network; inputting the image characteristics into a capsule network, extracting spatial correlation information of the image characteristics, and improving generalization capability; and distinguishing and classifying the clothes according to the image characteristics to obtain a clothes classification result. The invention also discloses a corresponding clothing classification system. The garment classification method is high in accuracy, does not depend on a large number of training samples, still has good classification and identification precision when the garment image is distorted and deformed, and is high in generalization capability.
Description
Technical Field
The invention belongs to the field of computer vision, and particularly relates to a garment classification method and a garment classification system based on a capsule network.
Background
At present, the clothes shopping of people gradually goes into the business platform from off-line, and the intelligent processing of clothes images becomes an essential important link. The e-commerce platform has huge demand on the clothes, and the manual identification of the clothes types cannot meet the timeliness requirement of the e-commerce platform. And the general types of clothes are various nowadays, and the difference between partial clothes types is only a slight difference of fine textures or spatial layout, and is not easy to distinguish by human eyes. Therefore, the way of manually distinguishing the kind of clothes has not been able to satisfy the needs of daily life.
The Chinese patent with publication number CN110210567A, namely 'a clothes image classification and retrieval method and system based on convolutional neural network', and the Chinese patent with application number 201510457010.2, namely 'a clothes classification method based on convolutional neural network', extract and classify image features by adopting convolutional neural network; the method cannot well process the problems of diversity of image samples and change of spatial structures of the samples, so that the detection accuracy of the detected images can drop vertically when large color difference or light-shade change occurs or shielding and the spatial structures are abnormally changed, if defects are overcome, a large number of training samples can be expanded only in training, however, extra calculation cost and collection cost are increased due to expansion of the image samples, models provided by the two methods are large, the models cannot be built on light-weight equipment, and the calculation speed is slow.
The Chinese patent with application number 201810784023.4, a fashionable woman dress image fine-grained classification method based on component detection and visual features, adopts an improved DPM (differential pulse width modulation) model to realize classification of woman dress images, and adopts 4 bottom layer features of HOG (histogram of colors), LBP (local histogram of colors) and edge operators of the images to perform feature characterization.
Disclosure of Invention
The invention aims to solve the problems, and provides a garment classification method and a garment classification system based on a capsule network.
The technical scheme of the invention is a garment classification method based on a capsule network, which comprises the following steps of:
step 1: carrying out image enhancement and normalization processing on the input clothing picture;
step 2: performing feature downsampling and feature fusion on the clothing image;
and step 3: performing characteristic enhancement on the clothing image;
step 3.1: amplifying the key vectors and the weights of the clothing image features by adopting an attention mechanism;
step 3.2: performing convolution and normalization processing on the image characteristics;
step 3.3: transforming the receptive field of the image features by using a spatial transformation network;
and 4, step 4: inputting the image characteristics into a capsule network, extracting spatial correlation information of the image characteristics, and improving generalization capability;
and 5: and distinguishing and classifying the clothes according to the image characteristics to obtain a clothes classification result.
Further, step 1 comprises the following substeps:
step 1.1: rotating, turning over, cutting, scaling, contrast enhancing and brightness adjusting the input clothing image;
step 1.2: and mapping the RGB parameters of the clothing image to a uniform interval.
Preferably, in step 2, the feature downsampling downsamples the image at the large scale 224x224 to the small scale of 25x25 best suited for feature enhancement.
Preferably, step 3.1 uses a squeeze and excitation network to amplify the key vectors and weights of the features of the garment image to screen out interfering non-target features.
The clothing classification system of the clothing classification method comprises the following steps: the image preprocessing module is used for carrying out image enhancement and normalization processing on the input clothing picture; the characteristic preprocessing module is used for carrying out characteristic downsampling and characteristic fusion on the clothing image; the characteristic enhancement module is used for amplifying the key vectors and the weights of the clothing image characteristics by adopting an attention mechanism and converting the receptive field of the image characteristics by utilizing a space transformation network, so that the obtained image characteristics are not limited to a single direction or posture; the capsule network unit extracts spatial correlation information of image features and improves generalization capability; and the distinguishing processing module is used for distinguishing and classifying the clothes according to the image characteristics to obtain the clothes classification result.
Further, the feature enhancement module comprises an extrusion and excitation network, a convolution and normalization module and a spatial transformation network, wherein the extrusion and excitation network amplifies the key vectors and the weights by using an attention mechanism and screens out interfering non-target features, and the spatial transformation network further transforms the receptive field of the image features to enable the obtained features to express a plurality of directions or a plurality of postures.
Compared with the prior art, the invention has the beneficial effects that:
(1) the clothing classification method of the invention enables the network model to achieve strong recognition capability on a small sample data set through the enhancement of the image and the enhancement change in the calculation process, and can deal with various forms of input images when in use without generating recognition errors caused by the changes of image direction, contrast and the like; through the attention mechanism module, image information irrelevant to a target can be removed in the process of training the model, and only useful clothing information is concerned; the schemes can play a role in relatively showing better identification and new energy advantages in actual production and can play a role in classification with high accuracy in an e-commerce platform or in individual use of users.
(2) The garment classification system provided by the invention adopts a capsule network structure, has spatial layout information which is not possessed by a common neural network, can play a good classification effect when the image layout is disordered or distorted, can express the forms of various samples by using only a small number of parameters, and has strong generalization capability.
Drawings
The invention is further illustrated by the following figures and examples.
Fig. 1 is a schematic diagram of a clothing classification method according to an embodiment of the invention.
Fig. 2 is a schematic diagram of a SENet network according to an embodiment of the present invention.
Fig. 3 is a schematic diagram of an STN network according to an embodiment of the present invention.
Detailed Description
The garment classification method based on the capsule network comprises the following steps of:
step 1: carrying out image enhancement and normalization processing on the input clothing picture;
step 1.1: rotating, turning over, cutting, scaling, contrast enhancing and brightness adjusting the input clothing image;
step 1.2: mapping RGB parameters of the clothing image to [0, 1 ];
step 2: performing feature downsampling and feature fusion on the clothing image, and downsampling the image with the large scale of 224x224 to a small scale of 25x25 suitable for feature enhancement;
and step 3: performing characteristic enhancement on the clothing image;
step 3.1: adopting an extrusion and excitation network to amplify the key vectors and the weights of the clothing image features, and screening out the non-target features of interference;
step 3.2: performing convolution and normalization processing on the image characteristics;
step 3.3: transforming the receptive field of the image features by using a spatial transformation network;
and 4, step 4: inputting the image characteristics into a capsule network, extracting spatial correlation information of the image characteristics, and improving generalization capability;
and 5: and distinguishing and classifying the clothes according to the image characteristics to obtain a clothes classification result.
As shown in fig. 1, the clothing classification system of the clothing classification method includes:
the image preprocessing module is used for carrying out image enhancement and normalization processing on the input clothing picture;
the characteristic preprocessing module is used for carrying out characteristic downsampling and characteristic fusion on the clothing image;
the characteristic enhancement module is used for amplifying the key vectors and the weights of the clothing image characteristics by adopting an attention mechanism and converting the receptive field of the image characteristics by utilizing a space transformation network, so that the obtained image characteristics are not limited to a single direction or posture;
the capsule network unit extracts spatial correlation information of image features, improves generalization capability and strengthens hierarchical relation expressed by internal knowledge in the neural network;
and the distinguishing processing module is used for distinguishing and classifying the clothes according to the image characteristics and calculating the confidence of the corresponding classification to obtain the classification result of the clothes.
The characteristic enhancement module comprises an extrusion-and-Excitation network (SE), a convolution and normalization module and a Spatial Transformer Network (STN), wherein the SE network amplifies the key vectors and the weights by using an attention mechanism and screens out the non-target characteristics of interference; the convolution and normalization module is used for feature fusion and normalization processing of the clothing image; the STN network further transforms the receptive field of the image features so that the resulting features can express multiple orientations or multiple poses.
In reality, various factors such as light brightness, shading, photographing azimuth distance and the like can influence the data sample for testing, so that the data sample cannot be well recognized by the network model. The clothing classification system of the invention preprocesses the input image to expand the diversity of the data set sample. The characteristic preprocessing module can well perform characteristic processing preprocessing on the image of the data to obtain a characteristic diagram with reduced scale and well retained key information, and is convenient for the next fine processing. The characteristic enhancement module can carry out careful weight training and enhancement on the obtained coarse characteristic diagram, so that only required key information can be trained and retained, and contents irrelevant to a target in the image are screened out; the capsule network module further extracts spatial information of the full-volume model, so that the image can keep a correct structure during judgment, and the accuracy and the correctness of the model are enhanced.
According to the clothing classification system, the network model of the clothing classification system can achieve strong identification capability on a small sample data set through image enhancement and enhancement change in the calculation process, and can be used for dealing with input images in various forms without generating identification errors caused by changes of image direction, contrast and the like; by means of an attention mechanism, image information irrelevant to a target can be eliminated in the process of training the model, and only useful clothing information is concerned; by combining the capsule network structure, the network model can have spatial layout information which is not possessed by a general neural network, a good classification effect can be achieved when the image layout is disordered or distorted, the shapes of various samples can be expressed only by a small number of parameters, and the generalization capability is strong.
The SENET network of the embodiment refers to the SENET network disclosed in "Squeeze-and-Excitation Networks" article Jie Hu et al published in IEEE Transactions on Pattern Analysis and Machine Analysis, 8.2020.
The STN network of an embodiment is described in Max Jaderberg et al, published 2015, 6, paper "Spatial Transformer Networks".
It will be understood by those skilled in the art that the foregoing is only a preferred embodiment of the present invention, and is not intended to limit the invention, and that any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention should be included in the scope of the present invention.
Claims (6)
1. The garment classification method based on the capsule network is characterized by comprising the following steps of:
step 1: carrying out image enhancement and normalization processing on the input clothing picture;
step 2: performing feature downsampling and feature fusion on the clothing image;
and step 3: performing characteristic enhancement on the clothing image;
step 3.1: amplifying the key vectors and the weights of the clothing image features by adopting an attention mechanism;
step 3.2: performing convolution and normalization processing on the image characteristics;
step 3.3: transforming the receptive field of the image features by using a spatial transformation network;
and 4, step 4: inputting the image characteristics into a capsule network, extracting spatial correlation information of the image characteristics, and improving generalization capability;
and 5: and distinguishing and classifying the clothes according to the image characteristics to obtain a clothes classification result.
2. The capsule network-based garment classification method according to claim 1, characterized in that said step 1 comprises the following sub-steps:
step 1.1: rotating, turning over, cutting, scaling, contrast enhancing and brightness adjusting the input clothing image;
step 1.2: and mapping the RGB parameters of the clothing image to a uniform interval.
3. The capsule network-based garment classification method according to claim 1, characterized in that in step 2, the feature downsampling downsamples the image of the large scale 224x224 to the small scale of 25x25 suitable for feature enhancement.
4. The capsule network-based garment classification method according to claim 1, characterized in that step 3.1 employs a compression and excitation network to enlarge key vectors and weights of garment image features to screen out interfering non-target features.
5. A garment categorization system according to any of claims 1 to 4, characterized in that it comprises:
the image preprocessing module is used for carrying out image enhancement and normalization processing on the input clothing picture;
the characteristic preprocessing module is used for carrying out characteristic downsampling and characteristic fusion on the clothing image;
the characteristic enhancement module is used for amplifying the key vectors and the weights of the clothing image characteristics by adopting an attention mechanism and converting the receptive field of the image characteristics by utilizing a space transformation network, so that the obtained image characteristics are not limited to a single direction or posture;
the capsule network unit extracts spatial correlation information of image features and improves generalization capability;
and the distinguishing processing module is used for distinguishing and classifying the clothes according to the image characteristics to obtain the clothes classification result.
6. The clothing classification system of claim 5, wherein the feature enhancement module comprises a compression and excitation network, a convolution and normalization module, and a spatial transformation network, wherein the compression and excitation network amplifies the key vectors and weights by using an attention mechanism to screen out interfering non-target features, and the spatial transformation network further transforms the receptive field of the image features so that the obtained features can express multiple orientations or multiple postures.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011573946 | 2020-12-28 | ||
| CN2020115739469 | 2020-12-28 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113205061A true CN113205061A (en) | 2021-08-03 |
Family
ID=77031655
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110541866.3A Pending CN113205061A (en) | 2020-12-28 | 2021-05-18 | Garment classification method and classification system based on capsule network |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113205061A (en) |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN205455455U (en) * | 2016-01-14 | 2016-08-17 | 中国水产科学研究院东海水产研究所 | Midwater trawl keed selectivity device |
| CN110728224A (en) * | 2019-10-08 | 2020-01-24 | 西安电子科技大学 | Remote sensing image classification method based on attention mechanism depth Contourlet network |
| CN111191660A (en) * | 2019-12-30 | 2020-05-22 | 浙江工业大学 | Rectal cancer pathology image classification method based on multi-channel collaborative capsule network |
-
2021
- 2021-05-18 CN CN202110541866.3A patent/CN113205061A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN205455455U (en) * | 2016-01-14 | 2016-08-17 | 中国水产科学研究院东海水产研究所 | Midwater trawl keed selectivity device |
| CN110728224A (en) * | 2019-10-08 | 2020-01-24 | 西安电子科技大学 | Remote sensing image classification method based on attention mechanism depth Contourlet network |
| CN111191660A (en) * | 2019-12-30 | 2020-05-22 | 浙江工业大学 | Rectal cancer pathology image classification method based on multi-channel collaborative capsule network |
Non-Patent Citations (2)
| Title |
|---|
| RAHUL KATARYA 等: "Study on Text Classification using Capsule Networks", 《2019 5TH INTERNATIONAL CONFERENCE ON ADVANCED COMPUTING & COMMUNICATION SYSTEMS》 * |
| 韩啸: "基于深度学习的服装检索与搭配的研究与应用", 《中国优秀硕士学位论文全文数据库 信息科技辑》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102102161B1 (en) | Method, apparatus and computer program for extracting representative feature of object in image | |
| CN103761531B (en) | The sparse coding license plate character recognition method of Shape-based interpolation contour feature | |
| CN106022375B (en) | A kind of clothes fashion recognition methods based on HU not bending moment and support vector machines | |
| CN106529499A (en) | Fourier descriptor and gait energy image fusion feature-based gait identification method | |
| CN105160310A (en) | 3D (three-dimensional) convolutional neural network based human body behavior recognition method | |
| CN108647625A (en) | A kind of expression recognition method and device | |
| CN107767416B (en) | Method for identifying pedestrian orientation in low-resolution image | |
| CN108021889A (en) | A kind of binary channels infrared behavior recognition methods based on posture shape and movable information | |
| CN108830312B (en) | Integrated learning method based on sample adaptive expansion | |
| CN110060273B (en) | Remote sensing image landslide mapping method based on deep neural network | |
| CN110163286A (en) | Hybrid pooling-based domain adaptive image classification method | |
| CN105718552A (en) | Clothing freehand sketch based clothing image retrieval method | |
| CN106022223B (en) | A kind of higher-dimension local binary patterns face identification method and system | |
| CN105718882A (en) | Resolution adaptive feature extracting and fusing for pedestrian re-identification method | |
| CN111563452A (en) | Multi-human body posture detection and state discrimination method based on example segmentation | |
| CN108734200A (en) | Human body target visible detection method and device based on BING features | |
| Ma et al. | Saliency detection based on singular value decomposition | |
| CN105760828B (en) | A kind of static gesture identification method of view-based access control model | |
| CN109190456A (en) | Pedestrian detection method is overlooked based on the multiple features fusion of converging channels feature and gray level co-occurrence matrixes | |
| CN106056132A (en) | Garment style identification method based on Fourier descriptor and support vector machine | |
| CN106529544A (en) | Fabric flatness objective evaluation method and fabric flatness objective evaluation device based on unsupervised machine learning | |
| CN109523514A (en) | To the batch imaging quality assessment method of Inverse Synthetic Aperture Radar ISAR | |
| CN109002771A (en) | A kind of Classifying Method in Remote Sensing Image based on recurrent neural network | |
| CN103605993B (en) | Image-to-video face identification method based on distinguish analysis oriented to scenes | |
| CN109299681A (en) | A kind of coal and rock face crack automatic identifying method based on support vector machines |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210803 |