CN108764302B - Bill image classification method based on color features and bag-of-words features - Google Patents
Bill image classification method based on color features and bag-of-words features Download PDFInfo
- Publication number
- CN108764302B CN108764302B CN201810434070.6A CN201810434070A CN108764302B CN 108764302 B CN108764302 B CN 108764302B CN 201810434070 A CN201810434070 A CN 201810434070A CN 108764302 B CN108764302 B CN 108764302B
- Authority
- CN
- China
- Prior art keywords
- bill
- color
- classification
- features
- words
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2411—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on the proximity to a decision surface, e.g. support vector machines
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
- G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/40—Extraction of image or video features
- G06V10/56—Extraction of image or video features relating to colour
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Multimedia (AREA)
- Bioinformatics & Computational Biology (AREA)
- Evolutionary Biology (AREA)
- Evolutionary Computation (AREA)
- Bioinformatics & Cheminformatics (AREA)
- General Engineering & Computer Science (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to the technical field of images, in particular to a bill image classification method based on color features and bag-of-words features. The method utilizes the classic Bag of Words in computer vision, namely, SIFT feature points of each bill are extracted from training samples and 128-dimensional feature descriptors are generated, then K mean value clustering is carried out to obtain K visual Words, the number of times of appearance of the visual Words of each bill is counted to form a visual word histogram of the bill as features, finally color features are blended to form a total feature vector, and the total feature vector is sent to an SVM classifier for training to obtain a bill classifier model. Because the bag-of-words model does not use the color features of the bill images, the method adds the global main color features of the images, and further improves the performance of the bill classifier. The method can train the bill classifier model only by a small amount of training samples without manually designing additional features, and the classifier has high classification speed and high accuracy.
Description
Technical Field
The invention relates to the technical field of images, in particular to a bill image classification method based on color features and bag-of-words features.
Background
In traditional bill management, the bills are classified by manpower, and the bills to be classified are large in number, so that a large amount of manpower and material resources are needed to finish the classification, and therefore, the automatic bill classification system is produced by taking machine vision as a technical background to solve the simple and repeated classification work. The existing automatic bill classification system needs to collect more various bill images as training samples, manually design specific features such as line segments, angular points, shapes, textures and the like for various bills, and then vectorize the features and send the vectorized features into a classifier such as an SVM for training. The classification system has certain limitations because it needs to collect a large number of training samples and spend a great deal of effort to design bill features manually to ensure the classification performance of the trained models. In addition, most of the bills which can be classified by the existing bill classification system are special financial invoices such as value-added tax, general-purpose machine invoice and the like, but the bills which are commonly used by the reimbursement system such as train tickets, high-speed railway tickets, taxi tickets, air tickets and the like cannot be classified, and the bill classification system has no universality.
Chinese patent No. CN106096667 authorizes an SVM-based bill classification method, which requires manual feature design of bills in advance, such as official seal extraction and linear extraction, and this method is only applicable to a small number of bills, and cannot classify most bills without linear lines or seals, and is too limited.
Chinese patent publication No. CN107633239 discloses a method for classifying bills and extracting bill fields based on deep learning and OCR, which requires first obtaining outline features of a stamp and collecting a large number of stamp samples as training samples for deep learning, and this method is not suitable for most bills without stamps but also needs collecting a large number of training samples.
Disclosure of Invention
The invention aims to overcome at least one defect in the prior art and provides a bill image classification method based on color features and bag-of-words features, the bill image classification method based on the color features and the bag-of-words features can be used for obtaining a bill classifier with excellent performance on a very small amount of training sets without designing corresponding features aiming at the characteristics of each type of bills, and the bill classifier is trained by only utilizing the generated global color features of the images and the bag-of-words features based on SIFT feature points and sending the characteristics into an SVM classifier, and when online bill classification prediction is carried out, a multi-level classification strategy based on the color features and the bag-of-words features is used, so that the classification performance is further improved, and a bill classification task is rapidly and accurately completed.
The technical scheme of the invention is as follows: a bill image classification method based on color features and bag-of-words features includes two parts of off-line bill classifier training and on-line bill rapid classification,
the off-line bill classifier training part comprises two parts of color feature extraction and bag-of-words model training: the color characteristic module firstly converts the pictures in the training set into HSV color space, quantizes the H component to generate a color histogram, records the main color of each type of bill, and stores the main color in a hard disk for storage; performing bag-of-words model training to obtain K clustering centers by SIFT feature extraction and K mean value clustering on bills of a training set, performing feature quantization to generate bags of words, performing statistics on visual word frequency of each type of bill training samples to generate visual word histograms of the types, generating corresponding feature vectors by the histograms, training by taking the feature vectors as input of a feature SVM classifier, and finally storing trained model parameter files in a hard disk for storage;
the on-line bill rapid classification part firstly needs to load a trained bill classifier model and a color classification parameter file, firstly converts an image into an HSV color space, generates a color histogram of the image and extracts a main color feature, judges that the main color of the bill exists and is unique in the existing bill category by using the feature, directly outputs a classification result if the main color exists in the existing bill category, and enters a word bag model classification process if the main color does not exist in the existing bill category; in the bag-of-words model classification process, firstly SIFT features of an image need to be extracted to generate visual words, the feature vectors are sent to an SVM for classification to obtain classification results, then secondary judgment needs to be carried out on the classification results according to color features, namely whether main color features corresponding to the results are the same as the previously obtained main color features or not is inquired according to the classification results, if yes, the classification results are output, and if not, the classification results are wrong, and the classification fails.
Furthermore, the bill classification model with excellent performance can be trained by only needing a plurality of sheets for each type of the training bill samples, and a large amount of training samples are not required to be collected.
Furthermore, the classification model adopts the main color feature of the bill as the classification feature of the bill, namely, firstly, the image is converted into an HSV color space, the color histogram of the H component of the image is counted, and the maximum component of the H component is extracted as the main color feature of the image.
Furthermore, the classification model adopts an SIFT feature point Bag of Words model for training and online classification, namely SIFT feature point extraction and K mean value clustering are firstly carried out on images of a training set to form K visual Words, statistics is carried out on the visual Words of each bill in the training set to form Bag-of-Words features, and the Bag-of-Words features are sent to an SVM classifier for training.
Further, the online bill classification is a hierarchical classification strategy, namely firstly, color features are utilized for online first classification, if the classification is successful, the result is directly output, and if the classification is failed, the result is compared with the color features of the prediction result of the SVM classifier, and the final classification result is obtained.
The offline bill classifier training module comprises the following specific steps:
(1) image conversion to HSV color space
(2) An image histogram of the H component of the training samples of each class is generated.
(3) And extracting the highest component in the image histogram of each class as the main color feature of the class of images and storing the main color feature in a file.
(4) Image graying
(5) And extracting SIFT feature points from each class of training samples to generate SIFT feature point descriptors.
(6) Clustering the K mean values to obtain K class cores as word bags
(7) And sequentially carrying out statistics on the visual words of the training samples of each type to obtain the visual word feature vector.
(8) The feature vectors are sent to an SVM classifier for training
(9) And saving the trained classifier model file.
The online classifier bill classification prediction module comprises the following specific steps:
(1) and initializing the bill classification system, and loading a model parameter file and a color feature file.
(2) The image to be classified is converted into HSV color space.
(3) A color histogram of the H component of the map is generated.
(4) And extracting main color features.
(5) And (4) judging whether the main color features of the graph exist and are unique in the color feature file, if so, directly outputting a classification prediction result, and if not, executing the step (6).
(6) And (5) graying the image.
(7) And extracting SIFT feature points from each class of training samples to generate SIFT feature point descriptors.
(8) And counting the visual words of the graph to obtain a visual word feature vector.
(9) And the SVM carries out classification prediction according to the visual word feature vector.
(10) And (4) carrying out secondary verification on the classification result, namely judging whether the main color features of the result are consistent with the main color features obtained in the step (4), if so, outputting the classification result, and if not, failing the classification.
Compared with the prior art, the beneficial effects are: the invention can automatically extract color characteristics and word bag characteristics under a small amount of bill training samples without manually designing characteristics aiming at the characteristics of each type of bills, and has the advantages of wide applicable bill range, high classification accuracy and high classification speed.
Drawings
FIG. 1 shows the algorithm framework of the present invention.
Fig. 2 shows the decision making process in on-line classification prediction.
Detailed Description
The drawings are for illustrative purposes only and are not to be construed as limiting the patent; for the purpose of better illustrating the embodiments, certain features of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product; it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted. The positional relationships depicted in the drawings are for illustrative purposes only and are not to be construed as limiting the present patent.
As shown in FIG. 1, the scheme is divided into two modules of offline bill classification model training and online bill rapid classification. The specific steps of the offline classification model training comprise: (1) the bill images of each class are collected, and because the method only has low requirement on the number of training samples, only a plurality of high-quality images need to be selected for each class of bills; (2) converting each image into HSV space; (3) because the hue component can reflect the intuitive feeling of a person on the color most, only H is quantized to the hue component H, and the hue component H is divided into 6 intervals at an interval of 60 degrees so as to generate a color histogram of the H component; (3) extracting the color with the highest interval as the main color characteristic of the bill, and storing the color to a file; (3) graying the sample image; (4) SIFT feature point extraction is carried out on each image of the training sample, a 128-dimensional feature descriptor of each feature point is generated and stored; (5) performing K-Means clustering on all the feature descriptor vectors obtained in the step (4), wherein the number of the class centers is 1000, namely 1000 visual words are obtained after clustering is completed; (6) counting the frequency of visual words of each type of bills, and generating data word characteristic vectors of each type; (7) and (5) taking the features obtained in the step (6) as input, sending the input into an SVM classifier for training to obtain an SVM classification model, and storing model parameters into a file for storage.
The classification decision process of the online bill rapid classification module is shown in fig. 2, and comprises the following specific steps: (1) initializing the system, loading the trained classifier model, and storing the color feature file on a color _ pool array if the color feature file is the color _ pool array; (2) converting the image to be classified into an HSV color space; (3) generating a color histogram; (4) extracting a main color component; (5) checking whether the main color of the graph exists in the color _ pool array and is unique, if so, indicating that the invoices with the colors exist in the bill category and only one type exists, so that the invoices with the colors can be judged to belong to the category, directly outputting a classification result, and if not, entering the next step; (6) graying the sample image; (7) SIFT feature point extraction is carried out on each image of the training sample, a 128-dimensional feature descriptor of each feature point is generated and stored; (8) counting the visual words of the graph to obtain a visual word feature vector; (9) carrying out classification prediction by using the trained SVM according to the visual word feature vector to obtain a pre-classification result; (10) and (3) carrying out secondary verification on the classification result, namely judging whether the main color of the type is consistent with the main color obtained in the step (4), if so, outputting the classification result, and if not, outputting the classification failure.
It should be understood that the above-described embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. Any modification, equivalent replacement, and improvement made within the spirit and principle of the present invention should be included in the protection scope of the claims of the present invention.
Claims (3)
1. A bill image classification method based on color features and bag-of-words features includes two parts of off-line bill classifier training and on-line bill rapid classification,
the off-line bill classifier training part comprises two parts of color feature extraction and bag-of-words model training: the color feature extraction includes: firstly, converting images in a training set into an HSV color space, quantizing the H component to generate a color histogram, recording the main color of each type of bill, and storing the main color in a hard disk for storage; the bag-of-words model training comprises: SIFT feature extraction is carried out on the bills of the training set, K mean value clustering is carried out on the bills to obtain K clustering centers, feature quantization is carried out on each type of bills to generate word bags, visual word frequency statistics is carried out on each type of bill training samples to generate visual word histograms of the types, corresponding feature vectors are generated according to the histograms, the feature vectors are used as input of an SVM classifier to carry out training, and finally, trained model parameter files are stored in a hard disk to be stored;
the online bill rapid classification part firstly loads a trained bill classifier model and a color classification parameter file, firstly converts an image to be classified into an HSV color space, generates a color histogram of the image and extracts a main color feature, judges whether the main color of the image to be classified exists in the existing bill category or not and is unique by using the feature, directly outputs a classification result if the main color exists in the existing bill category, and enters a word bag model classification process if the main color does not exist in the existing bill category; the bag-of-words model classification process firstly extracts SIFT features of an image to be classified and generates visual words, the visual words are counted to obtain visual word feature vectors, the feature vectors are sent to an SVM for classification to obtain classification results, secondary judgment is conducted on the classification results according to color features, namely whether main color features corresponding to the results are the same as the obtained main color features of the image to be classified or not is inquired according to the classification results, if yes, the classification results are output, if not, the classification results are wrong, and classification fails.
2. The bill image classification method based on the color feature and the bag-of-words feature according to claim 1, wherein: the bill classifier model can be trained by only needing a plurality of sheets for each type of the bill samples for training without collecting a large number of training samples.
3. The bill image classification method based on the color feature and the bag-of-words feature according to claim 1, wherein: and (3) taking the main color feature of the bill as the classification feature of the bill, namely converting the image into an HSV color space, counting the color histogram of the H component of the image, and extracting the largest component in the color histogram as the main color feature of the image.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810434070.6A CN108764302B (en) | 2018-05-08 | 2018-05-08 | Bill image classification method based on color features and bag-of-words features |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810434070.6A CN108764302B (en) | 2018-05-08 | 2018-05-08 | Bill image classification method based on color features and bag-of-words features |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108764302A CN108764302A (en) | 2018-11-06 |
| CN108764302B true CN108764302B (en) | 2021-09-28 |
Family
ID=64009283
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810434070.6A Active CN108764302B (en) | 2018-05-08 | 2018-05-08 | Bill image classification method based on color features and bag-of-words features |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108764302B (en) |
Families Citing this family (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109670513A (en) * | 2018-11-27 | 2019-04-23 | 西安交通大学 | A kind of piston attitude detecting method based on bag of words and support vector machines |
| CN110694941A (en) * | 2019-10-10 | 2020-01-17 | 昆明理工大学 | Intelligent sorting device and method for building solid waste |
| CN111160373B (en) * | 2019-12-30 | 2022-07-15 | 重庆邮电大学 | Method for extracting, detecting and classifying defect image features of variable speed drum part |
| CN111652309A (en) * | 2020-05-29 | 2020-09-11 | 刘秀萍 | Visual word and phrase co-driven bag-of-words model picture classification method |
| CN111797830A (en) * | 2020-07-07 | 2020-10-20 | 因凡科技(北京)有限公司 | Rapid red seal detection method, system and device for bill image |
| CN112613563A (en) * | 2020-12-25 | 2021-04-06 | 福建福清核电有限公司 | Nuclear power field equipment image classification method based on OpenCV |
| CN113657511B (en) * | 2021-02-02 | 2022-10-28 | 哈尔滨商业大学 | Commodity image feature description method |
| CN112907534A (en) * | 2021-02-18 | 2021-06-04 | 哈尔滨市科佳通用机电股份有限公司 | Fault detection method and device based on door closing part position image |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622607A (en) * | 2012-02-24 | 2012-08-01 | 河海大学 | Remote sensing image classification method based on multi-feature fusion |
| CN106022364A (en) * | 2016-05-13 | 2016-10-12 | 邓昌顺 | Novel note classifying method |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106203448B (en) * | 2016-07-08 | 2019-03-12 | 南京信息工程大学 | A kind of scene classification method based on Nonlinear Scale Space Theory |
-
2018
- 2018-05-08 CN CN201810434070.6A patent/CN108764302B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102622607A (en) * | 2012-02-24 | 2012-08-01 | 河海大学 | Remote sensing image classification method based on multi-feature fusion |
| CN106022364A (en) * | 2016-05-13 | 2016-10-12 | 邓昌顺 | Novel note classifying method |
Non-Patent Citations (3)
| Title |
|---|
| 基于HSV颜色空间和SIFT特征的近似图像检索;王亚如;《中国优秀硕士学位论文全文数据库信息科技辑》;20141015(第10期);第I138-803页 * |
| 基于决策树的瓷砖图像分类方法研究;邓亚芳;《中国优秀硕士学位论文全文数据库信息科技辑》;20150315(第03期);第I138-1943页 * |
| 层次型金融票据图像分类方法;殷绪成 等;《中文信息学报》;20051231;第19卷(第6期);第70-77页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108764302A (en) | 2018-11-06 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108764302B (en) | Bill image classification method based on color features and bag-of-words features | |
| CN107622255B (en) | Bill image field positioning method and system based on position template and semantic template | |
| Wu et al. | Harvesting discriminative meta objects with deep CNN features for scene classification | |
| Dixit et al. | Scene classification with semantic fisher vectors | |
| CN101887523B (en) | Method for detecting image spam email by picture character and local invariant feature | |
| CN110598800A (en) | Garbage classification and identification method based on artificial intelligence | |
| CN102156871B (en) | Image classification method based on category correlated codebook and classifier voting strategy | |
| CN103093240A (en) | Calligraphy character identifying method | |
| CN103761531A (en) | Sparse-coding license plate character recognition method based on shape and contour features | |
| JP2008159056A (en) | Classification through generative model of feature occurring in image | |
| CN102663401B (en) | Image characteristic extracting and describing method | |
| CN105930792A (en) | Human action classification method based on video local feature dictionary | |
| CN103106265A (en) | Method and system of classifying similar images | |
| CN102156885A (en) | Image classification method based on cascaded codebook generation | |
| CN105809205A (en) | Classification method and system for hyperspectral images | |
| Sikirić et al. | Classifying traffic scenes using the GIST image descriptor | |
| CN106960176A (en) | A kind of pedestrian's gender identification method based on transfinite learning machine and color characteristic fusion | |
| CN103279738A (en) | Automatic identification method and system for vehicle logo | |
| CN101655911B (en) | Mode identification method based on immune antibody network | |
| Liu et al. | Wagon number recognition based on the YOLOv3 detector | |
| CN103839074A (en) | Image classification method based on matching of sketch line segment information and space pyramid | |
| CN105389588A (en) | Multi-semantic-codebook-based image feature representation method | |
| CN106570514A (en) | Automobile wheel hub classification method based on word bag model and support vector machine | |
| CN103984965A (en) | Pedestrian detection method based on multi-resolution character association | |
| CN104008095A (en) | Object recognition method based on semantic feature extraction and matching |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |