CN110705613B - Object classification method - Google Patents
Object classification method Download PDFInfo
- Publication number
- CN110705613B CN110705613B CN201910885199.3A CN201910885199A CN110705613B CN 110705613 B CN110705613 B CN 110705613B CN 201910885199 A CN201910885199 A CN 201910885199A CN 110705613 B CN110705613 B CN 110705613B
- Authority
- CN
- China
- Prior art keywords
- classification
- result
- objects
- target object
- distance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Probability & Statistics with Applications (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Evolutionary Biology (AREA)
- Computational Linguistics (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses an object classification method, which comprises the following steps: firstly, detecting all targets in a scene by using a detection model, and carrying out primary coarse classification; secondly, constructing a position relation graph of the roughly classified targets according to the mutual position relation, and calculating the relation influence weight according to the relative distance; thirdly, graph convolution is carried out on the position relation graph by using the relation influence weight calculated in the second step to obtain a smoothed result; and repeating the steps until the change value is not changed. According to the invention, through constructing the position relation diagram of different objects, the final result is not only the classification result of a single object but also the classification result of surrounding objects is considered, so that objects which are difficult to distinguish by a machine learning model due to occlusion blurring and the like can be reclassified, and simple inference can be carried out, namely the classification result of the objects which are difficult to distinguish is deduced through the classification result of the surrounding objects.
Description
Technical Field
The invention relates to the technical field of product classification, in particular to a technology of an object classification method.
Background
With the development of artificial intelligence, detection and classification techniques play an increasingly important role in daily life, but the essence of the current mainstream detection and classification techniques is to classify each opposite object, and the interrelation between the objects is not considered. For some scenes needing to consider the mutual relation, the performance is poor, and the accuracy of human beings is difficult to achieve.
Disclosure of Invention
In order to overcome the defects of the prior art, the invention aims to provide an object classification method, which can be used for simply deducing the classification of an object which is difficult to distinguish to obtain the class of the object.
In order to achieve the purpose, the invention adopts the following technical scheme:
a method of classifying an object, characterized by: the object classification method comprises the following steps
Firstly, detecting all targets in a scene by using a detection model, and carrying out primary coarse classification;
secondly, constructing a position relation graph of the roughly classified targets according to the mutual position relation, and calculating the relation influence weight according to the relative distance;
thirdly, graph convolution is carried out on the position relation graph by using the relation influence weight calculated in the second step to obtain a smoothed result;
and repeating the steps until the change value is not changed.
The coarse classification is a convolutional neural network such as ResNet, resulting in a probability of belonging to each class.
The mutual position relation uses Euclidean pixel distance as distance, and the weight is e ^ - (distance/k), wherein k is a parameter which can be changed according to actual effect.
The convolution in the third step is weighting, the weighting is carried out according to the weight calculated in the second step, and the smoothed result is x1 sum (wi x0 i); the variation value is x1-x 0.
The invention has the following beneficial effects:
according to the invention, through constructing the position relation diagram of different objects, the final result is not only the classification result of a single object, but also the classification result of surrounding objects is considered, namely the target object and the surrounding objects are combined to carry out the inference of one classification result, so that objects which are difficult to distinguish by a machine learning model due to the existence of shielding blurring and the like can be reclassified, and thus, simple inference can be carried out, namely the classification results of the objects which are difficult to distinguish are inferred through the classification results of the surrounding objects.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required to be used in the embodiments of the present invention will be briefly described below. It is obvious that the drawings described below are only some embodiments of the invention, and that for a person skilled in the art, other drawings can be derived from them without inventive effort.
FIG. 1 is a schematic diagram of an object classification method of the present invention;
FIG. 2 is a schematic diagram of a classification module of the object classification method of the present invention.
Detailed Description
The technical scheme of the invention is further explained by the specific implementation mode in combination with the attached drawings.
Wherein the showings are for the purpose of illustration only and are shown by way of illustration only and not in actual form, and are not to be construed as limiting the present patent; to better illustrate the embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of an actual product; it will be understood by those skilled in the art that certain well-known structures in the drawings and descriptions thereof may be omitted.
The same or similar reference numerals in the drawings of the embodiments of the present invention correspond to the same or similar components; in the description of the present invention, it should be understood that if the terms "upper", "lower", "left", "right", "inner", "outer", etc. are used for indicating the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, it is only for convenience of description and simplification of description, but it is not indicated or implied that the referred device or element must have a specific orientation, be constructed in a specific orientation and be operated, and therefore, the terms describing the positional relationship in the drawings are only used for illustrative purposes and are not to be construed as limitations of the present patent, and the specific meanings of the terms may be understood by those skilled in the art according to specific situations.
The object classification method of the invention comprises the following steps:
firstly, detecting all targets in a scene by using a detection model such as Mask-RCNN, and carrying out primary rough classification, namely the probability that the result of the convolutional neural network belongs to each class, if objects are classified into three classes, the result of each object rough classification [ 0.10.10.8 ] respectively represents that the probability of the first class is 0.1, the probability of the second class is 0.1, and the probability of the third class is 0.8, namely the objects are classified into the third class. The convolutional neural network result such as a ResNet result; ResNet (residual Neural network) was proposed by four people, Kaiming He, of Microsoft institute, and a 152-layer Neural network was successfully trained using the ResNet Unit. The structure of ResNet can accelerate the training of the neural network very fast, and the accuracy of the model is greatly improved. Meanwhile, the popularization of ResNet is very good, and even the ResNet can be directly used in an IncepotionNet network. Mask R-CNN is a two-stage framework, the first stage scanning the image and generating proposals (i.e., areas that may contain an object), the second stage classifying the proposals and generating bounding boxes and masks. The Mask R-CNN was extended from Faster R-CNN and was proposed by the same author in the last year. The Faster R-CNN is a popular target detection framework, and the Mask R-CNN expands the target detection framework into an example segmentation framework.
Secondly, constructing a position relation graph of each roughly classified target according to a mutual position relation, calculating a relation influence weight according to a relative distance, and using Euclidean pixel distance as the distance, wherein the relation influence weight is e ^ - (distance/k), and k is a parameter changed according to an actual effect;
thirdly, graph convolution is carried out on the position relation graph by using the weights calculated in the second step, the convolution is weighting, and the smoothed result is x1 sum (wi x0 i);
the above steps are repeated until the variation value stops changing, namely the variation value x1-x0 stops changing, and the variation value x1-x0 is as small as possible. The technical meanings given for the parameters e ^ distance, k, (distance/k), sum, wi, x0i, x0, x1 by the aforementioned e ^ - (distance/k), x1 ═ sum (wi ^ x0i) are as follows: e ^ index; distance is the pixel distance of two objects (the pixel distance is the distance on the picture); sum; wi is the calculated weight; x is the calculated probability. x1 ═ sum (wi x0i) depends on the calculation result of e ^ - (distance/k); x1-x0 refers to the correction of probability.
For example: a detection model such as Mask-RCNN is used for detecting all targets in a scene, including a target object A and surrounding objects B, wherein the target object A has shielding blurring, so that the machine learning model is difficult to distinguish, and the surrounding objects B are easy to distinguish the categories of the targets by the machine learning model. The method comprises the following steps of carrying out primary rough classification on a target object A and a peripheral object B, wherein the classification principle is that a probability numerical value belonging to each class is obtained by utilizing a convolution neural network result. Where the surrounding object B is a neighboring object sufficiently close to the target object a.
Constructing a position relation graph of a target object A and a surrounding object B according to the mutual position relation of the target object A and the surrounding object B, calculating a relation influence weight according to a relative distance, and using a Euclidean pixel distance as a distance, wherein the relation influence weight is e ^ - (distance/k), and k is a parameter changed according to an actual effect;
the position map is subjected to graph convolution, that is, weighting, using the calculated weights, and a smoothed result x1 is sum (wi x0 i).
Repeating the above steps until the variation value stops changing, namely the variation value x1-x0 stops changing, and then the probability that the target object and the surrounding object B are the same is considered to be high. It is inferred that the class of the surrounding object B is the class of the target object a.
According to the object classification method, the mutual position relation of the objects is considered, so that the mutual relation of different objects can be considered in the object detection and classification result, and the classification result is more stable and is close to the result achieved by human beings. In the invention, the technical core is that a position relation graph of different objects is constructed, so that the final result is not only the classification result of a single target object, but also the classification result of surrounding objects is considered, and an inference is carried out. The above process is actually weighted by object periphery classification, and the target object and the similar object are considered to be in the same class with higher probability, so as to carry out inference. As described above, objects that are difficult to distinguish by the machine learning model due to occlusion blurring or the like are reclassified so that the model can be weighted by the calculated weight, and the smoothed result, that is, x1 (sum (wi x0i), is obtained; repeating the steps, and if the change value x1-x0 stops changing, the probability that the target object and the surrounding object are the same is considered to be high, so that the class of the surrounding object, namely the similar object, is inferred to be the class of the target object, namely the object which is difficult to distinguish by the machine learning model. It can be seen that the present invention infers the category of these target objects that are difficult to distinguish through the surrounding objects.
In summary, the object classification method of the present invention connects neighboring objects that are close enough together by using one edge, so that the final result not only is the classification result of a single object, but also an inference is performed in consideration of the classification result of surrounding objects, and therefore, objects that are difficult to be resolved by the machine learning model due to occlusion blurring and the like are reclassified, so that the model can perform simple inference, and the classes of the objects that are difficult to be resolved are inferred by the surrounding objects.
Referring to fig. 1, the principle of the object classification method of the present invention is shown: detecting and roughly classifying by using a detection model; constructing an object relation graph in the picture and calculating weight; performing convolution according to the calculated weight sum graph to obtain a smoothed result x 1-sum (wi x0 i); the above process is repeated until the variation values x1-x0 stop unchanged. FIG. 2 is a diagram of a classification module, i.e., a detection classification model, an object relationship diagram, and a graph convolution.
It should be noted that the above-mentioned embodiments are only preferred embodiments of the present invention and the technical principles used, and any changes or substitutions which can be easily conceived by those skilled in the art within the technical scope of the present invention disclosed herein should be covered within the protective scope of the present invention.
Claims (2)
1. An object classification method is characterized in that the object classification method constructs a position relation graph of different objects, so that the final result is not only the classification result of a single target object, but also the classification result of surrounding objects is considered, an inference is made that the target object and the surrounding objects are possibly in the same class by weighting the classification result of the surrounding objects, and the classification result of the object which is difficult to distinguish is deduced according to the classification result of the surrounding objects, wherein the target object is called the object which is difficult to distinguish,
comprises the following steps: firstly, detecting all targets in a scene by using a detection model, wherein all targets comprise a target object and surrounding objects, the target object is shielded and blurred to make the detection model difficult to distinguish, the surrounding objects are easy to distinguish the categories of the target object, the surrounding objects are adjacent objects close to the target object enough, the target object and the surrounding objects are roughly classified for one time, and the classification principle of the rough classification is to obtain a probability value belonging to each category by using a convolutional neural network result;
secondly, constructing a position relation graph of each roughly classified target according to a mutual position relation, calculating a relation influence weight according to a relative distance, and using Euclidean pixel distance as the distance, wherein the relation influence weight is e ^ - (distance/k), and k is a parameter changed according to an actual effect;
thirdly, graph convolution is carried out on the position relation graph by using the relation influence weight calculated in the second step to obtain a smoothed result, the second step and the third step are repeated until a variation value stops changing, the probability that the target object and the surrounding object are the same is considered to be high, the category of the surrounding object is inferred to be the category of the target object, the convolution in the third step is weighting, weighting is carried out according to the weight calculated in the second step, and the smoothed result is x1 sum (wi x0 i); the change value is x1-x 0; wherein e ^ index; distance is the pixel distance of two objects, and the pixel distance is the distance on the picture; sum; wi is the calculated weight; x is the calculated probability; x1 ═ sum (wi x0i) depends on the calculation result of e ^ - (distance/k); x1-x0 refers to the correction of probability.
2. The object classification method according to claim 1, characterized in that: the detection model is Mask-RCNN and is a two-stage framework, the image is scanned and a proposal is generated in the first stage, and a bounding box and a Mask are generated by classifying the proposal in the second stage;
the rough classification is [ 0.10.10.8 ], which respectively represents that the first class probability is 0.1, the second class probability is 0.1, and the third class probability is 0.8, namely the rough classification is classified into a third class; the convolutional neural network result is a ResNet result.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910885199.3A CN110705613B (en) | 2019-09-19 | 2019-09-19 | Object classification method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910885199.3A CN110705613B (en) | 2019-09-19 | 2019-09-19 | Object classification method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110705613A CN110705613A (en) | 2020-01-17 |
| CN110705613B true CN110705613B (en) | 2021-06-11 |
Family
ID=69196245
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910885199.3A Active CN110705613B (en) | 2019-09-19 | 2019-09-19 | Object classification method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110705613B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111414962B (en) * | 2020-03-19 | 2023-06-23 | 创新奇智(重庆)科技有限公司 | Image classification method introducing object relation |
| CN111612051B (en) * | 2020-04-30 | 2023-06-20 | 杭州电子科技大学 | Weak supervision target detection method based on graph convolution neural network |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109087245A (en) * | 2018-08-13 | 2018-12-25 | 长治学院 | Unmanned aerial vehicle remote sensing image mosaic system based on neighbouring relations model |
| CN109255791A (en) * | 2018-07-19 | 2019-01-22 | 杭州电子科技大学 | A kind of shape collaboration dividing method based on figure convolutional neural networks |
| CN109446413A (en) * | 2018-09-25 | 2019-03-08 | 上海交通大学 | Serializing recommended method based on item associations relationship |
| CN109918542A (en) * | 2019-01-28 | 2019-06-21 | 华南理工大学 | A kind of convolution classification method and system for relationship diagram data |
| CN110222611A (en) * | 2019-05-27 | 2019-09-10 | 中国科学院自动化研究所 | Human skeleton Activity recognition method, system, device based on figure convolutional network |
| CN110222771A (en) * | 2019-06-10 | 2019-09-10 | 成都澳海川科技有限公司 | A kind of classification recognition methods of zero samples pictures |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105279517A (en) * | 2015-09-30 | 2016-01-27 | 西安电子科技大学 | Weak tag social image recognition method based on semi-supervision relation theme model |
| KR101771044B1 (en) * | 2015-11-27 | 2017-08-24 | 연세대학교 산학협력단 | Apparatus and Method of Object Recognition based on Space-Object Relation Graph |
| US11853903B2 (en) * | 2017-09-28 | 2023-12-26 | Siemens Aktiengesellschaft | SGCNN: structural graph convolutional neural network |
| US20190122111A1 (en) * | 2017-10-24 | 2019-04-25 | Nec Laboratories America, Inc. | Adaptive Convolutional Neural Knowledge Graph Learning System Leveraging Entity Descriptions |
| CN110009093B (en) * | 2018-12-07 | 2020-08-07 | 阿里巴巴集团控股有限公司 | Neural network system and method for analyzing relational network graph |
| CN110084296B (en) * | 2019-04-22 | 2023-07-21 | 中山大学 | Graph representation learning framework based on specific semantics and multi-label classification method thereof |
| CN110111325A (en) * | 2019-05-14 | 2019-08-09 | 深圳大学 | Neuroimaging classification method, terminal and computer readable storage medium |
-
2019
- 2019-09-19 CN CN201910885199.3A patent/CN110705613B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109255791A (en) * | 2018-07-19 | 2019-01-22 | 杭州电子科技大学 | A kind of shape collaboration dividing method based on figure convolutional neural networks |
| CN109087245A (en) * | 2018-08-13 | 2018-12-25 | 长治学院 | Unmanned aerial vehicle remote sensing image mosaic system based on neighbouring relations model |
| CN109446413A (en) * | 2018-09-25 | 2019-03-08 | 上海交通大学 | Serializing recommended method based on item associations relationship |
| CN109918542A (en) * | 2019-01-28 | 2019-06-21 | 华南理工大学 | A kind of convolution classification method and system for relationship diagram data |
| CN110222611A (en) * | 2019-05-27 | 2019-09-10 | 中国科学院自动化研究所 | Human skeleton Activity recognition method, system, device based on figure convolutional network |
| CN110222771A (en) * | 2019-06-10 | 2019-09-10 | 成都澳海川科技有限公司 | A kind of classification recognition methods of zero samples pictures |
Non-Patent Citations (4)
| Title |
|---|
| Improving Object Detection with Relation Graph Inference;Chen-Hang He 等;《ICASSP 2019-2019 IEEE International Conference on Acoustics，Speech and Signal Processing（ICASSP）》;20190417;第3节、图1-3 * |
| Learning Semantics-Preserving Attention and Contextual Interaction for Group Activity Recognition;Yansong Tang 等;《IEEE Transactions on Image Processing》;20190508;第28卷(第10期);第4997-5012页 * |
| Local Spectral Graph Convolution for Point Set Feature;Chu Wang 等;《ECCV 2018：Computer Vision-ECCV2018》;20181006;第56-71页 * |
| Spatial-aware Graph Relation Network for Large-scale Object Detection;Hang Xu 等;《2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition（CVPR）》;20190620;第9290-9299页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110705613A (en) | 2020-01-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109886121B (en) | Human face key point positioning method for shielding robustness | |
| CN111797716B (en) | Single target tracking method based on Siamese network | |
| CN111723654B (en) | High-altitude parabolic detection method and device based on background modeling, YOLOv3 and self-optimization | |
| CN108416266B (en) | Method for rapidly identifying video behaviors by extracting moving object through optical flow | |
| Zhu et al. | RGB-D local implicit function for depth completion of transparent objects | |
| CN108961235A (en) | A kind of disordered insulator recognition methods based on YOLOv3 network and particle filter algorithm | |
| CN109241982A (en) | Object detection method based on depth layer convolutional neural networks | |
| CN116152267A (en) | Point cloud instance segmentation method based on contrast language image pre-training technology | |
| CN113706581B (en) | Target tracking method based on residual channel attention and multi-level classification regression | |
| CN112950780B (en) | Intelligent network map generation method and system based on remote sensing image | |
| CN113408584B (en) | RGB-D multi-modal feature fusion 3D target detection method | |
| CN111652317A (en) | Hyper-parameter image segmentation method based on Bayesian deep learning | |
| CN110705613B (en) | Object classification method | |
| CN113326735B (en) | YOLOv 5-based multi-mode small target detection method | |
| CN106780546A (en) | The personal identification method of the motion blur encoded point based on convolutional neural networks | |
| CN110084284A (en) | Target detection and secondary classification algorithm and device based on region convolutional neural networks | |
| Fang et al. | Laser stripe image denoising using convolutional autoencoder | |
| CN116385958A (en) | Edge intelligent detection method for power grid inspection and monitoring | |
| CN114332473A (en) | Object detection method, object detection device, computer equipment, storage medium and program product | |
| CN112668672A (en) | TensorRT-based target detection model acceleration method and device | |
| CN115171183A (en) | Mask face detection method based on improved yolov5 | |
| Kim et al. | Fruit tree disease classification system using generative adversarial networks | |
| Xing et al. | Multi-level adaptive perception guidance based infrared and visible image fusion | |
| CN112668662B (en) | Outdoor mountain forest environment target detection method based on improved YOLOv3 network | |
| CN110348311B (en) | Deep learning-based road intersection identification system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |