CN114429577A - Flag detection method, system and equipment based on high beacon strategy - Google Patents
Flag detection method, system and equipment based on high beacon strategy Download PDFInfo
- Publication number
- CN114429577A CN114429577A CN202210101441.5A CN202210101441A CN114429577A CN 114429577 A CN114429577 A CN 114429577A CN 202210101441 A CN202210101441 A CN 202210101441A CN 114429577 A CN114429577 A CN 114429577A
- Authority
- CN
- China
- Prior art keywords
- flag
- marking
- picture
- target
- core
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 120
- 238000012549 training Methods 0.000 claims abstract description 52
- 238000000034 method Methods 0.000 claims abstract description 47
- 238000002372 labelling Methods 0.000 claims description 33
- 239000011159 matrix material Substances 0.000 claims description 30
- 239000013598 vector Substances 0.000 claims description 19
- 238000012545 processing Methods 0.000 claims description 13
- 230000009193 crawling Effects 0.000 claims description 8
- 238000003064 k means clustering Methods 0.000 claims description 6
- 238000013507 mapping Methods 0.000 claims description 6
- 238000007781 pre-processing Methods 0.000 claims description 6
- 238000004590 computer program Methods 0.000 claims description 5
- 230000006870 function Effects 0.000 claims description 4
- 230000002159 abnormal effect Effects 0.000 claims description 3
- 230000003321 amplification Effects 0.000 claims description 3
- 238000000605 extraction Methods 0.000 claims description 3
- 238000003199 nucleic acid amplification method Methods 0.000 claims description 3
- 238000005457 optimization Methods 0.000 claims description 3
- 238000012216 screening Methods 0.000 claims description 3
- 238000010008 shearing Methods 0.000 claims description 3
- 238000012795 verification Methods 0.000 claims description 3
- 230000003014 reinforcing effect Effects 0.000 claims description 2
- 239000003550 marker Substances 0.000 claims 2
- 230000007306 turnover Effects 0.000 description 3
- 238000010586 diagram Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000010276 construction Methods 0.000 description 1
- 230000002708 enhancing effect Effects 0.000 description 1
- 230000003203 everyday effect Effects 0.000 description 1
- 238000003062 neural network model Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/10—Pre-processing; Data cleansing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
- G06F18/232—Non-hierarchical techniques
- G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
- G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Computational Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Computational Linguistics (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Probability & Statistics with Applications (AREA)
- Image Analysis (AREA)
Abstract
A flag detection method, a system and equipment based on a high beacon strategy detect flags in pictures through core characteristic information of different target flags. Firstly, collecting related flag pictures; secondly, expanding unbalanced flag category pictures; thirdly, constructing a core characteristic marking standard; fourthly, constructing a complete data set; fifthly, training a supervised flag detection model by using the constructed data set; and finally, the detection model identifies the flag in the unknown picture. According to the method, the core characteristic marking criterion is utilized to mark the core characteristic information of the target flag, the core characteristic information of the target flag is better obtained, the confidence of a marked sample is improved, the detection capability of a model on different flag picture categories is improved, the identification of a shielding flag and a deformation flag in a picture is better solved, meanwhile, the problem of unbalanced data category is processed by an effective data enhancement method, and the method has the advantages of high recall, strong robustness, high efficiency and the like.
Description
Technical Field
The invention relates to the field of target detection, in particular to a flag detection method, a system and equipment based on a high beacon strategy.
Background
Flag detection refers to a technique of detecting the position and category of a target flag from one picture. The flag detection technology is widely applied to the field of image content auditing, and a user uploads a large number of pictures every day on each large Internet platform, wherein the pictures may contain some sensitive flags. Flag detection belongs to the field of target detection, the requirement on original training data is high due to the characteristics of supervised learning of the flag detection, and the marking work is used as the early-stage basis of the target detection, so that the marking quality directly influences the effect of the target detection. At present, the data labeling criterion in the target detection task is mainly to label the target object globally, that is, to label the whole range of the target as much as possible by using a rectangular frame. However, unlike other target detection tasks, flags belong to non-rigid (non-rigid) targets, and have typical deformation characteristics; meanwhile, the flag detection task is often accompanied by problems such as shielding. Therefore, the global labeling of the target flag cannot sufficiently extract useful discrimination information, and even a large amount of noise information is introduced, which brings difficulty to flag detection. Therefore, a new flag detection method is needed.
At present, the prior art provides a flag detection method for detecting a flag in a video stream of a camera, which mainly includes: firstly, enhancing an original flag data set by utilizing a plurality of effective data enhancement methods; then, target detection is carried out in the first detection branch by combining an Opticalflow and GMM method; meanwhile, in a second detection branch, the video frame of the expanded data set is input as the input of a Darknet53 backbone network to extract the feature map layer of the multi-scaling video frame, and then a sample selection algorithm is adopted to select positive and negative samples to train a YOLOv3 deep neural network model and target detection; and finally, combining the detection results of the two detection branches to detect whether the flag exists in the video stream of the camera.
According to the flag detection method, the target flag is still marked by using a traditional global marking mode, core characteristics of different flags are not considered, useful judgment information cannot be fully extracted, meanwhile, a lot of noise information is introduced by using the global marking mode, so that the confidence of a marking sample is low, and the detection of the target flag is difficult particularly in scenes such as flag shielding and deformation.
Disclosure of Invention
The invention aims to provide a flag detection method, a system and equipment based on a high-confidence labeling strategy to solve the problems.
In order to achieve the purpose, the invention adopts the following technical scheme:
a flag detection method based on a high beacon strategy comprises the following steps:
step 1, acquiring a flag picture: crawling k different types of flag pictures by using an API (application programming interface) interface by taking an internet media website as a data source to obtain a picture data set consisting of n flag pictures
Step 2, reinforcing unbalanced sample data: for flag types with the sample quantity less than 100, performing data enhancement operation, generating extended samples with the number similar to that of flag samples of other types, and adding the extended samples into a picture data set;
step 3, labeling the high-confidence sample: determining the core characteristics of each type of flags for the collected k flags aiming at the occlusion and deformation of the flags in the picture, and determining the core characteristic marking standard aiming at the core characteristics;
step 4, marking the core characteristics of the target flag: marking the core characteristic region of the target flag and the flag category of the target flag according to the high-confidence sample marking strategy in the step 3 by using a marking tool to obtain a label vector Y of the picturei＝{ai,bi,wi,hiC }, wherein ai，biMarking the center point coordinates of the region, w, for the core featuresi，hiMarking the width and the height of a region for the core feature, wherein c is the flag category to which the core feature region belongs; adding the label vectors of all pictures into the data set in the step 2, and according to the following steps of 8: 2, dividing a training set verification set in proportion to construct a complete flag detection data set;
step 5, establishing a flag detection model: extracting training samples from the data set constructed in the step 4, inputting the training samples into a YOLOv3 target detection model, and constructing and training a supervised flag detection model;
step 6, flag detection: and inputting the picture p needing to be identified into the detection model trained in the step 5, judging whether the target flag is contained or not, and determining the flag type and the flag position.
Further, in step 2, the data enhancement operation includes: firstly, analyzing the quantity proportion of different flag categories of the data collected in the step 1; and then, for the classes with the sample number less than 100, respectively carrying out color enhancement, Gaussian noise addition, double amplification, random rotation, random shearing, horizontal overturning, vertical overturning and horizontal and vertical overturning operations on the pictures, generating extended samples with the number similar to that of other flag classes, and adding the extended samples into the original data set to obtain a new data set.
Further, step 3 determines that in the flag high-confidence sample labeling: firstly, comparing all types of flags in a data set, determining a most distinctive region of each target flag as a core feature of the target flag, and if no obvious core feature exists, taking a complete flag region as the core feature; secondly, constructing a flag high-confidence sample labeling strategy: when the core features of the target flag are completely displayed, marking is carried out by using a marking frame as small as possible under the condition that the core features are completely covered, and a non-core feature area of the flag is not marked; when the core characteristics of the target flag are deformed, normal marking is carried out on the deformed area; when the core feature of the target flag is shielded, if the core feature is shielded by more than half, the marking is not performed, otherwise, the marking is performed.
Further, in step 3, the core feature labeling standard includes the position, coverage area and deformation shielding abnormal condition processing of the label.
Furthermore, in the step 4 target flag core feature marking, marking information of data is represented by using Y, and the marking information is used for marking
Middle picture piBased on the core features of flags of different categories, marking the core feature region of the target flag and the category of the flag to which the target flag belongs by using a marking tool according to the marking strategy in the step 3, and recording the center point coordinate and the width and the height of each marking region in a marking file.
Further, in the flag detection model establishment step 5, k-means clustering is carried out on the mark frame regions in all the picture mark information aiming at the training sample data set established in the step 3, so as to obtain 9 anchors with different sizes; preprocessing and cutting the pictures in the training set to 416X 416 size, and converting the pictures into an RGB three-channel image matrix X; inputting the image matrix, the anchor and the image label into a detection network model for training to obtain the flag detection network model; the constructed supervised model based on the core characteristic labeling is used for training a coefficient matrix W and mapping a data matrix X to a labeling information matrix Y, and the training mode is as follows:
in the formula IboxTo predict the regression loss between the frame and the real frame, lobjTo predict confidence loss for a box,/clsIs the classification loss between different classes.
Further, the specific training process is as follows:
(1) reading in pictures and label information, preprocessing the pictures, constructing a data matrix X of a training set, performing k-means clustering on labeling frame regions in all picture labeling information to obtain 9 anchors with different sizes, and setting a termination threshold belonging to the scope of optimization convergence;
(2) inputting an image matrix X, anchor and an image label matrix Y into a YOLOv3 detection network model, extracting input image features by using a Darknet53 feature extraction network, and generating feature maps respectively containing 13 × 13, 26 × 26 and 52 × 52 grid units by adopting a similar FPN (feature Pyramid network) structure;
(3) taking a feature map grid unit in which the center of a real flag region of an input image is located as a prediction grid unit, taking the prediction grid unit as the center to obtain a prediction frame corresponding to an anchor, and screening out the prediction frame with the maximum IOU value of the real flag region as the prediction flag region;
(4) comparing the real flag region with the predicted flag region, updating model parameters through training errors, judging whether the descending amplitude of the objective function value is less than the epsilon, and if not, returning to the step (3) to continue training; otherwise, quitting training and storing the parameter matrix W of the final detection network model. .
Further, in the flag detection of step 6, inputting the picture p to be identified into the detection model trained in step 5, determining whether a target flag appears in the picture, obtaining a prediction tag vector y of the target picture p through mapping of a supervised flag detection model coefficient matrix W, and if so, obtaining a prediction tag vector y of the target picture p
Indicating that no flag is detected from the target picture; if y ═ ai,bi,wi,hiC, c is {1,2, …, k }, which indicates that the flag of the category c is detected from the target picture, and the coordinate of the position center point of the flag is ai，biThe width and height of the flag are wi，hi。
Further, a flag detection system based on a high-confidence labeling strategy comprises:
the system comprises a flag picture acquisition module, a flag picture processing module and a flag picture processing module, wherein the flag picture acquisition module is used for crawling different types of flag pictures to obtain a picture data set consisting of n flag pictures;
the sample enhancement processing module is used for carrying out data enhancement operation on the flag type with smaller sample quantity, generating extended samples with the number similar to that of flag samples of other types and adding the extended samples into the picture data set;
the sample marking module is used for determining the core characteristics of each type of flag for the collected k types of flags according to the occlusion and deformation of the flags in the picture, and determining the core characteristic marking standard according to the core characteristics;
the target flag core feature marking module is used for marking a target flag core feature region according to a high-confidence sample marking strategy by using a marking tool and marking the flag category to which the target flag core feature region belongs to obtain a label vector of a picture, and constructing the label vector into a complete flag detection data set;
the flag detection model establishing module is used for extracting training samples from the established data set, inputting the training samples into a YOLOv3 target detection model, and establishing and training a supervised flag detection model;
and the flag detection module is used for inputting the picture p to be identified into a trained detection model, judging whether the picture p contains a target flag and determining the type and position of the flag.
Further, a computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the high beacon policy-based flag detection method when executing the computer program.
Compared with the prior art, the invention has the following technical effects:
according to the method, the core characteristic marking criterion is utilized to mark the core characteristic information of the target flag, so that the core characteristic information of the target flag is better obtained, the confidence of a marked sample is improved, the detection capability of a model on different flag picture categories is improved, the identification of a shielding flag and a deformation flag in a picture is better solved, the problem of unbalanced data category is simultaneously processed by an effective data enhancement method, and the method has the advantages of high recall, strong robustness, high efficiency and the like;
the method provides a high beacon annotation strategy, pays more attention to core characteristics of different flags, fully excavates the most distinctive characteristics in the flags, and improves the accuracy and recall rate of detection; the method effectively solves the problem through data enhancement and improves the overall recognition performance of the model.
Drawings
Fig. 1 is an overall flowchart of a flag detection method based on a high beacon strategy.
Fig. 2 is a flow chart of a data acquisition process.
Fig. 3 is a flow chart of an unbalanced sample data enhancement process.
Fig. 4 is a diagram of the core features of the flag.
FIG. 5 is a flowchart of core feature labeling standard construction and image labeling.
FIG. 6 is a flow chart of a detection model training process.
Fig. 7 is a block diagram of an embodiment of the image flag detection of the present invention.
Detailed Description
The embodiments of the present invention will be described in detail below with reference to the drawings and examples. It should be noted that the embodiments described herein are only for explaining the present invention, and are not intended to limit the present invention. Furthermore, the technical features related to the embodiments of the present invention may be combined with each other without conflict.
The specific implementation process of the method comprises a data acquisition process, an unbalanced sample data enhancement process, a high-confidence sample marking strategy determining process, a target flag core characteristic marking process, a flag detection model establishing process and a flag detection process.
The invention discloses a flag detection method based on a high beacon strategy, which comprises the following steps that 1) flag data are obtained from an internet social media platform in a keyword crawler mode; 2) analyzing the data, performing data enhancement on the class of unbalanced samples, and increasing the number of samples to improve the performance of the model; 3) implementing a high-confidence sample labeling strategy, determining the core characteristics of each type of flag, and constructing core characteristic labeling standards to process abnormal conditions including labeled positions, coverage areas, deformation shielding and the like; 4) marking the core characteristic region of the target flag according to a marking standard by using a marking tool, and marking the flag category of the target flag to generate a picture label vector to construct a complete data set; 5) constructing a supervised flag detection model based on a YOLOv3 target detection model; 6) and inputting the pictures needing to be identified into the constructed detection model to detect the target flag. According to the flag detection method disclosed by the invention, the core characteristic information of the target flag is utilized, the flag sample with high confidence level is marked, the detection capability of the model on different flag picture categories is improved, and the identification of the shielding flag and the deformation flag in the picture is better solved. Meanwhile, the problem of unbalanced data categories is solved by using an effective data enhancement method, and the method has the advantages of high recall, strong robustness, high efficiency and the like, so that the method has obvious advantages compared with other flag detection methods.
FIG. 1 is a general flowchart of a flag detection method based on a high-confidence annotation strategy according to the present invention.
Data acquisition process
The specific process of data acquisition is as follows:
(1) and crawling pictures according to the keywords related to the flag categories by using a crawler technology. During crawling, relevant flag tags such as 'flag of China', 'flag of Japan', 'flag of South Korea' and the like can be used for crawling;
(2) performing duplicate removal processing on the k types of crawled flag pictures to obtain a picture data set consisting of n flag pictures
The data acquisition process described above is illustrated in fig. 2.
Unbalanced sample data enhancement process
The data collected in step 1 is analyzed for the proportion of the number of different flag categories. For the category with smaller sample number, the image is respectively subjected to operations of color enhancement, Gaussian noise addition, double amplification, random rotation, random shearing, horizontal turnover, vertical turnover, horizontal and vertical turnover and the like, so that expansion samples with the number similar to that of other flag categories are generated and added into the original data set to obtain a new data set.
The above unbalanced sample data enhancement process is shown in fig. 3.
Determining high confidence sample annotation policy process
Determining the core characteristics of the flag: and comparing all types of flags in the data set, determining the most distinctive region of each target flag as the core feature of the target flag, and if no obvious core feature exists, taking the complete flag region as the core feature. If the united country sign in the united country flag has strong distinguishability, namely the core characteristic of the flag, and the british flag does not contain a strong distinguishability region, the whole british flag is the core characteristic. Constructing a flag high-confidence sample labeling strategy: when the core features of the target flag are completely displayed, marking is carried out by using a marking frame as small as possible under the condition that the core features are completely covered, and a non-core feature area of the flag is not marked; when the core characteristics of the target flag are deformed, normal marking is carried out on the deformed area; when the core feature of the target flag is shielded, if the core feature is shielded by more than half, the marking is not performed, otherwise, the marking is performed.
The core features of the flag are schematically shown in fig. 4.
Target flag core feature labeling process
Using Y to represent the annotation information of the data, for
Middle picture piBased on the core characteristics of different types of flags, by using a marking tool according to the stepsStep 3, marking the core characteristic region of the target flag by the marking standard, marking the category of the flag to which the core characteristic region belongs, and recording the center point coordinate and the width and the height of each marking region in a marking file; generating label vector Y of picture according to label filei＝{ai,bi,wi,hiC }, wherein ai，biAs coordinates of the center point of the labeled region, wi，hiC represents the flag type of the marked region, wherein c is 1,2, …, and k respectively corresponds to k different flag types; adding the label vectors of all pictures into the original data set, and according to the ratio of 8: 2, dividing the training set verification set, thereby constructing a complete flag detection data set.
The process of constructing the labeling standard and labeling the picture is shown in fig. 5.
Flag detection model establishing process
The flag detection model herein is constructed based on YOLOv3 detection network. In the YOLOv3, a structure similar to FPN is adopted to enhance the accuracy of small target detection, the model has 3 detection layers with different scales, and each detection layer is provided with 3 anchors with different sizes. And (4) performing k-means clustering on the labeling frame regions in all the image labeling information aiming at the training sample data set constructed in the step (4) to obtain 9 anchors with different sizes. And preprocessing and cutting the pictures in the training set to 416 multiplied by 416, converting the pictures into an RGB three-channel image matrix X, and inputting the image matrix, the anchor and the image labels into a detection network model for training to obtain the flag detection network model. The constructed supervised model based on the core characteristic labeling is used for training a coefficient matrix W and mapping a data matrix X to a labeling information matrix Y, and the training mode is as follows:
in the formula IboxTo predict the regression loss between the frame and the real frame, lobjTo predict confidence loss for a box,/clsFor the classification loss between different classes, the specific training process is：
(1) Reading in pictures and label information, preprocessing the pictures, constructing a data matrix X of a training set, and performing k-means clustering on the labeling frame regions in all the picture labeling information to obtain 9 anchors with different sizes. Meanwhile, setting a termination threshold belonging to the scope of optimization convergence;
(2) inputting the image matrix X, anchor and the image label matrix Y into a YOLOv3 detection network model, extracting input image features by using a Darknet53 feature extraction network, and generating feature maps respectively comprising 13 × 13, 26 × 26 and 52 × 52 grid units by adopting an FPN-like structure;
(3) taking a feature map grid unit in which the center of a real flag region of an input image is located as a prediction grid unit, taking the prediction grid unit as the center to obtain a prediction frame corresponding to an anchor, and screening out the prediction frame with the maximum IOU value of the real flag region as the prediction flag region;
(4) comparing the real flag region with the predicted flag region, updating model parameters through training errors, judging whether the descending amplitude of the objective function value is less than the epsilon, and if not, returning to the step (3) to continue training; otherwise, quitting training and storing the parameter matrix W of the final detection network model.
The training process of the detection model is shown in fig. 6.
Flag detection process
And (5) inputting the picture p needing to be identified into the detection model trained in the step 5, and judging whether a target flag appears in the picture. Through the mapping of the supervised flag detection model coefficient matrix W, the prediction label vector y of the target picture p can be obtained, if so, the prediction label vector y of the target picture p can be obtained
Indicating that no flag is detected from the target picture; if y ═ ai,bi,wi,hiC, c is {1,2, …, k }, which indicates that the flag of the category c is detected from the target picture, and the coordinate of the position center point of the flag is ai，biThe width and height of the flag are wi，hi。
The flag detection process described above is illustrated in fig. 7.
The following are embodiments of systems of the present invention that may be used to perform method embodiments of the present invention.
In another embodiment of the present invention, a flag detection system based on a high-confidence tagging policy is provided, which can be used to implement the above flag detection method based on the high-confidence tagging policy, and specifically, the flag detection system based on the high-confidence tagging policy includes:
the system comprises a flag picture acquisition module, a flag picture processing module and a flag picture processing module, wherein the flag picture acquisition module is used for crawling different types of flag pictures to obtain a picture data set consisting of n flag pictures;
the sample enhancement processing module is used for carrying out data enhancement operation on the flag type with smaller sample quantity, generating extended samples with the number similar to that of flag samples of other types and adding the extended samples into the picture data set;
the sample marking module is used for determining the core characteristics of each type of flag for the collected k types of flags according to the occlusion and deformation of the flags in the picture, and determining the core characteristic marking standard according to the core characteristics;
the target flag core feature marking module is used for marking a target flag core feature region according to a high-confidence sample marking strategy by using a marking tool and marking the flag category to which the target flag core feature region belongs to obtain a label vector of a picture, and constructing the label vector into a complete flag detection data set;
the flag detection model establishing module is used for extracting training samples from the established data set, inputting the training samples into a YOLOv3 target detection model, and establishing and training a supervised flag detection model;
and the flag detection module is used for inputting the picture p to be identified into a trained detection model, judging whether the picture p contains a target flag and determining the type and position of the flag.
In yet another embodiment of the present invention, a computer device is provided that includes a processor and a memory for storing a computer program comprising program instructions, the processor for executing the program instructions stored by the computer storage medium. The Processor may be a Central Processing Unit (CPU), or may be other general-purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable gate array (FPGA) or other Programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc., which is a computing core and a control core of the terminal, and is specifically adapted to load and execute one or more instructions in a computer storage medium to implement a corresponding method flow or a corresponding function; the processor according to the embodiment of the invention can be used for the operation of the flag detection method based on the high beacon strategy.
Claims (10)
1. A flag detection method based on a high beacon strategy is characterized by comprising the following steps:
step 1, acquiring a flag picture: crawling k different types of flag pictures by using an API (application programming interface) interface by taking an internet media website as a data source to obtain a picture data set consisting of n flag pictures
Step 2, reinforcing unbalanced sample data: for flag categories with the number of samples less than 100, performing data enhancement operation, generating expansion samples with the number similar to that of flag samples of other categories, and adding the expansion samples into a picture data set;
step 3, labeling the high-confidence sample: determining the core characteristics of each type of flags for the collected k flags aiming at the occlusion and deformation of the flags in the picture, and determining the core characteristic marking standard aiming at the core characteristics;
step 4, marking the core characteristics of the target flag: marking the core characteristic region of the target flag and the flag category of the target flag according to the high-confidence sample marking strategy in the step 3 by using a marking tool to obtain a label vector Y of the picturei＝{ai,bi,wi,hiC }, wherein ai，biMarking the center point coordinates of the region, w, for the core featuresi，hiMarking the width and the height of a region for the core feature, wherein c is the flag category to which the core feature region belongs; adding the label vectors of all pictures into the data set in the step 2, and according to the ratio of 8: 2, dividing a training set verification set in proportion to construct a complete flag detection data set;
step 5, establishing a flag detection model: extracting training samples from the data set constructed in the step 4, inputting the training samples into a YOLOv3 target detection model, and constructing and training a supervised flag detection model;
step 6, flag detection: and inputting the picture p needing to be identified into the detection model trained in the step 5, judging whether the target flag is contained or not, and determining the flag type and the flag position.
2. The flag detection method based on the high beacon mark strategy as claimed in claim 1, wherein in step 2, the data enhancement operation comprises: firstly, analyzing the quantity proportion of different flag categories of the data collected in the step 1; and then, for the classes with the sample number less than 100, respectively carrying out color enhancement, Gaussian noise addition, double amplification, random rotation, random shearing, horizontal overturning, vertical overturning and horizontal and vertical overturning operations on the pictures, generating extended samples with the number similar to that of other flag classes, and adding the extended samples into the original data set to obtain a new data set.
3. The method for detecting the flag based on the high-confidence annotation strategy according to claim 1, wherein in the step 3, it is determined that the flag is labeled with the high-confidence sample: firstly, comparing all types of flags in a data set, determining a most distinctive region of each target flag as a core feature of the target flag, and if no obvious core feature exists, taking a complete flag region as the core feature; secondly, constructing a flag high-confidence sample labeling strategy: when the core features of the target flag are completely displayed, marking is carried out by using a marking frame as small as possible under the condition that the core features are completely covered, and a non-core feature area of the flag is not marked; when the core characteristics of the target flag are deformed, normal marking is carried out on the deformed area; when the core feature of the target flag is shielded, if the core feature is shielded by more than half, the marking is not performed, otherwise, the marking is performed.
4. The method for detecting the flag based on the high beacon marker strategy as claimed in claim 1, wherein in step 3, the core feature marking standard includes marking position, coverage area and deformation shielding abnormal condition processing.
5. The method for detecting the flag based on the high beacon mark strategy as claimed in claim 1, wherein in the step 4 core feature mark of the target flag, the mark information of the data is represented by Y, and the step is characterized in that
Middle picture piBased on the core features of flags of different categories, marking the core feature area of the target flag and the category of the flag to which the target flag belongs by using a marking tool according to the marking strategy in the step 3, and recording the center point coordinate and the width and the height of each marked area in a marking file.
6. The flag detection method based on the high beacon marker strategy according to claim 1, characterized in that in the flag detection model establishment of step 5, k-means clustering is performed on the marking frame regions in all the picture marking information aiming at the training sample data set established in step 3, so as to obtain 9 anchors of different sizes; preprocessing and cutting the pictures in the training set to 416X 416 size, and converting the pictures into an RGB three-channel image matrix X; inputting the image matrix, the anchor and the image label into a detection network model for training to obtain the flag detection network model; the constructed supervised model based on the core characteristic labeling is used for training a coefficient matrix W and mapping a data matrix X to a labeling information matrix Y, and the training mode is as follows:
in the formula IboxTo predict the regression loss between the frame and the real frame, lobjTo predict confidence loss for a box,/clsIs the classification loss between different classes.
7. The flag detection method based on the high beacon note strategy according to claim 6, wherein the specific training process is as follows:
(1) reading in pictures and label information, preprocessing the pictures, constructing a data matrix X of a training set, performing k-means clustering on labeling frame regions in all picture labeling information to obtain 9 anchors with different sizes, and setting a termination threshold belonging to the scope of optimization convergence;
(2) inputting an image matrix X, anchor and an image label matrix Y into a YOLOv3 detection network model, extracting input image features by using a Darknet53 feature extraction network, and generating feature maps respectively containing 13 × 13, 26 × 26 and 52 × 52 grid units by adopting a similar FPN (feature Pyramid network) structure;
(3) taking a feature map grid unit in which the center of a real flag region of an input image is located as a prediction grid unit, taking the prediction grid unit as the center to obtain a prediction frame corresponding to an anchor, and screening out the prediction frame with the maximum IOU value of the real flag region as the prediction flag region;
(4) comparing the real flag region with the predicted flag region, updating model parameters through training errors, judging whether the descending amplitude of the objective function value is less than the epsilon, and if not, returning to the step (3) to continue training; otherwise, quitting training and storing the parameter matrix W of the final detection network model. .
8. The flag detection method based on high beacon mark strategy as claimed in claim 1, wherein in step 6, the flag needs to be identifiedInputting the picture p into the detection model trained in the step 5, judging whether a target flag appears in the picture, mapping a coefficient matrix W of the supervised flag detection model to obtain a prediction tag vector y of the target picture p, and if so, obtaining a prediction tag vector y of the target picture p
Indicating that no flag is detected from the target picture; if y ═ ai,bi,wi,hiC, c is {1,2, …, k }, which indicates that the flag of the category c is detected from the target picture, and the coordinate of the position center point of the flag is ai，biThe width and height of the flag are wi，hi。
9. A flag detection system based on a high-confidence labeling strategy is characterized by comprising:
the system comprises a flag picture acquisition module, a flag picture processing module and a flag picture processing module, wherein the flag picture acquisition module is used for crawling different types of flag pictures to obtain a picture data set consisting of n flag pictures;
the sample enhancement processing module is used for carrying out data enhancement operation on the flag type with smaller sample quantity, generating extended samples with the number similar to that of flag samples of other types and adding the extended samples into the picture data set;
the sample marking module is used for determining the core characteristics of each type of flag for the collected k flags according to the occlusion and deformation of the flags in the picture, and determining the core characteristic marking standard according to the core characteristics;
the target flag core feature marking module is used for marking a target flag core feature region according to a high-confidence sample marking strategy by using a marking tool and marking the flag category to which the target flag core feature region belongs to obtain a label vector of a picture, and constructing the label vector into a complete flag detection data set;
the flag detection model establishing module is used for extracting training samples from the established data set, inputting the training samples into a YOLOv3 target detection model, and establishing and training a supervised flag detection model;
and the flag detection module is used for inputting the picture p to be identified into a trained detection model, judging whether the picture p contains a target flag and determining the type and position of the flag.
10. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor when executing the computer program implements the steps of the method for flag detection based on high beacon flag policy according to any one of claims 1 to 8.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210101441.5A CN114429577B (en) | 2022-01-27 | 2022-01-27 | Flag detection method, system and equipment based on high confidence labeling strategy |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210101441.5A CN114429577B (en) | 2022-01-27 | 2022-01-27 | Flag detection method, system and equipment based on high confidence labeling strategy |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114429577A true CN114429577A (en) | 2022-05-03 |
| CN114429577B CN114429577B (en) | 2024-03-08 |
Family
ID=81313455
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210101441.5A Active CN114429577B (en) | 2022-01-27 | 2022-01-27 | Flag detection method, system and equipment based on high confidence labeling strategy |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN114429577B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116012674B (en) * | 2023-01-30 | 2023-07-18 | 人民网股份有限公司 | Flag target detection method and device |
| CN116664988A (en) * | 2023-07-24 | 2023-08-29 | 广立微(上海)技术有限公司 | Picture automatic labeling method, device, computer equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106599938A (en) * | 2016-12-30 | 2017-04-26 | 南京睿观信息科技有限公司 | Hierarchy classification method based on depth network model model sensitive flag content |
| CN111222450A (en) * | 2020-01-02 | 2020-06-02 | 广州虎牙科技有限公司 | Model training method, model training device, model live broadcast processing equipment and storage medium |
| CN112381043A (en) * | 2020-11-27 | 2021-02-19 | 华南理工大学 | Flag detection method |
| WO2021139069A1 (en) * | 2020-01-09 | 2021-07-15 | 南京信息工程大学 | General target detection method for adaptive attention guidance mechanism |
| CN113657151A (en) * | 2021-07-07 | 2021-11-16 | 东南大学 | Water traffic violation detection method based on YOLO target detection algorithm |
-
2022
- 2022-01-27 CN CN202210101441.5A patent/CN114429577B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106599938A (en) * | 2016-12-30 | 2017-04-26 | 南京睿观信息科技有限公司 | Hierarchy classification method based on depth network model model sensitive flag content |
| CN111222450A (en) * | 2020-01-02 | 2020-06-02 | 广州虎牙科技有限公司 | Model training method, model training device, model live broadcast processing equipment and storage medium |
| WO2021139069A1 (en) * | 2020-01-09 | 2021-07-15 | 南京信息工程大学 | General target detection method for adaptive attention guidance mechanism |
| CN112381043A (en) * | 2020-11-27 | 2021-02-19 | 华南理工大学 | Flag detection method |
| CN113657151A (en) * | 2021-07-07 | 2021-11-16 | 东南大学 | Water traffic violation detection method based on YOLO target detection algorithm |
Non-Patent Citations (2)
| Title |
|---|
| 帖军;宋威;尹帆;郑禄;杨欣;: "基于遮挡标记的目标检测算法", 中南民族大学学报(自然科学版), no. 03 * |
| 高星;刘剑飞;郝禄国;董琪琪;: "基于YOLOv3算法的训练集优化和检测方法的研究", 计算机工程与科学, no. 01 * |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116012674B (en) * | 2023-01-30 | 2023-07-18 | 人民网股份有限公司 | Flag target detection method and device |
| CN116664988A (en) * | 2023-07-24 | 2023-08-29 | 广立微(上海)技术有限公司 | Picture automatic labeling method, device, computer equipment and storage medium |
| CN116664988B (en) * | 2023-07-24 | 2023-11-21 | 广立微(上海)技术有限公司 | Picture automatic labeling method, device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN114429577B (en) | 2024-03-08 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Song et al. | Automated pavement crack damage detection using deep multiscale convolutional features | |
| Chen et al. | Pothole detection using location-aware convolutional neural networks | |
| Shi et al. | Automatic road crack detection using random structured forests | |
| TWI651662B (en) | Image annotation method, electronic device and non-transitory computer readable storage medium | |
| CN111179217A (en) | Attention mechanism-based remote sensing image multi-scale target detection method | |
| CN111767878B (en) | Deep learning-based traffic sign detection method and system in embedded device | |
| CN107330027B (en) | Weak supervision depth station caption detection method | |
| CN114429577B (en) | Flag detection method, system and equipment based on high confidence labeling strategy | |
| CN109919002B (en) | Yellow stop line identification method and device, computer equipment and storage medium | |
| CN111582175A (en) | High-resolution remote sensing image semantic segmentation method sharing multi-scale countermeasure characteristics | |
| Ye et al. | A two-stage real-time YOLOv2-based road marking detector with lightweight spatial transformation-invariant classification | |
| Nordeng et al. | DEBC detection with deep learning | |
| TWI405145B (en) | Pixel region-based image segmentation method, system and machine-readable storage medium | |
| CN112101386B (en) | Text detection method, device, computer equipment and storage medium | |
| Li et al. | A review of deep learning methods for pixel-level crack detection | |
| CN112749673A (en) | Method and device for intelligently extracting stock of oil storage tank based on remote sensing image | |
| CN112329559A (en) | Method for detecting homestead target based on deep convolutional neural network | |
| CN115273115A (en) | Document element labeling method and device, electronic equipment and storage medium | |
| CN106845458A (en) | A kind of rapid transit label detection method of the learning machine that transfinited based on core | |
| CN112541372B (en) | Difficult sample screening method and device | |
| CN114462469B (en) | Training method of target detection model, target detection method and related device | |
| Liu et al. | WSRD-Net: A convolutional neural network-based arbitrary-oriented wheat stripe rust detection method | |
| CN115082781A (en) | Ship image detection method and device and storage medium | |
| CN113378642B (en) | Method for detecting illegal occupation buildings in rural areas | |
| CN116843983A (en) | Pavement disease recognition method, model training method, electronic equipment and medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |