CN111897984A - Picture labeling method and device, terminal equipment and storage medium - Google Patents
Picture labeling method and device, terminal equipment and storage medium Download PDFInfo
- Publication number
- CN111897984A CN111897984A CN202010468139.4A CN202010468139A CN111897984A CN 111897984 A CN111897984 A CN 111897984A CN 202010468139 A CN202010468139 A CN 202010468139A CN 111897984 A CN111897984 A CN 111897984A
- Authority
- CN
- China
- Prior art keywords
- picture
- roi
- index
- list
- similarity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/55—Clustering; Classification
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
- G06F16/51—Indexing; Data structures therefor; Storage structures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/213—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
- G06F18/2135—Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/22—Matching criteria, e.g. proximity measures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Abstract
The invention discloses a picture marking method, a picture marking device, terminal equipment and a storage medium, wherein the method comprises the following steps: performing target detection on each acquired picture to be marked through a pre-trained target detection model to obtain an ROI picture and ROI position information corresponding to each picture to be marked; classifying all ROI pictures according to the similarity between every two ROI pictures to obtain a plurality of ROI picture sets; for each ROI picture set, taking the label of a target picture, of which the similarity with any picture in the ROI picture set meets a preset condition, in a preset picture set as the label of all ROI pictures in the ROI picture set; and for each picture to be labeled, labeling the picture to be labeled according to the label of the ROI picture corresponding to the picture to be labeled and the ROI position information to generate a labeled picture. By adopting the embodiment of the invention, the automatic marking of the picture data can be realized.
Description
Technical Field
The invention relates to the technical field of computers, in particular to a picture marking method and device, terminal equipment and a storage medium.
Background
In the field of computer vision, picture data serving as the most important production raw material plays an important role in the precision of neural network models in various deep learning. However, disordered image data can not be directly used for the neural network in most cases, and a large amount of marked and accurate image data can train the high-precision neural network.
At present, the image data required by training the neural network basically still adopts a manual labeling mode. The inventor finds that the following technical problems exist in the prior art in the process of implementing the invention: the image marking is carried out by adopting a manual marking mode, the time consumption is long, the efficiency is low, the labor cost is high, compared with the situation that a deep learning model frequently requires dozens of or even millions of marking data, the image marking sometimes becomes the biggest bottleneck restricting the rapid development of the deep learning, particularly in the fast-consumption field, a client often requires to carry out rapid modeling on a specific commodity, if the model learning in 1 day exceeds 200 categories, the rapid modeling can not be realized basically if the traditional manual marking is still used.
Disclosure of Invention
The embodiment of the invention provides a picture marking method, a picture marking device, terminal equipment and a storage medium, which can effectively solve the problems of low picture marking efficiency and long time consumption caused by a manual marking mode in the prior art.
An embodiment of the present invention provides a method for labeling a picture, including:
acquiring all pictures to be marked;
performing target detection on each picture to be marked through a pre-trained target detection model to obtain an ROI picture and ROI position information corresponding to each picture to be marked;
classifying all the ROI pictures according to the similarity between every two ROI pictures to obtain a plurality of ROI picture sets;
for each ROI picture set, if a target picture with the similarity meeting a preset condition with any picture in the ROI picture set exists in a preset picture set, taking the label of the target picture as the label of all ROI pictures in the ROI picture set;
and for each picture to be labeled, if the situation that the label of the ROI picture corresponding to the picture to be labeled exists is judged, labeling the picture to be labeled according to the label of the ROI picture corresponding to the picture to be labeled and the ROI position information, and generating the labeled picture.
As an improvement of the above scheme, the classifying all the ROI pictures according to the similarity between every two ROI pictures to obtain a plurality of ROI picture sets specifically includes:
acquiring a feature vector of each ROI picture;
and classifying all the ROI pictures according to the similarity between the feature vectors of every two ROI pictures to obtain a plurality of ROI picture sets.
As an improvement of the above scheme, the obtaining the feature vector of each ROI picture specifically includes:
extracting features of each ROI picture to obtain a high-dimensional feature vector of each ROI picture;
and carrying out dimensionality reduction on the high-dimensional feature vector of each ROI picture to obtain the feature vector of each ROI picture.
As an improvement of the above scheme, the classifying all the ROI pictures according to the similarity between the feature vectors of each two ROI pictures to obtain a plurality of ROI picture sets specifically includes:
step S01: for each ROI picture, establishing an index of the ROI picture according to the feature vector of the ROI picture;
step S02: establishing an index list according to indexes of all the ROI pictures;
step S03: calculating the similarity between every two indexes in the index list;
step S04: for each index in the index list, searching M indexes with the highest similarity to the index in the index list to obtain a similar index set, deleting indexes with the similarity to the index smaller than K in the similar index set to obtain an updated similar index set, and constructing a sub-list according to the updated similar index set and the index; wherein K is a preset similarity threshold value, and M is a positive integer;
step S05: combining the obtained sub-lists to generate a total list;
step S06: merging the sub-lists with intersection in the general list to obtain a final general list;
step S07: judging whether n is equal to a preset clustering frequency, if so, entering step S08, otherwise, making K equal to K/2, making n equal to n +1, taking the final total list as the index list, and returning to step S04; wherein the initial value of n is 1;
step S08: and classifying all the ROI pictures according to the final total list to obtain a plurality of ROI picture sets.
Accordingly, another embodiment of the present invention provides an image annotation device, including:
the image to be marked acquisition module is used for acquiring all images to be marked;
the target detection module is used for carrying out target detection on each picture to be marked through a pre-trained target detection model to obtain an ROI picture and ROI position information corresponding to each picture to be marked;
the ROI image classification module is used for classifying all the ROI images according to the similarity between every two ROI images to obtain a plurality of ROI image sets;
the image label acquisition module is used for taking the label of the target image as the label of all ROI images in the ROI image set if the target image with the similarity meeting the preset condition with any one image in the ROI image set exists in the preset image set;
and the marking data generation module is used for marking the picture to be marked according to the label of the ROI picture corresponding to the picture to be marked and the ROI position information if the label of the ROI picture corresponding to the picture to be marked is judged to exist in each picture to be marked, and generating a marked picture.
As an improvement of the above scheme, the ROI picture classification module specifically includes:
the feature vector acquisition sub-module is used for acquiring a feature vector of each ROI picture;
and the image set acquisition sub-module is used for classifying all the ROI images according to the similarity between the feature vectors of every two ROI images to obtain a plurality of ROI image sets.
As an improvement of the above scheme, the feature vector obtaining sub-module specifically includes:
the characteristic extraction unit is used for extracting the characteristics of each ROI picture to obtain a high-dimensional characteristic vector of each ROI picture;
and the feature dimension reduction unit is used for carrying out dimension reduction processing on the high-dimensional feature vector of each ROI picture to obtain the feature vector of each ROI picture.
As an improvement of the above scheme, the picture set obtaining sub-module specifically includes:
the image index establishing unit is used for establishing an index of the ROI image according to the feature vector of the ROI image and triggering the index list establishing unit for each ROI image;
the index list establishing unit is used for establishing an index list according to the indexes of all the ROI pictures and triggering an index similarity calculating unit;
the index similarity calculation unit is used for calculating the similarity between every two indexes in the index list and triggering the sub-list construction unit;
the sub-list building unit is used for searching M indexes with the highest similarity to each index in the index list to obtain a similar index set, deleting the indexes with the similarity to the index smaller than K in the similar index set to obtain an updated similar index set, and building a sub-list according to the updated similar index set and the index; wherein K is a preset similarity threshold value, and M is a positive integer;
the sub-list combining unit is used for combining the obtained sub-lists to generate a total list and triggering the sub-list combining unit;
the sub-list merging unit is used for merging the sub-lists with intersection in the general list to obtain a final general list and triggering the judgment unit;
the judging unit is used for judging whether n is equal to a preset clustering frequency, if so, triggering the image classifying unit, if not, enabling K to be K/2 and n to be n +1, taking the final total list as the index list, and triggering the sublist constructing unit; wherein the initial value of n is 1;
and the picture classification unit is used for classifying all the ROI pictures according to the final total list to obtain a plurality of ROI picture sets.
Another embodiment of the present invention provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, the processor implements the picture annotation method as described in any one of the above.
Another embodiment of the present invention provides a computer-readable storage medium, where the computer-readable storage medium includes a stored computer program, where when the computer program runs, the computer-readable storage medium controls a device to execute the image annotation method according to any one of the above items.
Compared with the prior art, the image labeling method, the image labeling device, the terminal device and the storage medium disclosed by the embodiment of the invention have the advantages that firstly, target detection is carried out on each acquired image to be labeled through a pre-trained target detection model, and ROI images and ROI position information corresponding to each image to be labeled are obtained; classifying all the ROI pictures according to the similarity between every two ROI pictures to obtain a plurality of ROI picture sets; then, for each ROI picture set, if a target picture with the similarity meeting a preset condition with any picture in the ROI picture set exists in a preset picture set, taking the label of the target picture as the label of all ROI pictures in the ROI picture set; and then, for each picture to be labeled, if the situation that the label of the ROI picture corresponding to the picture to be labeled exists is judged, labeling the picture to be labeled according to the label of the ROI picture corresponding to the picture to be labeled and the ROI position information, and generating a labeled picture. The embodiment of the invention can realize automatic marking of the picture data, thereby effectively solving the problems of low picture marking efficiency and long time consumption caused by a manual marking mode in the prior art, and further improving the picture marking efficiency.
Drawings
Fig. 1 is a schematic flow chart of a picture labeling method according to an embodiment of the present invention.
Fig. 2 is a schematic structural diagram of a picture labeling apparatus according to an embodiment of the present invention.
Fig. 3 is a schematic structural diagram of a terminal device according to an embodiment of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Referring to fig. 1, which is a schematic flow chart of a picture labeling method according to an embodiment of the present invention, the method includes:
s11, acquiring all pictures to be labeled;
s12, performing target detection on each picture to be labeled through a pre-trained target detection model to obtain an ROI picture and ROI position information corresponding to each picture to be labeled;
s13, classifying all the ROI pictures according to the similarity between every two ROI pictures to obtain a plurality of ROI picture sets;
s14, for each ROI picture set, if a target picture with the similarity meeting a preset condition with any picture in the ROI picture set exists in a preset picture set, taking the label of the target picture as the label of all ROI pictures in the ROI picture set;
and S15, for each picture to be labeled, if the situation that the label of the ROI picture corresponding to the picture to be labeled exists is judged, labeling the picture to be labeled according to the label of the ROI picture corresponding to the picture to be labeled and the ROI position information, and generating a labeled picture.
In order to facilitate understanding of the present embodiment, a process of labeling a picture may specifically be illustrated as follows:
firstly, training a target detection model without distinguishing categories according to previously labeled picture data or an open data set in advance to obtain a pre-established target detection model, and labeling a plurality of pictures corresponding to the categories to be labeled according to the names of the categories to be labeled in advance to establish a preset picture set; as can be understood, each picture in the preset picture set is labeled with a corresponding type of label; then, when the pictures are marked, all the pictures to be marked are obtained; then, performing target detection on each picture to be marked through a pre-trained target detection model, outputting position information of a Region of Interest (ROI) corresponding to each picture to be marked by the target detection model, cutting the ROI of each picture to be marked into ROI pictures according to the position information of the ROI corresponding to each picture to be marked, and simultaneously keeping each original picture to be marked; then, calculating the similarity between every two ROI pictures, and classifying all the ROI pictures according to the similarity between every two ROI pictures, so as to divide similar ROI pictures together to obtain a plurality of ROI picture sets, wherein the classification mode of all the ROI pictures is various and is not limited herein, for example, two ROI pictures with the similarity between every two pictures larger than a certain threshold value can be used as a set, so as to classify all the ROI pictures into a plurality of ROI picture sets, or a plurality of pictures which are similar to each other can be used as a set, so as to classify all the ROI pictures into a plurality of ROI picture sets; then, for each ROI picture set, randomly selecting one of the ROI pictures, for example, selecting a first ROI picture, calculating similarity between the first ROI picture and each picture in a preset picture set, and determining whether a target picture whose similarity with the first ROI picture meets a preset condition exists in the preset picture set, if so, indicating that types of all ROI pictures in the ROI picture set are the same as those of the target picture, so that the target picture label can be used as a label of all ROI pictures in the ROI picture set, and if not, the ROI picture set can be stored in another region first for subsequent inspection or directly discarded; the preset condition may be set according to an actual situation, and is not limited herein, for example, the similarity may be greater than a certain threshold, or the similarity may be the highest; finally, for each picture to be marked, judging whether a label of the ROI picture corresponding to the picture to be marked exists or not, if so, indicating that the label of the picture to be marked is obtained, and therefore, marking the picture to be marked according to the label of the ROI picture corresponding to the picture to be marked and the position information of the ROI can be performed to generate a marked picture, and after marking is completed, the marked picture can be moved to a folder of a corresponding type, so that subsequent data sorting work is reduced, and if not, indicating that the label of the picture to be marked is not obtained, so that the picture to be marked can be stored in another area and is subjected to subsequent inspection or directly discarded; the format of the generated labeled picture can be common xml, json, csv, txt or the like.
In one embodiment, through testing, compared with manual labeling, by using the technical scheme provided by the embodiment of the invention, about 1 hour is required for labeling 15000 frames, and about 20 hours is required for manual labeling, so that the labeling time is greatly shortened.
Firstly, performing target detection on each acquired picture to be marked through a pre-trained target detection model to obtain an ROI picture and ROI position information corresponding to each picture to be marked; classifying all the ROI pictures according to the similarity between every two ROI pictures to obtain a plurality of ROI picture sets; then, for each ROI picture set, if a target picture with the similarity meeting a preset condition with any picture in the ROI picture set exists in a preset picture set, taking the label of the target picture as the label of all ROI pictures in the ROI picture set; and then, for each picture to be labeled, if the situation that the label of the ROI picture corresponding to the picture to be labeled exists is judged, labeling the picture to be labeled according to the label of the ROI picture corresponding to the picture to be labeled and the ROI position information, and generating a labeled picture. The embodiment of the invention can realize automatic marking of the picture data, thereby effectively solving the problems of low picture marking efficiency and long time consumption caused by a manual marking mode in the prior art, and further improving the picture marking efficiency.
As an optional embodiment, the step S13 specifically includes:
s131, acquiring a feature vector of each ROI picture.
The types of the feature vectors include, but are not limited to, SIFT feature points, SURF feature points, AKAZE feature points, global or regional pixel or color histograms, neural network classification model outputs.
S132, classifying all the ROI pictures according to the similarity between the feature vectors of every two ROI pictures to obtain a plurality of ROI picture sets.
In the embodiment, the feature vectors of each ROI picture are extracted, and the similarity between the feature vectors of each two ROI pictures is used as the similarity between each two ROI pictures, so that all the ROI pictures are classified to obtain a plurality of ROI picture sets, the accuracy between the similarities of each two ROI pictures can be ensured, the classification accuracy of the ROI pictures is ensured, and the accuracy of picture labeling is further ensured.
Further, the step S131 specifically includes:
s1311, feature extraction is conducted on each ROI picture, and high-dimensional feature vectors of each ROI picture are obtained.
In this case, a Scale-invariant feature transform (SIFT) or a neural network model may be used to perform feature extraction on each ROI picture, so as to obtain a high-dimensional feature vector of each ROI picture.
S1312, performing dimensionality reduction on the high-dimensional feature vector of each ROI picture to obtain the feature vector of each ROI picture.
The length of the high-dimensional feature vector generally obtained by feature extraction is longer, so that (2) PCA (Principal component analysis) dimension reduction or quantization may be performed on the high-dimensional feature vector of each ROI picture according to different feature extraction methods, so as to reduce the length of the feature vector of each ROI picture to below 5000, and optionally, the length of the feature vector used in the present invention is 64 or 2048.
In this embodiment, the extracted high-dimensional feature vector of each ROI picture is subjected to dimension reduction processing, so as to obtain the feature vector of each ROI picture, which can reduce the complexity of subsequent similarity calculation, thereby improving the efficiency of picture labeling.
Further, the step S132 specifically includes:
step S01: for each ROI picture, establishing an index of the ROI picture according to the feature vector of the ROI picture, and entering step S02;
step S02: establishing an index list according to the indexes of all the ROI pictures, and entering a step S03;
step S03: calculating the similarity between every two indexes in the index list, and entering step S04;
step S04: for each index in the index list, searching M indexes with the highest similarity to the index in the index list to obtain a similar index set, deleting indexes with the similarity to the index smaller than K in the similar index set to obtain an updated similar index set, constructing a sub-list according to the updated similar index set and the index, and entering step S05; wherein K is a preset similarity threshold value, and M is a positive integer;
step S05: combining the obtained sub-lists to generate a total list, and entering step S06;
step S06: merging the sub-lists with intersection in the total list to obtain a final total list, and entering step S07;
step S07: judging whether n is equal to a preset clustering frequency, if so, entering step S08, otherwise, making K equal to K/2, making n equal to n +1, taking the final total list as the index list, and returning to step S04; wherein the initial value of n is 1;
step S08: and classifying all the ROI pictures according to the final total list to obtain a plurality of ROI picture sets.
Combining the feature vectors of each ROI picture obtained in step S131 into indexes by bars, establishing an index list, calculating similarity between every two indexes in the index list according to euclidean distance or cosine distance, searching each index in the index list, searching M indexes with the highest similarity to the index to obtain a similar index set, filtering out indexes with the similarity to the index being smaller than a preset similarity threshold in the similar index set according to the preset similarity threshold to obtain an updated similar index set, combining each index in the updated similar index set and the index into a two-dimensional list to obtain a sub-list corresponding to the index, and thus obtaining a sub-list corresponding to each index; where M is a positive integer, and may be determined according to the actual number of pictures in specific implementation, which is not limited herein. After obtaining the sub-lists corresponding to each index, combining the obtained sub-lists to generate a total list, and solving an union set of all sub-lists with intersections in the sub-lists in the total list, wherein the specific algorithm is as follows: (1) starting from the first sub-list of the total list, searching whether the intersection exists between the total list and the rest sub-lists, if so, taking a union set, if not, adding the sub-list to a list to be processed, and continuing to go down until all sub-lists are processed; (2) and (4) re-executing the step (1) on the to-be-processed list until no sub-list which can be merged exists. And obtaining a final list according to the merging algorithm, then taking the final list as an index list, repeating the steps S04 to S06 for N times, reducing the preset similarity threshold value in the step S04 by one time when repeating the steps each time, obtaining the final list after repeating the steps N times, searching ROI pictures corresponding to all indexes contained in each sub-list in the final list, and moving the ROI pictures to a unified folder, thereby obtaining each ROI picture set. N is a preset clustering frequency, and in specific implementation, N may be determined according to the type and number of the pictures, which is not limited herein.
In the embodiment, all the ROI pictures are classified in an indexing mode, the data retrieval speed can be effectively improved, the picture labeling efficiency is improved, multiple iterations are performed in the classification process, the picture classification accuracy can be effectively improved, and the picture labeling accuracy is improved.
Fig. 2 is a schematic structural diagram of a picture labeling apparatus according to an embodiment of the present invention.
The image labeling device provided by the embodiment of the invention comprises:
the image to be marked acquisition module 21 is used for acquiring all images to be marked;
the target detection module 22 is configured to perform target detection on each picture to be labeled through a pre-trained target detection model, so as to obtain an ROI picture and ROI position information corresponding to each picture to be labeled;
the ROI image classification module 23 is configured to classify all the ROI images according to the similarity between every two ROI images to obtain a plurality of ROI image sets;
the image tag obtaining module 24 is configured to, for each ROI image set, if it is determined that a target image exists in a preset image set, where similarity between the target image and any one of the images in the ROI image set meets a preset condition, use a tag of the target image as a tag of all the ROI images in the ROI image set;
and the labeling data generating module 25 is configured to label, for each picture to be labeled, the picture to be labeled according to the label of the ROI picture corresponding to the picture to be labeled and the ROI position information if it is determined that the label of the ROI picture corresponding to the picture to be labeled exists, and generate a labeled picture.
The principle of implementing image annotation by the image annotation device provided by the embodiment of the invention is consistent with that of the above method embodiment, and is not described herein again.
Firstly, performing target detection on each acquired picture to be marked through a pre-trained target detection model to obtain an ROI picture and ROI position information corresponding to each picture to be marked; classifying all the ROI pictures according to the similarity between every two ROI pictures to obtain a plurality of ROI picture sets; then, for each ROI picture set, if a target picture with the similarity meeting a preset condition with any picture in the ROI picture set exists in a preset picture set, taking the label of the target picture as the label of all ROI pictures in the ROI picture set; and then, for each picture to be labeled, if the situation that the label of the ROI picture corresponding to the picture to be labeled exists is judged, labeling the picture to be labeled according to the label of the ROI picture corresponding to the picture to be labeled and the ROI position information, and generating a labeled picture. The embodiment of the invention can realize automatic marking of the picture data, thereby effectively solving the problems of low picture marking efficiency and long time consumption caused by a manual marking mode in the prior art, and further improving the picture marking efficiency.
As an optional embodiment, the ROI picture classification module specifically includes:
the feature vector acquisition sub-module is used for acquiring a feature vector of each ROI picture;
and the image set acquisition sub-module is used for classifying all the ROI images according to the similarity between the feature vectors of every two ROI images to obtain a plurality of ROI image sets.
Further, the feature vector obtaining sub-module specifically includes:
the characteristic extraction unit is used for extracting the characteristics of each ROI picture to obtain a high-dimensional characteristic vector of each ROI picture;
and the feature dimension reduction unit is used for carrying out dimension reduction processing on the high-dimensional feature vector of each ROI picture to obtain the feature vector of each ROI picture.
Further, the picture set obtaining sub-module specifically includes:
the image index establishing unit is used for establishing an index of the ROI image according to the feature vector of the ROI image and triggering the index list establishing unit for each ROI image;
the index list establishing unit is used for establishing an index list according to the indexes of all the ROI pictures and triggering an index similarity calculating unit;
the index similarity calculation unit is used for calculating the similarity between every two indexes in the index list and triggering the sub-list construction unit;
the sub-list building unit is used for searching M indexes with the highest similarity to each index in the index list to obtain a similar index set, deleting the indexes with the similarity to the index smaller than K in the similar index set to obtain an updated similar index set, and building a sub-list according to the updated similar index set and the index; wherein K is a preset similarity threshold value, and M is a positive integer;
the sub-list combining unit is used for combining the obtained sub-lists to generate a total list and triggering the sub-list combining unit;
the sub-list merging unit is used for merging the sub-lists with intersection in the general list to obtain a final general list and triggering the judgment unit;
the judging unit is used for judging whether n is equal to a preset clustering frequency, if so, triggering the image classifying unit, if not, enabling K to be K/2 and n to be n +1, taking the final total list as the index list, and triggering the sublist constructing unit; wherein the initial value of n is 1;
and the picture classification unit is used for classifying all the ROI pictures according to the final total list to obtain a plurality of ROI picture sets.
Fig. 3 is a schematic structural diagram of a terminal device according to an embodiment of the present invention.
The terminal device provided by the embodiment of the present invention includes a processor 31, a memory 32, and a computer program stored in the memory 32 and configured to be executed by the processor 31, where the processor 31 implements the picture annotation method according to any one of the above embodiments when executing the computer program.
The processor 31, when executing the computer program, implements the steps in the above-mentioned embodiment of the image annotation method, for example, all the steps of the image annotation method shown in fig. 1. Alternatively, the processor 31, when executing the computer program, implements the functions of each module/unit in the above-mentioned embodiment of the image annotation device, for example, the functions of each module of the image annotation device shown in fig. 2.
Illustratively, the computer program may be divided into one or more modules, which are stored in the memory 32 and executed by the processor 31 to accomplish the present invention. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which are used for describing the execution process of the computer program in the terminal device. For example, the computer program may be divided into a to-be-labeled picture acquiring module, a target detecting module, an ROI picture classifying module, a picture label acquiring module, and a labeled data generating module, where the specific functions of the modules are as follows: the image to be marked acquisition module is used for acquiring all images to be marked; the target detection module is used for carrying out target detection on each picture to be marked through a pre-trained target detection model to obtain an ROI picture and ROI position information corresponding to each picture to be marked; the ROI image classification module is used for classifying all the ROI images according to the similarity between every two ROI images to obtain a plurality of ROI image sets; the image label acquisition module is used for taking the label of the target image as the label of all ROI images in the ROI image set if the target image with the similarity meeting the preset condition with any one image in the ROI image set exists in the preset image set; and the marking data generation module is used for marking the picture to be marked according to the label of the ROI picture corresponding to the picture to be marked and the ROI position information if the label of the ROI picture corresponding to the picture to be marked is judged to exist in each picture to be marked, and generating a marked picture.
The terminal device can be a desktop computer, a notebook, a palm computer, a cloud server and other computing devices. The terminal device may include, but is not limited to, a processor 31, a memory 32. It will be appreciated by those skilled in the art that the schematic diagram is merely an example of a terminal device and does not constitute a limitation of a terminal device, and may include more or less components than those shown, or combine certain components, or different components, for example, the terminal device may also include input output devices, network access devices, buses, etc.
The Processor 31 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), an off-the-shelf Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may be any conventional processor or the like, and the processor 31 is a control center of the terminal device and connects various parts of the whole terminal device by using various interfaces and lines.
The memory 32 can be used for storing the computer programs and/or modules, and the processor 31 can implement various functions of the terminal device by running or executing the computer programs and/or modules stored in the memory 32 and calling the data stored in the memory 32. The memory 32 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application program required for at least one function, and the like; the storage data area may store data created according to the use of the terminal device, and the like. In addition, the memory may include high speed random access memory, and may also include non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), at least one magnetic disk storage device, a Flash memory device, or other volatile solid state storage device.
Wherein, the terminal device integrated module/unit can be stored in a computer readable storage medium if it is implemented in the form of software functional unit and sold or used as a stand-alone product. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium, and when the computer program is executed by a processor, the steps of the method embodiments may be implemented. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signals, telecommunications signals, software distribution medium, and the like.
It should be noted that the above-described device embodiments are merely illustrative, where the units described as separate parts may or may not be physically separate, and the parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. In addition, in the drawings of the embodiment of the apparatus provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, and may be specifically implemented as one or more communication buses or signal lines. One of ordinary skill in the art can understand and implement it without inventive effort.
The above description is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, several modifications and variations can be made without departing from the technical principle of the present invention, and these modifications and variations should also be regarded as the protection scope of the present invention.
Claims (10)
1. A picture marking method is characterized by comprising the following steps:
acquiring all pictures to be marked;
performing target detection on each picture to be marked through a pre-trained target detection model to obtain an ROI picture and ROI position information corresponding to each picture to be marked;
classifying all the ROI pictures according to the similarity between every two ROI pictures to obtain a plurality of ROI picture sets;
for each ROI picture set, if a target picture with the similarity meeting a preset condition with any picture in the ROI picture set exists in a preset picture set, taking the label of the target picture as the label of all ROI pictures in the ROI picture set;
and for each picture to be labeled, if the situation that the label of the ROI picture corresponding to the picture to be labeled exists is judged, labeling the picture to be labeled according to the label of the ROI picture corresponding to the picture to be labeled and the ROI position information, and generating the labeled picture.
2. The picture annotation method of claim 1, wherein the classifying all the ROI pictures according to the similarity between every two ROI pictures to obtain a plurality of ROI picture sets comprises:
acquiring a feature vector of each ROI picture;
and classifying all the ROI pictures according to the similarity between the feature vectors of every two ROI pictures to obtain a plurality of ROI picture sets.
3. The method of claim 2, wherein the obtaining the feature vector of each ROI picture comprises:
extracting features of each ROI picture to obtain a high-dimensional feature vector of each ROI picture;
and carrying out dimensionality reduction on the high-dimensional feature vector of each ROI picture to obtain the feature vector of each ROI picture.
4. The picture annotation method of claim 2, wherein the classifying all the ROI pictures according to the similarity between the feature vectors of every two ROI pictures to obtain a plurality of ROI picture sets specifically comprises:
step S01: for each ROI picture, establishing an index of the ROI picture according to the feature vector of the ROI picture;
step S02: establishing an index list according to indexes of all the ROI pictures;
step S03: calculating the similarity between every two indexes in the index list;
step S04: for each index in the index list, searching M indexes with the highest similarity to the index in the index list to obtain a similar index set, deleting indexes with the similarity to the index smaller than K in the similar index set to obtain an updated similar index set, and constructing a sub-list according to the updated similar index set and the index; wherein K is a preset similarity threshold value, and M is a positive integer;
step S05: combining the obtained sub-lists to generate a total list;
step S06: merging the sub-lists with intersection in the general list to obtain a final general list;
step S07: judging whether n is equal to a preset clustering frequency, if so, entering step S08, otherwise, making K equal to K/2, making n equal to n +1, taking the final total list as the index list, and returning to step S04; wherein the initial value of n is 1;
step S08: and classifying all the ROI pictures according to the final total list to obtain a plurality of ROI picture sets.
5. A picture labeling apparatus, comprising:
the image to be marked acquisition module is used for acquiring all images to be marked;
the target detection module is used for carrying out target detection on each picture to be marked through a pre-trained target detection model to obtain an ROI picture and ROI position information corresponding to each picture to be marked;
the ROI image classification module is used for classifying all the ROI images according to the similarity between every two ROI images to obtain a plurality of ROI image sets;
the image label acquisition module is used for taking the label of the target image as the label of all ROI images in the ROI image set if the target image with the similarity meeting the preset condition with any one image in the ROI image set exists in the preset image set;
and the marking data generation module is used for marking the picture to be marked according to the label of the ROI picture corresponding to the picture to be marked and the ROI position information if the label of the ROI picture corresponding to the picture to be marked is judged to exist in each picture to be marked, and generating a marked picture.
6. The picture labeling apparatus of claim 5, wherein the ROI picture classification module specifically comprises:
the feature vector acquisition sub-module is used for acquiring a feature vector of each ROI picture;
and the image set acquisition sub-module is used for classifying all the ROI images according to the similarity between the feature vectors of every two ROI images to obtain a plurality of ROI image sets.
7. The picture labeling apparatus of claim 6, wherein the feature vector obtaining sub-module specifically comprises:
the characteristic extraction unit is used for extracting the characteristics of each ROI picture to obtain a high-dimensional characteristic vector of each ROI picture;
and the feature dimension reduction unit is used for carrying out dimension reduction processing on the high-dimensional feature vector of each ROI picture to obtain the feature vector of each ROI picture.
8. The picture labeling apparatus of claim 6, wherein the picture set obtaining sub-module specifically comprises:
the image index establishing unit is used for establishing an index of the ROI image according to the feature vector of the ROI image and triggering the index list establishing unit for each ROI image;
the index list establishing unit is used for establishing an index list according to the indexes of all the ROI pictures and triggering an index similarity calculating unit;
the index similarity calculation unit is used for calculating the similarity between every two indexes in the index list and triggering the sub-list construction unit;
the sub-list building unit is used for searching M indexes with the highest similarity to each index in the index list to obtain a similar index set, deleting the indexes with the similarity to the index smaller than K in the similar index set to obtain an updated similar index set, and building a sub-list according to the updated similar index set and the index; wherein K is a preset similarity threshold value, and M is a positive integer;
the sub-list combining unit is used for combining the obtained sub-lists to generate a total list and triggering the sub-list combining unit;
the sub-list merging unit is used for merging the sub-lists with intersection in the general list to obtain a final general list and triggering the judgment unit;
the judging unit is used for judging whether n is equal to a preset clustering frequency, if so, triggering the image classifying unit, if not, enabling K to be K/2 and n to be n +1, taking the final total list as the index list, and triggering the sublist constructing unit; wherein the initial value of n is 1;
and the picture classification unit is used for classifying all the ROI pictures according to the final total list to obtain a plurality of ROI picture sets.
9. A terminal device comprising a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, the processor implementing the picture annotation method according to any one of claims 1 to 4 when executing the computer program.
10. A computer-readable storage medium, comprising a stored computer program, wherein when the computer program runs, the computer-readable storage medium controls a device to execute the picture annotation method according to any one of claims 1 to 4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010468139.4A CN111897984B (en) | 2020-05-28 | 2020-05-28 | Picture labeling method and device, terminal equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010468139.4A CN111897984B (en) | 2020-05-28 | 2020-05-28 | Picture labeling method and device, terminal equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111897984A true CN111897984A (en) | 2020-11-06 |
| CN111897984B CN111897984B (en) | 2022-06-10 |
Family
ID=73206556
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010468139.4A Active CN111897984B (en) | 2020-05-28 | 2020-05-28 | Picture labeling method and device, terminal equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111897984B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150356740A1 (en) * | 2014-06-05 | 2015-12-10 | Xerox Corporation | System for automated text and halftone segmentation |
| CN106339479A (en) * | 2016-08-30 | 2017-01-18 | 深圳市金立通信设备有限公司 | Picture naming method and terminal |
| CN108062377A (en) * | 2017-12-12 | 2018-05-22 | 百度在线网络技术（北京）有限公司 | The foundation of label picture collection, definite method, apparatus, equipment and the medium of label |
| CN108681731A (en) * | 2018-04-03 | 2018-10-19 | 中山大学 | A kind of thyroid cancer ultrasound picture automatic marking method and system |
| CN109492698A (en) * | 2018-11-20 | 2019-03-19 | 腾讯科技（深圳）有限公司 | A kind of method of model training, the method for object detection and relevant apparatus |
| CN110033018A (en) * | 2019-03-06 | 2019-07-19 | 平安科技（深圳）有限公司 | Shape similarity judgment method, device and computer readable storage medium |
| CN110413839A (en) * | 2019-07-05 | 2019-11-05 | 高新兴科技集团股份有限公司 | A kind of label data sharing method, device and the equipment of image capture device |
-
2020
- 2020-05-28 CN CN202010468139.4A patent/CN111897984B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20150356740A1 (en) * | 2014-06-05 | 2015-12-10 | Xerox Corporation | System for automated text and halftone segmentation |
| CN106339479A (en) * | 2016-08-30 | 2017-01-18 | 深圳市金立通信设备有限公司 | Picture naming method and terminal |
| CN108062377A (en) * | 2017-12-12 | 2018-05-22 | 百度在线网络技术（北京）有限公司 | The foundation of label picture collection, definite method, apparatus, equipment and the medium of label |
| CN108681731A (en) * | 2018-04-03 | 2018-10-19 | 中山大学 | A kind of thyroid cancer ultrasound picture automatic marking method and system |
| CN109492698A (en) * | 2018-11-20 | 2019-03-19 | 腾讯科技（深圳）有限公司 | A kind of method of model training, the method for object detection and relevant apparatus |
| CN110033018A (en) * | 2019-03-06 | 2019-07-19 | 平安科技（深圳）有限公司 | Shape similarity judgment method, device and computer readable storage medium |
| CN110413839A (en) * | 2019-07-05 | 2019-11-05 | 高新兴科技集团股份有限公司 | A kind of label data sharing method, device and the equipment of image capture device |
Non-Patent Citations (1)
| Title |
|---|
| 厉智等: "基于深度卷积神经网络的服装图像分类检索算法", 《计算机工程》 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111897984B (en) | 2022-06-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107944020B (en) | Face image searching method and device, computer device and storage medium | |
| CN110458245B (en) | Multi-label classification model training method, data processing method and device | |
| CN105144239B (en) | Image processing apparatus, image processing method | |
| US9418283B1 (en) | Image processing using multiple aspect ratios | |
| CN109886928B (en) | Target cell marking method, device, storage medium and terminal equipment | |
| US10949702B2 (en) | System and a method for semantic level image retrieval | |
| CN108205580B (en) | Image retrieval method and device and computer readable storage medium | |
| Guo et al. | Local directional derivative pattern for rotation invariant texture classification | |
| CN109117773B (en) | Image feature point detection method, terminal device and storage medium | |
| Wang et al. | Logo information recognition in large-scale social media data | |
| CN113221918B (en) | Target detection method, training method and device of target detection model | |
| Zhang et al. | Image annotation based on feature fusion and semantic similarity | |
| CN111695453A (en) | Drawing book identification method and device and robot | |
| CN112329810B (en) | Image recognition model training method and device based on significance detection | |
| CN111897984B (en) | Picture labeling method and device, terminal equipment and storage medium | |
| CN108229498B (en) | Zipper piece identification method, device and equipment | |
| CN110334731B (en) | Spectral image spatial information extraction method and device and electronic equipment | |
| CN112560463B (en) | Text multi-labeling method, device, equipment and storage medium | |
| Boulton et al. | Artistic domain generalisation methods are limited by their deep representations | |
| CN110287943B (en) | Image object recognition method and device, electronic equipment and storage medium | |
| CN112819953A (en) | Three-dimensional reconstruction method, network model training method and device and electronic equipment | |
| CN111930935A (en) | Image classification method, device, equipment and storage medium | |
| CN112765394A (en) | Data processing method and device, electronic equipment and storage medium | |
| Chaitra et al. | Flower classification in videos: a HOG-PCA-NN method | |
| Chamasemani et al. | Region-based surveillance video retrieval with effective object representation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |