CN103324937A

CN103324937A - Method and device for labeling targets

Info

Publication number: CN103324937A
Application number: CN2012100769274A
Authority: CN
Inventors: 刘国翌; 张洪明; 王峰; 唐绍鹏; 曾炜
Original assignee: NEC China Co Ltd
Current assignee: NEC China Co Ltd
Priority date: 2012-03-21
Filing date: 2012-03-21
Publication date: 2013-09-25
Anticipated expiration: 2032-03-21
Also published as: CN103324937B

Abstract

The invention discloses a method and device for labeling targets, and belongs to the technical field of picture processing. The labeling result of the targets can be used for training a target detector. The method comprises obtaining training data, using a classifier for carrying out target detection on the training data to obtain a first target position collection, calculating the precision of the classifier, selecting the targets not meeting assigned confidence coefficient requirements from the first target position collection and output the targets to carry out false drop labeling when the precision of the classifier does not meet an assigned precision requirement, obtaining a second target position collection, combining the targets meeting the assigned confidence coefficient requirements in the first target position collection and the second target position collection to form a target labeling collection, carrying out training on the classifier according to the target labeling collection, and executing the mentioned steps repeatedly until the precision of the classifier meets the assigned precision requirement. The device comprises a target detection module, a performance testing module, a learning module, a training module and an iteration module. The method and device for labeling the targets greatly reduces the cost of manually labeling, and improves efficiency of target labeling.

Description

The method and apparatus of mark target

Technical field

The present invention relates to technical field of image processing, particularly a kind of method and apparatus that marks target.

Background technology

Along with progress and the networking of electronic information technology are popularized, people use various image capture devices in daily life more and more at large, video sensor in rig camera, Digital Video, digital camera, web camera, mobile phone camera and the Internet of Things etc. for example, thus a large amount of images and video data can be obtained efficiently.Analyze fast and intelligently these data, become public administrative departments, industry member, especially IT (Information Technology, infotech) industry and internet industry etc., even the active demand in people's daily life.

Human Detection is a vital technological approaches of these data analyses, and is widely used in a plurality of fields, mainly comprises three aspects: at first be field of content analysis, can be used for management retrieval and the mark of large batch of picture video; Next is the intelligent vehicle field, can automatically detect the front pedestrian of car and take corresponding safety practice; Be the monitoring field at last, can be used for the intelligent monitor system that support is understood based on visual pattern.

The process of human detection is exactly to detect people's existence in a given width of cloth input picture, and orients its position.The sample that existing Human Detection generally needs given a large amount of marks utilizes these sample training, obtains describing the sorter of people's outward appearance, then uses this sorter to go to detect target in the picture.Wherein, given mark sample is to mark out by hand by the people that everyone position obtains in the picture, the time-consuming again effort of the mode of this artificial mark great amount of samples.When being that scene is when installing new video camera, the video of taking for this video camera also needs again to mark sample, and re-start training to improve precision and the speed of target detection, owing to being that a large amount of samples is marked, therefore, the mode of present pure manual mark is difficult to finish the mark of this large-scale video data.

Summary of the invention

The problem that wastes time and energy in order to solve existing manual mark sample, the embodiment of the invention provides a kind of method and apparatus that marks target.Described technical scheme is as follows:

On the one hand, the embodiment of the invention provides a kind of method that marks target, comprising:

Obtain training data, use sorter that described training data is carried out target detection, obtain the first object location sets;

Calculate the precision of described sorter, judge whether the precision of described sorter reaches the accuracy requirement of appointment, when the precision of described sorter does not reach the accuracy requirement of described appointment, in described first object location sets, choose and do not reach the target output of specifying degree of confidence to require, to carry out the flase drop mark, obtain the set of the second target location;

The target and the set of described the second target location that reach described appointment degree of confidence requirement in the described first object location sets are formed the set of target mark, mark set according to described target described sorter is trained;

Repeat above-mentioned steps until the precision of described sorter reaches the accuracy requirement of described appointment.

Under a kind of embodiment, described training data is video data;

Use sorter that described training data is carried out target detection, obtain also comprising after the first object location sets:

Target in the described first object location sets is followed the tracks of the track that obtains target;

In described first object location sets, choose not reach and specify the target of degree of confidence requirement to export, to carry out the flase drop mark, obtain the set of the second target location, comprising:

In described first object location sets, choose not reach and specify the target of degree of confidence requirement to export, to carry out the flase drop mark;

Result according to the flase drop mark obtains positive sample object, according to the track of described positive sample object, the described positive sample object that does not mark in the described video data is marked out, obtains the set of the second target location.

Wherein, according to the track of described positive sample object, the described positive sample object that does not mark in the described video data is marked out, comprising:

According to the track of described positive sample object, determine described positive sample object residing each frame in described video data;

The described positive sample object that does not mark in described each frame is marked out.

Under another kind of embodiment, the described training data that obtains comprises:

When obtaining training data first, in training data, choose a training data section;

When obtaining training data first, choose a training data section when non-in the data that in described training data, were not selected, and the data amount check in the training data section chosen greater than the last time of the data amount check in this training data section of choosing.

Under another embodiment, form after the target mark gathers reaching target that described appointment degree of confidence requires and the set of described the second target location in the described first object location sets, also comprise:

To export through the described training data that mark is processed, to carry out undetected mark;

Obtain the central point of undetected mark target;

Search the target of coupling in the set of described target mark, callout box coordinate range corresponding to the target of described coupling comprises the coordinate of described central point;

Determine a callout box magnitude range according to the size of callout box corresponding to the target of described coupling;

Generate a plurality of interim callout box, the size of described interim callout box is in described callout box magnitude range, and the distance of the central point of described interim callout box and the described central point that obtains is less than default threshold value;

Use the degree of confidence of the described a plurality of interim callout box of described classifier calculated, and from described a plurality of interim callout box, choose a callout box as the callout box of described undetected target according to degree of confidence, in the frame at described undetected target place, described undetected target is marked out, and the described undetected target behind the mark is added in the set of described target mark.

Preferably, from described a plurality of interim callout box, choose a callout box as the callout box of described undetected target according to degree of confidence, comprising:

From described a plurality of interim callout box, choose the highest callout box of degree of confidence as the callout box of described undetected target.

In another embodiment, in described first object location sets, choose not reach and specify the target of degree of confidence requirement to export, comprising:

In described first object location sets, choose the target output of the ratio of appointment according to degree of confidence order from low to high.

Wherein, in described first object location sets, choose before the target output of ratio of appointment according to degree of confidence order from low to high, also comprise:

Ratio according to the described appointment of determine precision of described sorter.

Preferably, the ratio according to the described appointment of determine precision of described sorter comprises:

Calculate the ratio of described appointment according to formula H=K* (1-P);

Wherein, described H is the ratio of appointment, and described P is the precision of described sorter, and described K is the coefficient of appointment, and described K 〉=1.

Under another embodiment, calculate the precision of described sorter, comprising:

Obtain the target mark set of checking data and described checking data;

Use described sorter that described checking data is carried out target detection, obtain the set of the 3rd target location;

Common factor is got in the set of described target mark and the set of described the 3rd target location, divided by the target number in the set of described the 3rd target location, obtained the precision of described sorter with the target number among the result who gets common factor.

Under another embodiment, described method also comprises:

Using first before described sorter carries out target detection, with the initial training set described sorter is trained, and according to experimenter's operating characteristic ROC curve of described sorter, corresponding confidence value when choosing the recall rate that makes described sorter and reaching default recall rate threshold value, with the described confidence value of choosing as the confidence threshold value of described sorter so that described sorter carries out target detection according to described confidence threshold value.

Under another embodiment, judge that whether the precision of described sorter reaches the accuracy requirement of appointment, comprising:

Whether judge the precision of described sorter more than or equal to default precision threshold, if so, determine that then the precision of described sorter reaches the accuracy requirement of appointment, otherwise, determine that the precision of described sorter does not reach the accuracy requirement of described appointment; Perhaps,

The precision of judging described sorter with last when carrying out target detection the precision of sorter differ whether less than default differential threshold, if, determine that then the precision of described sorter reaches the accuracy requirement of described appointment, otherwise, determine that the precision of described sorter does not reach the accuracy requirement of described appointment.

On the other hand, the embodiment of the invention also provides a kind of device that marks target, comprising: module of target detection, performance test module, study module, training module and iteration module;

Described module of target detection is used for obtaining training data, uses sorter that described training data is carried out target detection, obtains the first object location sets;

Described performance test module for the precision of calculating described sorter, judges whether the precision of described sorter reaches the accuracy requirement of appointment;

Described study module is used for when the precision of described sorter does not reach the accuracy requirement of described appointment, chooses not reach the target output of specifying degree of confidence to require in described first object location sets, to carry out the flase drop mark, obtains the set of the second target location;

Described training module is used for described first object location sets is reached target and the set of described the second target location set composition target mark that described appointment degree of confidence requires, and set is trained described sorter according to described target mark;

Described iteration module is used for the described module of target detection of repeated trigger, performance test module, study module and training module until the precision of described sorter reaches the accuracy requirement of described appointment.

Under a kind of embodiment, described training data is video data;

Described device also comprises:

Tracking module is used for the target in the described first object location sets is followed the tracks of the track that obtains target;

Described study module is used for: choose not reach in described first object location sets and specify the target of degree of confidence requirement to export, to carry out the flase drop mark, result according to the flase drop mark obtains positive sample object, track according to described positive sample object, the described positive sample object that does not mark in the described video data is marked out, obtain the set of the second target location.

Under the another kind of embodiment, described study module comprises:

Flase drop mark unit is used for the track according to described positive sample object, determines described positive sample object residing each frame in described video data, and the described positive sample object that does not mark in described each frame is marked out.

Under another embodiment, described module of target detection comprises:

Acquiring unit, be used for when obtaining training data first, in training data, choose a training data section, when non-when obtaining training data first, choose a training data section in the data that in described training data, were not selected, and the data amount check in the training data section chosen greater than the last time of the data amount check in this training data section of choosing.

Under another embodiment, described training module also comprises:

Undetected mark unit, be used for after forming the set of target mark, to export through the described training data that mark is processed, to carry out undetected mark, obtain the central point of undetected mark target, in the set of described target mark, search the target of coupling, callout box coordinate range corresponding to the target of described coupling comprises the coordinate of described central point, determine a callout box magnitude range according to the size of callout box corresponding to the target of described coupling, generate a plurality of interim callout box, the size of described interim callout box is in described callout box magnitude range, and the distance of the central point of described interim callout box and the described central point that obtains is less than default threshold value, use the degree of confidence of the described a plurality of interim callout box of described classifier calculated, and from described a plurality of interim callout box, choose a callout box as the callout box of described undetected target according to degree of confidence, in the frame at described undetected target place, described undetected target is marked out, and the described undetected target behind the mark is added in the set of described target mark.

Wherein, described undetected mark unit is used for: choose the highest callout box of degree of confidence as the callout box of described undetected target from described a plurality of interim callout box.

Under the another kind of embodiment, described study module comprises:

Choose the unit, be used in described first object location sets, choosing according to degree of confidence order from low to high the target output of the ratio of appointment.

Further, described study module also comprises:

Determining unit was used for before the target output of the described ratio of choosing the described appointment of unit selection, according to the ratio of the described appointment of determine precision of described sorter.

Preferably, described determining unit specifically is used for: the ratio of calculating described appointment according to formula H=K* (1-P);

Under another embodiment, described performance test module comprises:

Test cell, the target mark set that is used for obtaining checking data and described checking data uses described sorter that described checking data is carried out target detection, obtains the set of the 3rd target location;

Computing unit is used for common factor is got in the set of described target mark and the set of described the 3rd target location, divided by the target number in the set of described the 3rd target location, obtains the precision of described sorter with the target number among the result who gets common factor.

Under another embodiment, described training module also comprises:

The initial training unit, be used for using first before described sorter carries out target detection in described target tracking module, with the initial training set described sorter is trained, and according to experimenter's operating characteristic ROC curve of described sorter, corresponding confidence value when choosing the recall rate that makes described sorter and reaching default recall rate threshold value, with the described confidence value of choosing as the confidence threshold value of described sorter so that described sorter carries out target detection according to described confidence threshold value.

Under another embodiment, described performance test module comprises:

The first judging unit, whether the precision that is used for judging described sorter if so, then determines the precision of described sorter reach the accuracy requirement of appointment more than or equal to default precision threshold, otherwise, determine that the precision of described sorter does not reach the accuracy requirement of described appointment; Perhaps,

The second judging unit, the precision that is used for judging described sorter with last when carrying out target detection the precision of sorter differ whether less than default differential threshold, if, determine that then the precision of described sorter reaches the accuracy requirement of described appointment, otherwise, determine that the precision of described sorter does not reach the accuracy requirement of described appointment.

The beneficial effect that the technical scheme that the embodiment of the invention provides is brought is: export to carry out the flase drop mark by choose the target that degree of confidence do not reach the accuracy requirement of appointment in the first object location sets of sorter, obtain the set of the second target location, the set of first object location sets and the second target location is combined as the set of target mark to be trained sorter, improved the precision of sorter, and only the output target that do not reach accuracy requirement manually marks, greatly reduced the workload of artificial mark, and the set of described target mark can be used for the training objective detecting device.

Further, follow the tracks of the track that obtains target by the detection target to sorter, the low target of output degree of confidence is carried out manual confirmation, in conjunction with result and the target trajectory of artificial mark, can automatically the target that does not mark in the video data be marked out, manually mark owing to only exporting the low target of degree of confidence, greatly reduced the sample size of artificial mark, reduce cost, reached time saving and energy saving effect, improved the efficient of target mark; And the target that the degree of confidence beyond the export target is high forms the set of target mark with the target that marks based on track, and sorter is trained, and can further improve the precision of sorter, thereby improve accuracy rate and precision that target marks.The technical scheme that the embodiment of the invention provides manually marks owing to only exporting the low target of degree of confidence, need not all samples are manually marked, therefore, can be competent at the target mark to large-scale video data, also can be applied to the target mark of the video data of new video camera shooting of installing, have broad application prospects.

Description of drawings

In order to be illustrated more clearly in the technical scheme in the embodiment of the invention, the accompanying drawing of required use was done to introduce simply during the below will describe embodiment, apparently, accompanying drawing in the following describes only is some embodiments of the present invention, for those of ordinary skills, under the prerequisite of not paying creative work, can also obtain according to these accompanying drawings other accompanying drawing.

Fig. 1 is a kind of method flow diagram that marks target that the embodiment of the invention provides;

Fig. 2 is the method flow diagram of the another kind mark target that provides of the embodiment of the invention;

Fig. 3 is the target following result schematic diagram that the embodiment of the invention provides;

Fig. 4 is the schematic diagram that undetected target is marked that the embodiment of the invention provides;

Fig. 5 is the progressive schematic diagram that obtains training data of increment that the embodiment of the invention provides;

Fig. 6 is a kind of structure drawing of device that marks target that the embodiment of the invention provides;

Fig. 7 is the structure drawing of device of the another kind mark target that provides of the embodiment of the invention.

Embodiment

For making the purpose, technical solutions and advantages of the present invention clearer, embodiment of the present invention is described further in detail below in conjunction with accompanying drawing.

Referring to Fig. 1, one embodiment of the invention provides a kind of method that marks target, comprising:

Step 101: obtain training data, use sorter that described training data is carried out target detection, obtain the first object location sets;

Step 102: the precision of calculating sorter, judge whether the precision of sorter reaches the accuracy requirement of appointment, when the precision of sorter does not reach the accuracy requirement of appointment, in the first object location sets, choose and do not reach the target output of specifying degree of confidence to require, to carry out the flase drop mark, obtain the set of the second target location;

Step 103: form the set of target mark with reaching target and the set of the second target location of specifying degree of confidence to require in the first object location sets, according to this target mark set sorter is trained;

Step 104: repeat above-mentioned steps until the precision of sorter reaches the accuracy requirement of appointment.

The method that the present embodiment provides, export to carry out the flase drop mark by in the first object location sets of sorter, choosing the target that degree of confidence do not reach the accuracy requirement of appointment, obtain the set of the second target location, the set of first object location sets and the second target location is combined as the set of target mark to be trained sorter, improved the precision of sorter, and only the output target that do not reach accuracy requirement manually marks, greatly reduced the workload of artificial mark, and the set of described target mark can be used for the training objective detecting device.

Referring to Fig. 2, another embodiment of the present invention also provides a kind of method that marks target, comprising:

Step 201: with the initial training set sorter is trained, and determine the confidence threshold value of this sorter.

In the present embodiment, using first before sorter carries out target detection, use first the initial training set that it is trained.The video data of described initial training set for gathering in advance preferably, for the relevant video data of scene that gather and video data to be marked, perhaps is the video data under the Same Scene, and the present invention does not limit this.

The detection that the embodiment of the invention relates to and the target of mark include but not limited to: human body, object etc., the present invention does not limit this.

The sorter that the embodiment of the invention relates to (classifier) is sorter or the mathematical model that object to be divided is incorporated into use for a certain class.Sorter is a kind of machine learning program, after by study, can automatically data be divided into known class.Sorter can be applied in search engine and the various search program, simultaneously also is widely used in data analysis and prediction field, and the artificial intelligence fields such as data mining, expert system, pattern-recognition.Sorter has multiple branch, includes but not limited to: Bayes sorter, BP neural network classifier, decision Tree algorithms, SVM (support vector machine) algorithm etc.

Generally, sorter can be described with following expression:

Y = \{\begin{matrix} 1 & F (x) &GreaterEqual; t \\ 0 & F (x) < t \end{matrix}; - - - (1)

Wherein, x is the input picture of sorter, the degree of confidence of the target that F (x) obtains for the detection of classifier input picture, and t is the confidence threshold value of sorter, Y is the output of sorter, i.e. the goal set of detection of classifier.The principle of sorter is exactly that sorter is exported this target when the degree of confidence that detects target during more than or equal to confidence threshold value t; When the degree of confidence that detects target during less than confidence threshold value t, do not export this target.The target of sorter output has formed the target location set, can comprise one or more target in this target location set, generally include a plurality of targets, each target in the set of target location has the positional information of oneself, and this positional information represents with coordinate information usually.

The confidence threshold value t of sorter can fix, and perhaps also can be unfixed.Generally, before the use sorter carries out target detection, need to determine first the confidence threshold value t of sorter, so that sorter carries out target detection according to this confidence threshold value, so that the testing result of sorter can be more approaching with expected result.The confidence threshold value t of sorter determines whether sorter will detect the critical index that target is exported, and determine that this confidence threshold value t is all relevant with precision and the recall rate of sorter.

In the present embodiment, determine the confidence threshold value of described sorter, can specifically comprise:

ROC (Receiver Operating Characteristic according to this sorter, experimenter's operating characteristic) curve, corresponding confidence value when choosing the recall rate that makes this sorter and reaching default recall rate threshold value, with this confidence value of choosing as the confidence threshold value of this sorter so that sorter carries out target detection according to this confidence threshold value.

Wherein, recall rate (Recall) is the ratio of detected correct number of targets and realistic objective sum, measurement be the recall ratio that detects.With the closely-related index of recall rate be precision (Precision), precision is the ratio of detected correct number of targets and detected target sum, measurement be the precision ratio that detects.Recall rate and precision are two metrics that are widely used in information retrieval and Statistical Classification field, are used for the quality of evaluation result.When recall rate was high, precision was low; When precision was high, recall rate was low.Recall rate and precision can not make the best of both worlds, and usually, in the situation that do not sacrifice precision, it is very difficult obtaining a high recall rate.

The adjustment of the confidence threshold value t of sorter can affect recall rate and precision simultaneously.The confidence threshold value t of sorter is less, and the result that detection of classifier goes out is just more, and recall rate is also just higher like this, but the spinoff flase drop that brings also can increase, so precision can descend.In order more reasonably to determine the confidence threshold value of sorter, the present embodiment is determined the confidence threshold value of sorter based on the ROC curve.

Described ROC curve is called again sensitivity curve (sensitivity curve), and it is according to a series of two different mode classifications, take True Positive Rate (sensitivity) as ordinate, and the curve that false positive rate (1-specificity) is drawn for horizontal ordinate.The ROC evaluating method for curve has widely to be used.On the ROC curve of sorter, each point corresponding one group of parameter (t, r, p), wherein, t is the confidence threshold value of sorter, and r is the recall rate of sorter, and p is the precision of sorter.When any one parameter in these three parameters of appointment, just can obtain all the other corresponding two parameters according to this ROC curve.For example, specifying confidence threshold value t is t ₁, then can obtain and t according to the ROC curve ₁Corresponding r ₁And p ₁

In the present embodiment, can specify in advance a recall rate threshold value, corresponding point when the ROC curve finds recall rate to reach this recall rate threshold value then, and determine confidence value corresponding to this point, with the confidence threshold value of this confidence value as this sorter.Determine in this way the confidence threshold value of sorter, can guarantee that the recall rate of sorter can reach the requirement of expectation.Described recall rate threshold value can arrange as required, as one embodiment of the present of invention, can be 0.9 with the recall rate threshold value setting, can guarantee that in this case the loss of sorter is below 0.1.Certainly can also be other value with the recall rate threshold value setting in other embodiments of the invention, such as 0.8 or 0.85 etc., the present invention limit this.

Step 202: obtain training data, use sorter that this training data is carried out target detection, obtain the first object location sets.

In the present embodiment, described training data can be view data, perhaps also can be video data.When training data was video data, the described training data that obtains referred to obtain in being input to the video data of sorter, and the described video data that is input to sorter can be online video data, the video data of namely taking in real time; Perhaps, can be the video data of off-line also, the video data of namely having taken, the present invention does not limit this.Generally, the video data of described input is one section video data taking for some scenes, and this video data comprises a plurality of frames, and each frame is a video image, each video image has reflected the scene content in the corresponding moment, comprises human body, object, background etc.Because As time goes on the people in the scene or object etc. can change, therefore, one section video data has reflected the variation of this scene in corresponding a period of time.

In the present embodiment, obtaining training data can be disposable whole training data that obtains; Perhaps can be to obtain several times yet, obtain a part of training data at every turn, the present invention limit this.Because the precision of sorter is not very high when initial, preferably, can take the progressive mode of increment to obtain several times, this process is specific as follows:

Step 203: the target in the first object location sets is followed the tracks of the track that obtains target.

In the present embodiment, when described training data is video data, can adopt target tracking algorism that the target in the first object location sets is followed the tracks of, obtain the track of target.The target following technology is widely used in the association areas such as navigation, traffic and military affairs.The target following technology can be collected each constantly data of target, and the testing result of inscribing same target during with difference associates, and merges in the set, obtains the track of target.Described target tracking algorism includes but not limited to: many hypothesis are followed the tracks of, the average drifting track algorithm, and particle filter tracking algorithm etc., the present invention does not limit this.

For example, referring to Fig. 3, be the schematic diagram that the target of detection of classifier is followed the tracks of.Wherein, comprise that take video data 3 two field pictures describe as example, shown in Fig. 3 a.Sorter carries out obtaining the first object location sets after the target detection to this 3 two field picture, shown in Fig. 3 b, comprises 6 targets in this first object location sets, and is labeled in the image with square callout box.After adopting target tracking algorism that these 6 targets are followed the tracks of, can obtain four tracks, shown in Fig. 3 c, each bar track is exactly a set, and this set has comprised 3 same targets of inscribing when different.

In the present embodiment, can preserve and follow the tracks of the target trajectory obtain, so that follow-uply mark accordingly according to target trajectory.Wherein, when preserving target trajectory, can give the different different ID of Target Assignment, different same targets constantly all have identical ID in a track.

Step 204: the precision of calculating this sorter.

In the present embodiment, calculate the precision of sorter, can specifically comprise:

Obtain the target mark set of checking data and this checking data; Use this sorter that this checking data is carried out target detection, obtain the set of the 3rd target location; Common factor is got in this target mark set and the set of the 3rd target location, divided by the target number in the set of the 3rd target location, obtained the precision of this sorter with the target number among the result who gets common factor.

Particularly, above-mentioned computation process can represent with following formula:

Precision＝|A∩B|/|B|； (2)

Wherein, the precision of Precision presentation class device, A represents the set of target mark, B represents the set of the 3rd target location.

In the present embodiment, the data of the precision that is used for the testing classification device that described checking data refers to gather in advance preferably, are video datas.Described video data can be the video data of off-line.The target mark set of described checking data refers to that annotation results is correct target location set.As one embodiment of the present of invention, the target mark set of described checking data can obtain by the mode of artificial mark, certainly, and in other embodiments of the invention, can obtain by the higher sorter of other precision, the present invention does not limit this yet.No matter adopt which kind of mode to obtain, the target in this target mark set is considered to be correct mark target, as standard the sorter among the present invention is carried out performance test, to test the precision of this sorter.The set of described the 3rd target location is to adopt the sorter among the present invention that described checking data is carried out the actual testing result that obtains after the target detection, comprising correct detection target, also may comprise the detection target of mistake.When the precision of sorter was higher, the false target in the set of the 3rd target location will be less, and when the precision of sorter was low, the false target in the set of the 3rd target location will be more.

For example, in the target of the checking data mark set A 100 targets are arranged, in the 3rd target location set B that the sorter test obtains 110 targets are arranged, namely sorter marks out 110 targets.A ∩ B=90 wherein, namely sorter marks out 90 correct targets, and then the precision of sorter is 90/110=82%, that is to say that 82% annotation results is correct.

The checking data that relates in the present embodiment preferably is different from the data of training set, described checking data is not trained sorter as training data, it is the independent data that is specifically designed to the classifier performance test, it is more objective, reasonable to guarantee the performance test of sorter like this, and test result can reflect the precision of sorter more realistically.

Step 205: judge whether the precision of this sorter that calculates reaches the accuracy requirement of appointment, if this precision does not reach the accuracy requirement of appointment, then execution in step 206; If this precision reaches the accuracy requirement of appointment, then flow process finishes.

Wherein, judge whether the precision of this sorter reaches the accuracy requirement of appointment, can adopt following any mode to realize:

First kind of way: whether judge the precision of this sorter more than or equal to default precision threshold, if so, determine that then the precision of this sorter reaches the accuracy requirement of appointment, otherwise, determine that the precision of this sorter does not reach the accuracy requirement of appointment; Perhaps,

The second way: the precision of judging this sorter with last when carrying out target detection the precision of sorter differ whether less than default differential threshold, if, determine that then the precision of this sorter reaches the accuracy requirement of appointment, otherwise, determine that the precision of this sorter does not reach the accuracy requirement of appointment.

The precision threshold that relates in the described first kind of way is the precision boundary value that sets in advance.When the precision of sorter during more than or equal to this precision threshold, think that the precision of sorter has reached requirement, can directly use this sorter to carry out target detection, need not to train again and learnt.When the precision of sorter during less than this precision threshold, think sorter precision not enough, fail to reach the requirement of appointment, therefore, need to proceed training and study to sorter.

For example, default precision threshold is 0.8, if the actual precision that tests out sorter is 0.7, thinks that then this sorter does not reach the requirement of appointment, if the precision of the actual sorter that tests out is 0.9, thinks that then this sorter has reached the requirement of appointment.

Differential threshold in the described second way refers to the threshold value of the difference of the precision that sets in advance.If the difference that the precision of sorter was compared when the precision of current sorter was carried out target detection with the last time is less than this differential threshold, the precision that then shows sorter can not improve again, think that namely the precision of sorter has reached the requirement of appointment, can directly use this sorter to carry out target detection, need not to train again and learnt.If the difference that the precision of sorter was compared when the precision of current sorter was carried out target detection with the last time, shows then that the precision of sorter can also improve again more than or equal to this differential threshold, therefore, the needs continuation is trained sorter and is learnt.

For example, default differential threshold is 0.05, if the precision of the sorter of current actual test is 0.7, the precision of the sorter of last actual test is 0.6, then the two to differ be 0.1, and 0.1＞0.05, therefore, think that the precision of sorter can improve again, at this moment, also do not reach the requirement of appointment.If the precision of the sorter of current actual test is 0.9, the precision of the sorter of last actual test is 0.88, then the two to differ be 0.02, and 0.02＜0.05, therefore, think that the precision of sorter can not improve again, at this moment, the precision of sorter has reached the requirement of appointment, can directly use this sorter to carry out target detection, need not to train and learnt.

Step 206: in the first object location sets, choose the target output of the ratio of appointment according to degree of confidence order from low to high, to carry out the manual confirmation of flase drop.

In the present embodiment, carry out each target that target detection obtains for sorter, all have a degree of confidence, this degree of confidence has reflected the order of accuarcy that detects target, and degree of confidence is higher, shows that the correctness of this target is higher, degree of confidence is lower, shows that the correctness of this target is lower.For the target in the first object location sets, can sort according to the degree of confidence of each target, as sorting according to degree of confidence order from low to high, so that choose the target of the ratio of appointment.

Wherein, the ratio of described appointment can be in advance according to the determine precision of this sorter.

Particularly, the ratio according to this appointment of determine precision of this sorter can comprise:

Calculate the ratio of this appointment according to following formula:

H＝K*(1-P)； (3)

Wherein, described H is the ratio of appointment, and described P is the precision of this sorter, and described K is the coefficient of appointment, and K 〉=1.Particularly, the value of described K can set in advance as required, preferably be set to the number greater than 1, so that must be more slightly higher than the error rate of sorter with the ratio-dependent of appointment, thereby guarantee to obtain the more artificial correct target annotation results that marks.As one embodiment of the present of invention, can described COEFFICIENT K be set to 2, certainly, also can be set to 1.5 in other embodiments, perhaps 2.5, perhaps 3 etc., the present invention does not limit this.

For example, the precision P of actual testing classification device is 82%, 1-P=18% then, show that it is wrong that 18% annotation results is arranged in the annotation results, default K=2, the ratio that then calculates described appointment is 2*18%=36%, therefore, target according to degree of confidence Sequential output 36% is from low to high carried out manual confirmation, target by these outputs of manual confirmation is correct target, or wrong target, thereby so that the lower target of these degree of confidence can obtain by the mode of artificial mark correct target location set.

In this step, the target chosen of output can adopt various ways output when carrying out manual confirmation.As one embodiment of the present of invention, the frame output at the target place chosen can be carried out manual confirmation; Perhaps, in another embodiment of the present invention, the target of choosing can be extracted from the frame at place, export separately this target and carry out manual confirmation; Similarly, also have other mode, the present invention does not limit this.When manually the target of output being confirmed, also can adopt various ways to mark, include but not limited to: only mark out correct target, perhaps only mark out wrong target, perhaps correct target is marked to distinguish with different colors or shape with wrong target, etc., the present invention also is not specifically limited this.

In the embodiment of the invention, the result of detection of classifier target can export and show, and each target that detects can the form with callout box mark out in each frame, shape the present invention of described callout box is not specifically limited this, includes but not limited to: square, rectangle or circle etc.

Step 207: the result according to artificial mark obtains positive sample object, according to the track of this positive sample object, this positive sample object that does not mark in the described video data is marked out.

The result of the artificial mark of described basis obtains positive sample object and comprises several scenes: if artificial mark is correct target, then directly obtain this target; If artificial mark is wrong target, then obtain opposite target, i.e. the artificial not target of mark; If manually both marked correct target, also marked wrong target, then obtain correct target according to the difference of the two mark.

The positive sample object that relates in the embodiment of the invention can be one, perhaps also can be for a plurality of.When the result according to artificial mark gets access to a plurality of positive sample object, can determine the wherein track of each positive sample object according to the above-mentioned target trajectory that has obtained.

Wherein, according to the track of positive sample object, this positive sample object that does not mark in the described video data is marked out, can specifically comprise:

According to the track of this positive sample object, determine positive sample object residing each frame in described video data, the positive sample object that does not mark in described each frame is marked out.This process can be understood as an example of a positive sample object example and describes, and when a plurality of positive sample object is arranged, only is the simple repetition of said process, therefore, does not do too much explanation herein.This mode that target is marked based on pursuit path has greatly reduced the workload of artificial mark, tracking results for the some moment in the objective track, only need one of artificial mark, remaining all can realize automatically marking, and has greatly improved the efficient of target mark.For instance, suppose each target from average the continuing more than 30 frames that disappear occurring, the track for track algorithm carries out Active Learning and marks and mark work can be reduced 30 times so.

Particularly, can also target L be marked combining target ID, for example, comprise 6 different tracking results constantly in the track of target L, these 6 moment are respectively t ₁, t ₂, t ₃, t ₄, t ₅And t ₆When manually to wherein t ₁After target L constantly marks, have identical ID based on difference same target constantly, therefore, other the target L constantly that has this ID in the described video data all can be marked out, namely moment t ₂, t ₃, t ₄, t ₅And t ₆Under target L mark out, thereby greatly saved the cost of artificial mark, improved efficient.

Step 208: will reach the target of specifying degree of confidence to require in the first object location sets, with the described positive sample object that has marked in the described video data, form the set of target mark, according to this target mark set this sorter is trained, then return execution in step 202.

Wherein, the described target of specifying degree of confidence to require that reaches refers to the target that degree of confidence is higher, namely the target except export target.For example, 100 targets are arranged in the first object location sets, after choosing the output of 20 targets according to degree of confidence order from low to high, it is higher that remaining 80 targets are considered to be degree of confidence, reliable target, these targets are added the set of target mark, sorter is trained the precision that can improve further sorter.

The described positive sample object that has marked in the described video data comprises this positive sample object in a certain moment of artificial mark, and other this positive sample object constantly of sorter automatic marking.

The set of the target that forms in this step mark is used for sorter is trained, and therefore, also can be called the training set, and certainly, as the Output rusults of sorter, this target mark set also can be used for the training objective detecting device.

In the present embodiment, when the precision of sorter reaches the accuracy requirement of appointment, need not to continue again sorter is trained and learnt, can directly use this sorter that the video data of input has been carried out target detection, described target mark can be online real-time mark, or the mark of off-line, the present invention does not limit this.For example, the video data that a camera is photographed carries out target detection in real time; Perhaps, one section video data having taken is carried out target detection etc.

In addition, in the embodiment of the invention, carry out in the process of target detection at sorter, also undetected situation may appear, therefore, and in order further to improve the accuracy of annotation results, described method also comprises the step of undetected target being replenished mark, and is specific as follows:

After the set of composition target mark, will export to carry out undetected artificial mark through the described training data that mark is processed, obtain artificial central point to undetected target mark;

Use the degree of confidence of the described a plurality of interim callout box of described classifier calculated, and from these a plurality of interim callout box, choose callout box as the callout box of this undetected target according to degree of confidence, in the frame at described undetected target place, should undetected target mark out, and this undetected target behind the mark is added in the set of described target mark.

Wherein, determine a callout box magnitude range according to the size of callout box corresponding to the target of described coupling, comprising:

In a plurality of callout box corresponding to the target of described coupling, choose maximum callout box and minimum callout box, generate a callout box magnitude range with the size of this maximum callout box and the size of this minimum callout box as boundary value.

Wherein, described central point is a pixel of artificial mark, such as a pixel of manually clicking by mouse.The target of described coupling refers to mark out and callout box coordinate range can comprise the target of this center point coordinate, the target of these couplings may be in the described video data some the time inscribe appear at same position other the mark target, therefore, can be used as the term of reference of current undetected target callout box size.The distance of the central point of described interim callout box and the described central point that obtains is less than default threshold value, error to occur for the central point that prevents from obtaining, it or not the true centre point of undetected target, therefore, give the scope of a translation of interim callout box that meets the callout box magnitude range by described default threshold value, when the distance of the central point of interim callout box and the central point that obtains during less than default threshold value, this central point that obtains may be positioned at the edge of this interim frame, there is error in the central point that obtains in this case, therefore, this interim frame that obtains according to described default threshold value translation also can be used as the references object of undetected target.In order to make callout box can mark more accurately undetected target, can in these a plurality of interim callout box, choose the higher target of degree of confidence as the foundation of mark as far as possible.As a preferred embodiment of the present invention, can in these a plurality of interim callout box, choose the highest callout box of degree of confidence as the callout box of described undetected target, in the frame at this undetected target place, should undetected target mark out, put forward high-precision purpose thereby reach.In this way, artificial needs to determine a central point, just can realize the automatic marking of undetected target, compare with the mode of bottom right angle point with the manually definite upper left angle point of existing needs, effectively reduced the mark number of times of undetected target, greatly save artificial cost, improved the efficient of target mark.

Further, behind the mark that carries out undetected target, can also follow the tracks of the undetected target that has marked, so that in follow-up frame, track according to this undetected target, the same target that occurs in the corresponding frame is marked, thereby save further the cost of artificial mark and the efficient that improves the target mark.

For example, referring to Fig. 4, be the schematic diagram that undetected target is marked.What Fig. 4 a showed is the frame of video of output, and a undetected target is wherein arranged; What Fig. 4 b showed is manually undetected target to be determined a central point; What Fig. 4 c showed finds the coupling target to determine a callout box magnitude range from the set of target mark; Fig. 4 d generates a plurality of interim callout box and chooses callout box according to degree of confidence; Fig. 4 e marks undetected target according to the callout box of choosing; Fig. 4 f should undetected target mark out in other frame after the undetected target that marks is followed the tracks of.

The said method that the present embodiment provides, export to carry out the flase drop mark by in the first object location sets of sorter, choosing the target that degree of confidence do not reach the accuracy requirement of appointment, obtain the set of the second target location, the set of first object location sets and the second target location is combined as the set of target mark to be trained sorter, improved the precision of sorter, and only the output target that do not reach accuracy requirement manually marks, greatly reduced the workload of artificial mark, and the set of described target mark can be used for the training objective detecting device.

Referring to Fig. 5, be the progressive schematic diagram that obtains video data of the present embodiment increment.Wherein, the inputting video data of sorter divides and obtains for four times, 1,2,3,4 four number of times that digitized representation obtains that marks among the figure, the rectangle of each numeral top represents the video data of a period of time, the duration of the video data that the duration that can find out the video data that obtains each time all obtains greater than the last time.Each represents the rectangle that comprises in the rectangle of one section video data and represents the manual video data that marks of needs.For example, the duration of the 1st, 2,3 and 4 time video data was respectively 1 hour, 3 hours, 5 hours, 8 hours.From figure, can find out significantly, the gradual mode of obtaining video data of multiple incremental, can be so that manually the workload of mark be successively decreased step by step, when obtaining video data the 4th time, the workload of artificial mark is compared with the first time and is reduced a lot, thereby has greatly reduced the workload of artificial mark.

Referring to Fig. 6, the embodiment of the invention also provides a kind of device of target mark, comprising: module of target detection 601, performance test module 602, study module 603, training module 604 and iteration module 605;

Module of target detection 601 is used for obtaining training data, uses sorter that described training data is carried out target detection, obtains the first object location sets;

Performance test module 602 for the precision of calculating sorter, judges whether the precision of sorter reaches the accuracy requirement of appointment;

Study module 603 is used for when the precision of sorter does not reach the accuracy requirement of appointment, chooses not reach the target output of specifying degree of confidence to require in the first object location sets, to carry out the flase drop mark, obtains the set of the second target location;

Training module 604 is used for that the first object location sets is reached target and the set of the second target location of specifying degree of confidence to require and forms the set of target mark, and set is trained sorter according to described target mark;

Iteration module 605 is used for repeated trigger module of target detection 601, performance test module 602, study module 603 and training module 604 until the precision of sorter reaches the accuracy requirement of appointment.

As one embodiment of the present of invention, described training data is video data, and described device also comprises:

Tracking module 606 is used for the target in the first object location sets is followed the tracks of the track that obtains target;

Study module 603 is used for: choose not reach in the first object location sets and specify the target of degree of confidence requirement to export, to carry out the flase drop mark, result according to the flase drop mark obtains positive sample object, track according to described positive sample object, the described positive sample object that does not mark in the described video data is marked out, obtain the set of the second target location.

In the present embodiment, tracking module 606 can be preserved and follow the tracks of the target trajectory obtain, so that follow-uply mark accordingly according to target trajectory.Wherein, when preserving target trajectory, can give the different different ID of Target Assignment, different same targets constantly all have identical ID in a track.

Further, study module 603 can comprise:

Flase drop mark unit 603a is used for the track according to described positive sample object, determines described positive sample object residing each frame in described video data, and the described positive sample object that does not mark in described each frame is marked out.

As an alternative embodiment of the invention, module of target detection 601 can comprise:

Acquiring unit 601a, be used for when obtaining training data first, in training data, choose a training data section, when non-when obtaining training data first, choose a training data section in the data that in described training data, were not selected, and the data amount check in the training data section chosen greater than the last time of the data amount check in this training data section of choosing.

As another embodiment of the present invention, training module 604 also comprises:

Undetected mark unit 604a, be used for after forming the set of target mark, to export through the described training data that mark is processed, to carry out undetected mark, obtain the central point of undetected mark target, in the set of described target mark, search the target of coupling, callout box coordinate range corresponding to the target of described coupling comprises the coordinate of described central point, determine a callout box magnitude range according to the size of callout box corresponding to the target of described coupling, generate a plurality of interim callout box, the size of described interim callout box is in described callout box magnitude range, and the distance of the central point of described interim callout box and the described central point that obtains is less than default threshold value, use the degree of confidence of the described a plurality of interim callout box of described classifier calculated, and from described a plurality of interim callout box, choose a callout box as the callout box of described undetected target according to degree of confidence, in the frame at described undetected target place, described undetected target is marked out, and the described undetected target behind the mark is added in the set of described target mark.

Wherein, undetected mark unit 604a can be used for: choose the highest callout box of degree of confidence as the callout box of described undetected target from described a plurality of interim callout box.

In this way, artificial needs to determine a central point, just can realize the automatic marking of undetected target, compare with the mode of bottom right angle point with the manually definite upper left angle point of existing needs, effectively reduced the mark number of times of undetected target, greatly save artificial cost, improved the efficient of target mark.

As another embodiment of the present invention, study module 603 comprises:

Choose unit 603b, be used in the first object location sets, choosing according to degree of confidence order from low to high the target output of the ratio of appointment.

Further, study module 603 also comprises:

Determining unit 603c was used for before choosing unit 603b and choosing the target output of ratio of described appointment, according to the ratio of the described appointment of determine precision of described sorter.

Particularly, determining unit 603c specifically is used for: the ratio of calculating described appointment according to formula H=K* (1-P);

Wherein, described H is the ratio of appointment, and described P is the precision of described sorter, and described K is the coefficient of appointment, and described K 〉=1.Particularly, the value of described K can set in advance as required, preferably be set to the number greater than 1, so that must be more slightly higher than the error rate of sorter with the ratio-dependent of appointment, thereby guarantee to obtain the more artificial correct target annotation results that marks.As one embodiment of the present of invention, can described COEFFICIENT K be set to 2, certainly, also can be set to 1.5 in other embodiments, perhaps 2.5, perhaps 3 etc., the present invention does not limit this.

As an alternative embodiment of the invention, performance test module 602 comprises:

Test cell 602a, the target mark set that is used for obtaining checking data and described checking data uses sorter that described checking data is carried out target detection, obtains the set of the 3rd target location;

Computing unit 602b is used for common factor is got in the set of described target mark and the set of the 3rd target location, divided by the target number in the set of described the 3rd target location, obtains the precision of sorter with the target number among the result who gets common factor.

In the present embodiment, the data of the precision that is used for the testing classification device that described checking data refers to gather in advance preferably, are video datas.Described video data can be the video data of off-line.The target mark set of described checking data refers to that annotation results is correct target location set.As one embodiment of the present of invention, the positive sample object location sets of described checking data can obtain by the mode of artificial mark, certainly, and in other embodiments of the invention, can obtain by the higher sorter of other precision, the present invention does not limit this yet.No matter adopt which kind of mode to obtain, the target in this positive sample object location sets is considered to be correct mark target, as standard the sorter among the present invention is carried out performance test, to test the precision of this sorter.

Initial training unit 604b, be used for using first before described sorter carries out target detection in target tracking module 601, with the initial training set described sorter is trained, and according to experimenter's operating characteristic ROC curve of described sorter, corresponding confidence value when choosing the recall rate that makes described sorter and reaching default recall rate threshold value, with the described confidence value of choosing as the confidence threshold value of described sorter so that described sorter carries out target detection according to described confidence threshold value.

Wherein, the video data of described initial target mark set for gathering in advance preferably, for the relevant video data of scene that gather and video data to be marked, perhaps is the video data under the Same Scene, and the present invention does not limit this.

Described precision threshold is the precision boundary value that sets in advance.When the precision of sorter during more than or equal to this precision threshold, think that the precision of sorter has reached requirement, can directly use this sorter to carry out target detection, need not to train again and learnt.When the precision of sorter during less than this precision threshold, think sorter precision not enough, fail to reach the requirement of appointment, therefore, need to proceed training and study to sorter.

Described differential threshold refers to the threshold value of the difference of the precision that sets in advance.If the difference that the precision of sorter was compared when the precision of current sorter was carried out target detection with the last time is less than this differential threshold, the precision that then shows sorter can not improve again, think that namely the precision of sorter has reached the requirement of appointment, can directly use this sorter to carry out target detection, need not to train again and learnt.If the difference that the precision of sorter was compared when the precision of current sorter was carried out target detection with the last time, shows then that the precision of sorter can also improve again more than or equal to this differential threshold, therefore, the needs continuation is trained sorter and is learnt.

The device that the embodiment of the invention provides can be carried out the either method among the said method embodiment, and detailed process sees the description in the embodiment of the method for details, does not give unnecessary details herein.Described device can be arranged in the equipment such as computing machine, and the present invention does not limit this.

The said apparatus that the present embodiment provides, export to carry out the flase drop mark by in the first object location sets of sorter, choosing the target that degree of confidence do not reach the accuracy requirement of appointment, obtain the set of the second target location, the set of first object location sets and the second target location is combined as the set of target mark to be trained sorter, improved the precision of sorter, and only the output target that do not reach accuracy requirement manually marks, greatly reduced the workload of artificial mark, and the set of described target mark can be used for the training objective detecting device.

The all or part of step that one of ordinary skill in the art will appreciate that realization above-described embodiment can be finished by hardware, also can come the relevant hardware of instruction to finish by program, described program can be stored in a kind of computer-readable recording medium, the above-mentioned storage medium of mentioning can be ROM (read-only memory), disk or CD etc.

The above only is preferred embodiment of the present invention, and is in order to limit the present invention, within the spirit and principles in the present invention not all, any modification of doing, is equal to replacement, improvement etc., all should be included within protection scope of the present invention.

Claims

1. a method that marks target is characterized in that, described method comprises:

2. method according to claim 1 is characterized in that, described training data is video data;

3. method according to claim 2 is characterized in that, according to the track of described positive sample object, the described positive sample object that does not mark in the described video data is marked out, comprising:

4. method according to claim 1 is characterized in that, the described training data that obtains comprises:

5. method according to claim 1 is characterized in that, forms after the target mark gathers reaching target that described appointment degree of confidence requires and the set of described the second target location in the described first object location sets, also comprises:

Obtain the central point of undetected mark target;

6. method according to claim 5 is characterized in that, chooses a callout box as the callout box of described undetected target according to degree of confidence from described a plurality of interim callout box, comprising:

7. method according to claim 1 is characterized in that, chooses not reach the target output of specifying degree of confidence to require in described first object location sets, comprising:

8. method according to claim 7 is characterized in that, chooses before the target output of ratio of appointment according to degree of confidence order from low to high in described first object location sets, also comprises:

9. method according to claim 8 is characterized in that, the ratio according to the described appointment of determine precision of described sorter comprises:

Calculate the ratio of described appointment according to formula H=K* (1-P);

10. method according to claim 1 is characterized in that, calculates the precision of described sorter, comprising:

Obtain the target mark set of checking data and described checking data;

11. method according to claim 1 is characterized in that, described method also comprises:

12. method according to claim 1 is characterized in that, judges that whether the precision of described sorter reaches the accuracy requirement of appointment, comprising:

13. a device that marks target is characterized in that, described device comprises: module of target detection, performance test module, study module, training module and iteration module;

14. device according to claim 13 is characterized in that, described training data is video data;

Described device also comprises:

15. device according to claim 14 is characterized in that, described study module comprises:

16. device according to claim 13 is characterized in that, described module of target detection comprises:

17. device according to claim 13 is characterized in that, described training module also comprises:

18. device according to claim 17 is characterized in that, described undetected mark unit is used for: choose the highest callout box of degree of confidence as the callout box of described undetected target from described a plurality of interim callout box.

19. device according to claim 13 is characterized in that, described study module comprises:

20. device according to claim 19 is characterized in that, described study module also comprises:

21. device according to claim 20 is characterized in that,

Described determining unit specifically is used for: the ratio of calculating described appointment according to formula H=K* (1-P);

22. device according to claim 13 is characterized in that, described performance test module comprises:

23. device according to claim 13 is characterized in that, described training module also comprises:

24. device according to claim 13 is characterized in that, described performance test module comprises: