CN106355182A

CN106355182A - Methods and devices for object detection and image processing

Info

Publication number: CN106355182A
Application number: CN201510412850.7A
Authority: CN
Inventors: 纪新; 胥立丰
Original assignee: Canon Inc
Current assignee: Canon Inc
Priority date: 2015-07-14
Filing date: 2015-07-14
Publication date: 2017-01-25

Abstract

The invention provides methods and devices for object detection and image processing. The method for object detection includes the steps of acquiring first feature points of an image according to position of a preset feature point; determining second feature points in a local area covering the first feature points; calculating feature values of the first feature points and the second feature points on the basis of the position; and determining confidence coefficient of the local area according to a preset lockup table and the calculated feature value based on the position. By the method for object detection, object detection precision can be improved.

Description

Method and apparatus for object detection and image procossing

Technical field

The present invention relates to the method and apparatus being used for image procossing, more particularly, to object detection Method and apparatus.

Background technology

In recent years, to detect and/or to identify image by object detector (that is, object classifier) Or the object (such as face, human body, house pet, automobile etc.) in video becomes noticeable use Method.Due to rapid charater and the high accuracy characteristic of boosting method, therefore by improve method Lai Obtain most of object detector, such as in " high-performance rotation invariant multiview face detection(chang huang,haizhou ai,yuan li,shihong lao；ieee transactions on pattern analysis and machine intelligence, Vol.29, no.4, pp.671-686, april 2007) " disclosed in method.

Fig. 1 shows the exemplary multi-stage of the usual object detection systems 100 in object detection technique Structure.As shown in figure 1, object detection systems include the multiple weak object trained by raising method Grader, and each weak object classifier comprises threshold value (th) and at least one look-up table (lut). By this object detection systems 100, only input picture passes through from first weak object classifier to The all weak object classifier of a weak object classifier afterwards, it is right just to be judged as expecting by input picture As.If input picture is refused by any one weak object classifier, input picture is judged as Unexpected object, and next weak object classifier will not be used.

For one of weak object classifier shown in Fig. 1, its generally operation includes: first, Weak object classifier determines some features in input picture based on some predetermined characteristic point positions Point；Secondly, weak object classifier be directed to determined by characteristic point calculate eigenvalue；Again, weak right As the lut based on the eigenvalue calculating and training in advance for the grader, for determined by characteristic point Determine confidence level；Then, weak object classifier based on determined by confidence level and training in advance th, Judge whether input picture is expectation object.

Because location-based feature reflects the local location information of object, therefore in object detection Period, it is obtained in that the exact position of object by using location-based feature, so, right As, in detection technique, location-based feature is widely used in the eigenvalue calculating characteristic point.Above-mentioned Location-based feature can be for example local binary patterns (lbp) feature, uniform lbp feature, Locally three value pattern (ltp) feature.Additionally, in " object detection using non-redundant local binary patterns(nguyen,d.,zong,z.,ogunbona,p.&li,w； proceedings of 2010ieee 17^thinternational conference on image processing (pp.4609-4612) .new york, ny, usa:ieee) " in disclose using lbp feature detection A kind of example technique of object.

However, normally, location-based feature is sensitive for the positional information of characteristic point. If not accurately determining characteristic point in the input image, distinguished point based calculate based on position The eigenvalue put will lead to obtain incorrect confidence level, and this can make weak object classifier finally make Incorrect judgement.In other words, if the object in input picture have the skew of little position or Deformation, and/or in shooting image, illumination has little direction skew, then be based on predetermined characteristic point position The characteristic point put and determine may with the real features point of the object in actual representing input images not Match.Then, the confidence level of acquisition may be incorrect.Therefore, weak object classifier will do Go out incorrect judgement, it is unexpected right to be for example judged as the input picture of actually expectation object As, or the input picture of actually unexpected object is judged as expecting object.

Content of the invention

Therefore, in view of the foregoing background in description, the problem to be solved in the present invention is: it is right to make As detection for the little position skew of the object in input picture and/or deforms more robust (robust), and for the little direction of the illumination of input picture offset more robust.

According to an aspect of the present invention, there is provided a kind of method for checking object, comprising: first is special Levy a determination step, based at least one predetermined characteristic point position, determine and obtain in image at least One fisrt feature point；Second feature point determines step, by least one first regional area At least one characteristic point is defined as second feature point, wherein, one of described first regional area Cover one of described fisrt feature point；Eigenvalue calculation step, calculates described fisrt feature point Location-based eigenvalue with described second feature point；And confidence level determines step, based on extremely A few predetermined look-up table and described fisrt feature point and described second feature point described based on position The eigenvalue put, to determine the confidence level of described first regional area.

In order that object detection for determined by characteristic point positional information more insensitive, in base After predetermined characteristic point position determines fisrt feature point, the present invention is calculating location-based spy Value indicative and determining during confidence level it is considered to the surrounding features point of fisrt feature point.Therefore, if obtained Take the little position skew that there is object in image and/or deform, or deposit when shooting and obtaining image In the little direction skew of illumination, then can optimize confidence level using the eigenvalue of surrounding features point Determination.That is, the present invention is obtained in that more accurate confidence level, and finally can Make correct judgement.Therefore, object detection precision can be improved by using the present invention.

By description referring to the drawings, other features and advantages of the present invention will be clear from.

Brief description

Comprise in the description and constitute description a part accompanying drawing exemplified with the present invention enforcement Example, and be used for explaining the principle of the present invention together with explanatory note.

Fig. 1 shows the exemplary multi-stage structure of the usual object detection systems in object detection technique.

Fig. 2 is the entirety illustrating to apply the object detection systems of the object detection technique according to the present invention The block diagram of construction.

Fig. 3 is the exemplary control architectures illustrating object test equipment according to an embodiment of the invention Block diagram.

The flow chart that Fig. 4 schematically shows method for checking object according to an embodiment of the invention.

Fig. 5 schematically shows and is used for according to an embodiment of the invention updating a predetermined lut Process flow chart.

Fig. 6 schematically shows and is used for according to an embodiment of the invention judging to obtain with multi-level approach Take the flow chart whether image comprises the process of object.

Fig. 7 a and Fig. 7 b schematically shows and is used for according to an embodiment of the invention from acquisition figure The two methods of subimage are obtained in picture.

The flow chart that Fig. 8 schematically shows image processing method according to an embodiment of the invention.

Fig. 9 is the example functional architecture illustrating object test equipment according to an embodiment of the invention Block diagram.

Specific embodiment

Describe the exemplary embodiment of the present invention below with reference to accompanying drawings in detail.It should be pointed out that it is following Description is substantially merely illustrative and is exemplary description, be in no way intended to limit the present invention and Its application or purposes.Unless specifically stated otherwise, the part being given in these embodiments and step Positioned opposite, numerical expression and numerical value do not limit the scope of the invention.In addition, will not be to this Technology known to the skilled person, method and apparatus are discussed in detail, but in due course, These technology, method and apparatus are meant as a part for description.

Note that similar reference and letter refer to the similar item of in figure, therefore once exist One in figure defines project, then for figure below, this project need not be discussed.

(object detection systems)

Fig. 2 is the object detection systems 200 illustrating to apply the object detection technique according to the present invention The block diagram of unitary construction.As shown in Fig. 2 object detection systems 200 can include Image Acquisition setting For 210 and object test equipment 220.

Image acquisition equipment 210 captures or obtains to be judged and wherein whether there is object to be detected Image.Additionally, image acquisition equipment 210 can be any type of electronic equipment, as long as its Can capture or obtain image, such as photographing unit, digital camera, television camera, take the photograph Camera, mobile phone, personal digital assistant (pda), notebook computer or other are suitably electric Sub- equipment.

Object test equipment 220 is sent to by the image that image acquisition equipment 210 obtains.Additionally, According to the embodiments of the invention describing in detail referring below to Fig. 3 to Fig. 9, object detection fills Put 220 to judge to obtain whether image comprises object to be detected.Normally, object test equipment 220 Two major parts can be included, i.e. image processing section (that is, characteristic extraction part) and judgement Part.First, as shown in Fig. 2 in image processing section, object test equipment 220 is (i.e., Image processing apparatus 221) obtain from acquisition image or extract feature (for example location-based feature Value and/or other kinds of eigenvalue).Then, in judgment part, object test equipment 220 is (i.e., Judgment means 222) based on the feature obtaining or extract, judge whether acquisition image comprises to be detected Object.

Then, object test equipment 220 exports testing result to subsequent operation.

In an example it is assumed that the object test equipment 220 shown in Fig. 2 is applied in Digital photographic In machine, an object (such as people, face, house pet etc.) is detected in object test equipment 220 After occurring in the shooting area of digital camera, focus frame is presented on the display of digital camera In picture, wherein, focus frame is corresponding with the image block being judged as comprising object according to the present invention. Additionally, image acquisition equipment 210 can be the existing part of digital camera, object test equipment 220 can be realized by the hardware in digital camera and/or software.In one embodiment, energy Enough the functional module of execution object detection or functional device can be integrated in digital camera, because This digital camera has corresponding object detection functions.In another embodiment, it is able to carry out The software program of object detection can be stored in the storage device of digital camera, therefore numeral Photographing unit also has corresponding object detection functions.

(method for checking object)

Fig. 3 be illustrate Fig. 2 shown in object test equipment 220, be capable of hereafter described in The exemplary control architectures 300 of technology block diagram.The control structure 300 of object test equipment 220 CPU (cpu) 310, random access memory (ram) 320, only can be included Read memorizer (rom) 330, hard disk 340, input equipment 350, outut device 360, network Interface 370 and system bus 380.

Cpu 310 can be any appropriate programmable control device, and can be deposited by execution Various application programs in rom 330 or hard disk 340 for the storage, to execute be described below each Plant function.Ram 320 is used for program or the number that interim storage loads from rom 330 or hard disk 340 According to, and also it is used as following space, in described space, cpu 310 executes various programs (for example The technology that execution describes in detail referring below to Fig. 4 to Fig. 9) and by object test equipment 200 The other functions carrying out.Hard disk 340 can store much information, such as operating system (os), each Plant application, control program and the data being previously generated or trained by manufacturer, wherein, data is for example Can be described below lut, th, characteristic point position.

In one embodiment, input equipment 350 can be input interface, and can receive The image of image acquisition equipment 210 output shown in for example from Fig. 2.In addition, outut device 360 Can be output interface, and testing result can be exported to subsequent operation, for example, make photographing unit Assume the focus frame of above-mentioned detection object.

In another embodiment, input equipment 350 can allow the user to and object test equipment 220 interactions, for example user can pass through input equipment 350 input picture.Additionally, input equipment 350 can take various forms, and such as button, keypad, driver plate, click type touch rotating disk or touch Touch screen.Outut device 360 can include cathode ray tube (crt) or liquid crystal display, and Testing result can be displayed to the user that.In addition, if object test equipment 220 be so-called such as The equipment of intelligent mobile phone, pda, panel computer or other suitable personal devices etc., then defeated Enter equipment 350 and outut device 360 can be integrated with being integrated.If additionally, object detection Device 220 is so-called such as conventional mobile phone, notebook computer, computed table or other conjunctions The equipment of suitable personal device etc., then input equipment 350 and outut device 360 can be by dividually Integrate.

Network interface 370 is provided for connecting object test equipment 220 to network (not shown) Interface.For example, object test equipment 220 can connect with via network via network interface 370 Other electronic equipments (image acquisition equipment 210 for example shown in Fig. 2) of connecing and/or server is (not Illustrate) enter row data communication.Alternatively, wave point can be provided for object test equipment 220, To carry out RFDC.System bus 380 can provide data transfer path, by data In cpu 310, ram 320, rom 330, hard disk 340, input equipment 350, outut device Mutually transmit between 360 and network interface 370 etc..Although referred to as bus, but system bus 380 are not limited to any specific data transmission technology.

The flow chart that Fig. 4 schematically shows method for checking object according to an embodiment of the invention 400.

Program needed for object detection shown in the flow chart of Fig. 4 and other programs (for example will under The program needed for the image procossing shown in the flow chart of Fig. 8 describing in detail in literary composition) it is collectively stored in In hard disk 340.When cpu 310 needs the flow chart executing Fig. 4, it is stored in hard disk 340 Program be loaded in ram 320.The process of the flow chart being described later on also in the same manner by It is loaded in ram 320, and executed by cpu 310.

As described above, first, the input equipment 350 shown in Fig. 3 receives the image shown in from Fig. 2 Acquisition equipment 210 output or an image by user input.Secondly, input equipment 350 Transmit obtaining image to cpu 310 via system bus 380.

Then, as shown in figure 4, determining in step s420 in fisrt feature point, cpu 310 passes through System bus 380 obtains this acquisition image from input equipment 350, and based at least one predetermined spy Levy a position and determine at least one of acquisition image fisrt feature point, wherein, predetermined characteristic point position Put and can be stored in the rom 330 shown in Fig. 3 or hard disk 340, and/or can be stored in via In the server that network (not shown) is connected with the object test equipment 220 shown in Fig. 2.

In one embodiment, can rule of thumb or priori (priori knowledge), By manufacturer's hand labeled predetermined characteristic point position.For example using face as the object of detection, permissible Key position (position of the position of such as eyes, the position of nose and face) is labeled as making a reservation for Characteristic point position.In another embodiment, can determine when training above-mentioned weak object classifier Predetermined characteristic point position.

Additionally, optionally, first, cpu 310 by obtain image scaled be with Predetermined characteristic point position is corresponding and sample that determine together and store with predetermined characteristic point position The size of image block.Then, cpu 310 based on predetermined characteristic point position in sample image block Coordinate, determines fisrt feature point in image after scaling.Additionally, in this alternative, by The amount of fisrt feature point that cpu 310 determines is identical with the amount of predetermined characteristic point position.

Determine in step s430 in second feature point as shown in Figure 4, cpu 310 determines at least one At least one of individual first regional area characteristic point, as second feature point, the wherein first local One of region covers one of fisrt feature point.Wherein, second feature point can be first Arbitrary characteristics point in regional area, and generally by the equally distributed spy in the first regional area Levy and be a little defined as second feature point.

In order to reduce amount of calculation and obtain more preferable object detection precision, preferably, cpu 310 can determine the size of the first regional area according to the distance between fisrt feature point.Additionally, The first regional area determining does not overlap with each other.

In one embodiment, cpu 310 can rule of thumb or priori determines first game Portion region.In an example, first regional area can be by with a fisrt feature Put the circle as point circumference on determine as the center of circle or using a fisrt feature point.In another reality In example, first regional area can be by diagonal as polygon using a fisrt feature point The center of line or the polygon being determined as the point on polygonal side using a fisrt feature point.

In order to ensure cpu 310 in any case (for example, the little position skew of object and/or Deformation or the little direction skew of illumination) always can determine in the first regional area determining With the second feature point of fisrt feature Point matching, preferably, the first regional area can be Make by using fisrt feature point as the center of circle and with the size of several pixels (such as one pixel) The circle determining for the radius of circle.

As shown in figure 4, after cpu 310 determines fisrt feature point and second feature point, in spy In value indicative calculation procedure s440, cpu 310 calculates the first base of fisrt feature point and second feature point Eigenvalue in position.As described above, location-based feature can be for example lbp feature, all Even lbp feature and ltp feature.

Then, determine in step s450 in confidence level, cpu 310 is based at least one predetermined lut And the first location-based eigenvalue of fisrt feature point and second feature point, determine the first local The confidence level in region, wherein, predetermined lut is stored in rom 330 or hard disk shown in Fig. 3 In 340, and/or can be stored in via network (not shown) and the object detection shown in Fig. 2 In the server that device 220 connects.In one of predetermined lut and predetermined characteristic point position one Individual corresponding, and one of predetermined lut comprises at least one eigenvalue-confidence level pair, this spy Value indicative-confidence level characteristic point corresponding with eigenvalue to expression belongs to the confidence level of object.Additionally, Cpu 310 is by testing result (that is, confidence level determined by the first regional area) via system Bus 380 is transmitted to outut device 360, and then outut device 360 exports testing result to follow-up Operation.

Taking first regional area as a example, the step that determines the confidence level of this first regional area S450 may comprise steps of:

First, in confidence level obtaining step, cpu 310 can be based on one of predetermined lut, Obtain location-based with the first of the fisrt feature point in this first regional area and second feature point The corresponding confidence level of eigenvalue；Wherein, this predetermined lut correspond to by this first regional area The predetermined characteristic point position of the fisrt feature Point matching covering.

In some cases, if the fisrt feature point in this first regional area and second feature point Some first location-based eigenvalues in this predetermined lut, there is no individual features value, then exist In one embodiment, cpu 310 can determine the first location-based spy using interpolation algorithm The individual features value of value indicative.In another embodiment, cpu 310 can be by this predetermined lut Approximate eigenvalue be defined as the individual features value of the first location-based eigenvalue.

Secondly, in confidence level deciding step, cpu 310 can determine according to the confidence level obtaining The confidence level of this first regional area fixed.

In one embodiment, first, cpu 310 can to obtain and this first partial zones Fisrt feature point in the domain confidence corresponding with the first location-based eigenvalue of second feature point Degree is compared；Then, cpu 310 can select the first big confidence level or second largest confidence level to make Confidence level for this first regional area.In another embodiment, cpu 310 can be to acquisition With this first regional area in fisrt feature point and second feature point the first location-based spy The corresponding confidence level of value indicative is averaging, and the meansigma methodss of acquisition are considered as this first regional area Confidence level.

In order to obtain more preferable object detection precision, preferably, cpu 310 can will obtain The in the middle of confidence level the first big confidence level taking is determined as the confidence level of this first regional area.Meanwhile, Cpu 310 can obtain eigenvalue and the position of the characteristic point corresponding with the first big confidence level, and And can be stored in the rom 330 shown in Fig. 3 or hard disk 340.

Additionally, in order to obtain better object detection precision, manufacturer is it is also possible that the present invention Can update and be stored in advance in the rom 330 shown in Fig. 3 or hard disk 340 and/or prestore In the server being connected with the object test equipment 220 shown in Fig. 2 via network (not shown) Predetermined lut.Then, in object detection, the present invention can the above-mentioned confidence shown in Fig. 4 Degree determines that it is better right to be obtained by this present invention using the lut after updating in step s450 As accuracy of detection.Fig. 5 schematically shows and is used for according to an embodiment of the invention updating one The flow chart 500 of the process of predetermined lut.

As shown in figure 5, being directed to a predetermined lut, in step s510, cpu 310 is by the At least one of two regional areas characteristic point is defined as third feature point, the wherein second regional area One of be in Positive training sample in the storage device that can be stored in manufacturer and/or server Or in one of Negative training sample, and cover one of fourth feature point, fourth feature point With the above-mentioned point location matches of the predetermined characteristic corresponding to this predetermined lut.Additionally, third feature point can To be the arbitrary characteristics point in the second regional area, and generally will be uniform in the second regional area The characteristic point of distribution is defined as third feature point.

Similar to the description in the determination first game portion region in step s430 shown in Fig. 4, in order to obtain Obtain better object detection precision, preferably, for each training sample, cpu 310 Can according in this training sample and the fourth feature point of predetermined characteristic point location matches between away from From determining the size of the second regional area.Additionally, determined by the second regional area not mutual Overlapping.

Because the determination of the size and shape to the second regional area is similar to the first regional area Determine, be therefore not repeated here specific description.

In step s520, determine third feature point and the 4th in step s510 in cpu 310 After characteristic point, it is second location-based that cpu 310 calculates third feature point and fourth feature point Eigenvalue.As described above, location-based feature for example can also be lbp feature, uniform lbp Feature and ltp feature.

Then, in step s530, cpu 310 is based on this predetermined lut and third feature point With the second location-based eigenvalue of fourth feature point, to determine the confidence level of the second regional area.

Taking second regional area as a example, the step that determines the confidence level of this second regional area S530 may comprise steps of:

First, cpu 310 can be based on this predetermined lut, in acquisition and this second regional area The third feature point confidence level corresponding with the second location-based eigenvalue of fourth feature point.

Similar to the description of step s450 shown in Fig. 4, in some cases, if this second game Some second location-based eigenvalues of third feature point in portion region and fourth feature point are at this There is no in predetermined lut individual features value, then cpu 310 can determine using interpolation algorithm The individual features value of two location-based eigenvalues, or, cpu 310 can be by this predetermined lut In approximate eigenvalue be defined as the individual features value of the second location-based eigenvalue.

Secondly, cpu 310 can by the maximum confidence in the middle of the confidence level of acquisition be determined as this The confidence level of two regional areas.Meanwhile, cpu 310 can also obtain corresponding with maximum confidence Eigenvalue.

Finally, in step s540, cpu 310 passes through to be existed according to the second location-based eigenvalue Lut is recalculated in distribution in Positive training sample and Negative training sample, to replace this predetermined lut, Wherein, these second location-based eigenvalues and confidence level determined by the second regional area Corresponding eigenvalue.

Referring back to Fig. 4, determine the first regional area in step s450 except determining in confidence level Confidence level outside, the present invention can also judge obtain image whether comprise object.Therefore, right As judging in step s460, cpu 310 is also by using at least one predetermined th and first local Confidence level determined by region, to judge to obtain whether image comprises object, wherein predetermined th can So that by manufacturer, rule of thumb or priori to be arranged, and predetermined th can be stored in Fig. 3 In shown rom 330 or hard disk 340, and/or can be stored in via network (not shown) In the server being connected with the object test equipment 220 shown in Fig. 2.

In order that object detection more robust, the present invention can obtain image with multi-level approach judgement is No comprise object, multi-level approach can have the multilevel hierarchy for example shown in Fig. 1.For with multistage Mode judges that illustrative methods obtaining image are as shown in Figure 6.Fig. 6 schematically shows Carrying out in step s460 shown in Fig. 4, for multi-level approach judge acquisition image whether wrap The flow chart of the process containing object.

As shown in fig. 6, the object judging step s460 shown in Fig. 4 may comprise steps of:

First, in step s461, cpu 310 can be according to the determining in step s450 The confidence level of one regional area, to determine the confidence level obtaining image.

In an example, cpu 310 can select confidence level determined by the first regional area Central maximum confidence, as the confidence level obtaining image.In another example, in order that using Information as much as possible simultaneously obtains better object detection precision, and cpu 310 can be to first game Confidence level determined by portion region is averaging, and the meansigma methodss of acquisition are considered as obtaining putting of image Reliability.

Then, cpu 310 can be by the acquisition figure that will be determined in step s461 with multi-level approach The confidence level of picture, compared with predetermined th, to judge to obtain whether image comprises object.Additionally, it is many The detailed step of level mode can be as follows shown in Fig. 6:

In step s462, cpu 310 judges whether the confidence level obtaining image is more than or equal to and works as Front th, the such as the first th.If it is current that cpu 310 judges that the confidence level obtaining image is less than Th, then, in step s463, cpu 310 is judged as that obtaining image is not expectation object, will detect Result is transmitted to outut device 360 via system bus 380, and terminates follow-up judgement operation.

Otherwise, if cpu 310 judges that the confidence level obtaining image is more than or equal to current th, Then cpu 310 is judged as that obtaining image is expectation object, and judges whether that next judges operation.

That is, first, in step s464, cpu 310 judges whether next judgement Operation.If no longer existing and judging operation, cpu 310 is judged as acquisition figure in step s465 It seem expectation object.Additionally, cpu 310 is by testing result (that is, obtain image comprise object) Transmit to outut device 360 via the system bus 380 in Fig. 3, then outut device 360 is permissible Testing result is exported to subsequent operation.Otherwise, cpu 310 makes to judge that operation is corresponding with next Th as current th, and execute further next judge operation.

First, in step s466, cpu 310 recalculates the confidence level obtaining image.One In individual example, cpu 310 can recalculate acquisition by using the said method shown in Fig. 4 The confidence level of image.In another example, cpu 310 can also be by using of the prior art Existing method, recalculates the confidence level obtaining image, for example, using lbp feature or direction ladder Spend rectangular histogram (hog) feature to calculate confidence level.

Then, in step s467, cpu 310 by the confidence level recalculating and obtains image At least one formerly confidence level combination.In an example, cpu 310 can be to recalculating The confidence level going out is averaging with the previous confidence level obtaining image, and the meansigma methodss of acquisition are considered as obtaining Take the combination confidence level of image.In another example, in order to obtain more stable confidence level, cpu 310 The confidence level recalculating can be averaging with all formerly confidence levels obtaining image, and will The meansigma methodss obtaining are considered as obtaining the combination confidence level of image.

Finally, cpu 310, again in step s462, judges that the combination confidence level obtaining image is No more than or equal to current th, such as the 2nd th.

As described above, the method for checking object described in Fig. 4 directly obtains image execution phase to whole The operation answered.However, the size obtaining image is bigger, more easily determines and do not comprise for obtaining image Object, therefore, object detection precision becomes poor.In order to reduce detecting and improving object of mistake Accuracy of detection, the present invention for example can obtain acquisition image by using image scan method first Subimage, then can to obtain each subimage execute the corresponding operating described in Fig. 4.Cause This, the flow chart 400 shown in Fig. 4 can also include subimage and obtain step s410.

As shown in figure 4, obtaining in step s410 in subimage, cpu 310 for example passes through to make first Obtain at least one subimage with image scan method from obtaining image, then cpu 310 is to obtaining The each subpicture operation above-mentioned fisrt feature point obtaining determines that step s420, second feature point determine step S430, eigenvalue calculation step s440, confidence level determine step s450 and object judging step s460.

The key concept of above-mentioned image scan method is to scan acquisition figure using the scanning window of predefined size Picture, and with predetermined step-length motion scan window from obtaining the initial point of image.Have respectively to detect Plant the object of image size, Fig. 7 a and Fig. 7 b shows two kinds of example images scan methods.Figure 7a and Fig. 7 b schematically shows and is used for according to an embodiment of the invention obtaining from acquisition image Obtain the two methods of subimage.

In one embodiment, obtain in step s410 in subimage, cpu 310 can pass through The size obtaining image is adjusted to different yardsticks and using the scanning window scanning with fixed size Image (as shown in Figure 7a) after being sized, to execute image scanning operation.In another enforcement In mode, obtain in step s410 in subimage, cpu 310 can pass through each in scan operation Using having different size of scanning window scanning acquisition image (as shown in Figure 7b) in circulation, to hold Row image scanning operates.Because above-mentioned image scan method commonly uses in the prior art, therefore It is not repeated here specific description.

As described above, in order that object detection for the little position skew of object and/or deformation more Robust, and the little direction skew more robust for illumination, that is, make object detection for Determined by characteristic point positional information more insensitive, determining based on predetermined characteristic point position After one characteristic point, the present invention also determines that other in the regional area determining by fisrt feature point Characteristic point (that is, second feature point).

Additionally, the present invention judges to obtain whether image comprises object using the confidence level of regional area, Wherein, (that is, the fisrt feature point and of the characteristic point according to all determinations in a regional area Two characteristic points) location-based eigenvalue, to determine the confidence level of this regional area.Namely Say, the present invention is determining location-based eigenvalue and considering fisrt feature point when determining confidence level Surrounding features point.

Therefore, if there is the little position skew of object in obtaining image and/or deforming, or There is the little direction skew of illumination when shooting and obtaining image, then can use surrounding features point Eigenvalue is optimizing the determination of confidence level.That is, the present invention is obtained in that more accurately putting Reliability, and finally can make correct judgement.Therefore, can improve by using the present invention Object detection precision.

(image processing method)

As shown in Fig. 2 normally, object detection process can include two major parts, that is, scheme As process part (that is, the image processing apparatus 221 shown in Fig. 2) and judgment part (that is, Fig. 2 Shown judgment means 222).In the present invention, due to above-mentioned object detection process, (Fig. 3 is to figure Shown in 7) major programme be to determine characteristic point, therefore above-mentioned image processing section can be independent of upper State object detection process.Additionally, the exemplary control architectures of image processing apparatus 221 are examined with object The control structure (shown in Fig. 3) surveying device 220 is identical.

The flow chart that Fig. 8 schematically shows image processing method according to an embodiment of the invention 800.

Then, as shown in figure 8, determining in step s820 in fisrt feature point, cpu 310 passes through System bus 380 obtains this acquisition image from input equipment 350, and based at least one predetermined spy Levy a position and obtain at least one of image fisrt feature point, wherein, predetermined characteristic point to determine Position can be stored in the rom 330 shown in Fig. 3 or hard disk 340, and/or can be stored In the server being connected with the object test equipment 220 shown in Fig. 2 via network (not shown).

Determine in step s830 in second feature point, cpu 310 determines at least one regional area At least one characteristic point, as second feature point, wherein one of regional area covers first One of characteristic point.

In order to reduce amount of calculation, preferably, cpu 310 can according to fisrt feature point it Between the size to determine regional area for the distance.Additionally, determined by the first regional area not mutual Overlapping.

Finally, after cpu 310 determines fisrt feature point and second feature point, in eigenvalue meter Calculate in step s840, cpu 310 calculates the location-based spy of fisrt feature point and second feature point Value indicative.As described above, location-based feature can be for example lbp feature, uniform lbp feature With ltp feature.Additionally, cpu 310 is by the base of the fisrt feature calculating point and second feature point Eigenvalue in position transmits to outut device 360, then, outut device via system bus 380 360 export the location-based eigenvalue calculating to subsequent operation, such as shown in Fig. 4 to Fig. 7 Above-mentioned object detection process.

As shown in figure 8, except calculating fisrt feature point and second in eigenvalue calculation step s840 Outside the location-based eigenvalue of characteristic point, the present invention also determines that the eigenvalue of regional area.Cause This, determine in step s850 in eigenvalue, and cpu 310 determines the eigenvalue of regional area, wherein The fisrt feature point that the eigenvalue of one regional area includes being covered by this regional area based on position Eigenvalue and this regional area in the second feature point of determination location-based eigenvalue.

As described above, the image processing method described in Fig. 8 directly obtains image execution phase to whole The operation answered.However, the size obtaining image is bigger, the location-based eigenvalue of acquisition is poorer. Therefore, in order to obtain preferably location-based eigenvalue, the present invention can for example pass through to make first Obtain the subimage obtaining image with image scan method, then can be to each subimage obtaining Corresponding operating described in execution Fig. 8.Therefore, the flow chart 800 shown in Fig. 8 also includes subgraph As obtaining step s810.

As shown in figure 8, obtaining in step s810 in subimage, cpu 310 for example passes through to make first Obtain at least one subimage with image scan method from obtaining image, then cpu 310 is to obtaining The each subpicture operation above-mentioned fisrt feature point obtaining determines that step s820, second feature point determine step S830, eigenvalue calculation step s840 and eigenvalue determine step s850.

Due to the subimage shown in Fig. 8 obtain step s810, fisrt feature point determine step s820, Second feature point determines step s830 and eigenvalue calculation step s840 and the subimage shown in Fig. 4 Obtain step s410, fisrt feature point determines that step s420, second feature point determine step s430 Similar with eigenvalue calculation step s440, therefore it is not repeated here specific description.

(object test equipment and image processing apparatus)

As shown in Fig. 2 because object test equipment 220 includes image processing section (that is, image Processing meanss 221) and judgment part (that is, it is judged that device 222), therefore hereinafter will be with object The description of detection means comes together to describe the corresponding description of image processing apparatus.

Fig. 9 is to illustrate the object test equipment 220 shown in Fig. 2 according to an embodiment of the invention The block diagram of example functional architecture 900.

As shown in figure 9, object test equipment 900 includes according to an embodiment of the invention: image Processing meanss 910, judgment means 920 and storage predetermined characteristic point position, the data of lut and th Storehouse 930.In addition, data base 930 can be directly the rom 330 shown in Fig. 3 or hard disk 340. Alternatively, data base 930 can also be via network (not shown) and object test equipment 220 The server connecting or External memory equipment.

First, the input equipment 350 shown in Fig. 3 receives the image acquisition equipment 210 shown in from Fig. 2 Output or an image by user input.Secondly, input equipment 350 will obtain image warp Transmitted to image processing apparatus 910 by system bus 380.Complete accordingly in image processing apparatus 910 Process and after judgment means 920 complete to judge (will be described below) accordingly, judge Device 920 by testing result (for example, the confidence level of the determination of the first regional area or obtain image Whether comprise object) transmit to outut device 360, then outut device via system bus 380 360 export testing result to subsequent operation.

As shown in figure 9, image processing apparatus 910 include: fisrt feature point determining unit 912, Two characteristic point determining units 913 and eigenvalue calculation unit 914.Additionally, image processing apparatus 910 Subimage obtaining unit 911 and eigenvalue determining unit (not shown in Fig. 9) can also be included.

More specifically, fisrt feature point determining unit 912 is configured to based in data base 930 At least one predetermined characteristic point position of storage, to determine that at least one of acquisition image first is special Levy point (corresponding to step s420 shown in Fig. 4 or step s820 shown in Fig. 8).

Second feature point determining unit 913 is configured at least at least one regional area Individual characteristic point is defined as second feature point, and wherein, one of regional area covers fisrt feature point One of (corresponding to step s430 shown in Fig. 4 or step s830 shown in Fig. 8).

Eigenvalue calculation unit 914 be configured to calculate fisrt feature point and second feature point based on The eigenvalue (corresponding to step s440 shown in Fig. 4 or step s840 shown in Fig. 8) of position.

Additionally, eigenvalue determining unit is configured to determine the eigenvalue of regional area, wherein, one The fisrt feature point that eigenvalue in individual regional area includes being covered by this regional area based on position Eigenvalue and this regional area in the second feature point of determination location-based eigenvalue (corresponding to step s850 shown in Fig. 8).

Subimage obtaining unit 911 is configured to that to obtain at least one subimage (right from obtaining image Should step s410 shown in Fig. 4 or step s810 shown in Fig. 8).

As shown in figure 9, judgment means 920 include confidence level determining unit 921.Moreover, it is judged that dress Put 920 and can also include object judging unit 922.

More specifically, confidence level determining unit 921 is configured to store based in data base 930 At least one predetermined lut and fisrt feature point and second feature point location-based feature Value, to determine the confidence level (corresponding to step s450 shown in Fig. 4) of regional area.

Additionally, object judging unit 922 is configured to by using storage in data base 930 extremely The confidence level of the determination of a few predetermined th and regional area, to judge to obtain whether image comprises Object (corresponding to step s460 shown in Fig. 4).

Each unit (that is, image processing apparatus 910 and judgment means in object test equipment 900 920) may be constructed such that each step shown in the flow chart carrying out in Fig. 4 to Fig. 8.

Above-mentioned all units are all for realizing the exemplary and/or preferred of process described in the disclosure Module.These units can be hardware cell (such as field programmable gate array (fpga), number Word signal processor, special IC etc.) and/or software module (such as computer-readable program). Unit for realizing various steps is not below at large described.However, being used for carrying out in presence In the case of the step of a certain process, there may be the corresponding function module for realizing same treatment Or unit (being realized by hardware and/or software).Described step and corresponding with these steps The technical scheme of all combinations of unit be included in disclosure herein, as long as these combinations The technical scheme constituting is complete, applicable.

Additionally, the program corresponding with the said method in Fig. 4 to Fig. 8 can be stored in Fig. 3 In shown hard disk 340.

If additionally, the said apparatus 900 shown in the Fig. 9 being made up of various units are part or all of By software construction, then it can be stored in the hard disk 340 shown in Fig. 3.On the other hand, If, partly or entirely by hardware or firmware configuration, it is acceptable for the said apparatus in Fig. 9 900 It is incorporated into as functional module in electronic equipment (such as digital camera), as long as needing Detection object in electronic equipment.Additionally, in addition to this device 900, electronic equipment is certain Also there is other hardware or software part.

Methods and apparatus of the present invention can be realized in many ways.For example, it is possible to by software, Hardware, firmware or its combination in any are realizing methods and apparatus of the present invention.Unless in addition specifically Bright, otherwise the order of the step of said method is only used for illustrating, and the step of the method for the present invention does not limit In order described in detail above.Additionally, in certain embodiments, the present invention can also be carried out For record program in the recording medium, including can for realizing the machine of the method according to the invention Reading instruction.Therefore, the present invention also covers storage for realizing the program of the method according to the invention Recording medium.

Although some specific embodiments of the present invention are described in detail using example, this area It will be appreciated by the skilled person that above example is only used for illustrating, and do not limit the scope of the invention.This Field it will be appreciated by the skilled person that can be in the case of without departing substantially from the scope and spirit of the present invention Modification is carried out to above example.The scope of the present invention is defined by the following claims.

Claims

1. a kind of method for checking object, described method for checking object includes:

Fisrt feature point determines step, based at least one predetermined characteristic point position, determines and obtains figure At least one of picture fisrt feature point；

Second feature point determines step, by least one of at least one the first regional area feature Point is defined as second feature point, and wherein, one of described first regional area covers described first One of characteristic point；

Eigenvalue calculation step, calculates the first base of described fisrt feature point and described second feature point Eigenvalue in position；And

Confidence level determines step, based at least one predetermined look-up table and described fisrt feature point and Described first location-based eigenvalue of described second feature point, to determine described first partial zones The confidence level in domain.

2. method for checking object according to claim 1, described method for checking object also includes:

Object judging step, by using at least one predetermined threshold and described first regional area Determination confidence level, to judge described acquisition image whether comprise object.

3. method for checking object according to claim 2, described method for checking object also includes:

Subimage obtains step, obtains at least one subimage from described acquisition image,

Wherein, fisrt feature point described in the subpicture operation being obtained is determined step, described second Characteristic point determines that step, described eigenvalue calculation step, described confidence level determine step and described Object judging step.

4. method for checking object according to claim 1, wherein, according to described fisrt feature The distance between point, to determine the size of described first regional area.

5. method for checking object according to claim 4, wherein, described first regional area Do not overlap with each other.

6. method for checking object according to claim 1, wherein, for the described first local One of region, described confidence level determines that step includes:

Confidence level obtaining step, based on one of described predetermined look-up table, obtains and this first game Described fisrt feature point in portion region and the described first location-based spy of described second feature point The corresponding confidence level of value indicative；And

Confidence level deciding step, according to acquired confidence level, to determine this first regional area Confidence level.

7. method for checking object according to claim 6, wherein, described confidence level determines step Suddenly comprise the following steps:

By the maximum confidence in the middle of acquired confidence level, it is determined as putting of this first regional area Reliability.

8. method for checking object according to claim 1, wherein, is updated by following steps Described predetermined look-up table:

For one of described predetermined look-up table,

At least one of second regional area characteristic point is defined as third feature point, wherein, One of described second regional area is in Positive training sample or Negative training sample, and covers One of fourth feature point, described fourth feature point and the predetermined spy corresponding to this predetermined look-up table Levy a location matches；

Calculate the second location-based feature of described third feature point and described fourth feature point Value；

Institute based on this predetermined look-up table and described third feature point and described fourth feature point State the second location-based eigenvalue, to determine the confidence level of described second regional area；And

By according to the described second location-based eigenvalue in described Positive training sample and described Look-up table is recalculated in distribution in Negative training sample, to replace this predetermined look-up table, wherein, institute It is corresponding with the confidence level of the determination of described second regional area for stating the second location-based eigenvalue Eigenvalue.

9. method for checking object according to claim 8, wherein, for the described second local One of region, determines that the step of the confidence level of this second regional area comprises the following steps:

Based on this predetermined look-up table, obtain with the described third feature point in this second regional area and The corresponding confidence level of described second location-based eigenvalue of described fourth feature point；And

By the maximum confidence in the middle of acquired confidence level, it is determined as putting of this second regional area Reliability.

10. method for checking object according to claim 8, wherein, for each training sample, According in this training sample and the described fourth feature point of described predetermined characteristic point location matches between Distance, to determine the size of described second regional area, and described second regional area is not mutual Overlapping.

A kind of 11. image processing methods, described image processing method includes:

Second feature point determines step, will be true at least one of at least one regional area characteristic point It is set to second feature point, wherein, one of described regional area covers in described fisrt feature point One；And

Eigenvalue calculation step, calculate described fisrt feature point and described second feature point based on position The eigenvalue put.

12. image processing methods according to claim 11, described image processing method is also wrapped Include:

Eigenvalue determines step, determines the eigenvalue of described regional area, wherein, described partial zones The eigenvalue of one of domain regional area includes: the described fisrt feature being covered by this regional area The location-based eigenvalue of point, and the described second feature point of the determination in this regional area Location-based eigenvalue.

13. image processing methods according to claim 12, described image processing method is also wrapped Include:

Wherein, fisrt feature point described in the subpicture operation being obtained is determined step, described second Characteristic point determines that step, described eigenvalue calculation step and described eigenvalue obtain step.

14. image processing methods according to claim 11, wherein, special according to described first Levy the distance between a little, to determine the size of described regional area, and described regional area not phase Mutually overlap.

A kind of 15. object test equipments, described object test equipment includes:

Fisrt feature point determining unit, it is configured to based at least one predetermined characteristic point position, Determine and obtain at least one of image fisrt feature point；

Second feature point determining unit, it is configured at least at least one regional area Individual characteristic point is defined as second feature point, and wherein, one of described regional area covers described the One of one characteristic point；

Eigenvalue calculation unit, it is configured to calculate described fisrt feature point and described second feature The location-based eigenvalue of point；And

Confidence level determining unit, it is configured to based at least one predetermined look-up table and described One characteristic point and the described location-based eigenvalue of described second feature point, to determine described local The confidence level in region.

16. object test equipments according to claim 15, described object test equipment also wraps Include:

Object judging unit, it is configured to by using at least one predetermined threshold and described office Confidence level determined by portion region, to judge whether described acquisition image comprises object.

17. object test equipments according to claim 16, described object test equipment also wraps Include:

Subimage obtaining unit, it is configured to obtain at least one subgraph from described acquisition image Picture,

Wherein, to fisrt feature point determining unit described in the subpicture operation being obtained, described second Characteristic point determining unit, described eigenvalue calculation unit, described confidence level determining unit and described Object judging unit.

A kind of 18. image processing apparatus, described image processing meanss include:

Second feature point determining unit, it is configured at least at least one regional area Individual characteristic point is defined as second feature point, and wherein, one of described regional area covers described the One of one characteristic point；And

Eigenvalue calculation unit, it is configured to calculate described fisrt feature point and described second feature The location-based eigenvalue of point.

19. image processing apparatus according to claim 18, described image processing meanss are also wrapped Include:

Eigenvalue determining unit, it is configured to determine the eigenvalue of described regional area, wherein, The eigenvalue of one of described regional area regional area includes: the institute being covered by this regional area State the location-based eigenvalue of fisrt feature point, and described of the determination in this regional area The location-based eigenvalue of two characteristic points.

20. image processing apparatus according to claim 19, described image processing meanss are also wrapped Include:

Wherein, to fisrt feature point determining unit described in the subpicture operation being obtained, described second Characteristic point determining unit, described eigenvalue calculation unit and described eigenvalue determining unit.