CN101311965A - Photographic subject tracking method, computer program product and photographic subject tracking device - Google Patents
Photographic subject tracking method, computer program product and photographic subject tracking device Download PDFInfo
- Publication number
- CN101311965A CN101311965A CNA2008100928607A CN200810092860A CN101311965A CN 101311965 A CN101311965 A CN 101311965A CN A2008100928607 A CNA2008100928607 A CN A2008100928607A CN 200810092860 A CN200810092860 A CN 200810092860A CN 101311965 A CN101311965 A CN 101311965A
- Authority
- CN
- China
- Prior art keywords
- photography target
- target position
- reliability
- similar
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Image Analysis (AREA)
- Studio Devices (AREA)
Abstract
This invention provides a photographic subject tracking method, a computer program product and a photographic subject tracking apparatus. A photographic subject tracking method comprises: calculating (S4) similarity levels of images input sequentially to a template image, determining (S4) a photographic subject position in each of the input images based upon the similarity levels, tracking (S6) the photographic subject position through the input images, judging (S5) whether reliability of the similarity level calculated for the photographic subject position is high or low, and changing ways of at least one of calculating the similarity levels, determining the photographic subject position and tracking the photographic subject position, if the reliability of the similarity level is judged to be low. The computer program product is used for executing a photographic subject tracking method. The apparatus includes: a similarity level calculation unit processing the calculating step; a photographic subject position determination unit processing the determining step; a photographic subject tracking unit processing the tracking step; a reliability judgment unit processing the judging step; and a processing control unit processing the change step.
Description
Technical field
The present invention relates to a kind of computer program and a kind of photography target tracing equipment that is used for the photography target tracing program photography target followed the trail of in interframe that comprise.
Background technology
In correlation technique, exist for everybody known pattern matching method, thus image division is become a plurality of image-regions and extraction to handle determined other zone of the highest similar level of realizing as matching area (Japanese patent application No.H5-81433) by zones of different being carried out template matches.
Yet, if the method in the employing correlation technique comprises the photography target position within the image of the image similar to template to determine its background by template matches, so that photography target is followed the trail of in interframe, can occur pseudo-coupling (false match) so, this makes can not carry out object tracing accurately.
Summary of the invention
Photography target method for tracing according to first aspect present invention comprises: the image of computation sequence input and the similar rank of template image; Determine photography target position in each input picture according to this similar rank; By input picture follow shot object's position; Similar other reliability of level that judgement is calculated for this photography target position is a height or low; And if judge that other reliability of similar level is low, then change and calculate similar rank, determine at least one the mode in photography target position and the follow shot object's position.
According to a second aspect of the invention, in the photography target method for tracing of first aspect, by ferret out zone in each input picture being set, making search box move to a plurality of positions within the ferret out zone and the image that calculates within each locational search box calculates similar rank to the similar rank of template image, and definite photography target position is positioned at the position that calculates the highest similar other search box of level.
According to a third aspect of the invention we, in the photography target method for tracing of second aspect, if the absolute value of the difference between other mean value of similar level and the highest similar rank is greater than predetermined threshold, then judge the reliability height, if and the absolute value of this difference is equal to or less than threshold value, judge that then reliability is low.
According to a forth aspect of the invention, in the photography target method for tracing of the third aspect, if judge that reliability is low, then expectation is the qualification that the conduct of determined photography target position follows the trail of the objective in the corresponding input picture of cancellation.
According to a fifth aspect of the invention, in the photography target method for tracing of the third aspect, if judge that reliability is low, then preferably, the area size of regulating following ferret out zone accordingly to the similar rank of being calculated for this photography target position, during the template matches performed to input picture subsequently handled, search box moved on described ferret out zone.
According to a sixth aspect of the invention, in the photography target method for tracing of the third aspect, if judge that reliability is low, then regulate the search box area size in mobile ferret out zone thereon accordingly to the similar rank of being calculated for this photography target position, recomputate similar rank via the image within the search box to the template matches of template image, described template matches is being arranged within the current input image by making search box, have to move in the ferret out zone of the area size that has been conditioned and carry out, and the photography target position is appointed as in the position that has the highest similar other search box of level that recomputates within the input picture.
According to a seventh aspect of the invention, in the photography target method for tracing of second aspect, if utilize the previous maximal phase that input picture calculated like other mean value of level with utilize the current maximal phase that input picture calculated seemingly the absolute value of the difference between the rank greater than predetermined threshold, then can judge the reliability height, if and the absolute value of this difference is equal to or less than threshold value, can judge that then reliability is low.
According to an eighth aspect of the invention, in the photography target method for tracing aspect the 7th, if judge that reliability is low, then preferably, enlarge the area size in following ferret out zone, during the template matches performed to input picture subsequently handled, search box moved on described ferret out zone.
According to a ninth aspect of the invention, in the photography target method for tracing aspect the 7th, if judge that reliability is low, then enlarge the search box size in mobile ferret out zone thereon, recomputate similar rank via the image within the search box to the template matches of template image, described template matches is by moving search box to carry out within being arranged at current input image, in the ferret out zone that has been exaggerated, and the photography target position is appointed as in the position that has the highest similar other search box of level that recomputates within the input picture.
Any the photography target method for tracing that is used for carrying out first to the 9th aspect according to the computer-readable computer program product of tenth aspect present invention.
The photography target tracing equipment of the tenth one side according to the present invention comprises: similar level calculation unit, the image of computation sequence input and the similar rank of template image; The photography target position determination unit is determined photography target position in each input picture according to similar rank; The photography target tracing unit is followed the trail of the photography target position by input picture; The reliability judging unit judges that similar other reliability of level of being calculated for the photography target position is a height or low; And processing and control element (PCE), if judge that other reliability of similar level is low, then change similar level calculation unit to similar level other calculate, the photography target position determination unit to the photography target position determine and the photography target tracing unit at least one the mode in the tracking of photography target position.
Description of drawings
The block scheme of the structure that is adopted in the camera that Fig. 1 shows in an embodiment to be realized;
Fig. 2 shows the specific example of target image I, template image A and search box B;
How Fig. 3 A and 3B when search box B is mobile in the particular range of target image I can change if showing the similar factors of calculating by template matches;
Fig. 4 shows the process flow diagram of processing performed in the camera 100 in first embodiment;
How Fig. 5 can be provided with the ferret out zone if showing in a second embodiment;
Fig. 6 shows the process flow diagram of processing performed in a second embodiment the camera 100;
Fig. 7 shows the variation of the observable similar factors that occurs in interframe in the 3rd embodiment;
How Fig. 8 can be provided with the ferret out zone if showing in the 3rd embodiment;
Fig. 9 shows the process flow diagram of processing performed in the camera 100 in the 3rd embodiment;
Figure 10 shows the process flow diagram of processing performed in the camera 100 in variation;
How Figure 11 can be provided with the ferret out zone if showing in variation;
Figure 12 A and 12B show how search box can move in variation within the ferret out zone; And
How Figure 13 shows on personal computer the follow shot object.
Embodiment
First embodiment
Fig. 1 shows the block scheme of the camera that is used for being illustrated in the photography target tracing equipment that the first embodiment of the present invention realizes.Camera 100 has control member 101, lens 102, imageing sensor 103, control device 104, memory card slot 105 and monitor 106.Control member 101 comprises the various input links such as the such user's operation of power knob, shutter release switch, zoom button, cross key (cross key), selector button, playback button and delete button.
Though lens 102 are made of a plurality of optical lens group, Fig. 1 shows single typical lens 102.Can obtain image by catching by the imageing sensor 103 that CCD or CMOS constitute via lens 102 formed object images.Data (view data) with its acquired image output to control device 104 then.Control device 104 will become via the Image Data Compression that imageing sensor 103 is obtained such as the such particular image format of jpeg format, the image file of the predetermined format that generation such as Exif (exchangeable image file format that is used for digital frequency still camera) is such, and the image file that will be produced thus outputs to memory card slot 105.
On the memory card slot 105 that has loaded the storage card that is used as storage medium, will write and record the storage card from the image file that control device 104 is exported.In addition, response is read the image file that is stored in the storage card from the instruction of control device 104 outputs at the memory card slot place.
On as the monitor 106 of LCD monitor (rear portion monitor) that is assemblied in the back of camera 100, the setup menu etc. that is stored in image in the storage card, selects to be used for the setting of camera 100 is therein shown.In addition, control device 104 obtains image and thus obtained image is outputed to monitor 106 from imageing sensor 103 according to time sequencing.Consequently, on monitor 106, each and the corresponding image of given frame are shown successively with predetermined time interval.In other words, on monitor 106, will show by image (throughimage).
The control device 104 that is made of CPU, storer and other peripheral circuits comprises such as template matches unit 104a and the such functional unit of photography target tracing unit 104b.Should be noted that storer in the control device 104 comprise can be flash memory as the ROM of memory buffer and such as the such RAM of SDRAM.Handle so that the zone of determining the photography target within this frame on it is taken by template matches to carrying out from each of the order of passing through image (sequential) frame of imageing sensor 103 input for template matches unit 104a among this embodiment.Photography target tracing unit 104b follows the trail of the zone of determining thus in interframe.More specifically, carry out following the processing.
When from the passing through image and begin of imageing sensor 103 input, the search box B that template matches unit 104a is appointed as the image in each frame target image I and has the area size that equates with the area size of template image in the ad-hoc location setting of target image I.Template matches unit 104a makes the search box B that is provided with thus move and carry out coupling in target image I and calculates to find the coupling of template image A in the middle of the image (region of search image) within the search box B that adopts diverse location.According to this coupling result of calculation, determine the represented photography target position of following coordinate figure of being adopted within target image I then, described coordinate figure is corresponding to the highest similar other matching area of level between the template image A with the image in can realizing search box B.
Should be noted that template matches unit 104a goes up the execution template matches in specific objective scope (ferret out zone), rather than carry out template matches on whole target image I, described specific objective scope is included in the position that its place has extracted template image A.By these means, the template matches scope can be limited in the scope that may comprise photography target among the target image I, consequently can carry out this processing apace.In addition, because can carry out the template matches processing, therefore omit its detailed description by being adopted as any template matching method known in the correlation technique.For example, can calculate similar factors by expressed standardization correlativity (normalized correlation) in residual sum (residual sum) expressed in for example following (1) or following (2) carries out template matches and handles.
When the r as similar factors that expressed residual sum calculates in supposing according to following (1) was smaller value, template image A was higher to the similar rank between the image of region of search.On the contrary, when the r as similar factors that expressed standardization correlativity is calculated in supposing according to following (2) was higher value, template image A was higher to the similar rank between the image of region of search.
(expression formula 1)
(expression formula 2)
Fig. 3 A and 3B show by making and move the template matches carried out within the particular range of search box B in target image I and handle the similar factors of being calculated and can how to change.In Fig. 3 A and 3B, represent similar factors and represent to carry out the number of times of calculating along transverse axis along the longitudinal axis.That is to say, the figure shows by to move as search box B and carry out the similar factors that template matches calculates and mark and draw the result who is obtained at every turn when repeatedly carrying out template matches.Should be noted that in the description of this embodiment to have standardized value in 0 and 1 scope, will be called the similar factors value in other value of the similar level of the expression in 0 to 1 the scope along the represented similar factors of the longitudinal axis.When the similar factors value near 0 the time, similar rank is lower, and when the similar factors value near 1 the time, similar rank is higher.
For example, if by the template matches processing similar factors is changed as shown in Figure 3A, the position of being adopted by the search box B among the target image I when calculating the some 3a that can realize the highest similar factors value is confirmed as the photography target position so.On the other hand, if similar factors changes shown in Fig. 3 B, the position of being adopted by the search box B among the target image I when calculating the some 3c that can realize the highest similar factors value is confirmed as the photography target position so.
To be shown among Fig. 3 A the result be shown in comparison shows that result among Fig. 3 B carries out, though the similar factors value (the highest similar factors value) of point on the 3a is owing to significantly differ 0.94 and obviously high with similar factors value mean value (similar factors candidate value mean value) 3b among Fig. 3 A, the highest similar factors value on the some 3c among Fig. 3 B is owing to differ very little difference 0.12 and not obvious other similar factors value that is higher than with respect to similar factors candidate value mean value 3d.This means, can with according to all as shown in Figure 3A the result and definite photography target position is assumed to highly reliably simultaneously, can be with according to the result shown in Fig. 3 B and the reliability of definite photography target position is assumed to low.
If satisfy expressed relation in following (3), the template matches unit 104a among this embodiment judges the reliability height of fixed photography target position so, and if low in the situation that does not satisfy the relation in (3) reliability that judges.
| the highest similar factors value-similar factors candidate value mean value |＞threshold value ... (3)
When judging that fixed photography target position height is reliable, template matches unit 104a specifies the scope that is complementary with search box B corresponding to the photography target position with as the scope that has photography target among the target image I in it.So photography target tracing unit 104b shows the target frame of expression following ranges, object is present in the described scope on this frame, makes the user can determine that photography target is present in the position in this frame.
On the other hand, if judge when the reliability of fixed photography target position is low, photography target tracing unit 104b does not show with corresponding target frame in photography target position and template matches unit 104a and moves in the template matches to next frame.That is to say that when the reliability of judging fixed photography target position was low, photography target tracing unit 104b cancelled the qualification that specific photography target position follows the trail of the objective as photography target.
When each that imported carried out above-mentioned processing by picture frame, have only when the corresponding photographic object's position place that determines handling via the performed template matches of template matches unit 104a realizes high similar rank, just can comprise this target frame that follows the trail of the objective and at interframe follow shot object by demonstration.
Fig. 4 shows the process flow diagram of processing performed in the camera 100 of first embodiment.Processing among Fig. 4 is carried out based on following program by control device 104, when imported from imageing sensor 103 pass through image and begin the time, start described program.
In step S1, template matches unit 104a at first reads by picture frame and operation after this and advances to step S2.In step S2, template matches unit 104a extracts template image A according to the instruction that the user sends from first frame that is read.Subsequently, operation advances to step S3.
In step S3, template matches unit 104 reads next and advances to step S4 by picture frame and operation after this.In step S4, when search box B as previously described was mobile within the preset range of target image I, template matches unit 104a calculated the photography target position by making image within the search box B on the diverse location and the template image A similar factors of carrying out that is complementary.Subsequently, operation advances to step S5.
In step S5, template matches unit 104a reference expression formula (3) judges whether the reliability of fixed photography target position is high.If judge that reliability is low, operation is got back to step S3 to read next frame so.On the other hand, if judge the reliability height, operation advances to step S6 so, and photography target tracing unit 104b shows the target frame of expression following ranges in step S6, and photography target is present on the present frame in described scope.After this operation advances to step S7.
In step S7, template matches unit 104a selects the required various parameter settings of template matches of frame (subsequent frame) subsequently.For example, can select region of search parameter setting to specify the position or the area size of the scope (hunting zone) that search box B will move accordingly in next frame with the similar factors value of being calculated of determined photography target position.
More specifically, if the calculated similar factors value height of determined photography target position, the possibility that this photography target is suitably discerned is high and therefore, the region of search parameter that next frame is set is so that at the more enterprising line search that with determined object's position is the center so.On the other hand, if the calculated similar factors value of determined photography target position is low, omit this photography target and therefore so, the region of search parameter of next frame is set so that be the wider enterprising line search at center with determined object's position.After this this operation advances to step S8.
In step S8, whether template matches unit 104a is passed through image and has finished to make a determination what import.If make negates to judge, step S3 is got back in operation so, and template matches unit 104a reads next frame in this step S3.On the other hand, if make sure judgement, this processing finishes so.
In aforesaid first embodiment, can realize following advantage.
(1) within search box is being arranged on ferret out zone in each input picture frame of order input when mobile, the image in the search box and template image are carried out template matches with similar grade of other similar factors between the image among the search box that calculates expression and be included in employings (assume) particular search frame position and the template image.After this, be included in to input picture among it calculates the highest other search box of similar level accordingly in the position be defined as the photography target position, and judge the reliability of the locational similar factors of this photography target.If judge that the reliability of the similar factors of being calculated for this photography target position is low, cancel the qualification that fixed photography target position follows the trail of the objective as photography target in this frame so.Consequently, only show the target frame of the photography target position that can realize high level similar factors reliability, and can carry out the photography target tracking of pin-point accuracy.
(2) as judge the reliability of similar factors according to the expression formula in (3).That is to say, if for template image the highest similar factors value with for the absolute value of the difference between the mean value of the similar factors value that image calculated in the search box that adopts different search boxes position greater than predetermined threshold, judge the reliability height of the similar factors of being calculated for this photography target position so.Yet,, judge that so the reliability of the similar factors of being calculated for this photography target position is low if absolute value is equal to or less than predetermined threshold.Because this can judge the similar factors reliability according to the mode of unified standard.
Second embodiment
In aforesaid first embodiment, if judge that by decision-making expressed in (3) (decision-making) reliability of fixed photography target position is low, template matches unit 104a continues next frame is handled and do not shown the target frame of specific photography target position so.In other words, if judge that the reliability of fixed photography target position is low, cancel the qualification that specific photography target position follows the trail of the objective as photography target so.
Judge when the reliability of fixed photography target position is low even the template matches unit difference among the template matches unit 104a among second embodiment and first embodiment is in by (3) expressed decision-making, also show the target frame of fixed photography target position in the present frame and specify this photography target position to follow the trail of the objective as photography target.After this template matches unit 104a to previous described region of search parameter reset with at wider enterprising line search so that improve the reliability of determined photography target position in frame subsequently.Should be noted that because of aspect also to can be applicable to second embodiment, so its repeat specification is not provided referring to figs. 1 to 3 described first embodiment.
Second embodiment is further elaborated, if judge that the reliability of determined photography target position is low, template matches unit 104a enlarges the ferret out zone so, and target frame B moves on described ferret out zone in target image I.For example, ferret out zone C within the target image I can be set as shown in Figure 5.At this moment, template matches unit 104a enlarges the ferret out zone or regulates the amplification degree in this ferret out zone accordingly with the similar factors value of being calculated for the photography target position with estimated rate.
Fig. 6 shows the process flow diagram of processing performed in the camera 100 of second embodiment.Processing among Fig. 6 is carried out based on following program by control device 104, when beginning during by image, to start described program from imageing sensor 103 input.Should be noted that identical number of steps is distributed to the step of carrying out the processing identical with the processing among Fig. 4, be described with reference to 4 couples first embodiment of figure, and following explanation concentrates on different with first embodiment.
In step S4, as previously described, template matches unit 104a calculates to determine the photography target position by performed similar factors.Template matches unit 104a among this embodiment determine with the corresponding scope of search box B that is arranged on the photography target position with as the scope that comprises the photography target among the target image I, and do not consider the reliability class of fixed photography target position.After this operation advances to step S6, and wherein, before operation advanced to step S7, photography target tracing unit 104b showed that expression comprises the target frame of the scope of the photography target on this frame.
In step S7, template matches unit 104a selects all required region of search parameters that is used for the template matches of next frame as mentioned previously that so various parameter settings are set.Subsequently, operation advances to step S5, and wherein, template matches unit 104a judges according to expression formula (3) whether the reliability of fixed photography target position is high.If judge the reliability height, operation advances to step S8 so.
On the other hand, if judge that reliability is low, so operation advance to step S11 with according to and the similar factors value calculated accordingly of photography target position reset region of search parameter set among the step S7.For example, if the reliability of the calculated similar factors of determined photography target position is low, omit this photography target so and therefore, must search for determined position be the wider of center in next frame.Therefore, reply region of search parameter is provided with so that carry out search on the scope that set scope is bigger in than step S7 in these cases.After this operation advances to step S8.
Except the advantage of first embodiment, also can realize following advantage by above-mentioned second embodiment.
(1) even the reliability of the similar factors of calculating accordingly with specific photography target position is low, also can show the target frame of fixed photography target position.Therefore, show that each does not consider that by the target frame on the picture frame reliability is a height or low, this makes the user can determine the variation of photography target position.
(2) if the reliability of the similar factors of calculating accordingly with fixed photography target position is low, reset the region of search parameter so.When the reliability of the similar factors of being calculated for determined photography target position is hanged down, omit this photography target.In this case, must be to carry out search on center wider with the fixed photography target position in the next frame.In a second embodiment, introduced this conception and for next frame, make the area size optimization in the ferret out zone that will search for.
The 3rd embodiment
In aforesaid first and second embodiment, template matches unit 104a judges that according to the performed decision-making of expression formula in (3) reliability of the fixed photography target position that is used for present frame is low.As an alternative, in the 3rd embodiment, calculating corresponding to the ferret out in previous frame zone and the maximal phase that calculates like the mean value of factor values and the maximal phase in the ferret out zone in the present frame absolute value of the difference between the factor values seemingly, if and in this result of calculation represented absolute value greater than certain threshold level, judge the reliability height of the fixed photography target position that is used for present frame so, if yet represented absolute value is equal to or less than predetermined threshold in this result of calculation, judge that so the reliability of the fixed photography target position that is used for present frame is low.If the reliability of the fixed photography target position in the judgement present frame is low, template matches unit 104a enlarges the ferret out zone in the present frame so.
More specifically, with the same among first and second embodiment, the pre-position of 104a in target image I, template matches unit is provided with following search box B, and described search box B takies the zone that its size equals the template image size.After this template matches unit 104a carries out coupling and calculates with by search box B is complementary with template image A moving the image (region of search image) that makes within the search box B that takies diverse location within the target image I.It according to coupling result of calculation determine with the corresponding target image I in following coupling district within the represented position of preferred coordinate value, described coupling district is included in the image within the highest similar grade other search box B that can realize to template image A.Also calculate the similar factors value of this concrete photography target position.After this to being that the new search frame B at center is provided with and calculates by the coupling of utilizing newly-installed search box B to carry out next frame with fixed photography target position.
As shown in Figure 7, now to for the image calculation in the 4th frame maximal phase describe like the example of factor values.In this case, template matches unit 104a calculates the maximal phase that calculates corresponding to the frame before three the mean value 7a like factor values, calculate then this mean value 7a with for the absolute value 7c of the difference between the factor values 7b seemingly of the maximal phase that image calculated in the present frame (the 4th frame).If the absolute value 7c that represents this difference greater than predetermined threshold, judges the reliability height of fixed photography target position in the present frame so.On the other hand, be equal to or less than predetermined threshold, judge that so the reliability of fixed photography target position in the present frame is low if represent the absolute value 7c of this difference.
That is to say that as long as satisfy expressed relation in following (4), the template matches unit 104a among the 3rd embodiment just judges the reliability height of determined photography target position, if but do not satisfy this relation, judge that then reliability is low.
| for maximal phase that previous frame calculated like the mean value of factor values-for maximal phase that present frame calculated like factor values |＞threshold value ... (4)
Therefore if do not satisfy the relation in the expression formula (4) and judge for the reliability of the fixed photography target position of present frame lowly, the 104a template matches of carrying out next frame (frame 5 among Fig. 8) on the searching target zone of as shown in Figure 8 expansion in template matches unit is handled so.The position identical with position included in frame before (frame 3 among Fig. 8) can be adopted in set photography target position (center in ferret out zone) in next frame.
Now following example is described, wherein, via by utilizing first to fourth (current) frame and the ferret out zone C among each target image I being provided with the template matches of carrying out, can judge for the reliability of the fixed photography target position of present frame (the 4th frame) low.In this case, template matches unit 104a is provided with new ferret out region D by the ferret out zone C that enlarges in the present frame, carries out the template matches of next frame (frame 5) then and handle on newly-installed ferret out region D.
Fig. 9 shows the process flow diagram of processing performed in the camera 100 of the 3rd embodiment.Processing among Fig. 9 is carried out based on following program by control device 104, when beginning during by image, to start described program from imageing sensor 103 input.It should be noted that, identical number of steps is distributed to the step of having carried out the processing identical with the processing among Fig. 4 and Fig. 6, be described and be described with reference to 6 couples second embodiment of figure with reference to 4 couples first embodiment of figure, and following explanation concentrates on different with first and second embodiment.
In step S51, template matches unit 104a reference expression formula (4) judges whether the reliability of fixed photography target position is high.If judge the reliability height, step S6 is got back in operation so, the target frame in step S6 on photography target tracing unit 104b demonstration and the corresponding scope in fixed photography target position.
On the other hand, before operation advanced to step S8, if judge that in step S51 reliability is low, operation advanced to step S11 so, and template matches unit 104a resets search parameter so that enlarge the ferret out zone in this step S11.
Except the advantage of first and second embodiment, also can realize following advantage by aforesaid the 3rd embodiment.That is to say, determine following absolute value, the maximal phase that this absolute value representation calculates corresponding to previous frame like the mean value of factor values with for maximal phase that present frame the calculated difference between the factor values seemingly, then this absolute value and threshold value are compared, and judge that the reliability for the fixed photography target position of present frame is a height or low.By these means, according to can the pin-point accuracy rank for similar factors value that previous frame calculated judging that the reliability for the fixed photography target position of present frame is a height or low.
Modified example
The photography target tracing equipment that should be noted that specific the invention described above can be considered following distortion.
(1) among formerly illustrated second embodiment, even the reliability of the similar factors of calculating corresponding to specific photography target position is low and be adjusted in employed search parameter in the template matches process of subsequently frame, also can show the locational target frame of this photography target.Otherwise, when the reliability of judging the similar factors of being calculated for initial determined photography target position was low, template matches unit 104a reset search parameter and re-executes similar factors and calculates with by utilizing the search parameter of being regulated to recomputate photography target position for present frame.So photography target tracing unit 104b shows the locational target frame of photography target that is recomputated.
More specifically, control device 104 can be carried out the processing shown in Figure 10.It should be noted that, in Figure 10, identical number of steps is distributed to the step of the execution processing identical with the processing among Fig. 4, Fig. 6 and Fig. 9, be described with reference to 4 couples first embodiment of figure, be described with reference to 6 couples second embodiment of figure, and be described with reference to 9 couples the 3rd embodiment of figure.As shown in figure 10, template matches unit 104a judges according to expression formula (3) whether the reliability of fixed photography target position is high in step S5.If judge the reliability height, operation advances to step S6 so, the target frame in step S6 on photography target tracing unit 104b demonstration and the corresponding scope in determined photography target position.
On the other hand, if judge that in step S5 reliability is low, operation advances to step S11 so, and in step S11, template matches unit 104a resets search parameter so that enlarge the ferret out zone, and operation advances to step S12 then.In step S12, template matches unit 104a re-executes that similar factors is calculated and by utilizing the search parameter of being regulated to recomputate photography target position for present frame.Subsequently, operation advances to step S6, the target frame in step S6 on photography target tracing unit 104b demonstration and the new corresponding scope of determining in photography target position.
Because the region of search parameter is regulated, and if judge that reliability by present frame being carried out the photography target position that template matches determines at first is low then retry capable template matches, improved so photography target is followed the trail of accuracy.
Equally, in above-mentioned the 3rd embodiment, re-execute template matches after the ferret out zone in having enlarged present frame (frame 4 among Fig. 8).For example, template matches unit 104a can be provided with such as the bigger new search target area D of the ferret out zone C in the present frame shown in Figure 11 (frame 4).That is to say, can be provided with as frame 4 ' in represented ferret out region D.After this on the ferret out region D present frame (frame 4 ') being re-executed template matches handles.
(2) if on the ferret out zone of the expansion in present frame in above-mentioned distortion (1), re-execute template matches, so, the scope of searching for by (session) during first template matches, promptly, scope by processing performed among the step S4 was searched for needn't stand this processing once more.In other words, only need promptly, only to have exceeded on the zone of initial ranging target area on the scope during not mating, carry out search subsequently in expansion through original template.
For example, be assumed to set ferret out zone during the original template coupling and be the scope 8a among Figure 12 A, and the scope that is assumed to the set expansion of subsequently search comprises additional range 8a and scope 8b, so only needs re-execute search on the scope 8b of region of search.More specifically, template matches unit 104a must carry out template matches by search box B is moved with helicon mode (pattern) from the outside of hunting zone (8a+8b) again, and stops this processing immediately before search box B entry range 8a.
(3) with reference to the 3rd embodiment following example is described, wherein, template matches unit 104a comes the absolute value and the threshold value of following difference are compared according to expression formula (4), so that the reliability of fixed photography target position in the judgement present frame, described difference be formerly the maximal phase within the ferret out zone in the frame like the difference between the factor values seemingly of the maximal phase in the mean value of factor values and the ferret out zone set in present frame.Yet, if comprise similar factors value like factor values with low reliability class for the maximal phase that previous frame calculated, be maximal phase like factor values less than predetermined value, the similar factors value that has a low reliability by eliminating is calculated maximal phase within the ferret out zone set in the frame formerly like the mean value of factor values so.By these means, can improve carried out about accuracy for the judgement of the reliability of the fixed photography target position of present frame.
(4) with reference to first to the 3rd embodiment, following example is described, in described example, the photography target tracing equipment is embodied as camera, and in by image photography target is followed the trail of.Yet,, so, the different frame that constitutes the dynamic image of having photographed is carried out template matches, and comes photography target is followed the trail of by the frame that constitutes dynamic image if camera 100 has the dynamic image capturing function.In addition, but the program of the processing shown in execution graph 4, Fig. 6 or Fig. 7 is recorded in such as can be in such another equipment of the personal computer that dynamic image is reset, and when personal computer is carried out this program, on this personal computer, carries out photography target and follow the trail of.
Can be via the program that can make personal computer etc. carry out the photography target tracking process according to the invention described above is provided such as the such recording medium of CD-ROM or such as the such communication line in internet.How Figure 13 can provide this program if having illustrated.Personal computer 200 receives this program by CD-ROM 202.Personal computer 200 also can link to each other with following communication line 201, but the program that is provided by described communication line 201 reception servers 203.Communication line 201 can be the communication line that is used for internet communication, personal computer communication etc., perhaps can be dedicated communication line.Server 203 is sent to personal computer 200 via communication line 201 with program.That is to say, transmit via 201 pairs of programs that convert the data-signal on the carrier wave to of communication line from server 203.Therefore, this program can be comprised as employing that the computer-readable computer program product of any different mode of recording medium or carrier wave issues.
Only should be noted that otherwise sacrifice the function that is used to characterize this agreement, this agreement in no case can be in office the face CONSTRUCTED SPECIFICATION that is subject in the above-described embodiments to be adopted where.
Claims (11)
1. photography target method for tracing comprises:
Calculate the similar rank of image to the template image of the input of (S4) order;
Determine photography target position in (S4) each described input picture according to described similar rank;
Follow the trail of (S6) described photography target position by described input picture;
Judge that similar other reliability of level that (S5) calculated for described photography target position is a height or low; And
If judge that described other reliability of similar level is low, then change and calculate described similar rank, determine described photography target position and follow the trail of at least one mode in the described photography target position.
2. according to the photography target method for tracing of claim 1, wherein:
By in each described input picture, the ferret out zone being set, making search box move to a plurality of positions within the described ferret out zone and the image that calculates within the search box of each described position calculates described similar rank to the similar rank of described template image, and
Determine that described photography target position is positioned at the position that calculates the highest similar other described search box of level.
3. according to the photography target method for tracing of claim 2, wherein:
If the absolute value of the difference between described other mean value of similar level and the highest similar rank is then judged described reliability height greater than predetermined threshold, and if the absolute value of this difference is equal to or less than described threshold value, judge that then described reliability is low.
4. according to the photography target method for tracing of claim 3, wherein:
If judge that described reliability is low, then cancel the qualification that the conduct of determined photography target position follows the trail of the objective in the corresponding input picture.
5. according to the photography target method for tracing of claim 3, wherein:
If judge that described reliability is low, then, be adjusted in the area size in the described ferret out zone that will move for described search box during the performed template matches processing of input picture subsequently thereon corresponding to the similar rank of being calculated for described photography target position.
6. according to the photography target method for tracing of claim 3, wherein:
If judge that described reliability is low;
Then regulate the area size in the described ferret out zone that described search box will move thereon corresponding to the similar rank of being calculated for described photography target position;
Recomputate described similar rank by the image within the described search box and the template matches of described template image, described template matches by make described search box in being arranged at current input image, have to move in the described ferret out zone of the area size of having regulated and carry out; And
Described photography target position is appointed as in the position that has the highest similar other search box of level that recomputates within the described input picture.
7. according to the photography target method for tracing of claim 2, wherein:
If utilize the previous maximal phase that input picture calculated like other mean value of level with utilize the current maximal phase that input picture calculated seemingly the absolute value of the difference between the rank greater than predetermined threshold, then judge described reliability height, if and the absolute value of this difference is equal to or less than described threshold value, judge that then described reliability is low.
8. according to the photography target method for tracing of claim 7, wherein:
If judge that described reliability is low, then enlarge the area size in described ferret out zone, wherein during the template matches performed for input picture subsequently handled, described search box will be moved on described ferret out zone.
9. according to the photography target method for tracing of claim 7, wherein:
If judge that described reliability is low;
Then enlarge the area size in the described ferret out zone that described search box will move thereon;
Recomputate described similar rank by the image within the described search box and the template matches of described template image, described template matches is by moving search box to carry out in being arranged at current input image, in the ferret out zone of having amplified; And
Described photography target position is appointed as in the position that has the highest similar other search box of level that recomputates within the described input picture.
10. each computer-readable computer program product (202) of photography target method for tracing that is used for carrying out according to claim 1 to 9.
11. a photography target tracing equipment (100) comprising:
Similar level calculation unit (104a), the image of computation sequence input and the similar rank of template image;
Photography target position determination unit (104a) is determined photography target position in each described input picture according to described similar rank;
Photography target tracing unit (104b) is followed the trail of described photography target position by described input picture;
Reliability judging unit (104a) judges that similar other reliability of level of being calculated for described photography target position is a height or low; And
Processing and control element (PCE) (104), if judge that described other reliability of similar level is low, then this processing and control element (PCE) change described similar level calculation unit to described similar level other calculate, described photography target position determination unit to described photography target position determine and described photography target tracing unit at least one the mode in the tracking of described photography target position.
Applications Claiming Priority (3)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2007121584 | 2007-05-02 | ||
| JP2007121584 | 2007-05-02 | ||
| JP2008049404 | 2008-02-29 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN101311965A true CN101311965A (en) | 2008-11-26 |
Family
ID=40100601
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CNA2008100928607A Pending CN101311965A (en) | 2007-05-02 | 2008-05-04 | Photographic subject tracking method, computer program product and photographic subject tracking device |
Country Status (2)
| Country | Link |
|---|---|
| JP (1) | JP5024116B2 (en) |
| CN (1) | CN101311965A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101859438A (en) * | 2009-04-08 | 2010-10-13 | 株式会社尼康 | Subject tracking device and camera |
| CN102194236A (en) * | 2010-03-15 | 2011-09-21 | 欧姆龙株式会社 | Object tracking apparatus, object tracking method, and control program |
| CN108182393A (en) * | 2017-12-22 | 2018-06-19 | 上海信耀电子有限公司 | A kind of automobile and its front truck tracking and system of application |
| CN109284673A (en) * | 2018-08-07 | 2019-01-29 | 北京市商汤科技开发有限公司 | Method for tracing object and device, electronic equipment and storage medium |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP6654789B2 (en) * | 2016-11-11 | 2020-02-26 | Kddi株式会社 | Apparatus, program, and method for tracking object considering multiple candidates at change points |
| CN114359265B (en) * | 2022-03-04 | 2022-05-24 | 广东顺德富意德智能包装科技有限公司 | Screw counting method and system based on target tracking |
Family Cites Families (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH08287216A (en) * | 1995-04-18 | 1996-11-01 | Sanyo Electric Co Ltd | In-face position recognizing method |
| JP3994445B2 (en) * | 1995-12-05 | 2007-10-17 | ソニー株式会社 | Motion vector detection apparatus and motion vector detection method |
| JP3934279B2 (en) * | 1999-06-01 | 2007-06-20 | 富士通株式会社 | Moving area tracking method and apparatus, and recording medium |
| JP2001155164A (en) * | 1999-11-26 | 2001-06-08 | Ntt Communications Kk | Device for tracing mobile object |
| JP2001195582A (en) * | 2000-01-12 | 2001-07-19 | Mixed Reality Systems Laboratory Inc | Device and method for detecting image, device and system for three-dimensional display, display controller, and program storage medium |
| JP2002074358A (en) * | 2000-08-31 | 2002-03-15 | Mitsubishi Heavy Ind Ltd | Device and method for processing road extraction picture |
| JP2006254321A (en) * | 2005-03-14 | 2006-09-21 | Matsushita Electric Ind Co Ltd | Person tracking apparatus and program |
| JP2006323437A (en) * | 2005-05-17 | 2006-11-30 | Sony Corp | Object tracking method, program for object tracking method, recording medium for recording program for object tracking method, and object tracing device |
| JP2007041730A (en) * | 2005-08-01 | 2007-02-15 | Central Res Inst Of Electric Power Ind | Electric wire abnormality detection method, device, and program |
-
2008
- 2008-02-29 JP JP2008049404A patent/JP5024116B2/en active Active
- 2008-05-04 CN CNA2008100928607A patent/CN101311965A/en active Pending
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101859438A (en) * | 2009-04-08 | 2010-10-13 | 株式会社尼康 | Subject tracking device and camera |
| US8594371B2 (en) | 2009-04-08 | 2013-11-26 | Nikon Corporation | Subject tracking device and camera |
| CN101859438B (en) * | 2009-04-08 | 2014-05-28 | 株式会社尼康 | Subject tracking device and camera |
| CN102194236A (en) * | 2010-03-15 | 2011-09-21 | 欧姆龙株式会社 | Object tracking apparatus, object tracking method, and control program |
| CN108182393A (en) * | 2017-12-22 | 2018-06-19 | 上海信耀电子有限公司 | A kind of automobile and its front truck tracking and system of application |
| CN109284673A (en) * | 2018-08-07 | 2019-01-29 | 北京市商汤科技开发有限公司 | Method for tracing object and device, electronic equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| JP5024116B2 (en) | 2012-09-12 |
| JP2008299834A (en) | 2008-12-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106254756B (en) | Filming apparatus, information acquisition device, system and method and recording medium | |
| CN101373477B (en) | Subject tracking method, and subject tracking device | |
| TWI582710B (en) | The method of recognizing the object of moving image and the interactive film establishment method of automatically intercepting target image | |
| US9076069B2 (en) | Registering metadata apparatus | |
| CN101311965A (en) | Photographic subject tracking method, computer program product and photographic subject tracking device | |
| US20150124059A1 (en) | Multi-frame image calibrator | |
| US20130279813A1 (en) | Adaptive interest rate control for visual search | |
| US10674066B2 (en) | Method for processing image and electronic apparatus therefor | |
| CN108073890A (en) | Action recognition in video sequence | |
| CN102611826A (en) | Image processing device, image processing method, and program | |
| US20120127276A1 (en) | Image retrieval system and method and computer product thereof | |
| KR20160020498A (en) | Tracker assisted image capture | |
| US8400520B2 (en) | Subject tracking program and camera using template matching processing | |
| CN113473025A (en) | Holder adjusting method, holder adjusting device, mobile platform and medium | |
| CN109767453A (en) | Information processing unit, background image update method and non-transient computer readable storage medium | |
| CN110379017B (en) | Scene construction method and device, electronic equipment and storage medium | |
| CN105163034A (en) | Photographing method and mobile terminal | |
| CN103763480A (en) | Method and equipment for obtaining video dubbing | |
| CN102479220A (en) | Image retrieval system and method thereof | |
| WO2022000210A1 (en) | Method and device for analyzing target object in site | |
| US8144934B2 (en) | Photographic subject tracking method, computer program product and photographic subject tracking device | |
| CN112106347A (en) | Image generation method, image generation equipment, movable platform and storage medium | |
| KR101273634B1 (en) | Tracking Method of Multiple Objects using Mobile Device in Augumented Reality Environment and System Using the same | |
| CN115623313A (en) | Image processing method, image processing apparatus, electronic device, and storage medium | |
| US20210392268A1 (en) | Image processing device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C02 | Deemed withdrawal of patent application after publication (patent law 2001) | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Open date: 20081126 |