WO2013077562A1 - Apparatus and method for setting feature points, and apparatus and method for object tracking using same - Google Patents

Apparatus and method for setting feature points, and apparatus and method for object tracking using same Download PDF

Info

Publication number
WO2013077562A1
WO2013077562A1 PCT/KR2012/008896 KR2012008896W WO2013077562A1 WO 2013077562 A1 WO2013077562 A1 WO 2013077562A1 KR 2012008896 W KR2012008896 W KR 2012008896W WO 2013077562 A1 WO2013077562 A1 WO 2013077562A1
Authority
WO
Grant status
Application
Patent type
Prior art keywords
frame
block
sad
feature point
sum
Prior art date
Application number
PCT/KR2012/008896
Other languages
French (fr)
Korean (ko)
Inventor
우대식
박재범
전병기
김종대
정원석
Original Assignee
에스케이플래닛 주식회사
시모스 미디어텍(주)
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/20Analysis of motion
    • G06T7/246Analysis of motion using feature-based methods, e.g. the tracking of corners or segments

Abstract

The present invention relates to an apparatus and method for setting feature points, and to an apparatus and method for object tracking using same. According to the present invention, when a user designates feature points, a representative value of variability for each block is obtained in consideration of the pixel matrix surrounding the user-designated feature points, and the user-designated feature points are reset on the basis of the representative value of the variability. Accordingly, in terms of a stereoscopic conversion, the accuracy of feature point tracking for the outer contour of the object may be increased. In addition, a key frame whose total sum of errors is at the minimum can be easily found and selected through only simple calculation without having to track each frame.

Description

Object tracking apparatus and method using the same apparatus and method, and feature point set

The present invention is a block that corresponds to a case given the feature point corresponding to the initial position of the object, to form a block of a predetermined size with respect to the designated feature point on the input image, a group formed of the above within the set movement range block size Once defined, the present invention relates to redirect the movement range change to each frequency conversion for each block defined within FIG obtain a representative value, respectively, the center point of the movement range of the largest free block is changed a representative value in the final characteristic point .

Also, after the present invention is obtained for SAD (sum of absolute difference) between divided into blocks of a predetermined number of predetermined frames of the input image, the current frame and an adjacent frame for each block, respectively, the SAD between the adjacent frames, each frame obtained by adding together the sum total SAD, and then using the sum SAD by the respective frames obtained by the frame tracking error value, relates to the selection of the tracking error is the smallest value of the frame to key frame.

Technology for detecting and tracking them for a specific object in a video image have evolved around the field of security, face recognition applications. For a secure system, it is necessary to detect the object enters the monitor area to automatically monitor the people fall within the monitor area, track the movement of the object.

A general object tracking technique is typically extracted feature points, and perform in a manner that keep track of these feature points in the next image. Extraction of the feature point is there is a strong edge component and the corner elements of this partial image is because the position is maintained even if there is also a possible scoring movement. And each of the feature points are all represented by scoring representing the value of the feature point, because characteristics of such a component.

Automatically you that finding the minutiae if immediately possible to select a feature point having the highest score by this score, but manually select the feature points can be selected for the feature points by the user, regardless of this scoring, specified by the user If you want to track the minutiae, the more likely an error.

On the other hand, I personally wide range of video camcorders intestine is possible, recorders and digital video searching in charge of professional features from the video player, recognition, tracking, monitoring, and blocking a representative frame of the video (key frame) to extract etc. system, highlighting and indexing to support such functionality should perform a scene change detection.

If hayeoteul given a specific object or detected in a continuous video while looking for the same type of objects as much as possible the object in successive images is that the object tracking technology that you want to go out and keep track of, including their shape, or orientation. Objects in the general video while being maintained or modified forms may be moved or move.

Typically applications are the object tracking motion detection and direction of movement associated with the movement and tracking of objects for their shape is not critical dimensional transform such as a camera, my man of the motion tracking is to be maintained as much as possible the shape of the object it is possible to obtain an accurate outer shape or representing the object.

Therefore, differently from the generic object tracking function to track the shape of the specific object to the three-dimensional transformation to be minimized is the error with respect to the shape of the outer frame to represent the object. As a rule, the movement of the object increases in a series of image errors for the outer shape of the track in the process of each scene is increased more and more.

With reference to Figures 1 and 2 it will be described in the tracking error stacked relation for a three-dimensional transformation.

Considering the track for the object movement of the one-way as shown in Figure 1, as the cumulative error of each scene and the increasing sum of the overall error, when the above predetermined error generated no more have at least to a track for the three-dimensional transformation.

However, in the field to track the direction of the common object is not affected by the tracking of the object, because regardless of the existence of such an error only if you know the direction.

And In order to further minimize the error by considering the two-way tracking of the object, even compared to the cumulative error which occurs while tracking a particular object from one accumulated error and 9 generated while tracking a particular object in the number seven frames, such as 2 the positive direction of the sum total of error, that is the total of the bottom curve in the graph area of ​​9 frames are smaller.

That is, normally the camera object, such as a trace of the case which seek out all objects If track the position of the direction the same for the results to specify the particular object, and track them at times, seven times the frame 9 frames, the object for three-dimensional conversion in tracing the most reasonable hayeoyaman selected as "key frame" to track the object as a starting point, nine scenes from the generation of the depth map for the stereo converted to the corresponding nine frames as can reduce errors at the beginning of the track.

However, in the case of selecting the scenes (Scene) each time within each frame, the left and the right as a whole tracing (tracking) for reporting thus selected as a key frame is smaller the sum of the resulting error as described above, to perform all possible combinations that there is practically not possible disadvantages.

The present invention seeks to as having been made in view, an object of the present invention is to manually users considering the surrounding pixels, the matrix of the feature point designated by the user to designate the characteristic point of each block by changing a representative value in order to address the above-described drawbacks, the gradient reset the feature point designated by the user based on the representative value, and provides the feature point setting apparatus and method and an object tracking apparatus using the same and method, which can increase the accuracy of feature point tracking of the outline of the object in the three-dimensional conversion perspective It is to.

It is another object of the present invention can find a simple operation the sum of the error the smallest frame alone without performing the tracking (tracking) for each frame to be easily selected for the key frame, for the SAD caused by movement of the object error use in estimating factors to provide a feature point setting apparatus and method for selecting key frames to minimize the accumulated error.

According to one aspect of the present invention for achieving the above object, a feature point definition unit for storing the user for the input image received specifying the feature point corresponding to the initial position of the object; Wherein the center of the designated feature points forming a predetermined size of the block, and the block group forming part defining block corresponding to the size of the block formed within a predetermined movement range; Changes to obtain a representative value of each frequency change to convert each block defined within the movable range of each of the representative value calculation unit; And the feature point set in the apparatus including the movement range change is also redirected to the center point of the largest free block is a representative value to the final characteristic point defining a feature point determination unit is provided.

Here, after dividing a frame of the input video into blocks of a predetermined fixed number, the SAD calculation SAD (sum of absolute difference) between the current frame and the adjacent frame for each block to obtain each portion; SAD sum calculation section by adding together the SAD between the adjacent frames to obtain the sum SAD of each frame; Tracking error calculation unit to obtain the frame-to-frame tracking error value using a sum SAD by the each frame; And it may further include a key frame selection of selecting the tracking error is the smallest value of the frame to key frame (Key Frame).

According to another aspect of the present invention for achieving the above object, the feature points on the input image checking whether the validity of the feature point corresponding to the initial position of the object specified by the user setting a feature point of the object setting unit; Image analysis unit which extracts motion vector information and the residual signal information of each frame in the input image; And the location information, the forward motion vector information after generating the forward motion vector information for each unit block from the extracted motion vector information, and restoring the pixel information on a certain block from the extracted residual signal, the set feature points and wherein the pixel information from the reconstructed using the feature points of a tracking object that produces the optimum position information for the object in each frame part object tracking apparatus.

According to another aspect of the present invention for achieving the above object, (a) step of storing the user for the input image received specifying the feature point corresponding to the initial position of the object; (B) defining a block corresponding to the size of the blocks formed in the movement region to form a block of a predetermined size with respect to the designated feature points, group set in the input image; (C) calculating a representative value of the change in each of the frequency conversion for each block defined within the movable range, respectively; And (d) a feature point setting method of setting a feature point comprises a determining step redirects the center point of the range of movement, the biggest change is the representative value of the block within the final characteristic point is provided.

Further, the feature point setting method is recorded to the program and provides a recording medium readable in the electronic device.

On the other hand, dividing the block in accordance with another aspect, the determined (a) a frame of an input image in advance a predetermined number of the present invention for achieving the above object; (B) calculating a SAD between the current frame and the adjacent frame for each block of the predetermined number, respectively; (C) obtaining a sum SAD of each frame by adding together the SAD between the adjacent frames; (D) obtaining a frame-to-frame tracking error value using a sum SAD by the each frame; And (e) a key-frame selection method of the feature point set comprises the step of selecting a frame with the smallest error value, the track as a key frame is provided.

Further, the keyframe selection method is recorded to the program and provides a recording medium readable in the electronic device.

According to another aspect of the present invention for achieving the above object, (a) with respect to the input image checking the validity whether the feature point corresponding to the initial position of the object specified by the user setting a feature point of the object .; (B) extracting the motion vector information and the residual signal information of each frame in the input image; And (c) the extracted motion from the vector information to generate a forward motion vector information for each unit block, after restoring the pixel information on a certain block from the extracted residual signal, location information of the set feature points, and the forward the motion vector information and the pixel information from the reconstructed using the feature points of the object-tracking device includes generating the optimal location of the object in each frame, the object-tracking method is provided.

Then, the object tracking method using the characteristic point is recorded as a program to provide a recording medium readable in the electronic device.

According to the invention, a manually user and each of the blocks by changing considering the surrounding pixels, the matrix of the feature point designated by the user to designate a feature point Fig obtain a representative value, based on a representative value of the change in the feature point designated by the user It can be reset.

In addition, it is possible to increase the accuracy of tracking the feature points on the outline of the object in terms of the three-dimensional transformation.

Also, do not perform tracking (tracking) for each frame to find the sum of the smallest error frame with a simple operation can be easily selected key frame.

Then, by using the SAD caused by movement of the object as a parameter for estimating the error it can be selected for the key frame to minimize the accumulated error.

1 and 2 are views for explaining a tracking error stacked relation for a conventional three-dimensional transformation,

Figure 3 is a block diagram schematically showing the configuration of the feature point set according to the present invention,

Figure 4 is a view for explaining a method of forming a block in accordance with the invention,

5 is a diagram showing a frequency transformation block according to the invention,

Figure 6 is an example for explaining how to obtain a SAD between frames according to the invention,

Figure 7 is a graph showing the sum SAD of the frames in a scene (Scene) to be tracked in accordance with the invention,

8 is a diagram representing the frame-to-frame tracking error values ​​in accordance with the present invention in a graph,

With reference to the accompanying drawings, the present will be described in more detail a preferred embodiment of the invention. Description of the same components in the following description with reference to the accompanying drawings or corresponding are assigned the same reference numerals and a duplicate thereof will be omitted.

Figure 3 illustrates a frequency transformation block according to the drawings, the present invention Figure 5 is illustrating a method of block diagram schematically showing the construction of a feature point setting unit in accordance with the present invention, Figure 4 is formed in the block according to the invention diagram.

3, the feature point setting unit 300 has a feature point definition part 310, block forming portion 320, the degree of change by the representative value calculation unit 330, a feature point determination unit (340), SAD (sum of absolute difference ) comprises a calculation unit (350), the sum SAD calculation unit 360, a tracking error computation unit 370, a key frame (key frame) selector 380.

Feature point definition part 310 for receiving and storing the input video that the user specifies the feature points corresponding to the initial position of the object. That is, the user specifies the feature points corresponding to the initial position of the object in the input image, the feature points defining unit 310 stores the feature point designated by the user. Here, the feature point refers to the edge or corner.

A block forming portion (320) defines the block corresponding to the size of the blocks formed in the movement to form a block of the size predetermined around a feature point from the feature point specified definition unit 310, a pre-set range.

That is, the block formation section 320 sets the range of movement possible to form a random block, around which the formed block, then the left or right movement, including peripheral pixels with respect to the designated feature point on the input image. Here, the moving range of the block may be a range defined by the user in advance. By the setting of the movement range are defined block that may be formed within the moving range.

If the block forming unit 320, the reference to Figure 4 for a method of forming a block, the feature points in the image (i, j) Given the 400, block forming portion 420 is a random block of about the feature points to form a block (n) (410). block (n) (410) is any block that contains the peripheral pixels corresponding to the feature point n.

block (n) as a feature point coordinate coordinates (i, j) (400) is defined by the user of the unit 410, which is moveable previously-defined size d (x), before as d (y), and then, left it is possible or right. Thus, the outermost upper left block 420 and the maximum likelihood at the bottom block 430 is block (n) (410) This refers to a movable range of movement back and forth. As a result, the block (n) is (2 * dx + 2 * dy) the total (2 * dx + 2 * dy) within the coordinate that is, is determined that the moving range of the block, the moving range of the space that can move as much as one block this may be defined (n).

Change calculation unit 330, a representative value to each frequency conversion every block defined by the travel range set in the block forming unit 320 obtains a representative value for each shift of the respective blocks. In the range of movement (2 * dx + 2 * dy) if one block is defined, the degree of change by the representative value calculation unit 330 (2 * dx + 2 * dy) of the block frequency transformation, and the degree representative value changes each each obtained.

That is, the degree of change by the representative value calculation unit 330 performs frequency transformation on each block, and in the frequency conversion block summing pixel values ​​corresponding to the high frequency range is obtained for a representative value of each block changes.

Score for the user and give the value of the feature point on the given space that can move relative to the (i, j) coordinates of the feature point, that is, reliability of the feature points is used in the FFT or DFT transform representing a frequency characteristic of the block (n).

That is the two-dimensional FFT (Fast Fourier Transform) or (Discrete Fourier Transform) DFT on the space corresponding to the low frequency region at the top left as shown in the case of a simple image is a boring 3 in accordance with the characteristics of the image pixels of the block (n) If the values ​​are increased, the degree of change is large (i.e., when the change between the pixel large) is larger that the value corresponding to the high frequency area in the lower right corner.

When using the radio frequency region to express the strength of the degree of change in the FFT or DFT Table transformed to the frequency domain, so as a result to obtain a representative value that is proportional to the degree of change of an image, the total sum of the pixel values ​​corresponding to the high frequency region changes in the block (n) is also calculated as a representative value.

That is, the change calculation unit 330, a representative value is 2 in any block (n) is moveable in space * dx + 2 * dy spaces, i.e., a degree representative value changes for each of the blocks defined in the moving range, respectively obtained.

Feature point determination unit 340 redirects the center point of the movement range of the largest free block is changed a representative value in the final characteristic point.

Important for tracking feature points between images is subjected to a large impact on whether the surrounding of the minutiae is what differentiates the different parts. I.e., if the peripheral is video or changed simply boring selected as feature points greater (unless that is the degree of change is greater), typically not easy to distinguish the trace from the subsequent image, if the feature differentiation is possible (that is, the change between the pixel the greater side) a comparison with such a changed portion of the subsequent image is easy. I.e. the degree of change by a large part of the track so is easy and is also small tracking error. Those based on two euros If the selected feature points in a particular portion in accordance with the user's intention, a feature point setting unit 300 may be selected as a material more easily track the feature point in the moving range is set to the feature point in advance.

SAD calculation unit 350 is then divided by a predetermined block (block) number of a predetermined frame of the input image, for each block between a current frame and a previous frame is obtained for each of SAD between the current frame and after the frame.

In this case, the block is a block of size N x N, e.g., it may also be any size block in the block diagram of a 16 x 16 block size of, 8 x 8 block of size, 4 x 4 size. Each block is called the Macro Block divide into smaller blocks of a suitable size according to the size of the entire image, typically separated by about the size of 16x16 ~ 32x32.

The sum of the absolute differences of any pixel within the block (sum of absolute difference) SAD, resulting in a representation of the difference block, such as the inter-frame position. Thus, the mean SAD value is large means that the change in the image is greater.

SAD calculation unit 350 calculates the SAD between the current frame and each adjacent frame for each block by using the equation (1). Here, the adjacent frame is a previous frame and a subsequent frame of the current frame in the current frame.

Equation 1

Figure PCTKR2012008896-appb-M000001

Here, fn is a frame number, bm is an m-th block of the frame, i is the order of the corresponding pixel block, abs means an absolute value.

Thus, SAD calculation unit 350 obtains each of the SAD, SAD between the current frame and after the frame between the current frame and the previous frame for each block.

If you SAD calculation unit 350, a reference to Figure 6 how to obtain the SAD, the current frame is the Frame (fn), and a previous frame, Frame (fn-1), since the frame is Frame (fn + 1), between the SAD calculation unit 350 includes a SAD that is between the block (bm) at the same position in each frame, current frame block bm and a previous frame block bm SAD (fn-1, bm), the current frame and after the frame block bm between SAD ( calculate the fn, bm), respectively.

Sum SAD calculation unit 350 calculates the SAD by adding together the sum SAD of each block for each frame.

That is, SAD total sum calculation unit 360 between the current frame and the previous frame combined the SAD for all of the blocks to obtain the sum of claim 1 SAD, the claim 2 SAD sum summing the SAD for the current frame and all the blocks between subsequent frames after obtaining, the sum SAD of claim 1 and the second combined sum SAD calculated by the SAD sum frame.

That is, the sum SAD calculation unit 360 calculates the SAD of all of the combined blocks between the current frame and after the frame, the sum of claim 2 SAD (SAD (fn)) as shown in equation (2).

Equation 2

Figure PCTKR2012008896-appb-M000002

Here, the index j refers to the number of Block.

SAD calculated from Equation 2 (fn) is to be the current sum of SAD between the frame and the subsequent frames, SAD total sum calculation unit 360, as well as frames since the prior frame is considered together to be obtain the SAD sum of the current frame.

Therefore, the sum SAD calculation unit 360 calculates the sum SAD by the frame (t SAD (fn)) by using the equation (3).

Equation 3

Figure PCTKR2012008896-appb-M000003

Here, SAD obtained (fn-1) are between the current frame and summing the SAD for all blocks between the previous frame obtained claim 1 SAD sum, SAD (fn) is the current frame and after the frame combined the SAD for all block the 2 refers to the sum SAD.

SAD found by the sum total SAD calculation unit 360 is the representative value of sizes, i.e. it represents the trace of the difficulty of variability between frames.

If the sum SAD calculation unit 360, a sum determined by representing the SAD frame graphically shown in FIG. 7, it represents the sum SAD (tSAD (fn)) of the frames in a scene (Scene) to be tracked.

Tracking error computation unit 370 using a sum SAD of each frame obtained by the sum total SAD calculation unit 360 calculates the frame-to-frame tracking error value.

That is, the tracking error computation unit 370 to accumulate the sum SAD and their sum SAD obtained from the previous frame for each frame of the scene to be tracked is determined for tracking error value.

For example, assuming that fn the key frame, from the tracking error value between the later in the current frame (fn) the frame (frame (fn + 1)) is, but ask using equation 3, frame (fn + 1) then the sum of the successive error between the frame (fn + 2) is of the as a result, in a frame (fn + 1) by take into account the errors reflected in the previous frame tracking error value to the frame (fn + 2) is the previous tracking error value is the sum.

That is, the tracking error value, in the frame fn is the tSAD (fn).

Then, the tracking error value, in the frame fn + 1, fn + 2 frame, the frame fn + N is equal to the equation (4).

Equation 4

Figure PCTKR2012008896-appb-M000004

Tracking error computation unit frame-by-frame tracking error values ​​calculated by formula (370) is represented by a graph like Fig.

8, the tracking error value of a case of frame fn the key frame, the frame f (n + 1) is the result of accumulating the sum in his SAD tracking error value of frame fn.

Further, the tracking error value of the frame f (n + 2) it can be seen that the result of the cumulative total of his SAD value of the tracking error f (n + 1).

Keyframe selection unit 380 selects a tracking error computation unit 370 Tracking error is the smallest frame track error value from the value obtained from a key frame.

For example, referring to Figure 8, it can be seen that the tracking error ohjeok value in a particular frame fn most small. Therefore, the key frame selection unit 380 selects a key frame of the scene to fn.

Consequently, the selection after deriving a graph similar to Figure 8 for each frame of each footprint, that is, a frame that minimizes the sum of the error curve as a key frame.

A feature point setting unit 300, the condition of the selected key frame using the SAD premise configured as described above is tracking (tracking) of the particular object to another that contains the movement in a video image are more the movement of the image relative a larger tracking error of the image. Of course, every increase is much in motion tracking error, but it is wise movement will average the volatility of a particular object to be tracked much more handsome This is because making it difficult to trace.

Therefore, the feature point setting unit 300 calculates the SAD between frames, it will be the selection of a key frame by using the SAD.

Figure 9 is a block diagram schematically showing the configuration of an object tracking apparatus using a feature point according to an embodiment of the invention.

9, the object tracking device 900 using the feature point includes a feature point setting section 910, the image analysis unit 920, object-tracking section 930.

A feature point setting section 910 checks whether the validity of the feature point corresponding to the initial position of the object specified by the user and sets the feature points of the object.

That is, the feature point setting section 910 to form a random block of about the feature point specified by the user in the input image, set the range of movement of the formed block, the frequency transform of each block within the range of movement of the block change for each block also calculates a representative value. Then, a feature point setting unit 910 sets the center point of the movement range of the largest free block is changed a representative value in the final characteristic point. Description of how a feature point setting unit 910 to set the feature point is the same with the operation of a feature point setting unit in FIG. 3 and the detailed description thereof will be omitted.

Image analysis unit 920 extracts the motion vector information and the residual signal information of each frame in the input image.

After the object tracking unit 930 will generate a forward motion vector information for each unit block from the extracted motion vector information, and restoring the pixel information on a certain block from the residual signal (Residual Signal) of the extraction, the feature point position information of the feature point set in the setting unit 910, and generates the optimum location information of the object in each frame from the forward motion vector information and the reconstructed pixel information.

In accordance with another embodiment of the invention, an object tracking unit 930 by using a pixel value difference between the current frame and a previous frame including the feature points set in the feature point setting section 910 determines the object feature point candidates of the current frame, and Searching for similar template with predetermined templates in a predetermined area surrounding the current object, the feature point candidate of the frame to determine the feature point object for the current frame.

That is, the object tracking section 930 calculates an optical flow (optical flow) of the object feature point set in the previous frame and the feature point set to the current using a pixel value difference between the frame portion 910, by using the calculated optical flow and the determining the object feature point candidate in the current frame. Then, the object tracking unit 930 is a template matching (template matching), the use by surrounding the similar template and including object feature points set in the feature point setting section 910 of the determined object feature point candidates of the current frame given searches in the area.

In the above, but with two kinds of dividing describe how the object tracking unit 930 for tracking the object by using a characteristic point, which can take advantage of a variety of conventional methods.

10 is a flowchart illustrating a feature point set according to the present invention showing a method of resetting the feature point designated by the user, Figure 11 is an exemplary view for explaining a case redirecting feature points in accordance with the present invention.

And 10, a feature point setting unit 300 receives and stores the specified feature point corresponding to the initial position of the object from a user for an input image (S1002).

And after execution of the S1002, a feature point setting unit 300 may form any of a block and (S1004) around a feature point in the input image specified above, set the range of movement of the formed block (S1006). That the feature point setting unit 300 sets the range of movement means that the said block is formed with a plurality of blocks defined by moving within the moving range. For example, a moving range is (2 * 2 * dx + dy) if it is defined as the range of movement, the (2 * 2 * dx + dy) of the block may be defined.

After execution of the S1006, a feature point setting unit 300, converts the frequency of each of the blocks defined within the range of movement is obtained for each of the representative value of the block is also changed (S1008). That is, the feature point setting unit 300, converts the frequency of each block, and calculates a representative value of a block by changing the value of the combined high-frequency area in the frequency transformed image.

After execution of the S1008, a feature point setting unit 300 redirects the center point of the block of the largest Ido representative value from the change in the block defined in the moving range in the final feature points (S1010).

A feature point setting unit 300 with reference to FIG 11 how to reset the feature point set by the user will be described.

11, when the user, such as a hand-selected by the outline and feature points, for example, a redirecting the feature point having a better score to the feature point setting unit navigate around as b to improve the accuracy of tracking.

If you track the object using the minutia redirected in this manner, it is possible to minimize errors in the subsequent object tracking.

Of course, if it is determined a re-designated as a feature point a different location from the user's intention, the feature point is also possible to set up to use the defined position. However, typically FULL HD or HD image resolution for the specified feature point material in the vicinity of 1280x720 or 1920x1080 is because the surrounding 3x3, 5x5 pixels are also the practical feasibility, due to the difficulty of distinguishing fact in the user's point of view.

12 is a flowchart illustrating an object tracking device according to the invention illustrating a method for tracking an object by using a feature point.

12, the object tracking unit 900 checks the validity whether the feature point corresponding to the initial position of the object specified by the user and sets the feature points of the object (S1202). That is, the object tracking device 900 to form a random block of about the feature point specified by the user in the input image, set the range of movement of the formed block, and each block is defined within the range of movement frequency conversion change for each block also calculates a representative value. Then, the object tracking device 900 sets the center point of the movement range of the largest free block is changed a representative value in the final characteristic point.

After execution of the S1202, the object tracking unit 900 extracts the motion vector information and the residual signal information of each frame in the input image (S1204).

After execution of the S1204, the object tracking unit 900 generates the optimum location information of the object in each frame by using the location information, the motion vector information and the residual signal information of the feature point set (S1206). That is, the object tracking device 900 of the after generating the forward motion vector information for each unit block from the extracted motion vector information, and restoring the pixel information on a certain block from the extracted residual signal, the feature points wherein the set position information, and generates the optimum location information of the object in each frame from the forward motion vector information and the reconstructed pixel information.

In other words, the object tracking unit 900 predicts the position coordinates of each characteristic point is moved in the next frame from the feature point information, the initial position and the forward motion vector information of the previous frame. At this time, the object tracking unit 900 extracts at least one candidate position coordinates from the position coordinates of the predicted feature points to find the position coordinates of the feature point more accurate. That is, the object tracking unit 900 extracts a candidate location by selecting the coordinates in a certain range around the position coordinates of the predicted feature points. Then, the object tracking apparatus 900 measures the similarity energy texture, shape similarity energy and motion energy similarity of each candidate feature point position coordinates, respectively.

The present invention can be realized as a code which the computer can read in a computer-readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system.

13 is a flowchart illustrating a feature point set according to the present invention showing a method of selecting a key frame.

13, the feature point setting unit 300 is divided into blocks of a predetermined number of predetermined frames of the input image (S1302).

Then, feature point set is obtained apparatus 300 are each a SAD between the current frame and the adjacent frame for each block (S1304).

That is, the feature point setting unit 300 obtains each of the SAD, SAD between the current frame and after the frame between the current frame and the previous frame for each block. The SAD is the sum of the absolute differences of any pixel in the block, resulting in the expression of the difference block in the same position between frames.

A feature point setting unit 300, after executing the S1304 is by adding together the SAD of each block for each frame, calculate the sum SAD (S1306).

That is, a feature point setting unit 300 is the current sum of the frame and all of the blocks of SAD between the previous frame to obtain the sum of claim 1 SAD, combining said current frame and all the blocks of the SAD between subsequent frames Sum claim 2 SAD, the claim 1 SAD set and said second combined sum SAD calculated by the SAD sum frame.

After execution of the S1306, a feature point setting unit 300 calculates the frame-to-frame tracking error value using a sum SAD by the each frame (S1308).

That is, the feature point setting unit 300 is to accumulate the total sum SAD and their sum SAD obtained from the previous frame for each frame of the scene to be tracked is determined for tracking error value.

In other words, the feature point setting unit 300 is to seek the track error values ​​for each frame of the scene to be tracked using equation (4) described in FIG.

After execution of the S1308, a feature point setting unit 300 selects a tracking error is obtained the value of the small frame to the key frame (S1310).

A feature point setting unit 300 to the tracking, because for the comparison and calculation to track a simple sum operation, and an object there is the selection of the key frames through the process, but the process as described above is complicated, such as the extremely small amount of calculation before we can predict the most optimal key frame.

Further, the feature point setting unit 300 may be easily selected for the key frame with a simple inspection, operation without performing the tracking (tracking) for each frame in a process as described above.

Keyframe selection method of a feature point setting unit 300 described above is possible to write programs, codes and code segments constituting the program can be easily inferred by programmers in the art.

Further, the program of the key frame selection process of a feature point setting unit 300 is stored in the electronic device the data storage medium (Readable Media) that can be read, to be tracked as a key frame selection unit by being read and executed by the electronic device scenes can be selected from among a keyframe in the frame.

Thus, persons skilled in the art will appreciate that the present invention without changing the technical spirit or essential features may be embodied in other specific forms. Therefore, the embodiment described in the above examples should be understood as illustrative and not be limiting in all aspects. The scope of the invention is intended to be included within the scope of the above description becomes than indicated by the claims, which will be described later, the spirit and scope, and all such modifications as derived from the equivalent concept of the appended claims the invention do.

The present invention is by manually users considering the surrounding pixels, the matrix of the feature point designated by the user to designate the characteristic point of each block by the degree of change to obtain a representative value, the change is also to reset the feature point designated by the user on the basis of the representative value a feature point set that may be applied to object tracking device and method using the same and an apparatus and method.

In addition, the present invention can find the sum of the error the smallest frame with a simple operation without performing a tracking (tracking) for each frame easily selected for the key frame, parameters for estimating the SAD caused by movement of the object error and it can be applied to a feature point setting apparatus and method for selecting key frames to minimize the accumulated error using a.

Claims (21)

  1. The characteristic point defining unit for storing the user for the input image received specifying the feature point corresponding to the initial position of the object;
    Wherein the center of the designated feature points forming a predetermined size of the block, and the block group forming part defining block corresponding to the size of the block formed within a predetermined movement range;
    Changes to obtain a representative value of each frequency change to convert each block defined within the movable range of each of the representative value calculation unit; And
    Feature point determination unit to the center point of the movement range of the largest free block is changed a representative value in the redirect to the final characteristic point;
    A feature point set including a device.
  2. According to claim 1,
    The block forming portion for defining the block as possible within the range of movement to form a random block, and that the formed block set up before, after, left and right movable range of movement including a peripheral pixels with respect to the designated feature points a feature point setting unit, characterized in that.
  3. According to claim 1,
    A feature point setting unit, characterized in that the degree of change by the representative value calculation unit performs a frequency transform on said each block respectively, in the frequency conversion block summing pixel values ​​corresponding to the high frequency region to obtain a representative value change in each block .
  4. According to claim 1,
    After dividing the input to the block a predetermined number of the predetermined frame of the image, calculating the SAD SAD (sum of absolute difference) between the current frame and the adjacent frame for each block to obtain each portion;
    SAD sum calculation section by adding together the SAD between the adjacent frames to obtain the sum SAD of each frame;
    Tracking error calculation unit to obtain the frame-to-frame tracking error value using a sum SAD by the each frame; And
    Key frame selector to select the value of the track error smallest frame as a key frame (Key Frame);
    A feature point setting device according to claim 1, further comprising.
  5. 5. The method of claim 4,
    A feature point setting unit, characterized in that using the equation to the SAD calculation unit to obtain the SAD between the current frame and the previous frame, or between the present frame and the subsequent frames.
    Formula;
    SAD (fn, bm) = Σabs (Frame (fn, bm) pixel (i) - Frame (fn + 1, bm) pixel (i))
    Here, fn is a frame number, bm is an m-th block of the frame, also i is the order of the corresponding pixel block, abs means an absolute value.
  6. 5. The method of claim 4,
    Then the SAD sum calculation unit calculated the claim 2 SAD sum between the current frame and the previous frame combined the SAD for all of the blocks to obtain the sum of claim 1 SAD, summing the SAD for the current frame and all the blocks between subsequent frames, wherein the feature point setting device according to claim 1 to the sum SAD and the second SAD combined to obtain the sum total SAD per frame.
  7. 5. The method of claim 4,
    A feature point setting unit, characterized in that the accumulation value of the SAD sum obtained from the previous frame for each frame of the scene to be tracked in accordance with the equation calculating unit said tracking error and to accumulate their SAD sum to obtain the tracking error value.
    Formula;
    Frame-to-frame tracking error value = Σ (fn + k) tSAD, k = 0 ~ N
    Here, the tSAD (fn) = SAD (fn-1) + SAD (fn), SAD (fn-1) is the 1SAD sum, SAD (fn) is the 2SAD sum, tSAD (fn) is SAD in the frame fn meaning the total.
  8. The feature points on the input image checking whether the validity of the feature point corresponding to the initial position of the object specified by the user setting a feature point of the object setting unit;
    Image analysis unit which extracts motion vector information and the residual signal information of each frame in the input image; And
    Above the extraction movement from the vector information to generate a forward motion vector information for each unit block, and restores the pixel information on a certain block from the extracted residual signal and then, the position information of the predetermined feature point, the forward motion vector information, and object tracking section for generating an optimum location information of the object in each frame from the reconstructed pixel information;
    Object tracking apparatus using a feature point that includes the.
  9. The method of claim 8,
    The feature point setting section, define blocks corresponding to the size of the block so formed in the movement to form a block of the size predetermined around the feature point specified by the user in the input image, a predetermined range is defined within the range of movement converts each of the blocks to obtain a representative value of the frequency change for each block, the object-tracking device using a characteristic point, characterized in that for setting the change in the center point of the block is also the largest representative value to the final characteristic point.
  10. (A) step of storing the user for the input image received specifying the feature point corresponding to the initial position of the object;
    (B) defining a block corresponding to the size of the blocks formed in the movement region to form a block of a predetermined size with respect to the designated feature points, group set in the input image;
    (C) calculating a representative value of the change in each of the frequency conversion for each block defined within the movable range, respectively; And
    And (d) redirecting the center point of the range of movement, the biggest change is the representative value of the block within the final characteristic point;
    A feature point of the feature point setting unit setting method comprising a.
  11. 11. The method of claim 10,
    The step (b), in the input image to form a random block of, including peripheral pixels with respect to the designated feature points, the formed block is set to before, after, it left and right movable range of movement the movement range feature point and up feature point setting apparatus, characterized in that defining the block as possible within.
  12. 11. The method of claim 10,
    Wherein the step (c), a feature point set and wherein performing frequency transformation on each of the block respectively, in the above-frequency transform block summing pixel values ​​corresponding to the high frequency region to obtain a representative value change in each block how to set the feature point.
  13. And storing the user for the input image received specifying the feature point corresponding to the initial position of the object;
    Defining the blocks corresponding to the size of the blocks formed in the movement to form a block of a predetermined size with respect to the designated feature points, the predetermined range from the input image;
    Obtaining a representative value is also changed in each frequency conversion for each block defined within the movable range, respectively; And
    The feature point movement range setting method for the degree of change in the central value comprises the step of redirecting the center point to the final characteristic point of the largest free block is written as a program-readable recording medium in an electronic device.
  14. (A) dividing the block into a predetermined number of predetermined frames of the input image;
    (B) calculating a SAD between the current frame and the adjacent frame for each block of the predetermined number, respectively;
    (C) obtaining a sum SAD of each frame by adding together the SAD between the adjacent frames;
    (D) obtaining a frame-to-frame tracking error value using a sum SAD by the each frame; And
    (E) step of selecting a frame with the smallest value of the tracking error as a key frame;
    Keyframe Selection of the feature point setting device comprising a.
  15. 15. The method of claim 14,
    The step (b), key frame selection method of a feature point setting device for the SAD, SAD between the current frame and after the frame between the current frame and the previous frame for each block of the characterized in that to obtain, respectively.
  16. 15. The method of claim 14,
    Wherein the step (c), and then between the current frame and the previous frame combined the SAD for all of the blocks to obtain the sum of claim 1 SAD, summing the SAD for the current frame and all the blocks between subsequent frames Sum claim 2 SAD , key frame selection method of a feature point setting unit, characterized in that the first combined wherein the sum SAD 2 and the sum SAD to obtain the sum SAD by the frame.
  17. 15. The method of claim 14,
    The step (d), key frame selection of a feature point setting unit, characterized in that by accumulating the accumulated values ​​and their SAD sum of SAD sum obtained from the previous frame for each frame of the scene to obtain the tracking error value method to track .
  18. Dividing the block of the input frame with a predetermined certain number of images;
    Calculating a SAD between the current frame and the adjacent frame for each block of the predetermined number, respectively;
    Stage by adding together the SAD between the adjacent frames to obtain the sum SAD of each frame;
    The sum using the SAD for each frame comprises: obtaining a frame-by-frame tracking error value; And
    The track error value is selected for the step of the small frame to the key frame, key frame selection process comprising the program is recorded in a recording medium readable in the electronic device.
  19. (A) with respect to the input image checking whether the validity of the feature point corresponding to the initial position of the object specified by the user setting a feature point of the object;
    (B) extracting the motion vector information and the residual signal information of each frame in the input image; And
    (C) the extracted motion vector information and generate a forward motion vector information for each unit block from, after restoring the pixel information on a certain block from the extracted residual signal, the location information, the forward movement of the set feature points vector information and generating the optimum location information of the object in each frame from the reconstructed pixel information;
    Object tracking method using a feature point of an object tracking device comprising a.
  20. 20. The method of claim 19,
    Wherein the step (a),
    The process of defining the block corresponding to the size of the blocks formed in the movement to form a block of a predetermined size with respect to the designated feature points, the predetermined range from the input image;
    A process of obtaining a representative value of each frequency change to convert each block defined within the movable range, respectively; And
    Object tracking method using a feature point of the object-tracking device characterized in that the center point of the range of movement, the biggest change is the representative value in the block including the redirection process to determine final characteristic point.
  21. A step of checking the validity for the input image if the feature point that corresponds to the initial position of the object specified by the user setting a feature point of the object;
    Extracting motion vector information and the residual signal information of each frame in the input image; And
    Above the extraction movement from the vector information to generate a forward motion vector information for each unit block, and restores the pixel information on a certain block from the extracted residual signal and then, the position information of the predetermined feature point, the forward motion vector information, and from the reconstructed pixel information using the feature point includes the step of generating the optimum location information of the object in each frame, the object-tracking method is recorded as a program-readable recording medium in an electronic device.
PCT/KR2012/008896 2011-11-24 2012-10-26 Apparatus and method for setting feature points, and apparatus and method for object tracking using same WO2013077562A1 (en)

Priority Applications (4)

Application Number Priority Date Filing Date Title
KR20110123376A KR101624182B1 (en) 2011-11-24 2011-11-24 Apparatus and Method for selecting key frame
KR20110123372A KR101700276B1 (en) 2011-11-24 2011-11-24 Apparatus and Method for setting feature point, tracking object apparatus and method usig that
KR10-2011-0123376 2011-11-24
KR10-2011-0123372 2011-11-24

Publications (1)

Publication Number Publication Date
WO2013077562A1 true true WO2013077562A1 (en) 2013-05-30

Family

ID=48469967

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2012/008896 WO2013077562A1 (en) 2011-11-24 2012-10-26 Apparatus and method for setting feature points, and apparatus and method for object tracking using same

Country Status (1)

Country Link
WO (1) WO2013077562A1 (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100466587B1 (en) * 2002-11-14 2005-01-24 한국전자통신연구원 Method of Extrating Camera Information for Authoring Tools of Synthetic Contents
KR20050085842A (en) * 2002-12-20 2005-08-29 더 파운데이션 포 더 프로모션 오브 인더스트리얼 사이언스 Method and device for tracing moving object in image
KR20080017521A (en) * 2006-08-21 2008-02-27 (주)휴먼세미컴 Method for multiple movement body tracing movement using of difference image
KR100958379B1 (en) * 2008-07-09 2010-05-17 (주)지아트 Methods and Devices for tracking multiple 3D object, Storage medium storing the same
KR100996209B1 (en) * 2008-12-23 2010-11-24 중앙대학교 산학협력단 Object Modeling Method using Gradient Template, and The System thereof

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100466587B1 (en) * 2002-11-14 2005-01-24 한국전자통신연구원 Method of Extrating Camera Information for Authoring Tools of Synthetic Contents
KR20050085842A (en) * 2002-12-20 2005-08-29 더 파운데이션 포 더 프로모션 오브 인더스트리얼 사이언스 Method and device for tracing moving object in image
KR20080017521A (en) * 2006-08-21 2008-02-27 (주)휴먼세미컴 Method for multiple movement body tracing movement using of difference image
KR100958379B1 (en) * 2008-07-09 2010-05-17 (주)지아트 Methods and Devices for tracking multiple 3D object, Storage medium storing the same
KR100996209B1 (en) * 2008-12-23 2010-11-24 중앙대학교 산학협력단 Object Modeling Method using Gradient Template, and The System thereof

Similar Documents

Publication Publication Date Title
US7324670B2 (en) Face image processing apparatus and method
US5103305A (en) Moving object detecting system
US20060215903A1 (en) Image processing apparatus and method
US5991428A (en) Moving object detection apparatus and method
US20090196492A1 (en) Method, medium, and system generating depth map of video image
US20050129311A1 (en) Object detection
Yu et al. Trajectory-based ball detection and tracking in broadcast soccer video
US6411339B1 (en) Method of spatio-temporally integrating/managing a plurality of videos and system for embodying the same, and recording medium for recording a program for the method
US20050129276A1 (en) Object detection
JP2008219570A (en) Camera topology information generating device
Liu et al. Key frame extraction from MPEG video stream
Venkatesh et al. Efficient object-based video inpainting
JP2006510240A (en) Media processing system
US20050129275A1 (en) Object detection
JP2000222584A (en) Video information describing method, method, and device for retrieving video
US20080253617A1 (en) Method and Apparatus for Determining the Shot Type of an Image
Li et al. Automatic detection and analysis of player action in moving background sports video sequences
JP2008252296A (en) Face index preparation apparatus for moving image and face image tracking method thereof
JP2009199322A (en) Monitoring system, and person retrieval method
EP1542154A2 (en) Object detection
EP2562715A1 (en) Portable electric equipment and method of processing a series of frames
JPH06231252A (en) Method for tracking moving object on monitoring picture
US20080013837A1 (en) Image Comparison
WO2006025272A1 (en) Video classification device, video classification program, video search device, and videos search program
JP2006508461A (en) Face detection and face tracking

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12851233

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase in:

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12851233

Country of ref document: EP

Kind code of ref document: A1