KR20120060458A - Method of tracking object and apparatus for the same - Google Patents

Abstract

PURPOSE: An object tracing method and an apparatus thereof are provided to trace an object through a code book of an object and to stably trace the object in various environments. CONSTITUTION: A modeling unit updates a pixel within a tracing target object area to a codeword. The modeling unit removes a pixel of low frequency from the codeword. A probability map generating unit generates a probability map through the codeword from which the pixel is removed. A tracing unit(105) traces the tracing target object through the generated probability map.

Description

Object tracking method and apparatus therefor {METHOD OF TRACKING OBJECT AND APPARATUS FOR THE SAME}

The present invention relates to an object tracking method and apparatus, and more particularly, to an object tracking method and apparatus for tracking an object using a codebook of the object to be tracked.

Moving object tracking is a field where much research is being conducted in computer vision and various applications. The moving object tracking system refers to tracking a moving object from image data obtained from a camera. Moving object tracking is classified into the case where the camera is fixed and the object moves according to the moving relationship between the camera and the object, when the object is fixed, the camera moves, and the object moves and the camera moves.

Conventional methods for tracking a moving object include a tracking method using a difference image and an optical flow method. The tracking method using the difference image is a method in which the camera is fixed and applicable when the object moves and tracks the object using the background image and the difference image. The method of using the difference image is very limited by the change of the surrounding environment, and if the camera is moved, it is difficult to obtain the background image, and thus the wrong result is obtained in the preprocessing process.

The method of tracking an object using light flow is applicable when one of the camera and the object is fixed. The optical flow method recognizes and tracks the movement of an object by using a space-time vector when the object obtained in the previous frame is in one position of the next frame through a change in time and space. However, there is a disadvantage in that the processing speed is slow due to the large amount of calculation.

A first object of the present invention for solving the above problems is to provide an object tracking device for tracking an object using a codebook of the object to be tracked.

A second object of the present invention for solving the above problems is to provide an object tracking method for tracking an object using a codebook of the object to be tracked.

An object tracking apparatus according to an embodiment of the present invention for achieving the first object of the present invention, a conversion unit for receiving image data from a plurality of image recording apparatus, and converts the received image data to a pixel value, Determination unit for determining whether the tracking object is modeled in the converted image data, If the determination result is not modeled when there is a pixel in the tracking object region in the codeword of the codebook of the converted image data A modeling unit for updating the pixels in the area of the tracking object to the codeword, removing the less frequent pixels from the codeword, and generating a probability map using a codeword from which the less frequent pixels are removed A tracking unit that tracks the object to be tracked using the unit and the generated probability map. The.

In accordance with another aspect of the present invention, there is provided a method for tracking an object, the method including receiving image data from a plurality of image photographing apparatuses, converting the received image data into pixel values, and converting the image data. Determining whether the tracking object is modeled from the image data, and if the pixel in the tracking object area exists in a codeword of the codebook of the converted image data when the determination result is not modeled, the tracking object Updating a pixel in an object region to the codeword, removing a less frequent pixel from the codeword, generating a probability map using a codeword from which the less frequent pixel is removed, and generating the generated probability map Tracking the object to be tracked using;

When using an object tracking method and apparatus for tracking an object using the codebook of the object to be tracked according to the present invention as described above, it is possible to track the object stably and consistently in various environments. In addition, the present invention can be used as a basic technology required to track a person or an object, and can also be used in the fields of intelligent robots, security and surveillance, smart environments, telematics, and the like.

1 is a block diagram showing the internal structure of an object tracking apparatus according to an embodiment of the present invention.
2 is a flowchart illustrating a process of tracking an object according to an embodiment of the present invention.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. It should be understood, however, that the invention is not intended to be limited to the particular embodiments, but includes all modifications, equivalents, and alternatives falling within the spirit and scope of the invention. Like reference numerals are used for like elements in describing each drawing.

The terms first, second, A, B, etc. may be used to describe various elements, but the elements should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

When a component is referred to as being "connected" or "connected" to another component, it may be directly connected to or connected to that other component, but it may be understood that other components may be present in between. Should be. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that there is no other component in between.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art and shall not be construed in ideal or excessively formal meanings unless expressly defined in this application. Do not.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing the internal structure of an object tracking apparatus according to an embodiment of the present invention.

Referring to FIG. 1, an object tracking apparatus includes a receiver 101, a converter 102, a determiner 103, a controller 104, a modeling unit 105, a probability map generator 106, and a tracking unit 107. ), And the modeling unit 105 may include a comparison unit 113, an update unit 123, and a codeword removal unit 133. The receiver 101 receives image data photographed from a plurality of image photographing devices of various types. Here, the image capturing apparatus may include, for example, a multi-view video camera or a stereo video camera and a depth camera.

The converting unit 102 outputs the image data received from the receiving unit 101 as it is under the control of the control unit 104 or converts the received image data into a color space and outputs the converted color space. Here, when the conversion unit 102 outputs the image data received from the reception unit 101 as it is, the setting of the object tracking device may be a case where the image data is not converted to the color space. That is, when the setting of the object tracking device does not convert the image data into the color space, the conversion unit 102 provides only the bypass path.

On the other hand, when the converter 102 converts the image data received from the receiver 101 into a color space and outputs the color data, the object tracking apparatus may be configured to not convert the image data into the color space. According to an exemplary embodiment of the present invention, the conversion unit 102 converts the image data of the image data received from the reception unit 101 into pixel values between 0 and 255, thereby converting the image data into the (R, G, B) color space. I can convert it. According to an exemplary embodiment of the present invention, the conversion unit 102 converts the color difference signal of the image data received from the reception unit 101 to the Y component, which is the brightness of color (IuminanCe), and the U (Cb) and V (Cr, which are chromatin Ce). ), And the Y component is sensitive to errors, so that the ratio of Y: U: V is 4: 2: 2 so that it can be coded into more bits than the productivity U and V.

The determination unit 103 may determine whether the image data converted from the conversion unit 102 is modeled under the control of the control unit 104. First, when the determination unit 103 determines that the tracking target object is modeled in the image data converted from the conversion unit 102 under the control of the control unit 104, the modeling unit 105 provides only a bypass path. Done. On the other hand, when the determination unit 103 determines that the tracking target object is not modeled in the image data converted from the conversion unit 102 under the control of the control unit 104, the modeling unit 105 controls the control unit 104. The codeword of the image data converted into the color space is received from the conversion unit 102 under the control of the control unit, and the tracking object is modeled using the codebook and the codeword. Here, the codebook is composed of a plurality of codewords, the i-th codeword Cw _i has the components I _{< min , i>} , I _{< max , i>} , f _i , C _i , I _{< min , i >} Denotes the brightness of the darkest color in codeword i, I _{< max , i>} denotes the brightness of the brightest color in codeword i, and f _i denotes the number of pixels in the i-th codeword. Frequency, and C _i represents a vector C _{1 , i} , C _{2 , i} , C _{3 , i} composed of three dimensions representing the color of the i-th codeword.

The controller 104 may determine whether to convert the color space of the image data received from the receiver 101 by using the setting of the object tracking device. First, when the setting of the object tracking device is set such that the image data does not need to be converted to the color space, the control unit 104 does not require color space conversion of the image data received from the receiving unit 101, so that the control unit 104 ) Is set so that the conversion unit 102 does not perform color space conversion of the image data. On the other hand, when the setting of the object tracking device is set that the image data should be converted to the color space, the control unit 104 needs to convert the color space of the image data received from the receiving unit 101, so that the control unit 104 converts the data. The control unit 102 controls the color space conversion of the image data.

The controller 104 may determine whether the image data converted from the converter 102 is modeled. First, when the controller 104 determines that the object to be tracked is modeled from the image data converted by the converter 102, the controller 104 determines that the object to be tracked is converted from the image data converted from the converter 102. In 105), the modeling is not performed. On the other hand, when the controller 104 determines that the object to be tracked is not modeled in the image data converted by the converter 102, the controller 104 models the object to be tracked in the image data converted by the converter 102. In step 105, the modeling is controlled.

The comparing unit 113, the modeling unit 105 compares the pixel V _new in the code word and a tracking object area of the image data and outputs the presence or absence of a code word of the pixel V _new is the image data in the tracking object area. According to an exemplary embodiment of the present invention, the comparison unit 113 uses the i th codeword of the N codeworks in the codebook of the image data converted from the V _new and the conversion unit 102 into the color space using Table 1 below. Perform a comparison on Cw _i . Here, the pixel of V _new has the components of C _new and I _new , and C _new represents the three-dimensional vector C _{1 , new} , C _{2 , new} , C _{3 , new} representing the color of the pixel, I _new Represents the brightness of the pixel.

Referring to Table 1, the comparison unit 113 performs comparison by the number N of code words in the codebook of the image data converted from the conversion unit 102 into the color space under the control of the control unit 104. If the comparing unit 113, the tracking object area pixel V _new color of the C _new and the color of the image data code word C _i is the same, and tracking the object pixel V _new brightness I _new and video data of in the region in the If the brightness of the codeword <I _{< min i>} , I _{< max i>>} is the same, and it is determined that pixel V _{new in} the result tracked object region exists in the codeword of the image data, the index i of the codeword is set to i _rep . Allocate it and return it to finish.

If the comparing unit 113 of the comparison result and tracking the object pixel V _new brightness I _new and video data of in the region of the color C _i of the code word of the pixel V _new color C _new and the image data within a tracked object area If the comparison result of the brightness <I _{< min i>} and I _{< max i>>} of the codewords is not the same, null is assigned to i _rep and the comparison is repeated. In this case, the comparison unit 113 compares the color C _new of the pixel V _new in the tracking object region with the color C _i of the codeword of the image data by using Equation 1 below, and uses Equation 2 below. and compares the tracked object to the brightness of the code word of the pixel V _new brightness I of the _new image data in the region _{<I <min i>, I} <max i>>.

를 나타내고, Ci는 영상 데이터 코드워드의 i번째 코드워드의 색상을 대표하는 C_{1 ,i,} C_{2 ,i,} C_{3 ,i} 3차원으로 구성된 벡터를 나타내고, Cnew는 추적 대상 물체 영역내의 픽셀 V_new의 색상 C_new 을 대표하는 3차원으로 구성된 벡터 C_{new ,1}, C_{new ,2}, C_{new , 3} 를 나타내고, TH_C는 미리 결정된 임계치를 나타낸다.
Referring to Equation 1, C is

Ci represents a vector consisting of C _{1 , i,} C _{2 , i,} C _{3 , i} three-dimensional vectors representing the color of the i th codeword of the image data codeword, and Cnew represents a pixel V in the tracking object region. vector constructed in three dimensions, representing the color of the _{new C new C new, 1,} C new, represents _{2, C new, 3, TH} C represents a predetermined threshold.

을 타나낸다. 비교 유닛(113)은 추적 대상 물체 영역내의 픽셀 V_new 의 밝기 I_new 가 영상 데이터의 코드워드의 가장 어두운 색의 밝기 I_{< min , i>} 와 미리 결정된 임계값 TH_min 의 차이보다 크고 영상 데이터의 코드워드의 가장 밝은 색의 밝기 I_{< max , i>} 와 미리 결정된 임계값 TH_min 의 합보다 작은지 비교한다.Referring to Equation 2, I

Appears. The comparing unit 113 is tracking the object area pixel V _new brightness I _new is large, the image data than the threshold difference TH _min pre-determined and the darkest brightness of the color I _{<min, i>} of the code words of the video data in the The brightness of the brightest color of the codeword I _{< max , i>} is compared with the sum of the predetermined threshold TH _min .

The update unit 123 updates the codeword according to the comparison result of the comparison unit 113. First, when the comparison unit 113 determines that the pixel V _{new in the} tracking object region exists in the codeword of the image data, the codeword is updated as shown in Equation 4 below.

Referring to Equation 3, the update unit 123 includes the pixels included in the codewords in the vectors C _{1 , i} , C _{2 , i} , C _{3 , i} of the color C _i of the codeword of the image data, respectively. Multiply the number / frequency f _i of by and add the vectors (C _{1 , new} , C _{2 , new} , C _{3 , new} ) of the color C _new of the tracked object. The update unit 123 performs the update by dividing the number / frequency f _i of the pixel by increasing the color C _new of the tracking object to the color C _i of the codeword of the image data.

In addition, the updating unit 123 compares the brightness I _min of the darkest color of the codeword of the image data with the brightness I _new of the tracking object and compares the small value to the brightness I _{< min ,} of the darkest color of the codeword of the image data _{. i>,} and compare the brightness I _max of the brightest color of the codeword of the image data with the brightness I _new of the tracked object and compare the large value to the brightness I _{< max , i>} Perform the update.

Second, when the comparison unit 113 determines that the pixel V _{new in the} tracking object region does not exist in the codeword of the image data, the update unit 123 generates a codeword as shown in Equation 4 below. .

Referring to Equation 4, the update unit 123 generates the color Ci of the i th codeword of the tracking object as (C _{1 , new} , C _{2 , new} , C _{3 , new} ), and The brightness of the darkest color in the codeword is generated as I, the brightness of the lightest color in the codeword i is generated as I, and the number / frequency of pixels included in the i-th codeword is generated as 1.

After the generation of the code word of the object to be tracked through the update unit 123 is completed, the codeword removal unit 133 uses <Table 2> to remove codewords having a small number of components.

Referring to Table 2, if the codeword removal unit 133 repeats the number N of codewords, the codeword removal unit 133 removes codewords having a small number of components. If the number / frequency of pixels included in the codeword is smaller than the predetermined threshold TH _p , the codeword removal unit 133 removes the corresponding codeword.

The probability map generator 106 generates a probability map by using a codebook of the tracking target object received from the modeling unit 105. According to an embodiment of the present invention, when the color value of the (x, y) position of the newly input image is C (x, y), the probability map generator 106 uses the codebook of the tracking object to determine the probability map. In the following description, the probability map generator 106 searches a codeword in the codebook including C (x, y) in the codebook of the tracking object received from the modeling unit 105, and the search result C. If a codeword exists in the codebook containing (x, y), the index i _selected is returned. If the codeword does not exist in the codebook containing C (x, y), the search returns null. To create a probability map. Here, the probability value of the probability map when the codeword does not exist in the codebook containing C (x, y) becomes 0, and is returned when the codeword exists in the codebook containing C (x, y). The frequency f _iselected of the codeword cw _selected is a probability value.

The tracking unit 107 tracks an object to be tracked using the probability map generated from the probability map generating unit 106. According to an embodiment of the present invention, the tracking unit 107 may apply the probability map generated from the probability map generator 106 to the Mean Shift TraCking algorithm to track the object to be tracked, and the Mean Shift TraCking algorithm may use the probability map. By using the probability map generated by the generation unit 106 to search the center of the largest probability distribution value of the area of the tracking object by repeatedly searching the slope value of the probability distribution to find the center value of the probability distribution in the frame. The object to be tracked can be tracked.

According to an embodiment of the present invention, the tracking unit 107 may apply a Kalman filter (Kaman FiIter) algorithm to the probability map generated from the probability map generator 106 to track the object to be tracked. If there is a linear relationship between the tracked object in the previous frame and the tracked object in the current frame, the position of the tracked object in the current frame is tracked and continuously based on the position of the tracked object tracked in the previous frame. The measured value can be used to track the position of the tracking object.

2 is a flowchart illustrating a process of tracking an object according to an embodiment of the present invention.

Referring to FIG. 2, the object tracking apparatus receives image data photographed from a plurality of image photographing apparatuses of various types and converts the received image data into a color space (S201). Here, the image capturing apparatus may include, for example, a multi-view video camera or a stereo video camera and a depth camera. According to an embodiment of the present invention, the object tracking device may convert the image data into the (R, G, B) color space by converting the primary color signal of the received image data into a pixel value between 0 and 255. According to an embodiment of the present invention, the object tracking device classifies the color difference signal of the received image data into Y component, which is the brightness of color (IuminanCe), and U (Cb) and V (Cr) component, which are chromatin (CrominanCe), and Y Since the component is sensitive to error, the ratio of Y: U: V can be converted to color space by 4: 2 to code more bits than the productivity U and V.

The object tracking apparatus determines whether the converted image data is modeled (S202). If it is determined that the converted image data is modeled (S202), the object tracking apparatus generates a probability map using a codebook (S207). The tracking target object is tracked using the generated probability map (S208). According to an embodiment of the present invention, when the color value of the (x, y) position of the newly input image is C (x, y), a process of generating a probability map using a codebook of a tracking object will be described. The object tracking device searches a codeword in a codebook including C (x, y) in the codebook of the object to be tracked, and if the codeword exists in the codebook including C (x, y) as a result of the search, i _selected If the codeword does not exist in the codebook containing the search result C (x, y), returns null to generate a probability map. Here, the probability value of the probability map when the codeword does not exist in the codebook containing C (x, y) becomes 0, and is returned when the codeword exists in the codebook containing C (x, y). The frequency f _iselected of the codeword cw _selected is a probability value.

On the other hand, if it is determined that the converted image data is not modeled (S202), the object tracking apparatus determines that the pixels in the tracking object region exist in the codeword of the image data (S203). If it is determined that a pixel in the speculative object region does not exist in the codeword of the image data (S203), the object tracking apparatus generates a codeword (S204), and the pixel in the tracking object region is a codeword of the image data. If it is determined to exist at (S203), the pixel in the tracking object region is updated to the codeword of the image data (S205). In this case, the object tracking apparatus determines whether pixels in the tracking object region exist in the codeword of the image data by the number N of codewords in the codebook of the image data.

If it is determined whether the number of pixels in the tracking object region exists in the codeword of the image data by the number N of codewords (S206), the object tracking apparatus generates a probability map using the codebook (S207). The tracking target object is tracked using the generated probability map (S208). According to an embodiment of the present invention, the object tracking device may track a target object by applying a probability map to a Mean Shift TraCking algorithm, and the Mean Shift TraCking algorithm finds a center value of a probability distribution in a frame by using a probability map. The object to be tracked can be tracked by repeatedly searching for the slope of the probability distribution and searching for the center of the largest probability distribution value of the area of the object to be tracked.

According to an embodiment of the present invention, the object tracking device may track a target object by applying a probability map to the KaIman FiIter algorithm, and the Kalman filter algorithm may track the target object in the previous frame and the current frame. If there is a linear relationship between the tracked objects, the tracked object position is tracked in the current frame based on the estimated position of the tracked object estimated in the previous frame, and the tracked object position is measured using the continuously measured values. It can be estimated.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the present invention as defined by the following claims It can be understood that

101: receiver 102: converter
103: modeling unit 113: comparison unit
123: update unit 133: code word removal unit
104: probability map generator 105: tracking unit

Claims (1)

A converter which receives image data from a plurality of image photographing apparatuses and converts the received image data into pixel values;
A determination unit which determines whether or not a model of a tracking target is modeled in the converted image data;
If the result of the determination is not modeled, when the pixel in the tracking object region exists in the codeword of the codebook of the image data, the pixel in the tracking object region is updated to the codeword, and the frequency in the codeword A modeling unit to remove a few pixels;
A probability map generator for generating a probability map by using a codeword from which pixels with less frequency are removed; And
And a tracking unit to track the object to be tracked using the generated probability map.

Images