KR20200050553A - Motion tracking system and method - Google Patents

Abstract

According to one embodiment of the present invention, a system for tracking motion comprises: an image photographing unit obtaining a plurality of video images photographed by a camera; an object recognition unit detecting an object to be tracked for motion from the video image and generating an elliptical tracking model defined as a nonlinear model corresponding to a target area of the detected object; and a motion analysis unit tracking motion of the target area of the object through motion analysis using the elliptical tracking model. The system may check brain diseases.

Description

Motion tracking system and method {MOTION TRACKING SYSTEM AND METHOD}

Embodiments of the present invention relates to a motion tracking system and method for tracking a movement of an object and observing a behavioral pattern, and through this, a brain disease such as a brain disease can be examined without inconvenience of visual inspection.

The detection and tracking of an image object refers to a process of separating an area of a single or multiple image objects meaningful from an input image from the background and giving meaning. Visual tracking algorithms allow you to study mouse movements in an objective and consistent way for a long time compared to manual methods.

Recording an action manually can be implemented at a low cost, but for certain actions, such as interactions between too many objects, a visual tracking algorithm can have significant advantages. A more objective and consistent analysis is possible because the observer does not have to look into the experiment for a long time. Therefore, computer vision algorithms (especially automatic tracking systems) that understand the behavior of objects are very important in various fields.

However, video tracking algorithms developed in the existing video vision field have too many limitations in attaching markers to high-cost experimental equipment, special-purpose cameras, and experimental animals to analyze the behavior of objects. There is a problem that satisfactory results cannot be obtained because it takes a lot of work and is difficult due to difficult tasks and objective and consistent measurements.

Related prior art is Republic of Korea Patent Publication No. 10-2004-0041297 (Invention name: a method for tracking and displaying the position and movement of a moving object using multiple camera images, published date: 2004.05.17) .

One embodiment of the present invention provides a motion tracking system and method for tracking brain motion, such as brain disease, without discomfort of visual inspection through observation of behavior patterns.

The problem to be solved by the present invention is not limited to the problem (s) mentioned above, and another problem (s) not mentioned will be clearly understood by those skilled in the art from the following description.

A motion tracking system according to an embodiment of the present invention includes an image capturing unit acquiring a plurality of video images captured by a camera; An object recognition unit for detecting an object to be tracked from the video image and generating an elliptical tracking model defined as a nonlinear model corresponding to the target area of the detected object; And a motion analysis unit tracking motion of a target region of the object through motion analysis using the elliptical tracking model.

The object recognition unit detects the skin surface of the object from the video image to generate a Gaussian distribution of the skin surface, and a skin model generation unit to generate a Gaussian skin model based on the generated Gaussian distribution; A target detection unit detecting a target region of the object using the Gaussian skin model; And a tracking model generator that generates the elliptical tracking model using an extended Kalman filter.

The skin model generation unit may generate the Gaussian distribution by cropping a sample image of the same size including only pure skin that is not part of the background from the video image.

The target detector acquires an observation vector set as a boundary point of the target region of the object using the Gaussian skin model, and associates the obtained observation vector with the state vector of the object using a function of a nonlinear observation model. Can be.

The state vector is composed of a plurality of parameters related to the elliptical tracking model for indicating the overall shape of the object, wherein the parameter is the center point of the ellipse corresponding to the overall shape of the object, the long axis length, the short axis length, and the rotation Angle and constant speed.

The motion analysis unit may estimate the state vector over time using the elliptical tracking model, and calculate the speed and distance of the object based on the estimated state vector to track the motion of the target area of the object. have.

The motion analysis unit analyzes an operation including at least one of stretching out, huddle up, and stand up of the object based on the estimated state vector to determine a difference in the operation. Quantification and tracking of the movement of the target area of the object may be performed based on the result of the quantification.

A motion tracking method according to an embodiment of the present invention includes the steps of acquiring a plurality of video images captured by a camera by an image capturing unit of a motion tracking system; An object recognition unit of the motion tracking system detects an object to be tracked from the video image, and generates an elliptical tracking model defined as a nonlinear model corresponding to the target area of the detected object; And tracking a movement of the target area of the object through a motion analysis using the elliptical tracking model.

The generating of the elliptical tracking model may include detecting a skin surface of the object from the video image to generate a Gaussian distribution of the skin surface, and generating a Gaussian skin model based on the generated Gaussian distribution; Detecting a target region of the object using the Gaussian skin model; And generating the elliptical tracking model using an extended Kalman filter.

The generating of the Gaussian skin model may include generating the Gaussian distribution by cropping a sample image of the same size including only pure skin that is not part of the background from the video image.

The detecting of the target region of the object may include obtaining an observation vector set as a boundary point of the target region of the object using the Gaussian skin model; And associating the obtained observation vector with the state vector of the object using a function of a nonlinear observation model.

Tracking the movement of the target area of the object may include estimating the state vector over time using the elliptical tracking model; And tracking the movement of the target region of the object by calculating the speed and the moving distance of the object based on the estimated state vector.

The step of tracking the movement of the target area of the object includes at least one of stretching out, huddle up, and stand up of the object based on the estimated state vector. It may further include the step of quantifying the difference in the motion by analyzing the, and tracking the movement of the target region of the object based on the result of the quantification.

Details of other embodiments are included in the detailed description and accompanying drawings.

According to an embodiment of the present invention, behavior patterns can be observed by tracking the movement of an object, and through this, brain diseases such as brain diseases can be examined without inconvenience of visual inspection.

1 is a block diagram illustrating a motion tracking system according to an embodiment of the present invention.
FIG. 2 is a block diagram illustrating a detailed configuration of the object recognition unit of FIG. 1.
3 to 6 are views illustrating a motion tracking method according to an embodiment of the present invention.
7 to 9 are graphs showing the movement path of the mouse based on the measured data (red star), manual tracking (green circle), and tracking algorithm (blue triangle), respectively.
10 and 11 are graphs showing average and standard deviations related to errors in the moving distance and the moving speed.
12 to 14 are graphs showing the entire motion path according to the tracking algorithm (red star) and manual tracking (green circle) in the actual video.
15 and 16 are graphs showing the mean and standard deviation of errors in values obtained using a tracking algorithm and values obtained using a manual tracking.

Advantages and / or features of the present invention and methods for achieving them will become apparent by referring to embodiments described below in detail together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but will be implemented in various different forms, and only the present embodiments allow the disclosure of the present invention to be complete, and common knowledge in the art to which the present invention pertains. It is provided to completely inform the person having the scope of the invention, and the present invention is only defined by the scope of the claims. The same reference numerals refer to the same components throughout the specification.

In addition, a preferred embodiment of the present invention to be carried out below is provided in each system functional configuration in order to efficiently describe the technical components constituting the present invention, or a system function commonly provided in the technical field to which the present invention pertains. The configuration is omitted as much as possible, and the functional configuration that should be additionally provided for the present invention will be mainly described. If a person having ordinary knowledge in the technical field to which the present invention pertains, it will be possible to easily understand the functions of components that have been used in the prior art among the omitted functional configurations not shown below, and also the omitted components as described above. The relationship between the elements and the components added for the invention will also be clearly understood.

In addition, in the following description, the terms "transmission", "communication", "transmission", "reception" of a signal or information, or other similar meanings means that a signal or information is directly transmitted from one component to another. Not only that, it also includes passing through other components. In particular, "transmitting" or "transmitting" a signal or information to a component indicates the final destination of the signal or information, not a direct destination. The same is true for the "reception" of a signal or information.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating a motion tracking system according to an embodiment of the present invention, and FIG. 2 is a block diagram illustrating a detailed configuration of the object recognition unit 120 of FIG. 1.

1 and 2, the motion tracking system 100 according to an embodiment of the present invention includes an image capture unit 110, an object recognition unit 120, a motion analysis unit 130, and a control unit 140 It may include.

The image capturing unit 110 may acquire a plurality of video images captured by a camera. Here, the camera may include a digital camera, and may be composed of one or two or more.

The object recognition unit 120 may detect an object to be tracked from the video image and generate an elliptical tracking model defined as a nonlinear model corresponding to the target area of the detected object.

Specifically, the object recognition unit 120 may include a skin model generation unit 210, a target detection unit 220, and a tracking model generation unit 230, as shown in FIG. 2.

The skin model generation unit 210 may detect the skin surface of the object from the video image to generate a Gaussian distribution on the skin surface, and generate a Gaussian skin model based on the generated Gaussian distribution.

In this case, the skin model generator 210 may crop a sample image of the same size including only pure skin that is not part of the background from the video image, and generate the Gaussian distribution based on the sample image.

The target detection unit 220 may detect the target region of the object using the Gaussian skin model.

The target detector 220 obtains an observation vector set as a boundary point of the target region of the object using the Gaussian skin model, and uses the function of a nonlinear observation model to obtain the observation vector as the state vector of the object. Can be associated with.

Here, the state vector is composed of a plurality of parameters related to the elliptical tracking model for indicating the overall shape of the object, wherein the parameter is the center point of the ellipse corresponding to the overall shape of the object, the long axis length, the short axis length , Rotation angle and constant speed.

The tracking model generation unit 230 may generate the elliptical tracking model using an extended Kalman filter. The complex dynamic / observation model caused by the complex environment in the tracking problem produces a non-Gaussian model. When the dynamic / observation model is a nonlinear model, the optimal properties of the Kalman filter do not apply to the tracking problem. Therefore, it is preferred that the analytical approximation, ie the extended Kalman filter, is used to track the object.

The dynamic / observation model of the extended Kalman filter can be expressed as Equations 1 and 2 below.

[Equation 1]

[Equation 2]

Here, the variable x is the state vector of the target at time t, and consists of 7 parameters to indicate the overall shape of the mouse. The variable y is an observation vector obtained in the detection step of collecting the boundary points of the mouse. The random variables w _t-1 and v _t represent process and observed noise, respectively. The nonlinear function model f is a function that evolves the target's state variable, and the nonlinear function observation model h is a function that associates the state vector with the observation vector y _t . If the f and h functions are linear, we can perform the calculation using the Kalman filter technique. However, if f and h are nonlinear functions, the state variable can be estimated using the extended Kalman filter technique. The extended Kalman filter can be obtained by approximating a predicted distribution, a posterior distribution, and a marginal likelihood distribution.

The elliptical tracking model can be used to represent the overall shape of the object. By tracking the overall shape of the object, it is possible to distinguish various changes in behavior, such as bending or erecting and stretching the body. In addition, using the elliptical tracking model, it is possible to calculate the velocity and the moving distance of the object based on the state vector of the ellipse estimated over time, including the average rate of change and the relative frequency of each state vector. Approximate values for the shape, size, and speed of the object during operation can be obtained at relative frequencies useful for obtaining information about the operation of the object.

On the other hand, the observation vector is closely related to the state vector, and may be set as a boundary point of an ellipse that can be obtained using a simple point detection algorithm. The relationship model between the state vector and the observation vector is expressed by Equation 3 below.

[Equation 3]

는 x축 주위의 회전각이다. 관측 벡터의 우도 함수는 아래의 수학식 4와 같은 가우시안 모델을 사용하여 근사할 수 있다.Here, the variable θ takes a value from 0 to 2π,

Is the rotation angle around the x-axis. The likelihood function of the observation vector can be approximated using a Gaussian model as shown in Equation 4 below.

[Equation 4]

Here, the Gaussian standard deviation σ can be set by the experiment designer. Dt is the Euclidean distance between the subject's predicted observation vector y and the observation vector obtained in the detection step. The equation of the model is shown in Equation 5 below.

[Equation 5]

Therefore, the model can be applied to the extended Kalman filter framework.

Hereinafter, as an example of a process for detecting an object, the object will be described with a mouse. This is for convenience of description and easy understanding, and is not intended to limit the scope of the present invention.

The most difficult problems for tracking objects are complex environments, as well as unpredictable fast movements and inconsistent shapes. Model presentation to track the overall shape of the mouse, including its tail and background (cage), is a major barrier to accurate tracking. Therefore, in order to successfully track the mouse, we propose a mouse tracking model consisting of repeated detection and tracking steps. The limitation of this simple detection method is that it does not work well in situations where the mouse's skin color and background color are similar. When tracking highly deformable objects, separating the tracked object's data from the background increases the robustness of the tracker.

Therefore, the first step is to create a Gaussian distribution of the mouse skin surface. In particular, a Gaussian distribution can be created by manually cropping a 30 * 30 sample image of the same size that contains only pure skin that is not part of the background. Once the skin model is generated, the predicted area in the mouse is calculated for each pixel of the input image by calculating the Maharan Novis distance from the mean. The skin likelihood value can then be displayed as a grayscale image. White areas are more likely to become skin areas. After the likelihood value (changing from 0 to 1) is calculated for each pixel, all pixel values where the likelihood value is higher than the threshold are set to 1 and the remaining pixels are set to 0. The initial threshold is initiated by the tracking procedure or automatically calculated by the program and then three types of actions are used to remove the areas representing the tail and ears of the mouse, barriers to tracking, using erosion, expansion, and filling.

The mouse tail of the binary image can be removed by the erosion using a small rectangle (optionally setting the initial rectangle size to n * n) that can peel the pixel layer from the boundary of the binary area. The expansion has the opposite effect of expansion, and adds a pixel layer to the border of the region. In addition, the filling can be realized by filling a hole in a binary area when a hole exists in the image. Proper use of this method can increase the accuracy of tracking and discriminate mice from the background in complex environments.

The motion analysis unit 130 may track motion of a target region of the object through motion analysis using the elliptical tracking model.

Specifically, the motion analysis unit 130 estimates the state vector over time using the elliptical tracking model, and calculates the speed and the moving distance of the object based on the estimated state vector to target the object You can track the movement of the area.

Further, the motion analysis unit 130 analyzes an operation including at least one of stretching out, huddle up, and stand up of the object based on the estimated state vector. By quantifying the difference in the motion, it is possible to track the movement of the target region of the object based on the result of the quantification.

The control unit 140 generally performs operations such as the motion tracking system 100 according to an embodiment of the present invention, that is, the image capturing unit 110, the object recognition unit 120, and the motion analysis unit 130. Can be controlled.

The apparatus described above may be implemented with hardware components, software components, and / or combinations of hardware components and software components. For example, the devices and components described in the embodiments include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor (micro signal processor), a microcomputer, a field programmable array (FPA), It may be implemented using one or more general purpose computers or special purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. The processing device may run an operating system (OS) and one or more software applications running on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, a processing device may be described as one being used, but a person having ordinary skill in the art, the processing device may include a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that may include. For example, the processing device may include a plurality of processors or a processor and a controller. In addition, other processing configurations, such as parallel processors, are possible.

The software may include a computer program, code, instruction, or a combination of one or more of these, and configure the processing device to operate as desired, or process independently or collectively You can command the device. Software and / or data may be interpreted by a processing device, or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. , Or may be permanently or temporarily embodied in the transmitted signal wave. The software may be distributed over networked computer systems, and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

3 to 6 are views illustrating a motion tracking method according to an embodiment of the present invention.

The motion tracking method described here is only one embodiment of the present invention. In addition, various steps may be added as necessary, and the following steps may also be performed by changing the order. It is not limited to each step described and the order.

First, referring to FIGS. 1 to 4, in step 310, the image capturing unit 110 of the motion tracking system 100 may acquire a plurality of video images captured by a camera.

Next, referring to FIGS. 1 to 3 and 5, the skin model generation unit 210 of the motion tracking system 100 detects the skin surface of the object from the video image in step 320. After generating the Gaussian distribution of, a Gaussian skin model may be generated based on the generated Gaussian distribution at step 330.

Next, referring to FIGS. 1 to 3 and 5, in operation 340, the target detection unit 220 of the motion tracking system 100 may detect the target region of the object using the Gaussian skin model. .

Next, referring to FIGS. 1 to 3 and 6, in step 350, the tracking model generator 230 of the motion tracking system 100 may generate the elliptical tracking model using an extended Kalman filter. have. Then, in step 360, the motion analysis unit 130 of the motion tracking system 100 may track motion of the target area of the object through motion analysis using the elliptical tracking model.

7 to 9 are graphs showing the motion path of the mouse based on the measured data (red star), manual tracking (green circle), and tracking algorithm (blue triangle), respectively. The green motion path (passive tracking) indicates the average of the values obtained from 8 experimental rats. 10 and 11 are graphs showing average and standard deviations related to errors in the moving distance and the moving speed. According to the graphs of FIGS. 7 to 11, it can be confirmed that the proposed tracking algorithm is much more accurate than manual tracking.

12 to 14 are graphs showing the entire motion path according to the tracking algorithm (red star) and the manual tracking (green circle) in the actual video. Although the overall motion path is similar, the two methods are different. The averages and standard deviations for errors in values obtained using a tracking algorithm and values obtained using a manual tracking are shown in FIGS. 15 and 16.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, or the like alone or in combination. The program instructions recorded in the medium may be specially designed and configured for the embodiments or may be known and usable by those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CDROMs, DVDs, and magneto-opticals such as floptical disks. And hardware devices specifically configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, etc., as well as machine language codes produced by a compiler. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described by a limited embodiment and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques are performed in a different order than the described method, and / or the components of the described system, structure, device, circuit, etc. are combined or combined in a different form from the described method, or other components Alternatively, even if replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

110: video recording unit
120: object recognition unit
130: motion analysis unit
140: control unit
210: skin model generation unit
220: target detection unit
230: tracking model generator

Claims (13)

An image capturing unit acquiring a plurality of video images captured by the camera;
An object recognition unit detecting an object to be tracked from the video image and generating an elliptical tracking model defined as a nonlinear model corresponding to the target area of the detected object; And
Motion analysis unit that tracks the movement of the target area of the object through motion analysis using the elliptical tracking model
Motion tracking system comprising a.
According to claim 1,
The object recognition unit
A skin model generation unit detecting a skin surface of the object from the video image to generate a Gaussian distribution on the skin surface, and generating a Gaussian skin model based on the generated Gaussian distribution;
A target detection unit detecting a target region of the object using the Gaussian skin model; And
A tracking model generator that generates the elliptical tracking model using an extended Kalman filter
Motion tracking system comprising a.
According to claim 2,
The skin model generator
A motion tracking system for generating the Gaussian distribution by cropping a sample image of the same size including only pure skin that is not part of the background from the video image.
According to claim 2,
The target detection unit
Obtaining an observation vector set as a boundary point of the target region of the object using the Gaussian skin model, and associating the obtained observation vector with the state vector of the object using a function of a nonlinear observation model. Motion tracking system.
The method of claim 4,
The state vector is
Consists of a plurality of parameters related to the elliptical tracking model for indicating the overall shape of the object, wherein the parameter is the center point of the ellipse corresponding to the overall shape of the object, the long axis length, the short axis length, the rotation angle and a constant speed. Motion tracking system comprising a.
The method of claim 4,
The motion analysis unit
A motion characterized by tracking the motion of the target region of the object by estimating the state vector over time using the elliptical tracking model, and calculating the speed and distance of the object based on the estimated state vector. Tracking system.
The method of claim 6,
The motion analysis unit
Quantify the difference between the motions by analyzing an operation including at least one of stretching out, huddle up, and stand up of the object based on the estimated state vector. A motion tracking system characterized by tracking the movement of the target area of the object based on the result of quantification.
Acquiring a plurality of video images captured by a camera by an image capturing unit of a motion tracking system;
An object recognition unit of the motion tracking system detects an object of a motion tracking object from the video image, and generates an elliptical tracking model defined as a nonlinear model corresponding to the target area of the detected object; And
The motion analyzing unit of the motion tracking system tracks the movement of the target area of the object through motion analysis using the elliptical tracking model.
Motion tracking method comprising a.
The method of claim 8,
The step of generating the elliptical tracking model
Detecting a skin surface of the object from the video image to generate a Gaussian distribution of the skin surface, and generating a Gaussian skin model based on the generated Gaussian distribution;
Detecting a target region of the object using the Gaussian skin model; And
Generating the elliptical tracking model using an extended Kalman filter
Motion tracking method comprising a.
The method of claim 9,
The step of generating the Gaussian skin model
Generating the Gaussian distribution by cropping a sample image of the same size including only pure skin that is not part of the background from the video image.
Motion tracking method comprising a.
The method of claim 9,
The step of detecting the target region of the object
Obtaining an observation vector set as a boundary point of the target region of the object using the Gaussian skin model; And
Associating the obtained observation vector with the state vector of the object using a function of a nonlinear observation model
Motion tracking method comprising a.
The method of claim 11,
Tracking the movement of the target area of the object
Estimating the state vector over time using the elliptical tracking model; And
Tracking the movement of the target region of the object by calculating the velocity and the moving distance of the object based on the estimated state vector
Motion tracking method comprising a.
The method of claim 12,
Tracking the movement of the target area of the object
Quantify the difference between the motions by analyzing an operation including at least one of stretching out, huddle up, and stand up of the object based on the estimated state vector. Tracking movement of the target area of the object based on the result of quantification
Motion tracking method further comprising a.

Images