KR20190078270A - System and method for tracking object - Google Patents

Abstract

According to the present invention, a method for tracking an object comprises the following steps of: estimating an occlusion region covered a the first object based on information of the first object; generating an occlusion map including the occlusion region; and determining whether the second object is occluded by the first object based on information of the second object that is being tracked and the occlusion map.

Description

[0001] SYSTEM AND METHOD FOR TRACKING OBJECT [0002]

The present invention relates to an object tracking system and method.

Object tracking is a technique for tracking the movement or path of an object based on object information measured from a sensor. In this object tracking technology, Active Safety System and Driver Assistance System (ADAS), which aims to prevent or prevent collision with objects existing in front of the vehicle, corresponds to important element technology of accurate and robust tracking of objects every week.

However, the limitations and noise of the current measurement performance of the sensor, as well as the highly complex real world environment, deteriorate the object tracking performance. Tracking performance degradation in tracking objects can lead to very dangerous situations from a system point of view.

For example, from the viewpoint of an active safety system in which a danger is judged by the driver himself / herself during a running operation, a collision can not be avoided because a wrong object tracking can not determine a risk of collision with a front obstacle (object) In the absence of a frontal obstacle, judging the risk of a false collision can lead to unintentional braking or steer-like movements that can interfere with the driver's driving or threaten safety.

Therefore, in order to solve such a problem, robust object tracking technology that can improve the tracking performance of an object in a complex real environment is needed.

In this regard, Korean Patent Laid-Open Publication No. 10-2010-0072779 (entitled "Vehicle Pedestrian Avoiding Apparatus and Method") discloses a technique of automatically detecting a moving direction of a pedestrian by using a three-dimensional sensor have.

An object of the present invention is to provide an object tracking system and method that enable robust object tracking through an occlusion prediction process and a track management process considering occlusion.

It should be understood, however, that the technical scope of the present invention is not limited to the above-described technical problems, and other technical problems may exist.

According to an aspect of the present invention, there is provided an object tracking method comprising: estimating an occlusion area covered by a first object based on information of a first object; Generating an occlusion map including the occlusion region; And determining whether or not occlusion due to the first object of the second object based on the information of the second object being tracked and the occlusion map.

Estimating an occlusion area covered by the first object based on information of the first object may include estimating an occlusion area covered by the first object based on the relative position and azimuth of the first object and the sensor, To estimate the occlusion region.

The occlusion map including the occlusion region may include a plurality of occlusion regions when the first object or the second object is in motion.

Wherein the step of determining whether or not occlusion is caused by the first object of the second object includes predicting a position of the next object at the current point of time based on at least one of the position, Whether or not the second object is occlusion can be determined.

Wherein the determining whether the second object is occlusive due to the first object comprises: measuring a time before entry of the second object into the occlusion region; Calculating a predicted time until the occlusion area advances based on the information of the second object; And tracking the second object for a predicted time from the pre-entry time to the advancement.

The object tracking method according to the present invention may further include tracking the second object, wherein the tracking state of the second object may include an initial state, an uncertain state, a confidential state, an obscured state, and a latent state.

The step of tracking the second object may include switching to the uncertain state when acquiring information of the second object in the initial state; Acquiring information of the second object in the uncertainty state and switching to the confident state when the second object is confirmed; And switching to the obscured state if the second object is not tracked in the asserted state.

The step of tracking the second object may further include maintaining the obscured state when it is determined that the occlusion is in the obscured state.

The step of tracking the second object may further include switching from the obscured state to the latent state when it is determined that the observer is not in the occlusion state.

According to a second aspect of the present invention, there is provided an object tracking system including a sensor unit for sensing information of an object, a memory for storing a program for tracking an object based on the sensed information, . At this time, as the program is executed, the processor generates an occlusion map by estimating an occlusion area covered by the first object based on the information of the first object sensed by the sensor unit, Whether or not the second object is occluded due to the first object based on the information of the second object sensed by the sensor unit and the occlusion map.

The sensor unit senses a relative position and an azimuth with respect to the first object, a size and a position of the first object, and the processor can estimate the occlusion region based on the sensed position.

Wherein the sensor unit senses at least one of the position, velocity, and magnitude of the second object, and the processor estimates the sensed information and the position at the next time point at the current time point to determine whether or not the second object is occluded It can be judged.

According to any one of the above-mentioned objects of the present invention, it is possible to solve the problem of misrecognition and unrecognition of the sensor due to occlusion and superposition situation between conventional objects through occlusion prediction.

FIGS. 1A to 1C are diagrams for explaining an object estimation technique.
2 is a block diagram of an object tracking system in accordance with an embodiment of the present invention.
3 is a flowchart of an object tracking method according to an embodiment of the present invention.
Fig. 4 is a diagram for explaining an occlusion map. Fig.
FIG. 5 is a diagram for explaining contents for determining whether occlusion has occurred. FIG.
6 is a diagram for explaining the tracking state of the second object in consideration of occlusion.
7 to 9 are views for explaining test results according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, which will be readily apparent to those skilled in the art. The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. In order to clearly explain the present invention in the drawings, parts not related to the description are omitted.

Whenever a component is referred to as "including" an element throughout the specification, it is to be understood that the element may include other elements, not the exclusion of any other element, unless the context clearly dictates otherwise.

FIGS. 1A to 1C are diagrams for explaining an object estimation technique.

The object tracking process improves the measurement accuracy by stochastically predicting the actual position based on the correlation between the past measured information and the currently measured information, and verifies the validity of the object and verifies the same object classification It is the process of managing the tracking state such as confirming or removing the existence of the object.

In general, the validity of an object is verified by verifying the similarity between the predicted state and the currently measured state based on the previous state (position, speed, etc.) of the object, A method of assuring the existence of

On the other hand, it is necessary to predict the next state through the previous state (position, velocity, shape, etc.) of the measured object from the sensor, using predictive filters such as Kalman filter and particle filter, Can be estimated.

If this process is repeated, it is necessary to manage the object tracking state such as creation of the object being tracked, state update, and disappearance.

In the conventional technology, when the object is not measured continuously for several times from the sensor, it is determined that the object is obscured and the tracking state is deleted.

Specifically, referring to FIG. 1A, in an existing object tracking process, a gating process for comparing an estimated value of a sensor and a predicted value based on a kinematic model to track an object to track the object, Verify validity.

As shown in FIG. 1B, the track management procedure (Track Management) for the validated object creates a new track (P2) in the initial state (P1), maintains and updates the existing track (P3) And performs the process (P4) to track the object.

However, in the conventional art, in the actual road environment in which a plurality of objects exist as shown in FIG. 1C, in the occlusion situation between the moving objects, the sensor deteriorates the object detection performance and the object is not detected Eventually, there is a problem that tracking of the object does not continue.

Such a problem can be further exerted in an active safety system such as AEB for collision avoidance by performing emergency braking.

As described above, deterioration in tracking performance of an object results in a problem that a collision can not be avoided or an erroneous emergency braking is performed in the system. Therefore, a method for solving the problem is required. The object tracking system 100 and the method according to the present invention will be described in detail.

2 is a block diagram of an object tracking system 100 in accordance with an embodiment of the present invention.

The object tracking system 100 according to an embodiment of the present invention includes a sensor unit 110, a memory 120, and a processor 130.

The sensor unit 110 is for sensing information of an object, and may include a camera or a rider sensor, and may include each or all of them.

In the memory 120, a program for tracking an object based on information sensed by the sensor unit 110 is stored. Herein, the memory 120 is collectively referred to as a nonvolatile storage device and a volatile storage device which keep the stored information even when power is not supplied.

For example, the memory 120 may be a compact flash (CF) card, a secure digital (SD) card, a memory stick, a solid-state drive (SSD) A magnetic computer storage device such as a NAND flash memory, a hard disk drive (HDD) and the like, and an optical disc drive such as a CD-ROM, a DVD-ROM, etc. .

The processor 130 may determine whether or not the second object is occluded by the first object by executing the program stored in the memory 120. [

2 may be implemented in hardware such as software or an FPGA (Field Programmable Gate Array) or ASIC (Application Specific Integrated Circuit), and may perform predetermined roles can do.

However, 'components' are not meant to be limited to software or hardware, and each component may be configured to reside on an addressable storage medium and configured to play one or more processors.

Thus, by way of example, an element may comprise components such as software components, object-oriented software components, class components and task components, processes, functions, attributes, procedures, Routines, segments of program code, drivers, firmware, microcode, circuitry, data, databases, data structures, tables, arrays, and variables.

The components and functions provided within those components may be combined into a smaller number of components or further separated into additional components.

Hereinafter, an object tracking method in the object tracking system 100 according to an embodiment of the present invention will be described in detail with reference to FIGS.

3 is a flowchart of an object tracking method according to an embodiment of the present invention.

The object tracking method according to an embodiment of the present invention first estimates an occlusion area covered by a first object based on information of a first object sensed by the sensor unit (S110). Next, an occlusion map including an occlusion region is generated (S120.).

Fig. 4 is a diagram for explaining an occlusion map. Fig.

The step of estimating the occlusion region in the embodiment of the present invention may include estimating the occlusion area based on the relative position and the azimuth angle between the first object A and the sensor unit 110, The clusogenic region can be estimated. In addition, the occlusion region is affected by the shape and shape of the first object A.

This occlusion region means an obscured region where the sensor unit 110 can not detect an object as shown in FIG. The occlusion map including the occlusion region may include a plurality of occlusion regions when the first object A is a moving object.

Meanwhile, the process of generating occlusion map will be described as follows.

)과 트랙의 상태 벡터(class, px, py, vx, vy, w, l)는 다음 수학식 1과 같이 표현할 수 있다.For example, in a time step k, a track generated based on a measurement value of an object sensed from a sensor unit 110 such as a camera sensor and a rider sensor through an object tracking algorithm

) And the track state vector (class, px, py, vx, vy, w, l) can be expressed by the following equation (1).

[Equation 1]

Here, class represents the type of object, px and py refer to the relative distance between the vertical and horizontal directions from the sensor unit 110, and vx and vy refer to the relative velocity between the vertical and horizontal directions. Also, w and l mean width and length, and k means time.

The occlusion map thus determined can be expressed as Equation (2).

&Quot; (2) "

Here, px represents the longitudinal relative distance between the sensor unit 110 and the object, and θc, θl, and θr represent the azimuth angle from the sensor unit 110 to the center of the bottom and the left and right corner portions of the object. I denotes an index of an existing track, and k + 1 denotes a next time point predicted from the previous state k.

The state vectors px,? C,? L and? R of the occlusion map can be calculated from the state vectors of existing objects using the following Equation (3).

&Quot; (3) "

Equation (3) shows that the position of the object is on the right side of the sensor unit 110, but it can be disposed on the left, right, and center of the sensor unit 110 in case of a moving object. Therefore, θc, θl, and θr may vary depending on the position of the object when creating occlusion maps.

Referring again to FIG. 3, whether the occlusion due to the first object of the second object is determined based on the information of the second object being tracked through the sensor unit 110 and the generated occlusion map (S130 ).

That is, whether or not occlusion occurs is determined based on whether the first object and the second object, which are objects different from each other, enter the occlusion region based on the generated occlusion map.

)의 상태 벡터를 통한 예측된 다음 상태와 비교함으로써 수행될 수 있다. 라이더 센서로부터 생성된 추적 중인 트랙과 트랙의 상태 벡터는 수학식 4와 같이 표현할 수 있다. For example, the determination of occlusion may be made by comparing the occlusion area generated through Equation (1) with the track generated by an individual sensor (e.g., a rider sensor)

≪ / RTI > with the next state predicted through the state vector of the state vector < RTI ID = 0.0 > The state vector of the track and track being generated from the rider sensor can be expressed by Equation (4).

&Quot; (4) "

A comparison with the occlusion region can be made through a method of predicting the future state based on the current state of the track generated by the rider sensor. In this example, a constant velocity (CV) model is used for the prediction.

&Quot; (5) "

The tracking position of the rider sensor predicted by the model can be converted to an azimuth angle as shown in Equation (6) for comparison with the occlusion map.

&Quot; (6) "

For prediction of occlusion, the predicted state of the track to be finally compared with the occlusion map can be defined by the following equation (7).

&Quot; (7) "

FIG. 5 is a diagram for explaining contents for determining whether occlusion has occurred. FIG.

The determination of occlusion by the first object A of the second object B may be made based on at least one of the position, velocity and size of the second object B, It can be judged by predicting the position.

On the other hand, the occlusion can be divided into a state in which only a part of the second object B is hidden and a state in which the entire object is hidden as shown in FIG.

5, the sensor unit 110 can measure the second object B because the second object B is not covered by the first object A in the initial state (a).

Next, in the states (b), (c), and (d), the object enters the occlusion region in accordance with the movement of the second object B, and in the state (e), it is out of the occlusion region.

An embodiment of the present invention can determine whether or not occlusion of the second object B based on the occlusion area (map) calculated in the previous procedure and the predicted state of the moving second object B by the following equation 8 to Equation (11).

Equations (8) to (11)

Equations (8) to (11) are for predicting the states of (b) to (d), where i represents the track index of the rider sensor and j represents the index of the occlusion map.

Finally, the equation for detecting occlusion for the second object B can be summarized as follows.

&Quot; (12) "

On the other hand, if the second object B is located on the left or the center of the sensor unit 110, as in the case of Equations (8) to (11), the determination of occlusion is determined according to the relative position of the second object The positional relationship of the sign in the above equation can be changed.

In one embodiment of the present invention, after the occlusion is predicted through the above process, the second object can be predicted not to be sensed from the sensor unit 110 for a certain time due to occlusion by the first object have. The tracking state of the second object can be managed in consideration of this.

That is, since it is possible to determine that measurement can be performed again after a certain period of time has elapsed based on the size and movement information of the second object before entering the occlusion region according to an embodiment of the present invention, 2 object is measured before the entry into the occlusion region, the predicted time until the entry of the occlusion region is calculated based on the information of the second object, 2 You can track objects.

At this time, the tracking state of the second object may include an initial state (Empty), an uncertain state (Initiator-1, Initiator-2), a state of being Comfirmed (Obscured), and a potential state (Potential).

6 is a diagram for explaining the tracking state of the second object in consideration of occlusion.

The tracking state management is a process for processing a missing or erroneous detection that may occur during the object tracking process. It is verified that the object measured by the sensor unit 110 exists, and when it is determined that the object is a fictitious object or an invalid object The object can be deleted from the tracked object.

Specifically, the initial state in FIG. 6A means a track object, that is, a state in which a track is not created or deleted.

In this state, when the sensor unit 110 measures an object (measurement), the track is moved to the start state 1, and if the information about the second object, which is related in the state, is repeatedly acquired, . In the case of the start state 2, it means an uncertain track state before the second object is finally confirmed. If measurement data irrelevant to the track is obtained in the start state 2, the track is deleted and then switched back to the initial state.

When the second object is finally confirmed, the state is switched to the asserted state. The state is managed by incrementing Age by 1 each time the track in the asserted state is repeatedly performed.

On the other hand, if the second object is not tracked in the asserted state, it is switched to the obstructed state. If it is determined that the second object is occluded in the occluded state, the occlusion state can be maintained.

Also, if it is judged that it is not in the occlusion state in the occluded state, it is converted into the latent state, and the state is managed by decreasing the age. Further, even if measurement data irrelevant to the information on the second object is obtained in the asserted state, the state can be switched to the latent state, and the asserted state can be switched again when related measurement data is acquired.

In the latent state, if the current Age is less than the predetermined Age, the state is switched back to the initial state.

As described above, according to an embodiment of the present invention, a track is generated, maintained, checked, and deleted while repeating a data association process for determining the relevance of the measurement data to the gating process, , It is possible to prevent tracks from being deleted in the occlusion state by applying the obscured state to the occlusion track described in Fig. 6 (a).

In the above description, steps S110 to S130 may be further divided into additional steps or combined into fewer steps, according to an embodiment of the present invention. Also, some of the steps may be omitted as necessary, and the order between the steps may be changed. In addition, the contents already described with respect to the object tracking system 100 in FIGS. 1 and 2 apply to the object tracking method of FIGS. 3 to 6 even if other contents are omitted.

7 to 9 are views for explaining test results according to an embodiment of the present invention.

7 is a diagram for explaining measurement and occlusion prediction results of the rider sensor.

Figure 7 (a) shows the relative lateral distance of the pedestrian measured by the rider sensor. In a situation where the pedestrian is covered by an adult, the sensor is operated at a specific time (about 13.5 to 14.3 The pedestrian can not be measured during the second.

In the test of the present invention, the occlusion map was used to predict the situation in which the pedestrian of the pedestrian was blocked by the pedestrian during the time, and the track status of the pedestrian controlled by the pedestrian is shown in FIG. 8 and Table 1 Respectively.

Index 0 One 2 State Empty Initiator-1 Initiator-2 Index 4 5 6 State Potential Obscured Confurmed

Prior to occlusion, the tracking status is assured because the measurement of the pedestrian is continued. It is then transiently transitioned from a state of assurance to a latent state because of the delay in the measurement of the sensor.

At the occlusion prediction time of 13.5 ~ 14.3 seconds, the state was switched to the obscured state, and the pedestrian was not sensed but the Age was maintained.

When passing through the occlusion region, the state is again associated with the measured value of the sensor, so it switches from the obscured state to the assured state.

The child pedestrian then goes out of the FOV area of the sensor and can not be measured, so the age of the track is gradually reduced and eliminated.

9 (a) and 9 (b) are diagrams showing the relative distance of the pedestrian measured by the rider sensor and the tracking result according to the present invention.

As described above, according to the embodiment of the present invention, it is possible to track the pedestrian of a child while maintaining the existing track record even when the pedestrian can not be detected through track management considering occlusion prediction and occlusion.

One embodiment of the present invention may also be embodied in the form of a computer program stored on a medium executed by a computer or a recording medium including instructions executable by the computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, the computer-readable medium may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically includes any information delivery media, including computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism.

While the methods and systems of the present invention have been described in connection with specific embodiments, some or all of those elements or operations may be implemented using a computer system having a general purpose hardware architecture.

It will be understood by those skilled in the art that the foregoing description of the present invention is for illustrative purposes only and that those of ordinary skill in the art can readily understand that various changes and modifications may be made without departing from the spirit or essential characteristics of the present invention. will be. It is therefore to be understood that the above-described embodiments are illustrative in all aspects and not restrictive. For example, each component described as a single entity may be distributed and implemented, and components described as being distributed may also be implemented in a combined form.

The scope of the present invention is defined by the appended claims rather than the detailed description and all changes or modifications derived from the meaning and scope of the claims and their equivalents are to be construed as being included within the scope of the present invention do.

100: Object Tracking System
110:
120: Memory
130: Processor

Claims (12)

An object tracking method in an object tracking system,
Estimating an occlusion area covered by the first object based on information of the first object;
Generating an occlusion map including the occlusion region; And
Determining whether or not occlusion due to the first object of the second object based on the information of the second object being tracked and the occlusion map. The method according to claim 1,
Estimating an occlusion area covered by the first object based on information of the first object,
Estimating the occlusion area based on a relative position and an azimuth of the first object and a sensor, and a size and a position of the first object. The method according to claim 1,
Wherein generating an occlusion map including the occlusion region comprises:
Wherein the occlusion map includes a plurality of occlusion regions when the first object or the second object is in motion. The method according to claim 1,
Wherein the step of determining whether or not occlusion due to the first object of the second object comprises:
And determining whether or not the second object is occlusion by predicting a position at the next point in time based on at least one of the position, velocity, and size of the second object. 5. The method of claim 4,
Wherein the step of determining whether or not occlusion due to the first object of the second object comprises:
Measuring a time before entry of the second object into the occlusion region;
Calculating a predicted time until the occlusion area advances based on the information of the second object; And
And tracking the second object for a predicted time from the pre-entry time to the advance. The method according to claim 1,
Further comprising tracking the second object,
Wherein the tracking state of the second object includes an initial state, an uncertain state, a confident state, an obscured state, and a latent state. The method according to claim 6,
Wherein tracking the second object comprises:
Switching to the uncertain state when acquiring information of the second object in the initial state;
Acquiring information of the second object in the uncertainty state and switching to the confident state when the second object is confirmed; And
And switching to the obscured state if the second object is not tracked in the confident state. 8. The method of claim 7,
Wherein tracking the second object comprises:
And maintaining the obscured state when it is determined that the observer is occlusion in the obscured state. 8. The method of claim 7,
Wherein tracking the second object comprises:
And switching from the obscured state to the latent state when it is determined that the observer is not in the occlusion state in the obscured state. An object tracking system comprising:
A sensor unit for sensing information of the object,
A memory for storing a program for tracking an object based on the sensed information,
And a processor for executing a program stored in the memory,
The processor generates an occlusion map by estimating an occlusion area covered by the first object based on the information of the first object sensed by the sensor unit as the program is executed, And determines whether or not occlusion due to the first object of the second object based on the information of the second object sensed by the second sensor and the occlusion map. 11. The method of claim 10,
Wherein the sensor unit senses a relative position and an azimuth with respect to the first object, a size and a position of the first object, and the processor estimates the occlusion area based thereon. 11. The method of claim 10,
Wherein the sensor unit senses at least one of the position, velocity, and magnitude of the second object, and the processor estimates the sensed information and the position at the next time point at the current time point to determine whether or not the second object is occluded Object tracking system.

Images