KR20180079880A - Apparatus and method for tracking object - Google Patents

Abstract

The present invention proposes an apparatus and a method for tracking an object, capable of robustly tracking the object based on data selected through validation verification between a predicted value and a measured value. An apparatus according to the present invention includes: an object information predicting unit for predicting information about an object based on first sensing data obtained at a previous time; a valid data determination unit that compares second sensing data obtained at a current time with information about the object to determine whether valid data exists in the second sensing data; and an object tracking unit for tracking the object based on at least one valid data if the existence of the valid data is determined.

Description

[0001] Apparatus and method for tracking object [0002]

The present invention relates to an apparatus and method for tracking an object. More particularly, the present invention relates to an apparatus and method for tracking an object in a vehicle.

Object tracking refers to a technique for tracking an object's movement path based on measured data from a sensor. Such object tracking is applied to vehicles and is widely used in an active safety system such as an automatic emergency braking system (AEBS).

However, conventional object tracking has a problem in that tracking performance of an object is degraded if an object is missed due to an environmental factor such as a defect or an occlusion of the sensor itself. If the vehicle is exposed to such a problem while driving, a traffic accident can not be avoided.

Korean Patent Publication No. 10-2016-0071162 (Publication date: Jun. 2016)

It is an object of the present invention to provide an apparatus and method for tracking an object robustly based on data selected through validation between a predicted value and a measured value.

However, the objects of the present invention are not limited to those mentioned above, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention, there is provided an information processing apparatus including an object information predicting unit for predicting information about an object based on first sensing data acquired at a previous time; A valid data determination unit for comparing second sensing data obtained at a current time with information about the object to determine whether valid data exists among the second sensing data; And an object tracking unit for tracking the object based on at least one valid data if it is determined that the valid data exists.

Preferably, the valid data determination unit determines whether the valid data exists based on whether an error between the second sensing data and the object information is less than or equal to a reference value.

Preferably, the valid data determination unit determines the presence or absence of the valid data based on at least one of a measurement error of the sensor, a traveling speed of the object, a traveling speed of the vehicle tracking the object, And information on a surrounding environment in which the vehicle is located.

Preferably, the object information predicting unit calculates distance information between the object and the vehicle tracking the object, velocity information between the object and the vehicle, and a state transition matrix to obtain information about the object Predict.

Preferably, the object information predicting unit further includes information on the type of the object to predict information on the object.

Preferably, the object tracking device further includes an object state generator for generating state information of the object corresponding to the unrecognized object based on a first determination result related to whether or not the valid data exists.

Preferably, when generating the object state information, the object state generator may generate the object state information based on a result of a second determination as to whether the first sensing data exists or not, As a result of the determination, as a result of the fourth determination as to whether or not the valid data exists, a fifth determination result related to whether or not the valid data continuously exists is determined. Or a sixth determination result related to whether or not a predetermined time has elapsed.

Preferably, the object state generation unit may generate the second determination result, the third determination result, the fourth determination result, the fifth determination result, and the second determination result when using the second determination result to the sixth determination result, 6 Use judgment results in order.

Preferably, the object tracking apparatus further includes an object tracking determination unit that determines whether to continue tracking the object based on the state information of the object.

Preferably, the object state generator is configured to acquire the first sensing data by one of a camera sensor, a radar sensor, and a ladder sensor, and generate second sensing data and seventh sensing data And generates state information of the object on the basis of at least one sensing data selected from the first sensing data, the sixth sensing data, and the seventh sensing data.

Preferably, the object state generation unit is configured to generate the object state data based on the sensed data obtained by the camera sensor and the sixth sensing data are sensed data obtained by the radar sensor, The sensing data obtained by the sensor may be generated by first generating state information of the object on the basis of the seventh sensing data and then, based on the first sensing data, the sixth sensing data, and the seventh sensing data Generates state information of the object, generates state information of the object based on the first sensing data and the sixth sensing data when the predetermined condition is satisfied, and then generates state information of the object based on the first sensing data Generates state information of the object, and then generates state information of the object based on the sixth sensing data And generates status information of the object.

Preferably, the object tracking apparatus further includes a travel control unit for controlling travel of the vehicle that tracks the object based on the state information of the object.

Preferably, the object tracking unit tracks an object based on the at least one valid data selected based on an error between the valid data and the information about the object.

Preferably, the effective data determination unit determines whether the valid data exists among second sensing data related to the object located in the ROI.

Preferably, the object tracking apparatus estimates information on the object on the basis of the first sensing data and third sensing data obtained by another sensor, and if the object is predicted based on the first sensing data, And a first range determination unit for determining a first reference range based on the information on the object and the information on the predicted object based on the third sensing data, Sequentially comparing the fourth sensing data obtained by the sensor and the fifth sensing data obtained by the same sensor with the third sensing data to the first reference range to generate the fourth sensing data and the fifth sensing data, It is determined whether the valid data exists among the data.

Preferably, the object tracking apparatus estimates information on the object on the basis of the first sensing data and third sensing data obtained by another sensor, and if the object is predicted based on the first sensing data, Further comprising a second range determination unit for determining a second reference range based on the information on the first sensing data and determining a third reference range based on information on the predicted object based on the third sensing data, The determination unit compares the fourth sensing data obtained by the same sensor with the first sensing data to the second sensing range to determine whether the valid data exists among the fourth sensing data, Comparing the fifth sensing data obtained by the same sensor with the data to the third reference range, Whether or not the valid data exists.

Preferably, when the third sensing data is acquired by a sensor different from the first sensing data, the object information predicting unit may predict the object based on the data obtained by combining the first sensing data and the third sensing data Predicts information about the user.

Preferably, when the third sensing data is acquired by the sensor different from the first sensing data, the object information predicting unit may select the same item among the first sensing data and the third sensing data, And predicts information on the object based on the sensing data.

Preferably, the object tracking apparatus includes an object speed determiner for determining whether the speed information of the object is included in the valid data if the information on the type of the object is not included in the valid data; And a first object type estimating unit for estimating information on the type of the object based on the speed information of the object if it is determined that the speed information of the object is included in the valid data.

Preferably, the object tracking apparatus includes a first object type determination unit determining whether information on a type of the object is a pedestrian if information on the type of the object is included in the first sensing data. A second object type determination unit for determining whether information on the type of the object is included in the valid data if it is determined that the information on the type of the object is a pedestrian; And if it is determined that information on the type of the object is not included in the valid data, information on the type of the object included in the first sensing data is included in the valid data And further includes an estimation unit.

According to another aspect of the present invention, there is provided an information processing method including: an object information prediction step of predicting information on an object based on first sensing data obtained at a previous time; A valid data determination step of determining whether valid data exists among the second sensing data by comparing second sensing data obtained at a current time with information about the object; And an object tracking step of tracking an object based on at least one valid data if it is determined that the valid data exists.

Preferably, the effective data determination step determines whether the valid data exists based on whether an error between the second sensing data and the object information is less than a reference value.

Preferably, the valid data determination step may include a step of determining whether the valid data exists, a measurement error of the sensor, a traveling speed of the object, a traveling speed of the vehicle tracking the object, Information based on at least one of information on the environment in which the vehicle is located and information on a surrounding environment in which the vehicle is located.

Preferably, the object information prediction step calculates distance information between the object and the vehicle tracking the object, velocity information between the object and the vehicle, and a state transition matrix to obtain information about the object .

Preferably, the object information prediction step further includes information on the type of the object to predict information on the object.

Preferably, state information of the object corresponding to the unrecognized object is generated between the valid data determination step and the object tracking step based on a first determination result related to whether or not the valid data exists And an object state generating step.

Preferably, the object state generating step may include a step of, when generating the state information of the object, determining whether or not the first sensing data exists and a second determination result related to whether the second sensing data is the first acquired data 3 is a result of a fourth determination as to whether or not the valid data is present as a first determination result, and a fifth determination result related to whether or not the valid data is continuously determined to be present, Or a sixth determination result related to whether or not a predetermined time has elapsed since the determination of whether or not the predetermined time has elapsed.

Preferably, the object state generation step may include the second determination result, the third determination result, the fourth determination result, the fifth determination result, and the fourth determination result when using the second determination result to the sixth determination result. And the sixth determination result.

Preferably, the method further includes an object tracking determination step between the object state generation step and the object tracking step to determine whether to continue tracking the object based on the object state information.

Preferably, the object state generation step may include a step of generating the object state data by obtaining the first sensing data by a sensor of a camera sensor, a radar sensor, and a ladder sensor, The state information of the object is generated based on at least one sensing data selected from the first sensing data, the sixth sensing data and the seventh sensing data.

Preferably, the object state generation step may be performed such that the first sensing data is the sensing data obtained by the camera sensor, the sixth sensing data is the sensing data obtained by the radar sensor, The first sensing data, the sixth sensing data, and the seventh sensing data are first generated based on the seventh sensing data, and then the first sensing data, the sixth sensing data, And generates state information of the object based on the first sensing data and the sixth sensing data when the predetermined condition is satisfied, and then generates the state information of the object based on the first sensing data and the sixth sensing data, To generate status information of the object, and then, based on the sixth sensing data, The state information of the object.

Preferably, a traveling control step of controlling the traveling of the vehicle that tracks the object based on the state information of the object is provided between the object state generating step and the object tracking step.

Advantageously, the tracking of the object tracks an object based on the at least one valid data selected based on an error between the valid data and the information about the object.

Preferably, the effective data determination step determines whether the valid data exists among second sensing data related to the object located in the ROI.

Preferably, when the information about the object is predicted based on the third sensing data obtained by the sensor different from the first sensing data, between the object information predicting step and the valid data determination step, Further comprising a first range determining step of determining a first reference range based on information about the object predicted based on the data and information about the object predicted based on the third sensing data, The determining step sequentially compares the first sensing data with the fourth sensing data obtained by the same sensor and the fifth sensing data obtained by the same sensor with the third sensing data sequentially with the first reference range And determines whether the valid data exists among the fourth sensing data and the fifth sensing data.

Preferably, when the information about the object is predicted based on the third sensing data obtained by the sensor different from the first sensing data, between the object information predicting step and the valid data determination step, Determining a second reference range based on information about the object predicted based on the data and determining a third reference range based on information on the predicted object based on the third sensing data, Wherein the valid data determination step includes comparing the fourth sensing data obtained by the sensor with the first sensing data to the second reference range to determine whether the valid data is included in the fourth sensing data Determining whether or not the third sensing data is present, and comparing the fifth sensing data obtained by the same sensor with the third sensing data, Compared to the standard range, it is determined whether the valid data are present in said fifth sensing data.

Preferably, the object information predicting step may include the step of predicting the object information based on the data obtained by combining the first sensing data and the third sensing data when the third sensing data is acquired by a sensor different from the first sensing data, Lt; / RTI >

Preferably, when the third sensing data is acquired by the sensor different from the first sensing data, the object information predicting step may predict the same item among the first sensing data and the third sensing data according to a priority order And predicts information on the object based on the selected sensing data.

Preferably, if information on the type of the object is not included in the valid data between the valid data determination step and the object tracking step, whether or not the speed information of the object is included in the valid data Determining an object speed; And a first object type estimating step of estimating information on the type of the object based on the speed information of the object if it is determined that the speed information of the object is included in the valid data.

Preferably, if the information on the type of the object is included in the first sensing data between the valid data determination step and the object tracking step, it is determined whether information on the type of the object is a pedestrian 1 object type determination step; A second object type determination step of determining whether information on the type of the object is included in the valid data if it is determined that the information on the type of the object is a pedestrian; And if it is determined that information on the type of the object is not included in the valid data, information on the type of the object included in the first sensing data is included in the valid data And an estimation step.

The present invention also proposes a computer program stored on a computer readable medium for executing an object tracking method in a computer.

The present invention can achieve the following effects through the above-described configurations.

First, it is possible to track the object robustly even if the object is temporarily unrecognized.

Second, using selected valid data makes it possible to track objects quickly and accurately even in a multi-sensor environment.

FIG. 1 is a conceptual diagram for explaining an object tracking management procedure according to an embodiment of the present invention.
2 is a flowchart sequentially illustrating an operation method of an object tracking system according to an embodiment of the present invention.
3 is a reference diagram for explaining validity verification according to an embodiment of the present invention.
4 is a reference diagram for explaining a validity verification method according to an embodiment of the present invention.
5 and 6 are reference diagrams for explaining object tracking results of an object tracking system according to an embodiment of the present invention.
7 is a flowchart illustrating a method of tracking an object in a multi-sensor environment according to a first embodiment of the present invention.
8 is a flowchart illustrating a method of tracking an object in a multi-sensor environment according to a second embodiment of the present invention.
9 is a conceptual diagram illustrating a second fusion strategy according to an embodiment of the present invention.
FIG. 10 is a block diagram schematically illustrating internal configurations of an object tracking apparatus according to a preferred embodiment of the present invention.
11 is a block diagram illustrating internal configurations that may be added to the object tracking apparatus of FIG.
12 is a flowchart schematically illustrating an object tracking method according to a preferred embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In the drawings, the same reference numerals are used to designate the same or similar components throughout the drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear. In addition, the preferred embodiments of the present invention will be described below, but it is needless to say that the technical idea of the present invention is not limited thereto and can be variously modified by those skilled in the art.

The present invention relates to a technology for robustly tracking an object even if the object can not be recognized temporarily. In the present invention, a potential track is used to cope with a temporary unrecognized situation of an object, and an object is continuously tracked through a potential track to improve tracking performance of the object.

The potential track defines a target that is sufficiently verified through a repetitive tracking process. These potential tracks are switched only on a fully confirmed track and disappear by themselves if they are not verified again for a certain period of time after switching. A verification method for determining the state of a track is performed through a gating process for verifying the validity of the predicted value and the measured value.

Hereinafter, the present invention will be described in detail with reference to the drawings.

FIG. 1 is a conceptual diagram for explaining an object tracking management procedure according to an embodiment of the present invention.

The life cycle for defining the state of the object to be tracked includes a track creation step, a track state update step, a track state maintenance step, a track extinction step, and the like. According to the progress of each stage of the life cycle, the state of the object to be tracked is defined as six types as shown in FIG.

Empty track 110 defines the state of an object that has no effect on the running of the vehicle. The state of this object is defined as the empty track 110 when the object is not located within a predetermined distance from the vehicle, when the object is excluded from the tracking object, and the like.

A first initiator track (Initiator 1 track) 120 defines the state of the object initially recognized by the vehicle sensor. The state of this object is defined as the first initiator track 120 when the object is within a predetermined distance from the vehicle and this object is sensed by the vehicle sensor.

The second initiator track 130 defines the state of the validated object in the first cycle after being recognized by the vehicle sensor. In the present invention, once the validity is verified once, the state of this object is defined as the second initiator track 130, and the number of validity verifications for defining the second initiator track 130 is not necessarily limited to one. If the validity is not verified in the first cycle after being recognized by the vehicle sensor, the state of this object is then initialized to the empty track 110.

In the meantime, the cycle means a measurement period of the sensor. A detailed description of the validation method will be given later.

The Confirmed track 150 defines the state of the validated object in the second cycle after the first cycle after being recognized by the vehicle sensor. In the present invention, when the validity is verified twice consecutively, the state of the object is defined as the conformed track 150, and the number of validity successive verifications to be defined as the concurrent track 150 is not necessarily limited to two times. When the state of the object is defined as the conformed track 150, the vehicle determines that the object is likely to be collided and controls the vehicle to travel by avoiding the object, or performs an emergency braking (EB) function .

The third initiator track 140 defines the state of the object that has not been validated in the current cycle for the object defined as the second initiator track 130 in the previous cycle. In one example, the third initiator track 140 defines the state of an object that has been validated in the first cycle but not yet validated in the second cycle after being recognized by the vehicle sensor.

If the object defined as the third initiator track 140 in the current cycle is validated in the next cycle, the state of this object is then defined as the concatenated track 150. [ On the other hand, if the object defined as the third initiator track 140 in the current cycle is not validated in the next cycle, the state of this object is then initialized to the empty track 110.

A potential track 160 defines a state of an object defined as a concert track 150 that does not meet conditions related to validation. For example, the potential track 160 may define the state of an object for which validity is not continuously verified during a cycle of N times (eg, three) for an object defined as a concert track 150, And defines the state of an object whose validity is not verified during the first time period with respect to the object defined by the track 150. [

If the object defined as the potential track 160 in the current cycle is validated in the next cycle, the state of this object is then defined as the concatenated track 150. [ On the other hand, if the object defined as the potential track 160 in the current cycle is not validated in the next cycle, the state of this object will be defined as the empty track 110 thereafter. It is also possible to define the state of this object as the empty track 110 if the object defined as the potential track 160 in the current cycle is not validated for a second time (e.g., 2 seconds).

In the present invention, it is also possible to determine the state of an object based on the FOV (Field Of View) of the sensor.

The measured value for the object is not measured by the sensor when the object is out of the sensor's FOV at the next cycle (K + 1 time point) when the state of the object at the previous cycle (K point) is the concert track 150. [ Therefore, in this case, the state of the object is changed from the conformed track 150 to the potential track 160. [ When the object is out of the sensor FOV at the next cycle (K + 1 time) when the state of the object is the potential track 160 in the previous cycle (K time point), the state of the object is not immediately followed by the potential track 160 to the empty track 110.

On the other hand, if the object state is within the FOV of the sensor but the state of the object is changed from the track 150 to the potential track 160, it can be determined that the object is not recognized by the sensor or that the sensor has failed. Therefore, in this case, the state of the object can be determined based on the time at which the state of the object is held in the concatenated track 150. [

For example, if the state of the object at point K is changed from the track 150 to the potential track 160, if the time at which the state of the object is maintained until the point K is equal to or longer than the reference time , The state of the object at the time K + 1 can be determined as a more potential track in the potential track 160. In the present invention, the mower potential track is defined as a state in which the validity is not continuously verified during a cycle of M times (where M < N) with respect to the object defined as the concert track 150, And defines the state of an object whose validity is not verified during a third time (third time < first time) with respect to an object defined by the track 150. [

Next, the validation method will be described.

Validation, or gating, refers to a method of determining the state of an object by extracting a valid current measurement value through a correlation between the predicted value and the current measured value based on the predicted value associated with the previous measurement and the current measured value. The previous measurement value means a measurement value obtained by the sensor at the previous time, and the current measurement value means the measurement value obtained by the sensor at the present time.

In the present invention, a system for tracking an object, that is, an object tracking system, can track objects and manage the tracking results according to the order shown in FIG. 2 is a flowchart sequentially illustrating an operation method of an object tracking system according to an embodiment of the present invention.

First, the object tracking system acquires sensing data using various sensors mounted on the vehicle (S210). For example, the object tracking system can acquire image data using a camera sensor, and can acquire relative distance data from the vehicle to each object using the distance sensor. It is also possible for the object tracking system to obtain relative speed data of each object with respect to the vehicle using a speed sensor.

Thereafter, the object tracking system performs validation (Gating) on each object based on the sensing data (S220). In the present invention, the object tracking system can perform validation on an object located in a region of interest (ROI). In this case, the object tracking system can set the region of interest before executing the validation. The object tracking system can set the region of interest based on the field of view (FOV) of the sensor.

Validation (S220) for each object can be performed in the following order.

3 is a reference diagram for explaining validity verification according to an embodiment of the present invention. The following description refers to Fig. 2 and Fig.

First, the object tracking system acquires the previous measurement value x _k-1 310 as shown in FIG. 3 (a) (S221). When using a camera sensor for example, the object tracking system previously measured value x _k-1 (310) class information (class) of the object, the longitudinal / lateral distance information of an object (px, py), the longitudinal direction of the object / Transverse velocity information (vx, vy), width information of the object (width or w), time information, and the like. In the case of using the radar sensor, the object tracking system uses the previous measurement value x _k-1 (310) to determine the longitudinal / lateral distance information of the object, the longitudinal / lateral velocity information of the object, Can be obtained.

The object tracking system then generates a predicted value x ' _k 320 (S222) associated with the object of the current time based on the previous measurement x _k-1 310 as shown in FIG. 3 (b). A detailed description of how the object tracking system generates the predicted value x ' _k (320) will be described later.

When the current measurement values z _k (331, 332, 333) are obtained (S223), the object tracking system compares the predicted value x ' _k (320) and the current measured value z _k (331 , 332, and 333 to determine whether the current measured values z _k (331, 332, and 333) are valid data (S224). In the present invention, the object tracking system can determine whether the current measurement values z _k (331, 332, 333) are valid data using a Kalman filter. In one example, the object tracking system, a predicted value x _'k (320), the current measured value to within a predetermined range 340 based on the z _k (331, 332, 333) is based on whether z _k (331 current measurements that position , 332, and 333 are valid data.

The object tracking system then determines a first measurement (x, y) that is within a specified range 340 based on the predicted value x ' _k (320) among the current measurements z _k (331, 332, 333) The value x _k (331) is determined to be valid data (S225).

The object tracking system, on the other hand, includes a second measurement 332 and a third measurement 332 that are not located within the specified range 340 based on the predicted value x ' _k 320 among the current measurements z _k 331, 332, 333) as invalid data (S226).

Then, the object tracking system tracks each object based on the sensing data (ex. X _k (331)) determined as valid data through validation (S220) (S230).

In addition, the object tracking system determines whether the current measurement values z _k (331, 332, 333) are valid data, and generates state information of the object based on the obtained result (S240). The state information of the object has been described above with reference to FIG. 1, and a detailed description thereof will be omitted here.

Thereafter, the object tracking system stores the sensing data determined as valid data and the state information of the object as the tracking information of the object (S250), and then utilizes the sensing data as a previous measurement value when tracking the object again.

In the present invention, gating may be used as a validation method. 4 is a reference diagram for explaining a validity verification method according to an embodiment of the present invention.

According to FIG. 4, the object tracking system generates a circular gate (gate) 350 based on the predicted value 320 generated based on the previous measurement.

The object tracking system then verifies the validity of the trace based on whether the current measurement is located within the gate 350.

Gate 350 in the at least two current measured values _{(o 1 (361), o} 2 (362), o 3 (363), o 4 (364)) If the position, the object tracking system Global Nearest Neighbor (GNN ) Method to select the closest current measurement (o ₄ (364)).

Since object tracking system to fuse the prediction value 320, and the closest current measured value o ₄ (364) generates an estimated value.

Referring back to FIG. 2 and FIG.

In the present invention, a gate size for validity verification can be used as a reference range (or reference value). The gate size is a value designed to reflect a factor that may affect the accuracy of the sensor such as the error of the sensor, the running speed, and the running environment, for example, 2 m may correspond to this. When the gate size is defined as 2 m, if the current measured value 341 exists within 2 m (350) of the predicted value 320 as shown in FIG. 3 (c), the predicted value 320 is determined to be valid. If the current measured values 342 and 343 do not exist within 2m (350) of the predicted value 320, the predicted value 320 is judged to be invalid.

After step S243, the object tracking system manages the objects corresponding to the tracking object based on the result of the validation (S244). The object tracking system includes a track list in which information on the relative distance from the vehicle to the object, such as the state of the object, the type of the object, the longitudinal relative distance, and the lateral relative distance, The relative velocity of the object relative to the vehicle such as the longitudinal relative velocity and the lateral relative velocity may be recorded to manage objects corresponding to the object to be tracked.

On the other hand, the object tracking system can manage the tracking object and the tracking object when managing the objects corresponding to the tracking object.

If the state of the object is the empty track 110, the object tracking system determines the object to be tracked and then excludes it from the management object. If the state of the object is not the empty track 110, that is, if the state of the object is the first initiator track 120, the second initiator track 130, the third initiator track 140, the concert track 150, The potential track 160, etc., the object tracking system determines the object to be tracked and continuously includes it in the object of management.

Next, how the object tracking system generates the predicted value 320 will be described.

First, a method for the object tracking system to generate the predicted value 320 based on the information obtained by the camera sensor will be described. Hereinafter, the predicted value 320 generated based on the information obtained by the camera sensor is defined as a first predicted value.

The object tracking system calculates a first predicted value through a kinematic prediction model based on the previous state. The prediction model applied when the object tracking system calculates the first predicted value is a CV (Constant Velocity) model. The object tracking system can calculate the first predicted value for the next time point k at the previous point-in-time k-1 using the following equation (1).

Means the predicted position (predicted state) of the tracking target object (object tracker) in the _k-1 is the next point k (time step k) when the present time one k-1 and, Class _{k | |} x ^c _k from the _{k -1} indicates the type of the object to be tracked at the next point k. px _{k | k-1} represents the longitudinal relative distance of the tracking object at the next point k, and py _{k | k-1} represents the lateral relative distance of the tracking object at the next point k. vxk _{| k-1} denotes the longitudinal relative speed of the object to be traced at the next point k, and vyk _{| k-1} denotes the transverse relative speed of the object to be traced at the next point k.

In order to obtain the latter determinant in Equation (1), the state transition matrix used to compute the determinant of the former is expressed by Equation (2).

Next, how the object tracking system generates the predicted value 320 based on the information obtained by the radar sensor will be described. The predicted value 320 generated based on the information obtained by the radar sensor is defined as a second predicted value.

Similar to the case of calculating the first predicted value, the object tracking system applies the CV model as a prediction model even when calculating the second predicted value. However, the information obtained by the camera sensor includes the class information of the object, but the information obtained by the radar sensor does not include the class information of the object. Therefore, when calculating the second predicted value, the class information of the object is excluded unlike the case of calculating the first predicted value.

The object tracking system can calculate the second predicted value for the next time point k at the previous time point (k-1) using the following Equation (3).

In the above, x ^r _{k | k-1} indicates the predicted position of the tracking object at the next point k when the current point of time is k-1.

Meanwhile, in order to obtain the latter determinant in Equation (3), the state transition matrix used for calculating the matrix of the former is expressed by Equation (2).

Next, a method for the object tracking system to generate the predictive value 320 based on the information obtained by the lidar sensor will be described. In the following description, the predicted value 320 generated based on the information obtained by the Lidar sensor is defined as a third predicted value.

Similar to the case of calculating the second predicted value, the object tracking system applies the CV model as a prediction model even when calculating the third predicted value. The object tracking system can calculate the third predicted value for the next time point k at the previous point-in-time k-1 using the equation (3). The object tracking system x ^r _{k | k-1} is applied to the equation 3 _{| k-1} instead of ^l x _k. x ^l _{k | k-1} is the predicted position of the object to be tracked at the next point k when the current point is k-1.

5 and 6 are reference diagrams for explaining object tracking results of an object tracking system according to an embodiment of the present invention.

5, an object tracking system (not shown) mounted on the child vehicle 410 recognizes the other vehicle 420 and the pedestrian 430 at predetermined time intervals using a camera sensor.

The object tracking system then uses the validation method to determine the state of the other vehicle 420 and the state of the pedestrian 430 as one of the tracks 110 to 160 suggested in FIG.

6A shows a process of determining the state of the other vehicle 420 and the state of the pedestrian 430 at predetermined time intervals based on the information obtained using the camera sensor. The state value of 0 in FIG. 6A means that the state of the object is the empty track 110. The state value of 1 means that the state of the object is the first initiator track 120 do. A state value of 2 means that the state of the object is the second initiator track 130, and a state value of 3 means that the state of the object is the third initiator track 140. The state value of 4 means that the state of the object is the potential track 160, and the state value of 6 means that the state of the object is the concrete track 150. [

6B is a diagram showing the longitudinal relative distances from the child vehicle 410 to the objects 420 and 430 in a time division manner and FIG. , 430) are shown separately by time. Conventionally, when the pedestrian 430 is hidden by another vehicle 420, the pedestrian 430 can not be traced. According to the present invention, as shown in FIG. 1, the state of an object is classified and displayed so that the object can be tracked robustly even if the object is temporarily hidden.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention described with reference to FIGS. 1 through 6 relates to a method for tracking an object through validation in a single sensor environment. However, it is also possible to track objects through validation in a multi-sensor environment. This will be described below.

In a multi-sensor environment, object tracking results by each sensor may be different for various reasons such as the error of the sensor itself. In order to solve such a problem, the present invention proposes a sensor fusion strategy. The sensor fusion strategy proposed in the present invention includes a method of determining the state of an object by fusing sensed data and a method of determining the state of the object by fusing the results of validity verification by each sensing data. In the following, the former method is defined as the first fusion strategy, and the latter method is defined as the second fusion strategy.

First, the first fusion strategy will be described. In the present invention, the first fusion strategy may have a centralized sensor fusion structure. A first fusion strategy to be described below will be described by way of example of an environment using two sensors, but it is also possible in the present invention to use three or more sensors as the first fusion strategy. 7 is a flowchart illustrating a method of tracking an object in a multi-sensor environment according to a first embodiment of the present invention.

First, the object tracking system collects the first sensing data acquired by the first sensor and the second sensing data acquired by the second sensor in the previous cycle (S510). First sensing data from the information about the tracking of the object obtained by the first sensor information (Detection from 1 ^st sensor object only) for the position of the object obtained by the first sensor in the previous cycle, until the previous cycle ( Detection from track only). The second sensing data may include information on the position of the object acquired by the second sensor in the previous cycle, information on the tracking of the object acquired by the first sensor up to the previous cycle, and the like, as well as the first sensing data .

The object tracking system then generates a first predicted value for the next cycle based on the first sensed data, and generates a second predicted value for the next cycle based on the second sensed data (S520).

The object tracking system then sets a first reference range in relation to the first predicted value and the second predicted value (S530). The object tracking system may then set an intersection cap between a first range associated with the first predicted value and a second range associated with the second predicted value to a first reference range, ) May be set to the first reference range.

The object tracking system then collects the third sensing data obtained by the first sensor and the fourth sensing data obtained by the second sensor in the current cycle (S540). The third sensing data may include information on the position of the object obtained by the first sensor in the current cycle, information on the tracking of the object obtained by the first sensor up to the current cycle, and the like. Like the third sensing data, the fourth sensing data may include information on the position of the object acquired by the second sensor in the current cycle, information on the tracking of the object acquired by the second sensor up to the current cycle, .

Thereafter, the object tracking system determines whether the third sensing data and the fourth sensing data are within a first reference range (S550).

If it is determined that the third sensing data and the fourth sensing data are within the first reference range, the object tracking system determines that the validity of the object tracking is verified (S560). On the other hand, if it is determined that at least one of the third sensing data and the fourth sensing data is not within the first reference range, the object tracking system determines that the validity of the object tracking is not verified (S570).

According to the present invention, if at least one of the third sensing data and the fourth sensing data is determined to be within the first reference range, it is also possible that the object tracking system determines that the validity of the object tracking is verified.

In the present invention, when the first sensing data and the second sensing data are collected in step S510, the sensing data is combined to generate combined data, and in step S520, one prediction value for the next cycle is generated based on the combined data It is possible.

For example, when the first sensing data is information obtained by the camera sensor and the second sensing data is information obtained by the radar sensor, the class information of the object, the longitudinal / lateral distance information of the object, / Lateral speed information, width information of an object, time information, and the like can be generated as combined data. In this case, since the combined data is the same as the information obtained by the camera sensor, a method of generating a predicted value based on information obtained by the camera sensor can be applied to a method of generating predicted values based on combined data.

In the present invention, it is also possible that the object tracking system collects the fifth sensing data when one fifth sensing data is obtained using the first sensor and the second sensor together. In this case, the fifth sensing data includes information on the position of the object obtained by the first sensor and the second sensor in a specific cycle (Detection from 1 ^st sensor and 2 ^nd sensor object) information ^{(Detection of associated track (1 st} sensor)), the tracking information ^{(Detection of associated track (2 nd} sensor)) for the object obtained by the second sensor to a specific cycle to the tracking of the object, claim to a certain cycle, (1 ^st sensor and 2 ^nd sensor) of the object obtained by the first sensor and the second sensor, and the like.

Meanwhile, in the present invention, when the object tracking system collects the fifth sensing data by using the first sensor and the second sensor together, the same content information (eg, longitudinal relative speed of the object with respect to the vehicle, Direction relative speed, and the like), it is possible to collect information of any one of the information obtained by the first sensor and the information obtained by the second sensor as the fifth sensing data.

Next, the second fusion strategy will be described. In the present invention, the second fusion strategy may have a decentralized sensor fusion structure. As in the first fusion strategy, the second fusion strategy will be described as an example in which two sensors are applied, but the second fusion strategy is not limited thereto. 8 is a flowchart illustrating a method of tracking an object in a multi-sensor environment according to a second embodiment of the present invention.

First, the object tracking system collects the first sensing data obtained by the first sensor and the second sensing data obtained by the second sensor in the previous cycle (S610).

Thereafter, the object tracking system generates a first predicted value for the next cycle based on the first sensed data, and generates a second predicted value for the next cycle based on the second sensed data (S620).

Thereafter, the object tracking system sets a second reference range in relation to the first predicted value, and sets a third reference range in relation to the second predicted value (S630).

The object tracking system then collects the third sensing data acquired by the first sensor and the fourth sensing data acquired by the second sensor in the current cycle (S640).

Thereafter, the object tracking system determines whether the third sensing data is within the second reference range (S650), and determines whether the fourth sensing data is within the third reference range (S660).

If it is determined that the third sensing data is within the second reference range and the fourth sensing data is within the third reference range, the object tracking system determines that the validity of the object tracking is verified (S670). On the other hand, if it is determined that the third sensing data is not within the second reference range or the fourth sensing data is not within the third reference range, the object tracking system determines that the validity of the object tracking is not verified (S680 ).

According to the present invention, when it is determined that the third sensing data is within the second reference range or the fourth sensing data is within the third reference range, it is also possible that the object tracking system determines that the validity of the object tracking is verified Do.

The first fusion strategy has an intensive sensor fusion structure, and data fusion of data obtained by each sensor such as a camera sensor, a lidar sensor, a radar sensor, etc. is performed before tracking an object. On the other hand, the second fusion strategy has a distributed sensor fusion structure. After the object is traced through each sensor such as a camera sensor, a lidar sensor, a radar sensor, etc., fusion of data obtained by these sensors is performed. In the present invention, the latter data fusion is defined as track to track data fusion.

9 is a conceptual diagram illustrating a second fusion strategy according to an embodiment of the present invention.

When the second fusion strategy has a distributed sensor fusion structure, it may have the following characteristics.

First, data fusion is performed at a higher level by a tracker of each sensor such as a camera sensor, a radar sensor, and a Lidar sensor. In the present invention, this is defined as high level data fusion.

The camera sensor usually has an update rate of 90 msec (S710a). The radar sensor usually has an update rate of 50 msec (S710b), and the lidar sensor usually has an update rate of 80 msec (S710c). Here, the update rate means the time taken to newly acquire the sensing data.

Since the update rates of the camera sensor, the radar sensor, the Lidar sensor, and the like are different from each other, the sensing data obtained by the camera sensor and the sensing data obtained by the radar sensor have substantially similar types, The time taken to track the object and track management of the tracking result for the object may be matched to each other, for example, 10 ms (S720a, S720b).

However, since the types of sensing data obtained with the camera sensor and the radar sensor are different from each other in the case of the LIDAR sensor, the time required to track the object and manage the tracking result for the object such as the camera sensor, the radar sensor, (S720c).

When the tracking result (track (C)) by the camera sensor, the tracking result (track (R) by the radar sensor) and the tracking result (track (L) A multi-sensor data fusion procedure is performed (S730).

First, the object tracking system synchronizes the time at which the tracking result by each sensor is obtained through a rate transition (S731). The object tracking system may set the synchronization time to, for example, 10 ms.

Thereafter, the object tracking system sequentially performs a high-level data fusion procedure S732, a tracking result management procedure S733, an object classification procedure S734, and the like.

The high-level data fusion procedure S732 is a step of integrating the tracking result by the camera sensor, the tracking result by the radar sensor, and the tracking result by the Raidar sensor. The high-level data fusion procedure S732 fuses the sensing data according to a track to track data fusion method, which will be described later in more detail.

The tracking result management procedure (S733) is a step of storing and managing the tracking result by the camera sensor, the tracking result by the radar sensor, and the tracking result by the Raidar sensor.

The object tracking system can store and manage the existence probability (existence probability or existence level) of the object when storing and managing the tracking result by each sensor in the tracking result management procedure (S733) as a result of tracking. Here, the existence probability of the object means the state information of the object which is expressed as shown in FIG. 1 according to the validity check, and monitors the trace history of the object through the gating process.

The object classification procedure (object classification S734) refers to a step of sorting objects by type based on information obtained by the camera sensor. In the present invention, when the camera sensor does not operate normally, it is also possible to classify objects based on information (e.g., vehicle speed, object speed, etc.) obtained by a radar sensor, a Lidar sensor, or the like.

The object tracking system then tracks the object based on the result obtained by fusing the information obtained by the multiple sensors (S735). In the present invention, this is defined as a multi-sensor fusion track.

Second, in the track management procedure (S720a, S720b, S720c), the tracker of each sensor stores and manages the information acquired by each sensor, the result obtained by tracking the object, and the like, and the status information of the object obtained by using the Kalman filter such as a potential track.

Third, the final state information of the object by the multiple sensors is determined in an upper level data fusion procedure (S732).

Next, the high-level data fusion procedure S732 will be described. Hereinafter, a multi-sensor environment including a camera sensor, a radar sensor, a radar sensor, etc. will be described as an example, but the multi-sensor environment is not limited thereto.

① Lada standard gating and association (gating & association)

The object tracking system performs gating based on the Raider sensor to generate a first track. The object tracking system can thereby generate a first independent track, a first fusion track, and the like. In the above description, the first independent track means a track generated based only on the sensing data obtained through the Lydia sensor, and the first fusion track is generated based on the sensing data obtained through the Lydia sensor, the camera sensor, and the radar sensor Means a track.

② Camera-based gating and association

The object tracking system generates a second track by performing a fusion between the camera sensor and the radar sensor with respect to the confirmed track among the tracks not fused with the Raider sensor. The object tracking system can thereby create a second fusion track. The second fusion track is a track generated based on sensing data obtained through a camera sensor and a radar sensor.

③ Single track

The object tracking system generates the third track by independently composing the unfused verification tracks through (1) and (2). The object tracking system can thereby create a second independent track, a third independent track, and the like. The second independent track means a track generated based only on sensing data obtained through a camera sensor and the third independent track means a track generated based on only sensing data obtained through a radar sensor.

④ Priority-based data fusion of clustering-based sensor through gating process

The priority of each sensing data when fusing the sensing data obtained by the multiple sensors can be represented as shown in Table 1 below as an example.

Priority Class Px Py Vx Vy Width One camera Lada Lada Lada Lada camera 2 Estimation Radar camera Radar camera Default value 3 Estimation camera Radar camera Radar -

In the above, Class represents the class information of the object, and Width represents the width information of the object. Px denotes the longitudinal distance information of the object, and Py denotes the lateral distance information of the object. Vx denotes the longitudinal velocity information of the object, and Vy denotes the lateral velocity information of the object.

The class information of the object, the width information of the object, and the like are contained only in the sensing data obtained by the camera sensor. Accordingly, in the case of the class information of the object, if the sensing data acquired by the camera sensor is not fused during the data fusion, the object type is estimated through the object classification procedure S734. On the other hand, in the case of object width information, a predetermined default value is applied if the sensing data obtained by the camera sensor is not fused when data fusion is performed. In the present invention, the default value can be set to, for example, 0.3 m to 1.5 m.

Meanwhile, when the previous state of the object in the multiple sensor environment is the potential track 160, a method of determining the current state of the object may be performed as follows.

Since the potential track 160 indicates a high probability of existence of an object, if the plurality of sensors show the result of the potential track 160 for one object, it can be determined that the probability of existence of the object is high.

For example, in an environment composed of three sensors, the current state of an object may be determined in the following manner.

(1) When the tracking result of one or more sensors is the concurrent track 150, the corresponding track is determined as the controlled track (that is, the concurrent track 150).

(2) When the tracking result of two or more kinds of sensors is the potential track 160, the track is determined as the control target track (that is, the concert track 150).

(3) When only the tracking result of the sensor 1 is the potential track 160, the track is determined as the warning target track (i.e., the potential track 160).

On the other hand, if the class identification value of the object is calculated as a pedestrian at the previous time and the sensing data is not acquired by the camera sensor at the current time or the sensing data acquired by the camera sensor is not valid, It is also possible to keep the class identification value of the object as a pedestrian.

The class identification value of an object refers to information on the type of object such as a pedestrian, a bicycle, a Vulnerable Road Users (VRU), a vehicle, and the like. The class identification value of the object is necessary to determine whether the object to be validated is the same object.

The object tracking system includes class information of the object (e.g., traffic abbreviation, vehicle, etc.), state information of the object (e.g., potential track or concert track), position information of the object, velocity information of the object, Information, and the like, to generate a fusion tracker.

Then, the object tracking system acquires the measured values using the camera sensor, the radar sensor, the radar sensor, etc. at the K + 1 time point, and also generates the predicted value at the K + 1 time point based on the fusion track generated at the K point.

The object tracking system then identifies valid measurement values among the measured values through gating, and then fuses the valid measurement values and predicted values through an estimation procedure using a Kalman filter.

However, when the class information of the object is the pedestrian at the point K and the measured value effective at the point K + 1 is the sensing data obtained by the radar sensor, the sensing data obtained by the radar sensor does not include the class information of the object not.

If one object is located within the region of interest (ex. At least one point located on the traveling path of the vehicle), the class information of the object need not be provided. However, if a plurality of objects are located within the region of interest, class information of the object is required to determine whether the object to be validated is the same object.

Therefore, in this case, the class information of the object should be estimated based on the speed information of the object. If the result of estimating the class information of the object is unknown, the class information of the object is also maintained as a pedestrian at the time of K + 1. Here, unreal means that the class information of an object is unclear as to whether it is a pedestrian, a bicycle, or a vehicle.

As described above, unlike the case of the camera sensor, the class information of the object can not be obtained when the radar sensor is used. Therefore, in the present invention, in this case, the class information of the object can be obtained by classifying the object based on the longitudinal / transverse speed information (e.g., longitudinal / transverse absolute speed) and the speed ratio of the object.

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention has been described with reference to Figs. Best Mode for Carrying Out the Invention Hereinafter, preferred forms of the present invention that can be inferred from the above embodiment will be described.

FIG. 10 is a block diagram schematically illustrating internal configurations of an object tracking apparatus according to a preferred embodiment of the present invention.

Referring to FIG. 10, the object tracking apparatus 800 includes an object information prediction unit 810, a valid data determination unit 820, an object tracking unit 830, a power supply unit 840, and a main control unit 850.

The power supply unit 840 performs a function of supplying power to each configuration of the object tracking device 800. [ When the object tracking device 800 is mounted on a vehicle, the object tracking device 800 can use the vehicle battery as a power source. In this case, the power source 840 may not be provided in the object tracking device 800.

The main control unit 850 controls the overall operation of each of the components constituting the object tracking apparatus 800.

The object information predicting unit 810 predicts information about the object based on the first sensing data obtained at the previous time.

In the case of using the radar sensor, the object information prediction unit 810 calculates distance information between the object and the vehicle that tracks the object, velocity information between the object and the vehicle, and state transition matrix, Can be predicted. In more detail, the object information prediction unit 810 can use the vertical distance information and the horizontal distance information as the distance information, and the vertical direction information and the horizontal direction information can be used as the speed information.

In the case of using a camera sensor, the object information prediction unit 810 calculates distance information between the object and the vehicle that tracks the object, velocity information between the object and the vehicle, and state transition matrix, In addition, information on an object can be predicted by including information on the type of the object.

The object information predicting unit 810 predicts the information about the object based on the data obtained by combining the first sensing data and the third sensing data when the third sensing data is acquired by the sensor different from the first sensing data have.

The object information predicting unit 810 predicts the sensed data selected in accordance with the priority order for the same item among the first sensing data and the third sensing data when the third sensing data is acquired by the sensor different from the first sensing data Information on the object can be predicted.

The valid data determination unit 820 compares the second sensing data obtained at the current time with the information about the object predicted by the object information prediction unit 810 to determine whether valid data exists in the second sensing data And performs a function of judging.

The valid data determination unit 820 may determine whether valid data exists based on whether an error between the second sensing data and the object information is less than a reference value. If it is determined that the error between the second sensing data and the object information is less than the reference value, the valid data determination unit 820 determines the second sensing data as valid data. On the other hand, if it is determined that the error between the second sensing data and the object information is greater than the reference value, the valid data determination unit 820 determines the second sensing data as invalid data (invalid data).

When judging whether valid data exists, the valid data judging unit 820 judges whether valid data exist or not based on the measurement error of the sensor, the traveling speed of the object, the traveling speed of the vehicle for tracking the object, And a reference value determined based on at least one of information on the surrounding environment.

The valid data determination unit 820 may determine whether valid data exists among the second sensing data related to the object located in the region of interest. In this case, the object tracking apparatus 800 may further include a region of interest setting unit (not shown) for setting a region of interest. The region of interest setting unit may set the region of interest based on the FOV of the sensor.

The object tracking unit 830 performs a function of tracking an object based on at least one valid data if the valid data determining unit 820 determines that valid data exists.

The object tracking unit 830 may track an object based on at least one valid data selected based on an error between valid data and information about the object. In the present invention, the object tracking unit 830 can use the valid data having the smallest error with the information on the object for the object tracking.

11 is a block diagram illustrating internal configurations that may be added to the object tracking apparatus of FIG.

Referring to FIG. 11, the object tracking apparatus 800 may further include an object state generation unit 861.

The object state generation unit 861 generates state information of an object corresponding to an unrecognized object based on a first determination result related to whether or not valid data exists. The object tracking apparatus 800 may further include an object information management unit (not shown) for storing and managing object status information and valid data.

The object state generation unit 861 generates a state information of the object by validating whether the first sensing data exists or not based on a result of the third determination as to whether or not the second sensing data is the first acquired data As a result of the fourth determination as to whether or not the data exists, it is determined as a result of the fifth determination as to whether or not the valid data is continuously judged. A result of at least one of the sixth determination results related to whether or not the predetermined time has elapsed.

The object state generation unit 861 sequentially outputs the second determination result to the sixth determination result in the order of the second determination result, the third determination result, the fourth determination result, the fifth determination result, and the sixth determination result Can be used. An example is as follows.

First, the object state generation unit 861 generates state information of the object based on the second determination result. That is, if it is determined that the first sensing data does not exist, the object state generation unit 861 generates the state information of the object as either an empty track or a first initiator track. On the other hand, if it is determined that the first sensing data exists, the object state generation unit 861 outputs the object state information to the second initiator 2 track, the third initiator 3 track, the potential track ) And a confirmed track.

Next, the object state generation unit 861 generates state information of the object based on the third determination result. That is, if it is determined that the second sensing data is not the first acquired data, the object state generation unit 861 generates state information of the object as an empty track. On the other hand, if it is determined that the second sensing data is the first acquired data, the object state generation unit 861 generates the state information of the object as the first initiator track.

Next, the object state generation unit 861 generates object state information based on the fourth determination result. That is, if it is determined that the valid data exists, the object state generation unit 861 generates the state information of the object as the second initiator track. On the other hand, if it is determined that the valid data does not exist, the object state generation unit 861 generates the state information of the object as one of the third initiator track, the potential track, and the concurrent track.

Next, the object state generation unit 861 generates state information of the object based on the fifth determination result. That is, if it is determined that the valid data does not exist continuously, the object state generation unit 861 generates the state information of the object as the third initiator track. On the other hand, if it is determined that the valid data continuously exist, the object state generation unit 861 generates the state information of the object as either a potential track or a conformed track.

Next, the object state generation unit 861 generates state information of the object based on the sixth determination result. That is, if it is determined that the predetermined period of time has not elapsed since the valid data is continuously determined, the object state generator 861 generates the state information of the object as a conformed track. On the other hand, if it is determined that a predetermined period of time has elapsed since the valid data is continuously determined, the object state generation unit 861 generates the state information of the object as a potential track.

On the other hand, the object state generation unit 861 acquires first sensing data by a sensor of a camera sensor, a radar sensor and a ladder sensor, and acquires sixth sensing data and seventh sensing data by the other two sensors The state information of the object can be generated based on at least one sensing data selected from the first sensing data, the sixth sensing data, and the seventh sensing data.

If the first sensing data are the sensing data obtained by the camera sensor, the sixth sensing data are the sensing data obtained by the radar sensor, and the seventh sensing data are the sensing data obtained by the LR sensor, the object state generator 861 ) Can be operated in the following order.

First, the object state generation unit 861 generates object state information based on the seventh sensing data, and generates object state information based on the first sensing data, the sixth sensing data, and the seventh sensing data .

The object state generation unit 861 then generates state information of the object based on the first sensing data and the sixth sensing data when the predetermined condition is met.

Then, the object state generation unit 861 generates state information of the object based on the first sensing data, and generates state information of the object based on the sixth sensing data.

The object tracking apparatus 800 may further include an object tracking determination unit 862 in addition to the object state generation unit 861.

The object tracking determination unit 862 performs a function of determining whether to continue to track the object based on the state information of the object.

The object tracking apparatus 800 may further include a driving control unit 863 in addition to the object state generating unit 861.

The travel control unit 863 controls the travel of the vehicle that tracks the object based on the state information of the object.

Referring to FIG. 11, the object tracking apparatus 800 may further include a first range determination unit 871.

If the information on the object is predicted based on the first sensing data and the third sensing data obtained by the other sensor, the first range determining unit 871 determines information on the predicted object based on the first sensing data, 3 &lt; / RTI &gt; sensing data based on information about the predicted object. In this case, the valid data determination unit 820 determines the fourth sensing data, the third sensing data, and the fifth sensing data obtained by the same sensor with the first sensing data, It is possible to determine whether valid data exists among the fourth sensing data and the fifth sensing data.

The object tracking apparatus 800 may further include a second range determination unit 872. [

When the information on the object is predicted based on the first sensing data and the third sensing data obtained by the other sensor, the second range determination unit 872 determines the information on the predicted object based on the first sensing data And determines a third reference range based on the information about the predicted object based on the third sensing data. In this case, the valid data determination unit 820 compares the first sensing data and the fourth sensing data obtained by the same sensor with the second reference range to determine whether valid data exists among the fourth sensing data, The third sensing data and the fifth sensing data obtained by the same sensor may be compared with a third reference range to determine whether there is valid data among the fifth sensing data.

Referring to FIG. 11, the object tracking apparatus 800 may further include an object rate determination unit 881 and a first object type estimation unit 882.

The object speed determination unit 881 determines whether the object speed information is included in the valid data if the object type information is not included in the valid data.

The first object type estimating unit 882 estimates information on the type of the object based on the speed information of the object if the object speed determining unit 881 determines that the speed information of the object is included in the valid data Function.

Referring to FIG. 11, the object tracking apparatus 800 may further include a first object type determination unit 891, a second object type determination unit 892, and a second object type estimation unit 893.

The first object type determination unit 891 determines whether information on the type of the object is a pedestrian if information on the type of the object is included in the first sensing data.

If the first object type determination unit 891 determines that the information on the type of the object is a pedestrian, the second object type determination unit 892 determines whether information on the type of the object is included in the valid data .

If it is determined by the second object type determination unit 892 that the information on the type of the object is not included in the valid data, the second object type estimation unit 893 determines whether the object included in the first sensing data And the information about the type is included in the valid data.

Next, an operation method of the object tracking apparatus 800 will be described. 12 is a flowchart schematically illustrating an object tracking method according to a preferred embodiment of the present invention.

First, the object information prediction unit 810 predicts information about the object based on the first sensing data obtained at the previous time (S910).

Thereafter, the valid data determination unit 820 compares the second sensing data obtained at the current time with the information about the object to determine whether valid data exists in the second sensing data (S920).

If it is determined that valid data exists, the object tracking unit 830 then tracks the object based on at least one valid data (S930).

Between step S920 and step S930, the object state generation unit 861 may generate object state information to correspond to an unrecognized object based on a first determination result related to whether or not valid data exists STEP A).

Between STEP A and S930, the object tracking determination unit 862 can determine whether to continue to track the object based on the state information of the object.

Between STEP A and S930, the travel control unit 863 can control the traveling of the vehicle that tracks the object based on the state information of the object.

Between steps S910 and S920, when the first range determination unit 871 predicts information about the object based on the third sensing data obtained by the sensor other than the first sensing data, The first reference range can be determined on the basis of the information about the predicted object based on the third sensing data and the information on the predicted object based on the third sensing data. In this case, the valid data determination unit 820 determines the fourth sensing data, the third sensing data, and the fifth sensing data obtained by the same sensor with the first sensing data, It is possible to determine whether valid data exists among the fourth sensing data and the fifth sensing data.

Between steps S910 and S920, if the second range determination unit 872 predicts the information about the object based on the third sensing data obtained by the sensor other than the first sensing data, The second reference range can be determined based on the information about the predicted object and the third reference range can be determined based on the information about the predicted object based on the third sensing data. In this case, the valid data determination unit 820 compares the first sensing data and the fourth sensing data obtained by the same sensor with the second reference range to determine whether valid data exists among the fourth sensing data, The third sensing data and the fifth sensing data obtained by the same sensor may be compared with a third reference range to determine whether there is valid data among the fifth sensing data.

Between steps S920 and S930, if the object speed determination unit 881 does not include information on the type of object among the valid data, it may determine whether or not the speed information of the object is included in the valid data . If it is determined that the speed information of the object is included in the valid data, the first object type estimating unit 882 can estimate the information about the object type based on the speed information of the object.

Between steps S920 and S930, if the first object type determination unit 891 includes information on the type of the object among the first sensing data, it can be determined whether or not information on the type of the object is a pedestrian . If it is determined that the information on the type of the object is a pedestrian, the second object type determination unit 892 can determine whether information on the object type is included in the valid data. If it is determined that information on the type of the object is not included in the valid data, the second object type estimating unit 893 stores information about the type of the object included in the first sensing data in the valid data .

It is to be understood that the present invention is not limited to these embodiments, and all elements constituting the embodiment of the present invention described above are described as being combined or operated in one operation. That is, within the scope of the present invention, all of the components may be selectively coupled to one or more of them. In addition, although all of the components may be implemented as one independent hardware, some or all of the components may be selectively combined to perform a part or all of the functions in one or a plurality of hardware. As shown in FIG. In addition, such a computer program may be stored in a computer readable medium such as a USB memory, a CD disk, a flash memory, etc., and read and executed by a computer to implement an embodiment of the present invention. As the recording medium of the computer program, a magnetic recording medium, an optical recording medium, or the like can be included.

Furthermore, all terms including technical or scientific terms have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs, unless otherwise defined in the Detailed Description. Commonly used terms, such as predefined terms, should be interpreted to be consistent with the contextual meanings of the related art, and are not to be construed as ideal or overly formal, unless expressly defined to the contrary.

It will be apparent to those skilled in the art that various modifications, substitutions and substitutions are possible, without departing from the scope and spirit of the invention as disclosed in the accompanying claims. will be. Therefore, the embodiments disclosed in the present invention and the accompanying drawings are intended to illustrate and not to limit the technical spirit of the present invention, and the scope of the technical idea of the present invention is not limited by these embodiments and the accompanying drawings . The scope of protection of the present invention should be construed according to the following claims, and all technical ideas within the scope of equivalents should be construed as falling within the scope of the present invention.

Claims (31)

An object information predicting unit for predicting information on an object based on first sensing data obtained at a previous time;
A valid data determination unit for comparing second sensing data obtained at a current time with information about the object to determine whether valid data exists among the second sensing data; And
An object tracking unit for tracking an object based on at least one valid data if it is determined that the valid data exists,
And an object tracking device for tracking the object. The method according to claim 1,
Wherein the effective data determination unit determines whether the valid data exists based on whether an error between the second sensing data and the object information is less than a reference value. 3. The method of claim 2,
The valid data determination unit determines whether or not the valid data exists by using the measurement error of the sensor, the traveling speed of the object, the traveling speed of the vehicle tracking the object, information on the surrounding environment in which the object is located, Wherein the reference value is determined based on at least one of information on a surrounding environment in which the object is located. The method according to claim 1,
The object information predicting unit predicts information about the object by calculating distance information between the object and the vehicle tracking the object, velocity information between the object and the vehicle, and a state transition matrix, . 5. The method of claim 4,
Wherein the object information predicting unit predicts information on the object further including information on the type of the object. The method according to claim 1,
An object state generating unit for generating state information of the object corresponding to the unrecognized object based on a first determination result related to whether or not the valid data exists,
And an object tracking device for tracking the object. The method according to claim 6,
The object state generation unit may generate a state information of the object based on a result of a second determination as to whether or not the first sensing data exists and a third determination as to whether the second sensing data is data obtained first, As a result of the fourth determination as to whether or not the valid data exists in the first judgment, a fifth determination result related to whether or not the valid data is continuously judged to exist, And a sixth determination result related to whether or not a predetermined time has elapsed. 8. The method of claim 7,
The object state generation unit may generate the second determination result, the third determination result, the fourth determination result, the fifth determination result, and the sixth determination result when using the second determination result to the sixth determination result. In order from the object side. The method according to claim 6,
An object tracking determination unit for determining whether to continue to track the object based on the state information of the object,
And an object tracking device for tracking the object. The method according to claim 6,
The object state generator may generate the object sensing data by using one of a camera sensor, a radar sensor, and a ladder sensor, and when the sixth sensing data and the seventh sensing data are obtained by the other two sensors, And generates the state information of the object based on at least one sensing data selected from among the first sensing data, the sixth sensing data, and the seventh sensing data. 11. The method of claim 10,
The object state generation unit may generate the object sensing data based on the sensing data obtained by the camera sensor and the sixth sensing data are sensing data obtained by the radar sensor, The state information of the object is first generated based on the seventh sensing data, and then, based on the first sensing data, the sixth sensing data, and the seventh sensing data, Generates state information of the object on the basis of the first sensing data and the sixth sensing data when the predetermined condition is satisfied, and thereafter generates state information of the object based on the first sensing data, And then, based on the sixth sensing data, Object-tracking device, characterized in that for generating the status information. The method according to claim 6,
A running control unit for controlling the running of the vehicle that tracks the object based on the state information of the object;
And an object tracking device for tracking the object. The method according to claim 1,
Wherein the object tracking unit tracks an object based on the at least one valid data selected based on an error between the valid data and information about the object. The method according to claim 1,
Wherein the valid data determination unit determines whether the valid data exists among second sensing data related to an object located in a ROI. The method according to claim 1,
If information on the object is predicted based on the third sensing data obtained by the sensor and the first sensing data, information on the object predicted based on the first sensing data and the third sensing data A first range determination unit for determining a first reference range based on information on the object predicted on a basis,
Further comprising:
The valid data determination unit may compare the fourth sensing data obtained by the same sensor with the first sensing data and the fifth sensing data obtained by the same sensor with the third sensing data to the first reference range sequentially And determines whether the valid data exists among the fourth sensing data and the fifth sensing data. The method according to claim 1,
If information on the object is predicted based on third sensing data obtained by a sensor different from the first sensing data, based on information about the object predicted based on the first sensing data, And determines a third reference range based on information on the predicted object based on the third sensing data,
Further comprising:
Wherein the valid data determination unit compares the fourth sensing data obtained by the same sensor with the first sensing data to the second reference range to determine whether the valid data exists among the fourth sensing data, And compares the third sensing data with fifth sensing data obtained by the same sensor with the third reference range to determine whether the valid data exists among the fifth sensing data. The method according to claim 1,
Wherein the object information predicting unit predicts information about the object based on data obtained by combining the first sensing data and the third sensing data when the third sensing data is acquired by a sensor different from the first sensing data, And the object tracking device. The method according to claim 1,
Wherein the object information predicting unit determines the third sensing data based on the first sensing data and the third sensing data based on the priority of the same item among the first sensing data and the third sensing data when the third sensing data is acquired by a sensor different from the first sensing data, And predicts the information about the object with the object. The method according to claim 1,
An object speed determiner for determining whether the valid data includes the speed information of the object if the valid data does not include information on the type of the object; And
Estimating information on the type of the object based on the velocity information of the object if it is determined that the velocity information of the object is included among the valid data;
And an object tracking device for tracking the object. The method according to claim 1,
A first object type determination unit for determining whether information on the type of the object is a pedestrian if information on the type of the object is included in the first sensing data;
A second object type determination unit for determining whether information on the type of the object is included in the valid data if it is determined that the information on the type of the object is a pedestrian; And
If it is determined that the information on the type of the object is not included in the valid data, a second object type adder that includes information on the type of the object included in the first sensing data, government
And an object tracking device for tracking the object. An object information prediction step of predicting information on an object based on first sensing data obtained at a previous time;
A valid data determination step of determining whether valid data exists among the second sensing data by comparing second sensing data obtained at a current time with information about the object; And
An object tracking step of tracking an object based on at least one valid data if it is determined that the valid data exists;
The object tracking method comprising: 22. The method of claim 21,
The object information prediction step may include estimating information on the object by calculating distance information between the object and the vehicle tracking the object, velocity information between the object and the vehicle, and a state transition matrix A method for tracking an object. 22. The method of claim 21,
An object state generating step of generating state information of the object corresponding to the unrecognized object based on a first determination result related to whether or not the valid data exists;
The object tracking method further comprising: 24. The method of claim 23,
An object tracking determination step of determining whether to continue to track the object based on the state information of the object
The object tracking method further comprising: 24. The method of claim 23,
A traveling control step of controlling traveling of the vehicle that tracks the object based on the state information of the object;
The object tracking method further comprising: 22. The method of claim 21,
Wherein the object tracking step tracks an object based on the at least one valid data selected based on an error between the valid data and the information about the object. 22. The method of claim 21,
If information on the object is predicted based on the third sensing data obtained by the sensor and the first sensing data, information on the object predicted based on the first sensing data and the third sensing data A first range determination step of determining a first reference range based on information on the object predicted on a basis;
Further comprising:
The valid data determination step may include comparing the fourth sensing data obtained by the same sensor with the first sensing data and the fifth sensing data obtained by the same sensor with the third sensing data, And determines whether the valid data exists among the fourth sensing data and the fifth sensing data. 22. The method of claim 21,
If information on the object is predicted based on third sensing data obtained by a sensor different from the first sensing data, based on information about the object predicted based on the first sensing data, And a second range determination step of determining a third reference range based on information on the predicted object based on the third sensing data
Further comprising:
The valid data determination step may include comparing the fourth sensing data obtained by the sensor with the first sensing data to the second reference range to determine whether the valid data exists among the fourth sensing data, And comparing the fifth sensing data obtained by the same sensor with the third sensing data to the third reference range to determine whether the valid data exists in the fifth sensing data . 22. The method of claim 21,
Determining whether the valid data includes the speed information of the object if the valid data does not include information on the type of the object; And
A first object type estimation step of estimating information on the type of the object based on the speed information of the object if it is determined that the speed information of the object is included among the valid data
The object tracking method further comprising: 22. The method of claim 21,
A first object type determination step of determining whether information on the type of the object is a pedestrian if information on the type of the object is included in the first sensing data;
A second object type determination step of determining whether information on the type of the object is included in the valid data if it is determined that the information on the type of the object is a pedestrian; And
If the information on the type of the object is not included in the valid data, information on the type of the object included in the first sensing data is included in the valid data, step
The object tracking method further comprising: 31. A computer program stored in a computer readable medium for executing an object tracking method according to any one of claims 21 to 30 in a computer.

Images