GB2608400A - Object tracking data association module and method - Google Patents

Object tracking data association module and method Download PDF

Info

Publication number
GB2608400A
GB2608400A GB2109395.0A GB202109395A GB2608400A GB 2608400 A GB2608400 A GB 2608400A GB 202109395 A GB202109395 A GB 202109395A GB 2608400 A GB2608400 A GB 2608400A
Authority
GB
United Kingdom
Prior art keywords
track
data set
parameter
object tracking
sensor measurement
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
GB2109395.0A
Other versions
GB202109395D0 (en
Inventor
Yalagam Srinivas
Sridhar Muralikrishna
Bergen Tobias
Prakash Padiri Bhanu
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Continental Automotive GmbH
Original Assignee
Continental Automotive GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Continental Automotive GmbH filed Critical Continental Automotive GmbH
Priority to GB2109395.0A priority Critical patent/GB2608400A/en
Publication of GB202109395D0 publication Critical patent/GB202109395D0/en
Priority to DE112022002063.6T priority patent/DE112022002063T5/en
Priority to PCT/EP2022/058404 priority patent/WO2022214375A1/en
Publication of GB2608400A publication Critical patent/GB2608400A/en
Withdrawn legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S3/00—Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received
    • G01S3/78—Direction-finders for determining the direction from which infrasonic, sonic, ultrasonic, or electromagnetic waves, or particle emission, not having a directional significance, are being received using electromagnetic waves other than radio waves
    • G01S3/782—Systems for determining direction or deviation from predetermined direction
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S13/00—Systems using the reflection or reradiation of radio waves, e.g. radar systems; Analogous systems using reflection or reradiation of waves whose nature or wavelength is irrelevant or unspecified
    • G01S13/86—Combinations of radar systems with non-radar systems, e.g. sonar, direction finder
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S5/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S5/16—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations using electromagnetic waves other than radio waves
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G06N3/0442—Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
    • G01S2205/00—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations
    • G01S2205/01—Position-fixing by co-ordinating two or more direction or position line determinations; Position-fixing by co-ordinating two or more distance determinations specially adapted for specific applications

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Multimedia (AREA)
  • Artificial Intelligence (AREA)
  • Data Mining & Analysis (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Electromagnetism (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Image Analysis (AREA)

Abstract

An object tracking data association module (100) comprising: a track data retriever (110) configured to provide at least one track data set of at least one track associated with at least one object detected; a sensor measurement data retriever (112) configured to provide at least one sensor measurement data set for the at least one track associated with the at least one object detected; and a difference parameter determiner (120) configured to determine at least one difference parameter that defines a relationship between the at least one sensor measurement data set and the at least one track data set of the at least one track associated with the at least one object detected; wherein the difference parameter determiner (120) comprises an exponential function module (122) configured to perform calculations using an exponential function, and the difference parameter determiner (120) is configured to use the exponential function module (122) to calculate the at least one difference parameter using the exponential function.

Description

OBJECT TRACKING DATA ASSOCIATION MODULE AND METHOD
FIELD OF THE INVENTION
The invention relates to an object tracking data association module and a corresponding object tracking data association method.
BACKGROUND
Object detection and object tracking may have several applications in various fields, such as, automobile driving, more specifically, on-road motor vehicles. Object detection and object tracking are complementary. Whilst object detection may involve the detection of an object in an image, it would not provide any information about the motion history of the object. Thus, object tracking may complement object detection by taking into account a motion history of an object. Hence, object tracking may be useful for tracking an object for a period of time, for instance, in a video. However, object tracking may be complicated if multiple objects were to be tracked, because a currently detected object may have to be associated with a previously detected object, for instance, because the previously detected object may have moved to a new location, a previously detected object may no longer be detected by a sensor, for instance, because it has left a camera's view, or a newly detected object may currently be detected by a sensor, for instance, because it has entered the camera's view.
SUMMARY
An objective is to provide an object tracking data association module that is effective in performing data association for object tracking or an effective object tracking data association method According to a first aspect of the invention, there is provided an object tracking data association module comprising: a track data retriever configured to provide at least one track data set of at least one track associated with at least one object detected; a sensor measurement data retriever configured to provide at least one sensor measurement data set for the at least one track associated with the at least one object detected; and a difference parameter determiner configured to determine at least one difference parameter that defines a relationship between the at least one sensor measurement data set and the at least one track data set of the at least one track associated with the at least one object detected; wherein the difference parameter determiner comprises an exponential function module configured to perform calculations using an exponential function, and die difference parameter determiner is configured to use the exponential function module to calculate the at least one difference parameter using the exponential function.
The object tracking data association module comprises the difference parameter determiner that is configured to determine the at least one difference parameter using the exponential function, which, advantageously, would allow data association to be performed effectively.
Optionally, the difference parameter determiner comprises a correlation parameter determiner configured to determine at least one correlation parameter that defines a correlation between the at least one sensor measurement data set and the at least one track data set.
The correlation parameter determiner is configured to determine the at least one correlation parameter that defines a correlation between the at least one sensor measurement data set and the at least one track data set, which, advantageously, would allow a correlation between the at least one sensor measurement data set and the at least one track data set to be taken into account.
Optionally, the difference parameter determiner comprises a statistical distance calculator configured to statistically calculate the at least one correlation parameter.
The statistical distance calculator is configured to statistically calculate the at least one correlation parameter, which, advantageously, would allow a correlation between the at least one sensor measurement data set and the at least one track data set to be reliably and appropriately analysed.
Optionally, the exponential function comprises an exponent comprising the at least one correlation parameter.
The difference parameter determiner is configured to determine the at least one difference parameter using the exponential function that comprises the exponent comprising the at least one correlation parameter, which, advantageously, would allow data association to be performed effectively.
Optionally, the exponent comprises a negative multiple of the at least one correlation parameter.
The difference parameter determiner is configured to determine the at least one difference parameter using the exponential function that comprises the exponent comprising the negative multiple of the at least one correlation parameter, which, advantageously, would allow data association to be performed effectively.
Optionally, the difference parameter determiner comprises a Mahalanobis Distance calculator configured to calculate the at least one correlation parameter using a Mahalanobis Distance formula, wherein the at least one correlation parameter comprises a Mahalanobis Distance between the at least one sensor measurement data set and the at least one track data set The correlation parameter determiner comprises the Mahalanobis Distance calculator that is configured to calculate the at least one correlation parameter using the Mahalanobis Distance formula, which, advantageously, would allow a correlation between the at least one sensor measurement data set and the at least one track data set to be reliably and appropriately analysed.
Optionally, the exponential function comprises an exponent comprising the Mahalanobis Distance between the at least one sensor measurement data set and the at least one track data set.
The difference parameter determiner is configured to determine the at least one difference parameter using the exponential function that comprises the exponent comprising the Mahalanobis Distance between the at least one sensor measurement data set and the at least one track data set which, advantageously, would allow a correlation between the at least one sensor measurement data set and the at least one track data set to be reliably and appropriately analysed.
Optionally, the exponent comprises a negative multiple of the Mahalanobis Distance between the at least one sensor measurement data set and the at least one track data set.
The difference parameter determiner is configured to determine the at least one difference parameter using the exponential function that comprises the exponent comprising the negative multiple of the Mahalanobis Distance between the at least one sensor measurement data set and the at least one track data set, which, advantageously, would allow a correlation between the at least one sensor measurement data set and the at least one track data set to be reliably and appropriately analysed.
Optionally, the difference parameter determiner is configured to calculate the at least one difference parameter by subtracting a value of the exponential function from one The difference parameter determiner is configured to determine the at least one difference parameter by subtracting a value of the exponential function from one, which, advantageously, would allow data association to be performed effectively.
The object tracking data association module may further comprise an existence probability detemfiner configured to accept the at least one difference parameter as input and to output at least one existence probability parameter for the at least one track associated with the at least one object detected.
The existence probability determiner is configured to output the at least one existence probability parameter, which, advantageously, would allow data association to be performed effectively.
Optionally, the existence probability determiner comprises a machine learning system configured to accept the at least one difference parameter as input and to output the at least one existence probability parameter for the at least one track associated with the at least one object detected.
The machine learning systcm is configured to output the at least one existence probability parameter, which, advantageously, would allow data association to be performed effectively.
Optionally, the existence probability determiner comprises an LSTM network module configured to accept the at least one difference parameter as input and to output the at least one existence probability parameter for the at least one track associated with the at least one object detected
S
The LSTM network module is configured to output the at least one existence probability parameter, which, advantageously, would allow data association to be performed effectively. Moreover, the LSTM network module is, advantageously, trainable to allow the object tracking data association module, together with an estimator, such as Kalman filter, to perform end-toend object tracking with back propagation Optionally, the track data retriever is configured to provide the at least one track data set comprising an active track data set of an active track associated with a previously detected object.
The track data retriever is configured to provide the at least one track data set comprising the active track data set of the active track, which, advantageously, would allow data related to the previously detected object that is comprised in the active track data set to be taken into account.
Optionally, the track data retriever is configured to provide the at least one track data set comprising an inactive track data set of an inactive track associated with a newly detected object.
The track data retriever is configured to provide the at least one track data set comprising the inactive track data set, which, advantageously, would allow data related to the newly detected object to be taken into account.
The object tracking data association module may further comprise a threshold value comparator configured to compare the at least one existence probability parameter of the at least one track with an existence threshold value.
The object tracking data association module may further comprise a decision maker configured to perform an association for the active track if the at least one existence probability parameter of the active track is above the existence threshold value.
The decision maker is configured to perform an association for the active track if the at least one existence probability parameter of the active track is above the existence threshold value, which, advantageously, would allow a currently detected object to be associated with the active track effectively.
The object tracking data association module may further comprise a decision maker configured to terminate the active track if the at least one existence probability parameter of the active track is below the existence threshold value.
The decision maker is configured to terminate the active track if the at least one existence probability parameter of the active track is below the existence threshold value, which, advantageously, would effectively stop the object tracking data association module from tracking the previously detected object that is no longer detected by a sensor, for instance, because it has left a camera's view.
The object tracking data association module may further comprise a decision maker configured to initialise the inactive track into another active track if the at least one existence probability parameter of the inactive track is above the existence threshold value.
The decision maker is configured to initialise the inactive track into the another active track if the at least one existence probability parameter of the inactive track is above the existence threshold value, which, advantageously, would allow the newly detected object to be tracked effectively.
The object tracking data association module may further comprise a sensor reliability score retriever configured to obtain at least one sensor reliability parameter associated with at least one sensor used to provide the at least one sensor measurement data set for the at least one track associated with the at least one object detected.
The sensor reliability score retriever is configured to obtain at least one sensor reliability parameter, which, advantageously, would allow the at least one sensor reliability parameter to be taken into account.
The object tracking data association module may further comprise a threshold value comparator configured to compare the at least one sensor reliability parameter with a sensor reliability threshold value.
The object tracking data association module may further comprise a decision maker configured to disregard the at least one sensor measurement data set if the at least one sensor reliability parameter associated with the at least one sensor measurement data set is below the sensor rel i abi 1 ity threshold value.
The decision maker is configured to disregard the at least one sensor measurement data set if the at least one sensor reliability parameter associated with the at least one sensor measurement data set is below the sensor reliability threshold value, which, advantageously, would allow unreliable sensor data to be identified and disregarded.
An object tracking module may comprise the object tracking data association module.
A motor vehicle may comprise the object tracking data association module.
Any feature or step disclosed in the context of the first aspect of the invention may also be used, to the extent possible, in combination with and/or in the context of other aspects of the invention, and in the inventions generally. In addition, any feature or step disclosed in the context of any other aspect of the invention may also be used, to the extent possible, in combination with and/or in the context of the first aspect of the invention, mid in the inventions generally, According to a second aspect of the invention, there is provided an object tracking data association module comprising: a track data retriever configured to provide at least one track data set of at least one track associated with at least one object detected; wherein the track data retriever is configured to provide the at least one track data set comprising an active track data set of an active track associated with a previously detected object; wherein the track data retriever is configured to provide the at least one track data set comprising an inactive track data set of an inactive track associated with a newly detected object; a sensor measurement data retriever configured to provide at least one sensor measurement data set for the at least one track associated with the at least one object detected; a difference parameter determiner configured to determine at least one difference parameter that defines a relationship between the at least one sensor measurement data set and the at least one track data set of the at least one track associated with the at least one object detected; wherein the difference parameter determiner comprises an exponential function module configured to perform calculations using
S
an exponential function, and the difference parameter determiner is configured to use the exponential function module to calculate the at least one difference parameter using the exponential function wherein the difference parameter determiner comprises a correlation parameter determiner configured to determine at least one con-elation parameter that defines a correlation between die at least one sensor measurement data set and the at least one track data set; wherein the difference parameter determiner comprises a statistical distance calculator configured to statistically calculate the at least one correlation parameter; wherein the difference parameter determiner comprises a Mahalanobis Distance calculator configured to calculate the at least one correlation parameter using a Mahalanobis Distance formula, wherein the at least one correlation parameter comprises a Mahalanobis Distance between the at least one sensor measurement data set and the at least one track data set; wherein the difference parameter determiner is configured to calculate the at least one difference parameter by subtracting a value of the exponential function from one; wherein the exponential function comprises an exponent comprising the Mahalanobis Distance between the at least one sensor measurement data set and the at least one track data set; wherein the exponent comprises a negative multiple of the Mahalanobis Distance between the at least one sensor measurement data set and the at least one track data set: an existence probability determiner configured to accept the at least one difference parameter as input and to output at least one existence probability parameter for the at least one track associated with the at least one object detected; wherein the existence probability determiner comprises a machine learning system configured to accept the at least one difference parameter as input and to output the at least one existence probability parameter for the at least one track associated with the at least one object detected; wherein the existence probability determiner comprises an LSTM network module configured to accept die at least one difference parameter as input and to output die at least one existence probability parameter for the at least one track associated with the at least one object detected; a threshold value comparator configured to compare the at least one existence probability parameter of the at least one track with an existence threshold value; a decision maker configured to perform an association for the active track if the at least one existence probability parameter of the active track is above the existence threshold value; wherein the decision maker is configured to terminate an active track if the at least one existence probability parameter of the active track is below the existence threshold value; wherein the decision maker is configured to initialise the inactive track into an active track if the at least one existence probability parameter of the inactive track is above the existence threshold value; mid a sensor reliability score retriever configured to obtain at least one sensor reliability parameter associated with at least one sensor used to provide the at least one sensor measurement data set for the at least one track associated with the at least one object detected; wherein the threshold value comparator is configured to compare the at least one sensor reliability parameter with a sensor reliability threshold value; wherein the decision maker is configured to disregard the at least one sensor measurement data set if the at least one sensor reliability parameter associated with the at least one sensor measurement data set is below the sensor reliability threshold value.
The object tracking data association module would, advantageously, allow data association to be performed effectively. The correlation parameter determiner is configured to determine the at least one correlation parameter that defines a correlation between the at least one sensor measurement data set and the at least one track data set, which, advantageously, would allow a correlation between the at least one sensor measurement data set and the at least one track data set to be taken into account. The statistical distance calculator is configured to statistically calculate the at least one correlation parameter, which, advantageously, would allow a correlation between the at least one sensor measurement data set and the at least one track data set to be reliably and appropriately analysed. The correlation parameter determiner comprises the Mahalanobis Distance calculator that is configured to calculate the at least one correlation parameter using the Mahalanobis Distance formula, which, advantageously, would allow a correlation between the at least one sensor measurement data set and the at least one track data set to be reliably and appropriately analysed. The LSTM network module is, advantageously, trainable to allow the object tracking data association module, together with an estimator, such as Kalman filter, to perform end-to-end object tracking with back propagation. The track data retriever is configured to provide the at least one track data set comprising the active track data set of the active track, which, advantageously, would allow data related to the previously detected object that is comprised in the active track data set to be taken into account. The track data retriever is configured to provide the at least one track data set comprising the inactive track data set, which, advantageously, would allow data related to the newly detected object to be taken into account. The decision maker is configured to perform an association for the active track if the at least one existence probability parameter of the active track is above the existence threshold value, which, advantageously, would allow a currently detected object to be associated with the active track effectively. The decision maker is configured to terminate the active track if the at least one existence probability parameter of the active track is below the existence threshold value, which, advantageously, would effectively stop the object tracking data association module from tracking the previously detected object that is no longer detected by a sensor, for instance, because it has left a camera's view. The decision maker is configured to initialise the inactive track into the another active track if the at least one existence probability parameter of the inactive track is above the existence threshold value, which, advantageously, would allow the newly detected object to be tracked effectively. The sensor reliability score retriever is configured to obtain at least one sensor reliability parameter, which, advantageously, would allow the at least one sensor reliability parameter to be taken into account. The decision maker is configured to disregard the at least one sensor measurement data set if the at least one sensor reliability parameter associated with the at least one sensor measurement data set is below the sensor reliability threshold value, which, advantageously, would allow unreliable sensor data to be identified and disregarded.
Any feature or step disclosed in the context of the second aspect of the invention may also be used, to the extent possible, in combination with and/or in the context of other aspects of the invention, and in the inventions generally. In addition, any feature or step disclosed in the context of any other aspect of the invention may also be uscd, to the extent possible, in combination with and/or in the context of the second aspect of the invention, and in the inventions generally.
According to a third aspect of the invention, there is provided an object tracking data association method comprising the acts of: providing, by a processor, at least one track data set of at least one track associated with at least one object detected; providing, by the processor, at least one sensor measurement data set for the at least one track associated with the at least one object detected; and determining, by the processor, at least one difference parameter defining a relationship between the at least one sensor measurement data set and the at least one track data set of the at least one track associated with the at least one object detected, using an exponential function.
The object tracking data association method determines the at least one difference parameter using the exponential function, which, advantageously, would allow data association to be performed effectively.
The object tracking data association method may further comprise the act of determining, by the processor, at least one correlation parameter that defines a correlation between the at least one sensor measurement data set and the at least one track data set.
Advantageously, this would allow a correlation between the at least one sensor measurement data set and the at least one track data set to be taken into account.
Optionally, the act of determining the at least one correlation parameter comprises the act of statistically calculating, by the processor, the at least one correlation parameter.
Advantageously, this would allow a correlation between the at least one sensor measurement data set and the at least one track data set to be reliably and appropriately analysed.
Optionally, the act of determining the at least one difference parameter comprises the act of determining, by the processor, the at least one difference parameter using the exponential function comprising an exponent comprising the at least one correlation parameter.
Advantageously, this would allow data association to be performed effectively,.
Optionally, the act of determining the at least one difference parameter comprises the act of detennining, by the processor, the at least one difference parameter using the exponential function comprising the exponent comprising a negative multiple of the at least one correlation parameter.
Advantageously, this would allow data association to be performed effectively.
Optionally, the act of determining the at least one difference parameter comprises the act of calculating, by the processor, the at least one correlation parameter using a Mahalanobis Distance formula, wherein the at least one correlation parameter comprises a Mahalanobis Distance between the at least one sensor measurement data set and the at least one track data set.
Advantageously, this would allow a correlation between the at least one sensor measurement data set and the at least one track data set to be reliably and appropriately analysed.
Optionally, the act of determining the at least one difference parameter comprises the act of determining, by the processor, the at least one difference parameter using the exponential function comprising an exponent comprising the Mahalanobis Distance between the at least one sensor measurement data set and the at least one track data set.
Advantageously, this would allow a con-elation between the at least one sensor measurement data set and the at least one track data set to be reliably and appropriately analysed.
Optionally, the act of determining the at least one difference parameter comprises the act of detemiining, by the processor, the at least one difference parameter using the exponential function comprising the exponent comprising a negative multiple of the Mahalanobis Distance between the at least one sensor measurement data set and the at least one track data set Advantageously, this would allow a correlation between the at least one sensor measurement data set and the at least one track data set to be reliably and appropriately analysed.
Optionally, the act of determining the at least one difference parameter comprises the act of subtracting, by the processor, a value of the exponential function from one Advantageously, this would allow data association to be performed effectively.
The object tracking data association method may further comprise the act of providing, by the processor, the at least one difference parameter as input to an existence probability dctcrmincr.
Optionally, the act of providing the at least one difference parameter as input to the existence probability detenniner comprises the act of providing, by the processor, die at least one difference parameter as input to a machine learning system Optionally, the act of providing the at least one difference parameter as input to the existence probability determiner comprises the act of providing, by the processor, the at least one difference parameter as input to an LSTM network module The object tracking data association method may further comprise the act of determining, by the processor, using the existence probability determiner, at least one existence probability parameter for the at least one track associated with the at least one object detected.
Advantageously, this would allow data association to be performed effectively.
Optionally, the act of determining, using the existence probability determiner, the at least one existence probability parameter comprises the act of determining, by the processor, using the machine learning system, the at least one existence probability parameter for the at least one track associated with the at least one object detected.
Advantageously, this would allow data association to be performed effectively.
Optionally, the act of determining, using the existence probability determiner, the at least one existence probability parameter comprises the act of determining, by the processor, using the LSTM network module, the at least one existence probability parameter for the at least one track associated with the at least one object detected.
Advantageously, this would allow data association to bc performed effectively. Moreover, the LSTM network module is, advantageously, trainable to allow the object tracking data association module, together with an estimator, such as Kalman filter, to perfbmi end-to-end object tracking with back propagation.
Optionally, the act of providing the at least one track data set comprises the act of providing, by the processor, an active track data set of an active track associated with a previously detected object.
Advantageously, this would allow data related to the previously detected object that is comprised in the active track data set to be taken into account.
Optionally, the act of providing the at least one track data set comprises the act of providing, by the processor, the at least one track data set comprising an inactive track data set of an inactive track associated with a newly detected object.
Advantageously, this would allow data related to the newly detected object to be taken into account.
The object tracking data association method may further comprise the act of comparing, by the processor, the at least one existence probability parameter of the at least one track with an existence threshold value.
The object tracking data association method may further comprise the act of performing, by the processor, an association for the active track if the at least one existence probability parameter of the active track is above the existence threshold value.
Advantageously, this would allow a currently detected object to be associated with the active track effectively.
The object tracking data association method may further comprise the act of terminating, by the processor, an active track if the at least one existence probability parameter of the active track is below the existence threshold value Advantageously, this would effectively stop the tracking of the previously detected object that is no longer detected by a sensor, for instance, because it has left a camera's view.
The object tracking data association method may further comprise the act of initialising, by the processor, the inactive track into an active track if the at least one existence probability parameter of the inactive track is above the existence threshold value.
Advantageously, this would allow the newly detected object to be tracked effectively.
The object tracking data association method may -amber comprise the act of obtaining, by the processor, at least one sensor reliability parameter associated with at least one sensor used to provide the at least one sensor measurement data set for the at least one track associated with the at least one object detected.
Advantageously, this would allow the at least one sensor reliability parameter to be taken into account.
The object tracking data association method may further comprise the act of comparing, by the processor, the at least one sensor reliability parameter with a sensor reliability threshold value.
The object tracking data association method may further comprise the act of disregarding, by the processor, the at least one sensor measurement data set if the at least one sensor reliability parameter associated with the at least one sensor measurement data set is below the sensor reliability threshold value Advantageously, this would allow unreliable sensor data to be identified and disregarded.
Any feature or step disclosed in the context of the third aspect of the invention may also be used, to the extent possible, in combination with and/or in the context of other aspects of the invention, and in the inventions generally. In addition, any feature or step disclosed in the context of any other aspect of the invention may also be used, to the extent possible, in combination with and/or in the context of the third aspect of the invention, and in the inventions generally.
According to a fourth aspect of the invention, there is provided a computer-implemented object tracking data asso-ciation method comprising executing on a processor the acts of: providing at least one track data set of at least one track associated with at least one object detected; providing at least one sensor measurement data set for the at least one track associated with the at least one object detected: and de-termining at least one difference parameter defining a relationship between the at least one sensor measurement data set and the at least one track data set of the at least one track associated with the at least one object detected, using an exponential function.
The object tracking data association method determines the at least one difference parameter using the exponential function, which, advantageously, would allow data association to be performed effectively.
Any feature or step disclosed in the context of the fourth aspect of the invention may also be used, to the extent possible, in combination with and/or in the context of other aspects of the invention, and in the inventions generally. in addition, any feature or step disclosed in the context of any other aspect of the invention may also be used, to the extent possible, in combination with and/or in the context of the fourth aspect of the invention, and in the inventions gen-erally.
According to a fifth aspect of the invention, there is provided a non-transitory computer-readable medium with instructions stored thereon, that when executed on a processor, perform an object tracking data association method comprising the acts of: providing at least one track data set of at least one track associated with at least one object detected; providing at least one sensor measurement data set for the at least one track associated with the at least one object detected; and de-termining at least one difference parameter defining a re-lationship between the at least one sensor measurement data set and the at least one track data set of the at least one track associated with the at least one object detected, using an exponential function.
The object tracking data association method determines the at least one difference parameter using the exponential function, which, advantageously, would allow data association to be performed effectively.
Any feature or step disclosed in the context of the fifth aspect of the invention may also be used, to the extent possible, in combination with and/or in the context of other aspects of the invention, and in the inventions generally. in addition, any feature or step disclosed in the context of any other aspect of the invention may also be used, to the extent possible, in combination with and/or in the context of the fifth aspect of the invention, and in the inventions generally, According to a sixth aspect of the invention, there is provided an object tracking data association method comprising the acts of providing at least one thick data set of at least one track associated with at least one object detected; providing at least one sensor measurement data set for the at least one track associated with the at least one object detected; and determining at least one difference parameter defining a re-lationship between the at least one sensor measurement data set and the at least one track data set of the at least one track associated with the at least one object detected, using an exponential function.
The object tracking data association method determines the at least one difference parameter using thc exponential function which, advantageously, would allow data association to be performed effectively.
Any feature or step disclosed in the context of the sixth aspect of the invention may also be used, to the extent possible, in combination with and/or in the context of other aspects of the invention, and in the inventions generally. in addition, any feature or step disclosed in the context of any other aspect of the invention may also be used, to the extent possible, in combination with and/or in the context of the sixth aspect of the invention, and in the inventions generally.
In this summaiy, in the description below, in the claims below, and in the accompanying drawings, reference is made to particular features (including method steps) of the invention. it is to be understood that the disclosure of the invention in this specification includes all possible combinations of such particular features. For example, where a particular feature is disclosed in the context of a particular aspect or embodiment of the invention, or a particular claim, that feature can also be used, to the extent possible, in combination with and/or in the context of other particular aspects and embodiments of the invention, and in the inventions generally.
In this summary, in the description below, in the claims below, and in the accompanying drawings, where reference is made herein to a method comprising two or more defined steps, the defined steps can be carried out in any order or simultaneously (except where the context excludes that possibility), and the method can include one or more other steps which are carried out before any of the defined steps, between two of the defined steps, or after all the defined steps (except where the context excludes that possibility).
As used in this summary, in the description below, in the claims below, and in the accompanying drawings, the term "comprises" and grammatical equivalents thereof are used herein to mean that other components, ingredients, steps, et cetera are optionally present. For example, an article "comprising" (or "which comprises") components A, B, and C can consist of (that is, contain only) components A, B, and C, or can contain not only components A B, and C but also one or more other components.
As used in this summary, in the description below, in the claims below, and in the accompanying drawings, the term "at least" followed by a number is used in to denote the start of a range beginning with that number (which may be a range having an upper limit or no upper limit, depending on the variable being defined). For example, "at least 1" means 1 or more than 1. The term "at most" followed by a number is used herein to denote the end of a range ending with that number (which may be a range having 1 or 0 as its lower limit, or a range having no lower limit, depending on the variable being defined). For example, "at most 4" means 4 or less than 4, and "at most 40%" means 40% or less than 40%. When, in this specification, a range is given as "(a first number) to (a second number)" or "(a first number) -(a second number)", this means a range whose lower limit is the first number and whose upper limit is the second number. For example, 25 to 100 mm means a range whose lower limit is 25 mm, and whose upper limit is 100 mm.
As used in this summary, in the description below, in the claims below, and in the accompanying drawings, the term "volatile memory" means any type of computer memory where the contents of the memory are lost if there is no power to the computer. Random-access memory (RAM) is an example of a type of volatile memory. As used in the summary above, in this description, in the claims below, and in the accompanying drawings, the term "nonvolatile memory" o r the term "non-transitory computer-readable medium" means any type of computer memory where the contents of the memory are retained even if there is no power to the computer. Hard disk and solid-state drive (SSD) are examples of types of nonvolatile memory or non-transitory computer-readable medium.
As used in this summary, in the description below, in the claims below, and in the accompanying drawings, the term "machine learning system" means a computer system that is able to learn without direct programming instructions. A machine learning system applies statistical modelling to detect patterns and to improve performance, based on data input and without direct programming instructions. A machine learning system builds a statistical model through a training or learning process, which involves inputting data to the machine learning system. The four basic categories of learning process are supervised learning using labelled data sets, unsupervised learning using unlabelled data sets, semi-supervised learning using a mix of labelled data sets and unlabelled data sets, and reinforcement learning that involves learning by trial and error. Decision tree, support vector machine and neural network are examples of types of machine learning system.
As used in this summary, in the description below, in the claims below, and in the accompanying drawings, the term "neural network" or the term "artificial neural network" means a type of machine learning algorithm that uses a web of nodes, edges and layers. The first layer of a neural network comprises input nodes that accept data inputs from a data set.
The input nodes then send information through the edges to the nodes in the next laver. Each edge comprises an activation function that is alterable during a training process. The final layer of the neural network comprises the output nodes that provide data outputs of the neural network. During the training process, the data outputs of the neural network are compared to the actual outputs of the data set. The differences between the data outputs of the neural network and the actual outputs of the data set are measured and denoted as an error value. The error value is then fed back to the neural network, which changes its activation functions in order to minimise die error value. The training process is an iterative process. After the neural network has been trained, the trained neural network may then be used to predict a data output from a particular data input. LSTM network is an example of a type of artificial neural network.
As used in this summary, in the description below, in the claims below, and in the accompanying drawings, the term "object detected' or the term "detected object" means an object that has been detected, for instance, by a sensor, such as, an ultrasonic sensor, object and range detecting sensors or a camera, at a particular moment in time, for instance, in a particular frame of a video. Hence, a currently detected object would refer to an object that has been detected in the particular moment in time, for instance, in the particular frame of the video. Thus, a previously detected object would refer to an object that has been detected prior to the particular moment in time, for instance, in a previous frame of the video. Therefore, a newly detected object would refer to an object that has just been detected in the particular moment in time and not prior to the particular moment in time for instance, because the newly detected object has just entered a camera's view.
As used in this summary, in the description below, in the claims below, and in the accompanying drawings, the term "track" means a placeholder comprised in an object tracking data association module that may be used to link to a detected objcct. Hence, data associated with a detected object may be Finked to a track. Thus, not only data linked to a detected object that are currently obtained in a particular moment in time may be linked to a track, but previously obtained data linked to the detected object or predicted data linked to the detected object may also be linked to the track. An active track would be associated with a previously detected object and an inactive track would be associated with either no detected object or a newly detected object.
As used in this summary, in the description below, in the claims below, and in the accompanying drawings, the term "processor" means a computer component that is configured to perform calculations, make decisions, execute instructions, process data or control other computer components. Central processing unit (CPU) and graphics processing unit (GPU) are examples of types of processor.
BRIEF DESCRIPTION OF THE DRAWINGS
These and other features, aspects, and advantages will become better understood with regard to the following description, appended claims, and accompanying drawings where Figure I shows an object tracking module operationally connected to a sensor module; Figure 2 shows an object tracking data association module; Figure 3 shows a difference parameter table comprising values of difference parameters; Figure 4 shows an existence probability table comprising values of existence probability parameters; Figure 5 shows a motor vehicle comprising the object tracking data association module of Figure 2; and Figure 6 shows a diagram for an object tracking data association method.
In the drawings, like parts are denoted by like reference numerals.
DESCRIPTION
In the summary above, in this description, in the claims below, and in the accompanying drawings, reference is made to particular features (including method steps) of the invention. It is to be understood that the disclosure of the invention in this specification includes all possible combinations of such particular features. For example, where a particular feature is disclosed in the context of a particular aspect or embodiment of the invention, or a particular claim, that feature can also be used, to the extent possible, in combination with and/or in the context of other particular aspects and embodiments of the invention, and in the inventions generally.
Figure I shows an object tracking module 102 operationally connected to a sensor module 104. The object tracking module 102 may comprise an object tracking data association module 100 and an estimator, such as, a Kalman filter 106, operationally connected in a feedback loop. The sensor module 104 may comprise at least one sensor, such as, an ultrasonic sensor, object and range detecting sensors or a camera. The sensor module 104 is configured to obtain sensor measurement data and to provide the sensor measurement data to the object tracking module 102.
Figure 2 shows the object tracking data association module 100 comprising a track data retriever 110, a sensor measurement data retriever 112, a sensor reliability score retriever 114, a difference parameter determiner 120, an existence probability determiner 130, a threshold value comparator 140 and a decision maker 142.
The sensor measurement data retriever 112 is configured to obtain sensor measurement data associated with at least one object detected from the sensor module 104. Sensor measurement data associated with each object detected at a particular moment in time, for instance, in the particular frame of the video, may be comprised in a respective sensor measurement data set. A sensor measurement data set may comprise at least one of position data, velocity data or acceleration data. A sensor measurement data set would also be associated with a track. Hence, each track associated with a respective detected object may comprise more than one measurement data set because more than one sensor may have detected the same object. Therefore, the sensor measurement data retriever 112 is configured to provide at least one sensor measurement data set for at least one track associated with at least one object detected.
The track data retriever 110 is configured to provide at least one track data set of at least one track associated with at least one object detected. Each track data set may comprise at least one of position data, velocity data or acceleration data associated with a respective object detected. A track data set may either comprise an active track data set associated with an active track or an inactive track data set associated with an inactive track. The track data retriever 110 is configured to provide at least one track data set comprising an active track data set of an active track associated with a previously detected object. The track data retriever 110 is also configured to provide at least one track data set comprising an inactive track data set of an inactive track associated with a newly detected object.
The sensor reliability score retriever 114 is configured to obtain a respective sensor reliability parameter associated with each sensor used to provide sensor measurement data. A sensor may have a sensor reliability parameter, for instance, that is provided by its manufacturer or obtained from tests performed on the sensor to determine its detection reliability. Hence, the sensor reliability score retriever 114 is configured to obtain at least one sensor reliability parameter associated with at least one sensor used to provide at least one sensor measurement data set for at least one track associated with at least one object detected.
The difference parameter determiner 120 comprises an exponential function module 122 and a correlation parameter determiner 124 that comprises a statistical distance calculator 126 comprising a Mahalanobis Distance calculator 128.
The difference parameter determiner 120 is configured to determine at least one difference parameter that defines a relationship between at least one sensor measurement data set arid at least one track data set of at least one track associated with at least one object detected.
A difference parameter defines a relationship between each sensor measurement data set and a respective track data set of a corresponding track associated with an object detected. A difference parameter may be calculated using a difference parameter equation. A difference parameter equation may comprise an exponential function comprising an exponent that may comprise a correlation parameter that defines a correlation between each sensor measurement data set and a respective track data set. A difference parameter equation may also comprise an exponential function comprising an exponent that may comprise a negative multiple of a correlation parameter. A correlation parameter may be statistically calculated. A correlation parameter may be calculated using a Mahalanobis Distance formula, such that the correlation parameter comprises a Mahalanobis Distance between each sensor measurement data set and a respective track data set A difference parameter equation may comprise an exponential function comprising an exponent that may comprise a Mahalanobis Distance between each sensor measurement data set and a respective track data set A difference parameter equation may also comprise an exponential function comprising an exponent that may comprise a negative multiple of a Mahalanobis Distance between each sensor measurement data set and a respective track data set. A difference parameter may be calculated by subtracting a value of the exponential function from one.
The exponential function module 122 is configured to perform calculations using exponential functions. The difference parameter determiner 120 is configured to use the exponential function module 122 to calculate a difference parameter using an exponential function. The correlation parameter determiner 124 is configured to determine a correlation parameter that defines a correlation between each sensor measurement data set and a respective track data set. The statistical distance calculator 126 is configured to statistically calculate a correlation parameter. The Mahalanobis Distance calculator 128 is configured to calculate a correlation parameter using a Mahalanobis Distance formula, wherein each correlation parameter comprises a Mahalanobis Distance between a respective sensor measurement data set and a corresponding track data set.
A correlation parameter. CP, may be calculated using a Mahalanobis Distance formula in the following manner: CP = -OT S-1(711 -wherer-Ti is a measurement data vector comprising data from a sensor measurement data set; is a track data vcctor comprising data from a track data set; and S is a covariance matrix of ?Tr and E. CP calculated using a Mahalanobis Distance formula would be a positive number.
A difference parameter equation may be as follows: DP = 1-a-CP where a is a positive number, for instance, the Euler's number; and CP is a correlation parameter.
-CP is an example of an exponent suitable for calculating a difference parameter, and thus e{CP}is an example of an exponential fimction suitable for calculating a difference parameter. In addition, CP may be statistically calculated using a Mahalanobis Distance formula Since CP is a positive number, -CP would be a negative number.
If there were only one sensor and only one object detected, then there would only be one correlation parameter and correspondingly only one difference parameter. If there were more than one sensor and/or more than one object detected, then there would be more than one correlation parameter and correspondingly more than one difference parameter. In that case, values of the difference parameters may be provided in a difference parameter matrix or a difference parameter table 150, as shown in Figure 3.
A difference parameter table 150 comprises values of difference parameters. A difference parameter table 150 may comprise as many rows and as many columns as desired and practical.
The first row from the top of a difference parameter table 150 comprises track existence values of tracks. Each track comprises a respective track existence value. If a track existence value of a track is above a track existence threshold value, the track would be an active track, if a track existence value of a track is below a track existence threshold value, the track would be an inactive track. The first column from the left of a difference parameter table 150 comprises values of sensor reliability parameters associated with sensors used to provide sensor measurement data of tracks associated with objects detected. Each row, apart from the first row from the top, of a difference parameter table 150 represents one measurement attempt by a sensor at a particular moment in time. Further, a sensor may make more than one measurement attempt at a particular moment in time. Moreover, more than one object may be detected in a measurement attempt. Hence, the first cell from the left of each row would comprise a value of a sensor reliability parameter of a sensor mating a measurement attempt represented by that row. Therefore, the top left cell of a difference parameter table 150 would be empty, whilst all other cells, apart from cells in the first row from the top and the first column from the left, would comprise values of difference parameters.
The existence probability determiner 130 may comprise a machine learning system 132 that may comprise an LSTM network module 134.
The existence probability determiner 130 is configured to accept at least one difference parameter as input and to output at least one existence probability parameter for at least one track associated with at least one object detected. The existence probability determiner 130 is configured to output the same number of existence probability parameters as the number of difference parameters provided as input to the existence probability determiner 130. Hence, if there were more than one difference parameter for each track associated with a respective object detected, the existence probability determiner 130 would accept the more than one difference parameter as input and output a corresponding number of existence probability parameters.
The machine learning system 132 is configured to accept at least one difference parameter as input and to output at least one existence probability parameter for at least one track associated with at least one object detected. The machine learning system 132 is configured to output the same number of existence probability parameters as the number of difference parameters provided as input to the machine learning system 132. Hence, if there were more than one difference parameter for each track associated with a respective object detected, the machine learning system 132 would accept the more than one difference parameter as input and output a corresponding number of existence probability parameters.
The LSTM network module 134 is configured to accept at least one difference parameter as input and to output at least one existence probability parameter for at least one track associated with at least one object detected. The LSTM network module 134 is configured to output the same number of existence probability parameters as the number of difference parameters provided as input to the LSTM network module 134. Hence, if there were more than one difference parameter for each track associated with a respective object detected, the LSTM network module 134 would accept the more than one difference parameter as input and output a corresponding number of existence probability parameters. The LSTM network module is, advantageously, trainable to allow the object tracking data association module, together with the estimator, such as the Kalman filter 106, to perform end-to-end object tracking with back propagation.
Values in a difference parameter table 150 may be provided to the LSTM network module 134, and then, values of existence probability parameters obtained may be provided in an existence probability matrix or an existence probability table 152, as shown in Figure 4.
An existence probability table 152 comprises values of existence probability parameters. An existence probability table 152 may comprise as many rows and as many columns as a corresponding difference parameter table 150. The first row from the top of an existence probability table 152 comprises track existence values of tracks. Each track comprises a respective track existence value. If a track existence value of a track is above a track existence threshold value, the track would be an active track, lf a track existence value of a track is below a track existence threshold value, the track would be an inactive track. The first column front the left of an existence probability table 152 comprises values of sensor reliability parameters associated with sensors used to provide sensor measurement data of tracks associated with objects detected. Each row, apart front the first row from the top, of an existence probability table 152 represents one measurement attempt by a sensor at a particular moment in time. Further, the top left cell of an existence probability table 152 would be empty, whilst all other cells, apart front cells in the first row from the top and the first column from the left, would comprise values of existence probability parameters.
The threshold value comparator 140 is configured to compare at least one existence probability parameter of a track with an existence threshold value. The existence threshold value may be any suitable value, such as, 0.5. The threshold value comparator 140 is also configured to compare a sensor reliability parameter with a sensor reliability threshold value. The sensor reliability threshold value may be any suitable value, such as, 0.5. The threshold value comparator 140 may also be configured to compare a difference parameter with a difference parameter threshold value. The difference parameter threshold value may be any suitable value, such as, 0.5.
The decision maker 142 is configured to perform an association for an active track if an existence probability parameter of the active track is above the existence threshold value (for instance, a track associated with column 3 and a track associated with column 6 ofthe existence probability table 152 of Figure 4). The decision maker 142 is also configured to terminate an active track if an existence probability parameter of the active track is below an existence threshold value. The decision maker 142 is may also be configured to terminate an active track if all the existence probability parameters of the active track is below an existence threshold value (for instance, a track associated with column 7 of the existence probability table 152 of Figure 4). Further, the decision maker 142 is configured to initialise an inactive track into an active track if at least one existence probability parameter of the inactive track is above an existence threshold value (for instance, a track associated with column 2 of thc existence probability table 152 of Figure 4), Figure 5 shows a motor vehicle 108 comprising the object tracking data association module 100.
Figure 6 shows a diagram for an object tracking data association method 200 using the object tracking data association module 100. The acts or steps of the object tracking data association method 200 may be performed by a processor, such as a central processing unit of a computer.
At step 202, the object tracking data association method 200 initialises. At step 204, the sensor module 104 detects at least one object and obtains at least one sensor measurement data set for each object detected. At step 206, the at least one sensor measurement data set for each track associated with a respective object detected is provided. At step 208, at least one track data set is provided, wherein each track data set of a respective track is associated with a corresponding detected object. The at least one track data set may comprise at least one active track data set, wherein each active track data set of a respective active track is associated with a corresponding previously detected object. The at least one track data set may comprise at least one inactive track data set, wherein each inactive track data set of a respective inactive track is associated with a corresponding newly detected object.
At step 210, at least one difference parameter is determined, wherein each difference parameter defines a relationship between a respective sensor measurement data set and a corresponding track data set The process of step 210 starts at step 212. At step 214, at least one correlation parameter is determined, wherein each correlation parameter defines a correlation between a respective sensor measurement data set and a corresponding track data set. At step 216, the process of step 214 starts. At step 218, each correlation parameter is calculated using a Mahalanobis Distance formula, such that each correlation parameter comprises a Mahalanobis Distance between a respective sensor measurement data set and a corresponding track data set. The process of step 214 ends at step 220 Then, at step 222, a respective value of an exponential function comprising an exponent that comprises a negative multiple of each correlation parameter is calculated. At step 224, each difference parameter is calculated by subtracting a respective value of the exponential function from one. The process of step 210 ends at step 226 At step 228, at least one existence probability parameter is determined. At step 230, the process of step 228 starts. At step 232, the at least one difference parameter is provided to the machine learning system 132, such as, the LTSM network module 134. The machine learning system 132 or the LTSM network module 134 would have been trained to accept the at least one difference parameter as input and to output a respective existence probability parameter for each difference parameter provided. Hence, at step 234, the machine learning system 132 or the LTSM network module 134 outputs the at least one existence probability parameter. The process of step 228 ends at step 236.
At stcp 238, at least one sensor reliability parameter associated with at least one sensor used to provide the at least one sensor measurement data set is obtained. At step 240, the at least one sensor reliability parameter is compared with a sensor reliability threshold value. At step 242, the at least one existence probability parameter is compared with an existence threshold value.
At step 244, an association is petfonned for an active track if an existence probability parameter of the active track is above the existence threshold value. Hence, assuming the existence threshold value is 0.5, an existence probability parameter at row 3, column 3 of the existence probability table of Figure 4 would be linked to a track (for instance, track 2) associated with column 3 of the existence probability table of Figure 4, because a value of the existence probability parameter at row 3, column 3 is above 0.5.
At step 246, an active track is terminated if an existence probability parameter of the active track is below the existence threshold value. Hence, assuming the existence threshold value is 0.5, a track (for instance, track 6) associated with column 7 of the existence probability table of Figure 4 would be terminated, because none of the existence probability parameters of the track (for instance, track 6) has a value above 0.5.
At step 248, an inactive track is initialised into an active track if an existence probability parameter of the inactive track is above the existence threshold value. Hence, assuming the existence threshold value is 0.5, a track (for instance, track 1) associated with column 2 of the existence probability table of Figure 4 would be initialised, because a value of the existence probability parameter at row 2, column 2 is above 0.5.
At step 250, a sensor measurement data set is disregarded if a sensor reliability parameter associated with the sensor measurement data set is below the sensor reliability threshold value. Hence, assuming the sensor reliability threshold value is 0.5, all the existence probability parameters, together with their corresponding sensor measurement data sets, associated with rows 3, 5, 7 and 8 of the existence probability table of Figure 4 would be disregarded, because values of sensor reliability parameters in the first left cells (in column 1) of rows 3. 5. 7 and 8 are below 0.5.
At step 252, existence probability parameters of active tracks that have not been terminated and initialised tracks are provided to the estimator, such as, the Kalman filter 106. At step 254, the estimator, such as, the Kalman filter 106, would then update a respective track existence value for each active track that has not been terminated and each initialised track.
Finally, at step 256, the object tracking data association method 200 ends. However, it is to be understood that the object tracking data association method 200 illustrated in a sequential plurality of steps in Figure 6 merely illustrates the object tracking data association method 200 as applied to a detection in a particular moment in time for instance, in a particular frame of a video. Hence, step 256 may loop back to step 202 to perform the object tracking data association method 200 again in another detection in another moment in time after the particular moment in time, for instance, in a next frame of the video.
Thus, in accordance with the object tracking data association method 200 using the object tracking data association module 100, it is now possible to effectively perform data association for object tracking.
The LSTM network module 134 may be trained before it is used in the object tracking data association method 200. In order to produce labelled data sets for training the LSTM network module 134, values of difference parameters of a training set may first be obtained as described above. Then, difference parameters with values falling within a first range, for instance, 0 -0.01, may be considered to be linked to tracks that may potentially be initialised. In addition, difference parameters with values falling within a second range, for instance, 0.01 -0.2, may be considered to be linked to tracks for which association may be performed. Further, difference parameters with values falling within a third range, for instance, 0.2 -1, may be considered to be linked to tracks that may potentially be terminated. Hence, a training set of labelled data sets may be produced for training the LSTM network module 134.
Although the invention has been described in considerable detail with reference to certain embodiments or aspects, other embodiments or aspects are possible.
Therefore, the spirit and scope of the appended claims should not be limited to the description of the embodiments contained herein.
For example, the at least one sensor reliability parameter may be obtained (step 238) and compared (step 240) in earlier steps in the object tracking data association method 200.
All features disclosed in this specification (including the appended claims, abstract, and accompanying drawings) may be replaced by alternative features serving the same, equivalent, or similar purpose, unless expressly stated otherwise. Thus, unless expressly stated otherwise, each feature disclosed is one example only of a generic series of equivalent or similar features.

Claims (25)

  1. PATENT CLAIMS1. An object tracking data association module (100) comprising: a track data retriever (110) configured to provide at least one track data set of at least one track associated with at least one object detected; a sensor measurement data retriever (112) configured to provide at least one sensor measurement data set for the at least one track associated with the at least one object detected; and a difference parameter determiner (120) configured to determine at least one difference parameter that defines a relationship between the at least one sensor measurement data set and the at least one track data set of the at least one track associated with the at least one object detected; wherein the difference parameter determiner (120) comprises an exponential fimction module (122) configured to perform calculations using an exponential function, and the difference parameter determiner (120) is configured to use the exponential function module (122) to calculate the at least one difference parameter using the exponential function.
  2. 2. The object tracking data association module (100) as in claim 1, wherein the difference parameter determiner (120) comprises a correlation parameter determiner (124) configured to determine at least one correlation parameter that defines a correlation between the at least one sensor measurement data set and the at least one track data set.
  3. 3 The object tracking data association module (100) as in claim 2, wherein the difference parameter detenniner (120) comprises a statistical distance calculator (126) configured to statistically calculate the at least one correlation parameter.
  4. 4. The object tracking data association module (100) as in any one of claims 2-3, wherein the exponential function comprises an exponent comprising the at least one correlation parameter.
  5. 5. The object tracking data association module (100) as in claim 4, wherein the exponent comprises a negative multiple of the at least one correlation parameter.
  6. 6. The object tracking data association module (100) as in any one of claims 2-3, wherein the difference parameter determiner (120) comprises a Mahalanobis Distance calculator (128) configured to calculate the at least one correlation parameter using a Mahalanobis Distance formula, wherein the at least one correlation parameter comprises a Mahalanobis Distance between the at least one sensor measurement data set and the at least one track data set.
  7. 7 The object tracking data association module (100) as in claim 6, wherein the exponential function comprises an exponent comprising the Mahalanobis Distance between the at least one sensor measurement data set and the at least one track data set
  8. 8. The object tracking data association module (100) as in claim 7, wherein the exponent comprises a negative multiple of the Mahalanobis Distance between the at least one sensor measurement data set and the at least one track data set.
  9. 9. The object tracking data association module (100) as in any one of the preceding claims, wherein the difference parameter determiner (120) is configured to calculate the at least one difference parameter by subtracting a value of the exponential function from one.
  10. The object tracking data association module (100) as in any one of the preceding claims, further comprising an existence probability determiner (130) configured to accept the at least one difference parameter as input and to output at least one existence probability parameter for the at least one track associated with the at least one object detected.
  11. 11. The object tracking data association module (100) as in claim 10, wherein the existence probability determiner (130) comprises a machine learning system (132) configured to accept the at least one difference parameter as input and to output the at least one existence probability parameter for the at least one track associated with the at least one object detected.
  12. 12. The object tracking data association module (100) as in any one of claims 10-11, wherein the existence probability determiner (130) comprises an LSTM network module (134) configured to accept the at least one difference parameter as input and to output the at least one existence probability parameter for the at least one track associated with the at least one object detected.
  13. 13. The object tracking data association module (100) as in any one of claims 10-12, wherein the track data retriever (110) is configured to provide the at least one track data set comprising an active track data set of an active track associated with a previously detected object.
  14. 14. The object tracking data association module (100) as in any one of claims 10-13, wherein the track data retriever (110) is configured to provide the at least one track data set comprising an inactive track data set of an inactive track associated with a newly detected object.
  15. 15. The object tracking data association module (100) as in any one of claims 13-14, further comprising a threshold value comparator (140) configured to compare the at least one existence probability parameter of the at least one track with an existence threshold value.
  16. 16. The object tracking data association module (100) as in claim 15, further comprising a decision maker (142) configured to perform an association for the active track if the at least one existence probability parameter of the active track is above the existence threshold value.
  17. 17. The object tracking data association module (100) as in claim 15, further comprising a decision maker (142) configured to terminate the active track if the at least one existence probability parameter of the active track is below the existence threshold value.
  18. 18. The object tracking data association module (100) as in claim 15, further comprising a decision maker (142) configured to initialise the inactive track into another active track if the at least one existence probability parameter of the inactive track is above the existence threshold value.
  19. 19. The object tracking data association module (100) as in any one of claims 1-14, further comprising a sensor reliability score retriever (114) configured to obtain at least one sensor reliability parameter associated with at least one sensor used to provide the at least one sensor measurement data set for the at least one track associated with the at least one object detected.
  20. 20. The object tracking data association module (100) as in claim 19, further comprising a threshold value comparator (140) configured to compare the at least one sensor reliability parameter with a sensor reliability threshold value.
  21. 21. The object tracking data association module (100) as in claim 20, further comprising a decision maker (142) configured to disregard the at least one sensor measurement data set if the at least one sensor reliability parameter associated with the at least one sensor measurement data set is below die sensor reliability threshold value.
  22. 22. An object tracking module (102) comprising the object tracking data association module (100) of any one of the preceding claims.
  23. 23 A motor vehicle (108) comprising the object tracking data association module (100) of any one of the preceding claims
  24. 24. An object tracking data association module (100) of any one of the preceding claims comprising: a track data retriever (110) configured to provide at least one track data set of at least one track associated with at least one object detected; wherein the track data retriever (110) is configured to provide the at least one track data set comprising an active track data set of an active track associated with a previously detected object; wherein the track data retriever (110) is configured to provide the at least one track data set comprising an inactive track data set of an inactive track associated with a newly detected object; a sensor measurement data retriever (112) configured to provide at least one sensor measurement data set for the at least one track associated with the at least one object detected; a difference parameter determiner (120) configured to determine at least one difference parameter that defines a relationship between the at least one sensor measurement data set and the at least one track data set of the at least one track associated with the at least one object detected; wherein the difference parameter determiner (120) comprises an exponential function module (122) configured to perform calculations using an exponential function, and the difference parameter determiner (120) is configured to use the exponential function module (122) to calculate the at least one difference parameter using the exponential fiinction; wherein the difference parameter determiner (120) comprises a correlation parameter detenniner (124) configured to determine at least one correlation parameter that defines a correlation between the at least one sensor measurement data set and the at least one track data set; wherein the difference parameter determiner (120) comprises a statistical distance calculator (126) configured to statistically calculate the at least one correlation parameter; wherein the difference parameter determiner (120) comprises a Mahalanobis Distance calculator (128) configured to calculate the at least one correlation parameter using a Mahalanobis Distance formula, wherein the at least one correlation parameter comprises a Mahalanobis Distance between the at least one sensor measurement data set and the at least one track data set; wherein the difference parameter determiner (120) is configured to calculate the at least one difference parameter by subtracting a value of the exponential function from one; wherein the exponential function comprises an exponent comprising the Mahalanobis Distance between the at least one sensor measurement data set and the at least one track data set; wherein the exponent comprises a negative multiple of the Mahalanobis Distance between the at least one sensor measurement data set and the at least one track data set; an existence probability determiner (130) configured to accept the at least one difference parameter as input and to output at least one existence probability parameter for the at least one track associated with the at least one object detected; wherein the existence probability determiner (130) comprises a machine learning system (132) configured to accept the at least one difference parameter as input and to output the at least one existence probability parameter for the at least one track associated with the at least one object detected; wherein the existence probability determiner (130) comprises an LSTM network module (134) configured to accept the at least one difference parameter as input and to output the at least one existence probability parameter for the at least one track associated with the at least one object detected; a threshold value comparator (140) configured to compare the at least one existence probability parameter of die at least one track with an existence threshold value; a decision maker (142) configured to perform an association for the active track if the at least one existence probability parameter of the active track is above the existence threshold value; wherein the decision maker (142) is configured to terminate an active track if the at least one existence probability parameter of the active track is below the existence threshold value; wherein the decision maker (142) is configured to initialise the inactive track into an active track if the at least one existence probability parameter of the inactive track is above the existence threshold value; and a sensor reliability score retriever (114) configured to obtain at least one sensor reliability parameter associated with at least one sensor used to provide die at least one sensor measurement data set for the at least one track associated with the at least one object detected; wherein the threshold value comparator (140) is configured to compare the at least one sensor reliability parameter with a sensor reliability threshold value; wherein the decision maker (142) is configured to disregard the at least one sensor measurement data set if the at least one sensor reliability parameter associated with the at least one sensor measurement data set is below the sensor reliability threshold value.
  25. 25. An object tracking data association method comprising the acts of providing, by a processor, at least one track data set of at least one track associated with at least one object detected; providing, by a processor, at least one sensor measurement data set for the at least one track associated with the at least one object detected; and determining, by a processor, at least one difference parameter defining a relationship between the at least one sensor measurement data set and the at least one track data set of the at least one track associated with the at least one object detected, using an exponential function.
GB2109395.0A 2021-04-09 2021-06-30 Object tracking data association module and method Withdrawn GB2608400A (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
GB2109395.0A GB2608400A (en) 2021-06-30 2021-06-30 Object tracking data association module and method
DE112022002063.6T DE112022002063T5 (en) 2021-04-09 2022-03-30 MODULE AND METHOD FOR ASSOCIATION OF OBJECT TRACKING DATA
PCT/EP2022/058404 WO2022214375A1 (en) 2021-04-09 2022-03-30 Object tracking data association module and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
GB2109395.0A GB2608400A (en) 2021-06-30 2021-06-30 Object tracking data association module and method

Publications (2)

Publication Number Publication Date
GB202109395D0 GB202109395D0 (en) 2021-08-11
GB2608400A true GB2608400A (en) 2023-01-04

Family

ID=77179421

Family Applications (1)

Application Number Title Priority Date Filing Date
GB2109395.0A Withdrawn GB2608400A (en) 2021-04-09 2021-06-30 Object tracking data association module and method

Country Status (1)

Country Link
GB (1) GB2608400A (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190204411A1 (en) * 2017-12-29 2019-07-04 Automotive Research & Testing Center Method for analyzing error and existence probability of multi-sensor fusion of obstacle detection
CN111829506A (en) * 2020-06-12 2020-10-27 中国船舶重工集团公司第七二四研究所 Active and passive track motion characteristic correlation method based on sequential processing

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20190204411A1 (en) * 2017-12-29 2019-07-04 Automotive Research & Testing Center Method for analyzing error and existence probability of multi-sensor fusion of obstacle detection
CN111829506A (en) * 2020-06-12 2020-10-27 中国船舶重工集团公司第七二四研究所 Active and passive track motion characteristic correlation method based on sequential processing

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
CORNELL UNIVERSITY LIBRARY, 2020, YOUCHENG SUN ET AL, "Reliability Validation of Learning Enabled Vehicle Tracking" *
IEEE ACCESS, vol 8, 2019, WANG ZHANGJING ET AL, "Multi-Sensor Fusion in Automated Driving: A Survey", pages 2847-2868 *

Also Published As

Publication number Publication date
GB202109395D0 (en) 2021-08-11

Similar Documents

Publication Publication Date Title
Dong et al. A lightweight vehicles detection network model based on YOLOv5
CN108520530B (en) Target tracking method based on long-time and short-time memory network
US20180129934A1 (en) Enhanced siamese trackers
CN114638855A (en) Multi-target tracking method, equipment and medium
Lee et al. Ensemble bayesian decision making with redundant deep perceptual control policies
CN112015842B (en) Automatic driving vehicle risk assessment method and system for bicycle track prediction
CN111079342A (en) TBM tunneling performance prediction method based on online rock mass grade classification
CN107133970B (en) Online multi-target tracking method and device based on motion information
CN108573496A (en) Multi-object tracking method based on LSTM networks and depth enhancing study
WO2016026063A1 (en) A method and a system for facial landmark detection based on multi-task
CN111753300B (en) Method and device for detecting and defending abnormal data for reinforcement learning
US20200311527A1 (en) Residual semi-recurrent neural networks
EP3832550A1 (en) Device and method for training a classifier
WO2022214375A1 (en) Object tracking data association module and method
CN105976397A (en) Target tracking method based on half nonnegative optimization integration learning
Shao et al. Failure detection for motion prediction of autonomous driving: An uncertainty perspective
CN113635896B (en) Driving behavior determination method and related equipment thereof
Lemos et al. Convolutional neural network based object detection for additive manufacturing
Kiruthiga et al. Improved object detection in video surveillance using deep convolutional neural network learning
GB2608400A (en) Object tracking data association module and method
CN112881869A (en) Cable joint partial discharge ultrasonic sequence prediction method
Son et al. A Driving Decision Strategy (DDS) Based on Machine learning for an autonomous vehicle
CN116542715A (en) Advertisement click fraud detection method and system based on online outlier detection
WO2022099526A1 (en) Method for training lane change prediction regression model, and lane change predicton method and apparatus
Mane et al. Head and Tail Localization of C. elegans

Legal Events

Date Code Title Description
WAP Application withdrawn, taken to be withdrawn or refused ** after publication under section 16(1)