DE102010005290A1 - Vehicle controlling method for vehicle operator i.e. driver, involves associating tracked objects based on dissimilarity measure, and utilizing associated objects in collision preparation system to control operation of vehicle - Google Patents

Abstract

The method involves monitoring two input images i.e. camera images (104). Two tracked objects e.g. stationary objects, are monitored within the input images in two tracking cycles, respectively. A dissimilarity measure for comparing the tracked objects is determined, where the dissimilarity measure estimates whether the tracked objects represent a single tracked object proximate to a vehicle. The tracked objects are associated based upon the dissimilarity measure. The associated objects are utilized in a collision preparation system to control operation of the vehicle. An independent claim is also included for a system for operating a collision preparation system in a vehicle.

Description

CROSS-REFERENCE TO RELATED REGISTRATIONS

These Application claims priority to US Provisional Application No. 61 / 147,175, filed on January 26, 2009 and its disclosure herein by reference in its entirety is included.

TECHNICAL AREA

These Disclosure relates generally to object tracking systems, and more particularly Measuring systems associated with object tracking systems that related to vehicle operation.

BACKGROUND

The Statements in this background merely provide background information in terms of of the present disclosure and need not necessarily stand represent the technique.

modern Vehicles can be equipped with different detection devices and systems, a vehicle operator in managing the vehicle operation and support the navigation. Known systems for Ground vehicles can autonomous control systems for controlling vehicle operation. For example, vehicles use cruise control systems at a driving speed of the vehicle at a speed selected by an operator to keep. Furthermore, vehicles use systems of adaptive cruise control for controlling the speed based on both of selected by the operator Speed as well as the presence of and the distance to another vehicle on a projected driving path, for example another vehicle on the current lane of the vehicle in question Vehicle. Exemplary vehicle detection systems implemented by systems Adaptive cruise control systems include radar detection devices and LIDAR detection devices to the position and speed other vehicles on the projected driving path of the vehicle in question Detect vehicle.

Known Vehicle systems include autonomous and semi-autonomous vehicle control systems, the maneuvering skills include. Known autonomous vehicle control systems include and cruise control capabilities. An exemplary autonomous control system with maneuverability is an autonomous lane change system. Known autonomous lane change systems include on-board surveillance systems, which are designed to target vehicles on a current track of the vehicle concerned and target vehicles on lanes adjacent thereto to the vehicle in question and are close to this to detect. Known vehicle detection systems include camera-based systems, radar detection devices and LIDAR detection devices, around relative locations and trajectories of vehicles and other objects to identify. The autonomous lane change system comprises a human-machine interface device, to commands from the vehicle operator and actuators to the automatic Steering of the vehicle in question to receive the vehicle trajectory to change in response to the commands. Operation of the autonomous The lane change system includes a vehicle operator command activation the autonomous lane change system by the man-machine interface device, thereby a desire to change the lane of the vehicle in question, is specified. When the autonomous lane change system is activated, monitors the autonomous lane change system adjacent lanes and operates it is the steering system of the vehicle in question to the relevant Maneuver vehicle to the adjacent lane.

vehicles also include a number of security systems designed are to reduce the impact of a collision. An exact one Predicting an imminent collision event is preparing for a collision event is desired.

SUMMARY

A method of controlling vehicle operation during a dynamic vehicle event comprises monitoring a first input image, monitoring a first tracked object in the first input image in a first tracking cycle, monitoring a second input image, and tracking a second tracked object in the second input image second tracking cycle is monitored and a difference measure is determined that the first tracked object with the second tracked object compares. The difference measure estimates whether the first tracked object and the second tracked object represent a single tracked object in the vicinity of the vehicle. The method further comprises assigning the first tracked object and the second tracked object based on the difference measure and using the associated objects in a collision preparation system to control the operation of the vehicle.

BRIEF DESCRIPTION OF THE DRAWINGS

below become one or more embodiments described by way of example with reference to the accompanying drawings, in which:

1 schematically illustrates an exemplary arrangement of a vehicle equipped with a camera and a radar image acquisition system according to the present disclosure;

2 schematically illustrates an exemplary vehicle using sensors to produce a unified tracking of an object, in accordance with the present disclosure;

3 schematically illustrates an exemplary process for monitoring sensor inputs and generating a tracking list according to an embodiment of the present disclosure;

4 schematically illustrates an example system by which sensor inputs are merged into object traces useful in a collision preparation system in accordance with the present disclosure;

5 schematically illustrates an exemplary merge module according to the present disclosure;

6 an exemplary control scheme for identifying salient image areas and image areas associated with movement in accordance with the present disclosure;

7 schematically illustrates an exemplary bank of Kalman filters that operate to estimate the position and velocity of a group of objects in accordance with the present disclosure; and

8th 3 shows exemplary distance data superimposed on a corresponding image plane in accordance with the present disclosure.

DETAILED DESCRIPTION

Referring now to the drawings, in which the drawings are merely for the purpose of explaining certain example embodiments and not for the purpose of limiting the same, FIG 1 an exemplary arrangement of a camera 11 standing at the front of a vehicle 10 is arranged and on the ground in front of the vehicle 10 directed, according to the disclosure. The camera 11 stands with the control module 5 in communication that contains logic to inputs from the camera 11 to process. The vehicle 10 can also use a radar imaging system 13 equipped, which, if it is present, also with the control module 5 is in communication. Professionals will recognize that the vehicle 10 in addition to using the camera 11 and the radar imaging system 13 , or alternatively, could use a number of methods of identifying road conditions that include GPS information, information from other vehicles associated with the vehicle 10 are in communication, historical data regarding the particular lane, biometric information, such as systems that read the driver's point of view, a radar imaging system, or other similar systems. The disclosure herein can be applied to various device arrangements and is therefore not limited thereby.

The camera 11 is a device that is well known in the art and can translate visual inputs in the form of light, infrared or other electromagnetic radiation (EM radiation) into a data format that can be easily analyzed, e.g. B. a digital pixel image. In one embodiment, the camera uses 11 a charge coupled device (CCD) sensor for generating images indicative of a field of view. Preferably, the camera 11 for continuous imaging, e.g. B. 30 generated images per second, designed. Through the camera 11 generated images can be stored in memory in the camera 11 stored or for storage and / or analysis to the control module 5 be transmitted. Preferably, each is through the camera 11 generated image a two-dimensional Image with known pixel dimensions that includes multiple identifiable pixels. The plurality of identifiable pixels may be stored and analyzed using an array. Each pixel may be represented in the array as a set of bits or multiple sets of bits, the bits corresponding to a color on a predetermined pallet or color chart. Each pixel can be expressed as a function of multiple color intensity values, such as in a red-green-blue (RGB) color model or a cyan-magenta-yellow-key (CMYK) color model. Preferably, each pixel comprises a plurality of sets of bits, each set of bits corresponding to a color intensity and a color intensity value, e.g. For example, a first set of bits corresponds to a value of a red color intensity, a second set of bits corresponds to a value of a green color intensity, and a third set of bits corresponds to a value of a blue color intensity on the RGB color model.

The radar imaging device 13 is a device well-known in the art which can emit a transmitter capable of transmitting radio waves or other EM radiation, receiver means capable of detecting the transmitted waves reflected back from objects in front of the transmitter to the receiver, and means for Transmitting the detected waves into a data format that is capable of analysis and, for example, indicates the distance and angle of the objects from which the waves were reflected. Alternatively, the radar imaging device 13 by a Light Detection And Ranging (LIDAR) system designed to transmit and receive optical energy, replaced or supplemented. The received optical energy may be used to determine the geometric dimensions of an object and / or the geometric proximity to the vehicle 10 to investigate. It should be noted that the radar image detection device 13 is optional and unnecessary to perform many of the methods disclosed herein, wherein the processing of visual images may perform a clear path detection. As used herein, the term "free path" is intended to represent a common and ordinary meaning to one skilled in the art (and is not limited to any specific or specifically adapted meaning) and refers, without limitation, to a path that is free of objects exceeding a threshold, e.g. A path that is free of objects classified in the second and third categories mentioned above.

The control module 5 is in 1 shown and described herein as a discrete element. This illustration is to simplify the description and it should be noted that the control module 5 may take any suitable form that includes various combinations of one or more application specific integrated circuit (s) (ASIC), electronic circuits (s), central processing unit (s) (preferably microprocessor (s)). ) and associated memory (s) (read-only memory, programmable read only memory, random access memory, hard disk, etc.) executing one or more software or firmware programs, combinatorial logic circuits, input / Includes output circuitry (s) and devices, appropriate signal conditioning and buffer circuitry, and other suitable components to provide the described functionality. The control module has a set of control algorithms comprising resident software program instructions and calibrations stored in the memory and executed to provide the desired functions. The algorithms are preferably executed during predetermined loop cycles. The algorithms are implemented, for example, by a central processing unit and serve to monitor inputs of detectors and other networked control modules and to execute control and diagnostic routines for controlling the operation of the actuators. The loop cycles may be performed at regular intervals, for example, every 3.125, 6.25, 12.5, 25, and 100 milliseconds during ongoing machine and vehicle operation. Alternatively, algorithms may be executed in response to the occurrence of an event.

The control module 5 Executes an algorithmic code stored therein to related devices, such as the camera 11 and the radar imaging system 13 to monitor and execute commands or data transfers as indicated by the analysis performed in the processing module. The control module 5 may include algorithms and mechanisms for actuating autonomous driving by means known in the art and not described herein, or the control module 5 can simply provide information to a separate autonomous driving system. The control module 5 is adapted to receive input signals from other systems and the operator as needed, depending on the exact embodiment used in conjunction with the control module.

2 schematically shows the exemplary vehicle 10 using sensors to produce a unified tracking of an object, in accordance with the present disclosure. The exemplary vehicle includes a passenger car intended for use on expressways, noting Let the disclosure described herein be applicable to any vehicle or system that wishes to monitor the position and trajectory of distant vehicles and other objects. The vehicle includes a control system that includes various algorithms and calibrations that are executed at different times. The control system is preferably a subset of an overall vehicle control architecture and provides coordinated vehicle system control. The control system monitors inputs from various sensors, synthesizes corresponding information and inputs, and executes algorithms for controlling various actuators to achieve control objectives, including such parameters as collision avoidance and adaptive cruise control. The vehicle control architecture includes a plurality of distributed processors and devices that include a system controller that provides functionality such as anti-lock braking system, traction control, and vehicle stability.

The exemplary detection system preferably includes object location sensors that include at least two forward distance detection devices 14 and 16 and accompanying subsystems or processors. The object location sensors may be the radar imaging device 13 as described hereinabove, configured to include a near range radar subsystem, a far range radar subsystem, and a forward vision system. The object location detectors may include any distance sensors such as those described hereinabove, the FM CW (Frequency Modulated Continuous Wave) radars, Pulse and Frequency Shift Keying (FSK) radars, and LIDAR and ultrasonic devices based on effects , such as Doppler effect measurements, are based to localize forward-looking objects. The possible object locators include the camera 11 as described hereinabove, uses the methods of digital photography to "see" front objects. Such detection systems are used for detecting and locating objects in automotive applications that can be used with systems that are e.g. As an adaptive cruise control, a collision avoidance, a pre-crash and a side object detection include. The exemplary vehicle system may also include a global positioning system (GPS).

These sensors are preferably inside the vehicle 10 positioned at relatively obstacle-free positions relative to a view in front of the vehicle. It should also be noted that each of these sensors provides an estimate of an actual location or condition of a target object, the estimate comprising an estimated position and a standard deviation. Thus, sensor detection and measurement of object locations and conditions are typically referred to as "estimates." It should also be noted that the characteristics of these sensors are complementary in that some are more reliable in estimating certain parameters than others. Conventional sensors have different operating ranges and angle covers and can estimate various parameters within their operating range. For example, radar sensors may usually estimate the distance, range change and azimuth location of an object, but they are usually not stable in estimating the circumference of a detected object. A vision processor camera is more stable in estimating a shape and an azimuth position of the object, but is less efficient in estimating the distance and range change of the object. Scan-type LIDARs operate efficiently and accurately with respect to estimating range and azimuth position, but typically can not estimate the range change and are therefore not accurate in terms of acquiring / recognizing a new object. Ultrasonic sensors can estimate a distance, but generally can not estimate or calculate the range change and azimuth position. It should also be noted that the performance of each sensor technology is affected by different environmental conditions. Thus, conventional sensors represent parametric variations, with the effective overlap of these sensors creating opportunities for sensor integration.

Each object location sensor and subsystem provides an output that includes a distance R, a time-based range change R_dot, and an angle Θ, preferably with respect to a vehicle's longitudinal axis, that can be written as a measurement vector (o), ie, sensor data. An exemplary near range radar subsystem has a FOV of 160 degrees and a maximum range of thirty meters. An exemplary far-range radar subsystem has a field of view of 17 degrees and a maximum range of 220 meters. An exemplary forward vision subsystem has a 45 degree field of view and a maximum range of fifty (50) meters. For each subsystem, the field of view is preferably about the longitudinal axis of the vehicle 10 aligned. The vehicle is preferably on a coordinate system, referred to as XY coordinate system 20 , aligned with the longitudinal axis of the vehicle 10 determines the X-axis with a locus at a point suitable for the vehicle and signal processing, and wherein the Y-axis is defined by an axis orthogonal to the longitudinal axis of the vehicle 10 is and lies in a horizontal plane, which is thus parallel to the bottom surface.

Of the The disclosure of U.S. Patent No. 7,460,951 entitled SYSTEM AND METHOD OF TARGET TRACKING USING SENSOR FUSION is hereby Reference completely included so that the method and apparatus for Combine tracking data from multiple sensors for more accurate Estimate a location of an object need not be described in detail herein.

On 2 Referring to a vehicle 10 shown that a pair of detection systems 14 and 16 includes. The vehicle 10 can be used as a coordinate system (X, Y) 20 wherein the coordinates are defined with respect to a longitudinal axis of the vehicle and a lateral position, for example, compared to a centerline of the vehicle. The detection systems 14 and 16 may be any of the exemplary systems described above or the systems known in the art and capable of returning tracking of a detected object. The sensors can have their own coordinate system. An exemplary coordinate system (u, V) 21 is in relation to the sensor system 14 shown. An object 30 is shown at three different times. The sensor system 14 , which captures objects within the FOV (A), gives three locations of a tracked object for the object 30 back, which are denoted by r _a1 , r _a2 and r _a3 . The sensor system 16 detecting objects within the FOV (B) similarly gives three locations of a tracked object for the object 30 back, which are denoted by r _b1 , r _b2 and r _b3 . United positions for the object 30 can be based on the information available regarding the object 30 to be discribed. Methods are known in the art for merging multiple sets of data into a unified set. For example, example methods may apply weights or measured deviations in the various data points, and the contribution of each individual point to the merged set may depend on the weights or deviations. 2 Figure 14 shows an exemplary set of merged data points labeled r _f1 , r _f2, and r _f3 .

3 12 schematically illustrates an exemplary process for monitoring sensor inputs and generating a tracking list corresponding to detected objects, in accordance with the present disclosure. The exemplary vehicle 10 generally comprises a control system with an observation module 22 , a Data Assignment and Clustering Module (DAC Module) 24 , which is also a Kalman filter 24A and a track life management module (TLM module) 26 that has a chase list 26A track that includes multiple object pursuits. More specifically, the observation module includes the sensors 14 and 16 , their respective sensor processors and the connection between the sensors, the sensor processors and the DAC module 24 ,

The The above exemplary object tracking system shows a method by which one or more objects can be tracked. It It should be noted, however, that a number of different Sensors that provide information about the environment around the vehicle capture, could be used, and that the revelation are not limited to the particular embodiments described herein should. Further, the data pooling method described hereinabove an exemplary process through which the details of the various Input sensors for a single useful tracking of an object could be united. However, there are numerous data pooling techniques in the art and the disclosure is not intended to be as described herein certain exemplary embodiment be limited.

Here in are methods of controlling vehicle operation using a collision preparation system and a movement of identified Revealed objects. The identified object is analyzed to a likely imminent or imminent one Collision based on object movement relative to the vehicle to identify. The collision preparation system is designed, to a vehicle operator regarding an imminent collision to alert, and is configured to operate the vehicle, for example by controlling automatic braking when the collision is unavoidable is considered to reduce the collision severity.

Vision systems provide an alternative source of sensor input for use in vehicle control systems. It is known in the art that methods for analyzing visual information include pattern recognition, corner detection, vertical edge detection, vertical object detection, and other methods. It should be noted, however, that high resolution visual representations of the field in the vicinity of a vehicle, which are updated at a high rate necessary to perceive the movements in real time, involve a very large amount of information to be analyzed. A real-time analysis of visual information can be extremely expensive. A method is disclosed for uniting an input from a vision system with a unified tracking, generated by methods such as the exemplary tracking merging method described above to direct a visual analysis to a portion of the visual information most likely to present a collision hazard, and wherein the focused analysis is used to indicate a likely imminent collision event , Further described is a method that employs iterative tracking cycles to identify a tracked object, for example, using information determined in one cycle to aid in the detection of information in the next cycle. Further, a method is described for applying a bank of filters to a tracked object in an image and using a model to determine which filter output is the best estimate of the tracked object.

4 schematically shows an exemplary control scheme 100 for a collision preparation system according to the present disclosure. The control scheme is in 4 and described herein as comprising discrete elements. Such a representation is to simplify the description and it will be appreciated that the functions performed by these elements may be combined in one or more devices, e.g. B. realized in software, hardware and / or an application-specific integrated circuit. For example, the control scheme 100 in the control module 5 be realized. The Steuersche ma 100 includes monitoring sensor input data surrounding an environment of the vehicle 10 describe around, with distance sensors 102 , Camera images 104 and in-vehicle sensors 106 are included. Preferably, the control scheme monitors 100 continuously the environment of the vehicle using its distance sensors (such as radars and LIDARe) and cameras.

A data union module 110 enters the sensor data. The data union module 110 analyzes, filters or prioritizes the inputs relative to the reliability of the various inputs, and the prioritized or weighted inputs are summed to provide tracking estimates for objects in the vicinity of the vehicle 10 to create. These object traces then become the collision hazard evaluation module 120 where each track is evaluated for probability of a collision. It should be noted that for the implementation of the methods described herein, no collision needs to take place, but instead based on an evaluation of a dynamic vehicle event (eg, changing the relationship of a detected object to the host vehicle, which indicates an imminent collision ) a probability of a collision can be assessed. For example, this probability of a collision may be evaluated in terms of a threshold probability for a collision, and if a collision is determined to be likely, collision countermeasures may be initiated. The probability of a collision of an identified object is determined by a collision countermeasure module 150 analyzed.

Collision countermeasures can take many forms in technology. For example, at an attempt to slow the vehicle from impact, automatically vehicle braking systems are activated. Similarly The engine will be throttled to the speed of the vehicle and to reduce the kinetic energy of the vehicle before impact. For example, automated steering can be activated, to balance the vehicle before the collision to the impact a weird one Avoid impact. Similar can, if a side impact is projected as imminent, a steering and a forward movement the vehicle are controlled to the impact of the passenger compartment to adapt the vehicle away. Similar can if a rear-end impact projected as imminent will, the throttling of the vehicle will be increased to the difference the speeds of the vehicles at the time of impact to reduce. Furthermore, can other vehicle systems controlled in preparation for an impact become. For example, you can the seat belts of the occupants in preparation for the impact streamlined and / or locked. The headrests of the vehicle can be changed to best fit the neck and neck of the occupants of the vehicle protect. A moment before the projected impact, air-gags can preventively, for example, with a reduced pressure, to be deployed in this way reduce the likelihood that the inmate through the for usually rapid deployment of the airbags is impaired. Before the impact can automatically closed the windows and the doors are locked. The foot pedals can in a withdrawn Position adjusted to the likelihood of interaction with the vehicle operator during to reduce the impact. Through a wireless communication network can be an initial one Emergency call signal transmitted be to rescue workers to alarm, being an inability of the system to communicate when the impact of the operation of the Damaged system, is bypassed. It can be an audible alarm of an imminent Sound a collision, to alert the occupants of the vehicle. There will be a number of collision countermeasures considered and the disclosure is not intended to be herein be limited to certain specific exemplary embodiments described.

In one embodiment, the exemplary control scheme described hereinabove 100 used to control a vehicle operated during a dynamic vehicle event. The method includes monitoring distance sensor information and vision system information. Superimposing the range sensor information on the vision system information produces a modified image. Using edge detection, corner detection, and optical flow detection, prominent image areas in the modified image are identified, excluding the salient image areas corresponding to the floor. The method further comprises, in a first time step, identifying a first object in a vehicle field of view using the salient image areas, and identifying a second object in the vehicle field of view using the salient image areas in a subsequent time step. A position of the first object in the subsequent time step is predicted and matched based on comparing the predicted position of the first object with the identified second object with the second object. Object tracking of the matched objects is determined, and a probability of collision between the matched object and the vehicle is determined based on the object tracking. The vehicle is controlled based on the probability of a collision.

5 schematically shows the exemplary Bildvereinigungsmodul 120 according to the present disclosure. The union module 110 Integrates an input from various capture devices, the cameras 104 and distance sensors 102 and creates a unified tracking of an object in the vicinity of the vehicle 10 , The ranging sensor data includes position data of a list of objects in a two-dimensional vehicle coordinate system. In an exemplary embodiment, the objects are projected or superimposed on the camera data to form a modified image in a module 112 for motion analysis and salient area detection, preferably using predetermined camera calibration parameters that are adjusted according to the two-dimensional vehicle coordinate system. Distance information, such as radar returns, that is matched with corresponding areas of the image may be used to identify locations of potential objects in the image. Furthermore, illumination normalization may be applied to the camera data and the modified image. Normalization is a process that changes the range of pixel intensity values. The purpose of normalization is to bring the image into a range that is more appropriate for a machine process to improve reliability. For example, each pixel value is normalized to a mean of zero and unit variance to improve image contrast, particularly in a low-light environment, or when contrast is poor due to glare.

The module 112 motion analysis and salient area detection identifies image areas for further analysis having a threshold confidence for obtaining objects or parts of objects. The image areas corresponding to previously identified objects are determined using a priori information that includes object information of a previous time step, such as object tracking and position. In one embodiment, the objects on the modified image are projected using the object position, the elapsed time between iterations, and an associated object trace. The salient image areas can be identified using extraction algorithms, e.g. A scale-invariant feature transform (SIFT), an accelerated robust feature (SURF) algorithm, and / or a maximally stable extreme region (MSER) algorithm. In one embodiment, a gradient orientation histogram (HOG of oriented gradients) and eigen-image coefficients may be used.

Other Regions of interest of the modified image can be found under Use of edge, corner detection and detection methods of a salient blob. Under use a detection of an optical flow between successive ones modified images and previously identified by projecting Objects on the modified image as described hereinafter may be additional areas of interest are identified. Using a According to several known methods, an edge detection can be carried out. Using one of several algorithms using a Harris corner detection algorithm may include Corners are identified.

For the above described identification of salient blobs on the modified Image, a Gaussian operator is applied to the modified image. In operation, a modified image I (x, y), where x and y are line and column indexes of the image are through a Gauss kernel like folded the following.

The modified image is convolved by the Gauss kernel with a particular scale t to obtain a scale space representation, such as the following relationship. L (x, y, t) = g (x, y, t) · f (x, y) [2]

To The folding of the modified image becomes a scale normalized Gaussian difference operator based on the scale space representation determined. The scale-normalized Gauss difference operator (DoG operator of Difference of Gaussian operator) can be made using the following relationship be determined.

The salient blobs are located by highlighting points on the modified image that have simultaneous correspondence with local maxima / minima of the Gaussian difference operator with respect to both space and scale. The local maxima / minima of the Gaussian difference operator can be determined using the following relationship. (x ^, y ^, t ^) = arg max min ( x . y . t ) (DoG {I (x, y), t}) [4]

One of several optical flow detection methods may be used to identify image areas having motion between successive modified images. One method involves analyzing the successive modified images I _t-1 (x, y) and I _t (x, y) using a Lucas-Kanade optical flow operator to determine areas of interest corresponding to motion. In addition, 4-neighborhood connected component tracking thresholding and component tracking may be applied to the image of an optical flow to identify areas indicating significant motion. Applying a Lucas-Kanade optical flow operator between successive modified images, in one embodiment, provides an image of an optical flow comprising pixels associated with movement between modified images.

By analyzing pixels for visual characteristics, such as color and / or texture, that correspond to a ground in the area adjacent to the vehicle 10 In addition, regions of interest may be identified in the modified image. The pixel areas that correspond to the floor probably contain no objects and can be neglected correctly. Further, in one embodiment, image areas having a threshold quantity of pixels corresponding to ground may be neglected. Further, disassociation with objects identified by the range sensor data provides information regarding pixel correspondence with the ground. The pixel classifications of pixels in segmented image areas or pieces of the modified image are analyzed. Image areas containing a predetermined number of pixels assigned to a floor classification are less likely to contain an object.

After analyzing the modified image for salient areas and image areas associated with movement and features, the module identifies 112 for motion analysis and salient area detection, image areas associated with a confidence level greater than a predetermined threshold correspondence with an object and outputs of the identified image areas. In one embodiment, weights that indicate a confidence match with an object existence are assigned to certain identified features and applied quantitatively based on a feature slope in the image area. Likewise, weights may be applied to salient areas, image areas associated with a movement, and image areas containing previously identified objects. The weights associated with the image areas may be summed, with the image areas corresponding to a threshold weight being derived from the module 112 for a motion analysis and a salient area detection. In one embodiment, each extracted feature, the image regions associated with a motion, the objects identified in a previous time step, and the non-bottom image regions are merged together. The image areas associated with a threshold of merged pixels are derived from the module 112 for movement analysis and detection of a salient area.

6 shows an exemplary control scheme 600 for identifying salient image areas and image areas associated with movement using methods described hereinabove in accordance with the present disclosure. The control scheme 600 is in 6 and described herein as comprising discrete elements. This description is to simplify the description and it should be noted that the functions performed by these elements may be combined in one or more devices, e.g. B. realized in software, hardware and / or an application-specific integrated circuit. In one embodiment, the control scheme is 600 in the module 112 realized for a movement analysis and a detection of a salient area.

The control scheme 600 inputs and monitors continuously modified images generated using the camera data and range sensor data described hereinabove 602 , Using control techniques known in the art or described herein, the control scheme determines 600 an edge illustration 604 , a corner map using the Harris corner detection algorithm 606 and a Lucas-Kanade image of an optical flow 608 , It should be noted that the steps 604 to 608 Identify patterns in the image that are likely to include objects that are distinguishable from the ground or a lane on which the vehicle can travel. By methods known in the art, including analysis of pixel characteristics and analysis of areas of the image describing factors such as color and line patterns, areas of the image that are likely to encompass the floor may be identified. By comparing this area or areas with the identified areas with patterns describing objects, a determination can be made to define areas or areas of the image that are likely to include ground and indicate no objects to track. The process 600 defines areas that indicate the ground 610 on the basis of the input modified image as described hereinabove. The objects identified in an a priori iteration are projected on the modified image as described hereinabove 612 , The edge map, the corner map, the Lucas-Kanade optical flow map, the non-ground image areas, and the projected objects are merged to identify image areas on the modified image that correspond to an object 614 ,

Back on 5 Referring to the module gives 112 for motion analysis and salient area detection, after identifying image areas on the modified image, the object position (P) and the identified image areas comprising a defined range of features (I) for further analysis to a feature extraction module 114 out. The feature extraction module 114 analyzes the image area by applying known feature identification algorithms to the piece. The feature identification algorithms search the available visual information for characteristic patterns in the image area associated with an object, including features defined by line orientation, line location, color, corner properties, other visual attributes, and learned attributes. In one embodiment, the feature identification algorithms may be applied to sequential images to identify changes corresponding to vehicle movement, wherein the changes that are not associated with ground movement may be identified as a non-free path. The feature extraction module 114 monitors the image areas of the modified image and determines a feature descriptor for each image area. The feature descriptor may be constructed using feature extraction algorithms and transformations, such as a histogram of predetermined distinguishing features, such as vertical edges, symmetry, a gradient orientation histogram (HOG), and using a linear subspace projection, e.g. B. a principal component analysis can be determined. By methods known in the art, such as Haar wavelet, Gabor wavelet and Leung-Malik filter bank, additional distinguishing features can be defined.

The classification module 116 analyzes each image area and assigns a tag (L) to each identified object and associated image areas based on a pre-trained classifier. The classification module 116 takes the extracted features and the previously identified objects (a priori) and the associated data comprising a position, an image area, and a classification flag, which for the current time step in a prediction process module 117 to be updated as an input, and passes it to a classifier to determine if an image area is surrounding an object. The classification determines the tag of each identified object according to any number of classifiers. Exemplary classifications include e.g. A moving object, such as a moving vehicle, a slow moving object, such as a pedestrian, and a stationary object, such as a traffic sign. The classification module 116 uses statistical classifier methods with training classification for correspondence with features or attributes in the image area using a significant number of object and non-object image samples. The correspondence of learned attributes with features in the image area may be described by Ma however, for the most part, it is programmed offline and can be developed experimentally, empirically, predictively, through modeling, or other techniques suitable for accurately training distinguishing attributes. These attributes allow the control scheme to develop a confidence between zero and one that the features identified in the image area correspond to a predetermined classified object. A threshold confidence can be set that defines the confidence required to identify a particular image area as an object.

One Several methods can be used to train classifiers apply to the features extracted from the image areas. A first method analyzes the features using a Training set of pictures. An information from the trained Classifiers are used to identify the characteristic as a particular Specifying or weighting object specifying the particular Classification of the strength depends on comparisons with the trained data. A classification of the Feature, if the feature is the only feature in the image area is, can be applied directly to the image area. A classification An image area with multiple identified features can have many Take shapes that represent the image area covered by the feature defines the image area of a given object corresponds, the strongest indicates, or the image area, by a weighted sum of all Characteristics defined therein include.

One additional Method of applying trained classifiers to those of features extracted from the image areas for identification and classification of an object in the image areas includes using a support vector machine (SVM of support vector machine) and a neural network. As A person skilled in the art will recognize artificial neural networks or neural networks computer systems designed for a complex and adaptable Decision making are generated. While conventional computing means on a sequential processing of data by an algorithm that provides predictable results, it is known that neural Networks data in consecutive layers and parallel paths process within each layer by changing nodes. The neural network is initial trains with data that provides a known set of results. As a result of this training will be weights between the layers and applied under the nodes, with the network automatically adapts to the training data and adjusts the weights to a closer Model for to create the data. For later use, the neural Retain the training adjustments and use them over the To apply the life of the network, or the network can be different Use known methods for learning current data patterns. By training and tuning a neural network, input data, which include several identified characteristics, be analyzed, to identify and classify objects in the image areas.

After determining the tag in the classification module 116 For example, an object in an input image corresponding to P, I and L may be described according to the methods described above. A data mapping module 118 analyzes the current object list with P, I and L, the previous objects (a priori) and the associated data comprising a position, picture area features and a classification flag for the current time step in the prediction process module 117 be updated using an object dynamics model. The data mapping module 118 matches the current identified objects with the previous identified objects and outputs a list of matched object measurement pairs. The previous identified objects are using the prediction process module 117 as described below with the current time step.

einen binären Faktor beschreibt, der die Kennzeichen der beiden Objekte vergleicht. w_P, w_I und w_L sind Gewichte, die experimentell ermittelt und kalibriert werden. Durch Vergleichen der Position der verfolgten Objekte, der Bildmerkmale in der Nähe der verfolgten Objekte und der Kennzeichen der verfolgten Objekte kann geschätzt werden, ob die beiden Objekte das gleiche verfolgte Objekt in der Nähe des Fahrzeugs darstellen.By comparing one object (P, I and L) in a second tracking cycle analyzing a second input image with another object (P, I and L) in a first tracking cycle analyzing a first input image, it can be determined whether the two objects probably relate to the same tracked object near the vehicle. In an example method, the current object list may be matched with the predicted object list using a measure of difference between the descriptors of an object P, I, and L. The difference D (ô _β , x _α ) between the two objects α and β is defined as follows: Do β , x α ) = w P d (P ^ β , P α ) + w I d (I β , I α ) + w L d (L ^ β , L α ) [5] where d (P ^ _β, P _α) the position distance between the objects α and β, which are compared is, _(β, i _α) d is the difference between the two objects, calculated as the distance between two corresponding Merkmalsdeskriptorvektoren, and

describes a binary factor that compares the characteristics of the two objects. w _P , w _I and w _L are weights that are experimentally determined and calibrated. By comparing the position of the tracked objects, the image features in the vicinity of the tracked objects, and the characteristics of the tracked objects, it can be estimated whether the two objects represent the same tracked object in the vicinity of the vehicle.

In operation, an α-th predicted object from the predicted object list (a priori) with x _α = (I _α, P _α, L _α), where I is _α a feature descriptor of the corresponding image patch, P _α is the position of the object and L _{α is} the classification mark (eg vehicle, pedestrian, motorcycle, bicycle, deer). A β-th detected instantaneous object _(β i, p ^ _β, L ^ _β) with _β = ô designated. If the difference D (ô _β , x _α ) is less than a threshold, the observed object ô _{β is} assigned to the predicted object x _α .

An object tracker module 119 monitors the output of the data mapping module 118 to determine updated positions and velocity attributes of the objects using a bank of Kalman filters. 7 schematically shows the tracker module 119 , which is an exemplary bank of Kalman filters 206 1, which are configured to estimate the position and velocity of a group of objects, according to the present disclosure. Different filters are used for objects associated with different dynamic properties, such as objects traveling at constant speed, objects with high-length maneuvers, objects with high-transverse maneuvers, and stationary objects. It becomes a Markov decision process model (MDP model by Markov decision process model) 208 is used to select the Kalman filter with the largest weighted value used for correspondence with exact positions and a speed estimate for the object. Selecting a Kalman filter using the Markov decision process reduces tracking latency, allowing greater collision preparation system functionality. The selected Kalman filter is passed through the update object module 210 to update the object data, which includes position and velocity.

in Bezug auf einen Merkmalsdeskriptor

, eine Position und Geschwindigkeit des Objekts

und ein Klassifizierungskennzeichen

ermittelt. Das ausgewählte i-te Kalman-Filter zeigte eine größte Wahrscheinlichkeit aller i-ten Kalman-Filter, den Objektzustand

genau zu ermitteln. Eine Markov-Übergangsmatrix T kann verwendet werden, um eine Wahrscheinlichkeit zu beschreiben, dass ein anderes Kalman-Filter den Objektzustand

zu einem zukünftigen Zeitschritt t_k+1 genauer ermittelt. Daher stellt T_ij eine Wahrscheinlichkeit dar, dass sich der Objektzustand

von einem i-ten Kalman-Filter zu einem j-ten Kalman-Filter zu einem Zeitschritt t_k+1 ändert.Here is an exemplary Markov decision process with respect to 7 and the tracker module 119 described. The Markov Decision Process 208 is used to filter out a Kalman filter from the bank of Kalman filters 206 to update a position and speed of an object. Using a selected i th Kalman filter of a bank of Kalman filters 206 with an arbitrary number n, an object state becomes at a current time step t _k

with respect to a feature descriptor

, a position and speed of the object

and a classification mark

determined. The selected i-th Kalman filter showed a greatest probability of all i-th Kalman filters, the object state

to determine exactly. A Markov transition matrix T can be used to describe a probability that another Kalman filter will be the object state

to a future time step t _{k + 1} determined more accurately. Therefore, T _{ij represents} a probability that the object state

from an i-th Kalman filter to a j-th Kalman filter at a time step t _{k + 1} changes.

zu einem Zeitpunkt t_k mit einem bestimmten Modell auf der Grundlage von Bodenwirklichkeitsdaten gekennzeichnet (gesammelt durch ein anderes Messverfahren, z. B. GPS-Daten). Dann kann die Übergangswahrscheinlichkeit T_ij durch die folgende Beziehung ermittelt werden:

wobei

die Häufigkeit bezeichnet, dass ein Fahrzeugzustand zu einem Zeitschritt t_k+1 zu einem Modell j gehört, und

die Häufigkeit bezeichnet, dass ein Fahrzeugzustand zu Zeitschritten t_k bzw. t_k+1 zu einem Modell i bzw. einem Modell j gehört.The probabilities are predetermined by a data mining process that uses traffic data collected on a public road, or simply by heuristic rules. With vehicle data of multiple vehicles recorded over a substantially long sequence, each vehicle becomes

at a time t _k with a particular model based on ground reality data (collected by another measurement technique, eg GPS data). Then, the transition probability T _ij can be determined by the following relationship:

in which

the frequency denotes that a vehicle state belongs to a model j at a time step t _{k + 1} , and

the frequency denotes that a vehicle state at time steps t _k or t _{k + 1} belongs to a model i or a model j.

in Bezug auf einen Merkmalsdeskriptor

, eine Position und Geschwindigkeit des Objekts

und ein Klassifizierungskennzeichen

unter Verwendung eines ausgewählten i-ten Kaiman-Filters einer Bank von Kalman-Filtern 206 mit einer beliebi gen Anzahl n ermittelt. Das ausgewählte i-te Kalman-Filter zeigte eine größte Wahrscheinlichkeit aller i-ten Kalman-Filter, um den Objektzustand x_k genau zu ermitteln.Here's in terms of 7 and the tracker module 119 an exemplary Markov decision process described. The Markov Decision Process 208 is used to filter a Kalman filter from the bank of Kalman filters 206 to update a position and speed of an object. At an instantaneous time step t _k becomes an object state

in terms of a feature descriptor gate

, a position and speed of the object

and a classification mark

using a selected i th Kalman filter of a bank of Kalman filters 206 determined with any number n. The selected i-th Kalman filter showed a largest probability of all ith Kalman filters to accurately determine the object state x _k .

zu einem momentanen Zeitschritt k in Bezug auf einen Merkmalsdeskriptor

, eine Position und Geschwindigkeit des Objekts

und ein Klassifizierungskennzeichen

auf der Grundlage des Ausgangs des Datenzuordnungsmoduls 118 und der Fahrzeugkinematikdaten, die Fahrzeuggeschwindigkeit und Gierrate von den Fahrzeugsensoren 204 umfassen, zu aktualisieren. Ferner ist jedes Kalman-Filter ausgestaltet, um eine Positionsmessung

auf der Grundlage von Messungen von den Entfernungserfassungseinrichtungen 14 und 16 und der Kamera 11 zu aktualisieren. Jedes i-te Kalman-Filter der unabhängigen Bank von n Kalman-Filtern 206 kann dargestellt werden mit:

wobei Aⁱ und Bⁱ Systemmatrizen des i-ten Kaiman-Filters sind,

einen Fahrzeugbedienereingang und Fahrzeugkinematikdaten, die z. B. eine Brems- und eine Gaspedalstellung, einen Lenkradwinkel und eine Fahrzeuggierrate umfassen, zu einem Zeitschritt t_k darstellt, νⁱ einen Modellerstellungsfehler für das i-te Modell bezeichnet, verteilt als Gauss-Wahrscheinlichkeitsverteilungsfunktion (WVF), d. h. νⁱ = N(O, Qⁱ), wobei Qⁱ eine Kovarianzmatrix ist, Hⁱ die Messmatrix für ein i-tes Kalman-Filter ist und ωⁱ den Messfehler für das i-te Kalman-Filter bezeichnet, verteilt als Gauss-WVF, d. h. ωⁱ = N(O, Rⁱ), wobei Rⁱ die Kovarianzmatrix des Messfehlers des i-ten Kalman-Filters ist.Each Kalman filter is designed to be an object state

at a current time step k with respect to a feature descriptor

, a position and speed of the object

and a classification mark

based on the output of the data mapping module 118 and the vehicle kinematics data, vehicle speed, and yaw rate from the vehicle sensors 204 include, update. Further, each Kalman filter is configured to take a position measurement

based on measurements from the rangefinders 14 and 16 and the camera 11 to update. Every i-th Kalman filter of the independent bank of n Kalman filters 206 can be displayed with:

where A ⁱ and B ^{i are} system matrices of the i th Kalman filter,

a vehicle operator input and vehicle kinematic data, e.g. At a time step t _k , ν ⁱ denotes a model creation error for the ith model, distributed as a Gaussian Probability Distribution Function (WVF), ie, ν ⁱ = N (eg, a brake and an accelerator pedal position, a steering wheel angle and a vehicle _{yaw rate} . O, Q ⁱ ), where Q ^{i is} a covariance matrix, H ^{i is} the measurement matrix for an ith Kalman filter, and ω ⁱ denotes the measurement error for the i-th Kalman filter, distributed as Gauss-WVF, ie, ω ⁱ = N (O, R ⁱ ), where R ^{i is} the covariance matrix of the measurement error of the ith Kalman filter.

und die Positionsmessung

zu aktualisieren, die im Vergleich zu den Gewichten aller Kalman-Filter einem größten Gewicht wⁱ eines i-ten Kalman-Filters zugeordnet ist. Das Gewicht wⁱ beschreibt eine Wahrscheinlichkeit, dass ein bestimmtes Kalman-Filter i den Objektzustand x genau ermittelt. Im Betrieb initiiert der Markov-Entscheidungsprozess 208 jedes Kalman-Filter für die Bank von Kalman-Filtern 206 mit gemischten Variablen, die das Gewicht w i / 0, den Mittelwert x ^ i / 0 und die Kovarianz P i / 0 umfassen. Die Kalman-Filter werden nach dem Empfangen einer neuen Messung zu einem Zeitpunkt k aktualisiert, wobei vorherige Schätzwerte verwendet werden, die zu einem vorherigen Zeitschritt t_k-1 ermittelt werden und ein Gewicht

einen Mittelwert

und eine Kovarianzmatrix

für das i-te Kalman-Filter umfassen.The Markov Decision Process 208 selects a Kalman filter to set the object state

and the position measurement

which is associated with a largest weight w ^{i of} an i-th Kalman filter compared to the weights of all Kalman filters. The weight w ⁱ describes a probability that a specific Kalman filter i exactly determines the object state x. In operation, the Markov decision-making process initiates 208 every Kalman filter for the bank of Kalman filters 206 with mixed variables comprising the weight wi / 0, the mean x ^ i / 0 and the covariance P i / 0. The Kalman filters are updated after receiving a new measurement at a time k, using previous estimates obtained at a previous time step t _k-1 and a weight

an average

and a covariance matrix

for the ith Kalman filter.

wobei T_ij die Wahrscheinlichkeit darstellt, dass ein Objekt von einer bestimmten vorhergesagten Klassifizierung i, z. B. einer Klassifizierung eines stationären Objekts, in eine andere bestimmte Klassifizierung j, z. B. ein Fahren mit konstanter Geschwindigkeit, wechselt.Using a transition probability matrix T determined as described hereinabove and selected based on the compared Kalman filters, mixed variables are iteratively determined. The mixed variables will be in the bank of Kalman filters 206 entered. For each Kalman filter i, a mixed weighted value describing a time step k can be obtained using the following relationship:

where T _{ij represents} the probability that an object will depend on a particular predicted classification i, e.g. B. a classification of a stationary object, in another specific classification j, z. B. driving at a constant speed, changes.

For each Kalman filter can use the following relationship mixed average.

For each Kalman filter is using a feedback that has an estimated model condition and an estimated Includes error covariance using the methods described hereinbelow Relationships [19] and [20] are provided, iteratively a mixed one Covariance value determined. The mixed covariance value can be below Using the following relationship can be determined.

wobei Aⁱ und Bⁱ Systemmatrizen des i-ten Kalman-Filters sind;

einen Fahrzeugbedienereingang und Fahrzeugkinematikdaten, die z. B. eine Brems- und eine Gaspedalstellung, einen Lenkradwinkel und eine Fahrzeuggierrate umfassen, zu einem Zeitindex t_k darstellt; Qⁱ eine Kovarianzmatrix bezeichnet, die einen Modellvorhersagefehler für das i-te Kalman-Filter darstellt; Hⁱ die Messmatrix für das i-te Kalman-Filter ist; I eine Identitätsmatrix darstellt;

eine Kalman-Verstärkung für das i-te Kalman-Filter zu einem Zeitschritt t_k darstellt;

und

den geschätzten Mittelwert und die entsprechende Kovarianzmatrix für das Objekt zum Zeitschritt t_k bezeichnen; und Rⁱ eine Kovarianzmatrix bezeichnet, die einen Messfehler für das i-te Kalman-Filter darstellt.After determining the mixed variable, the bank of Kalman filters determines the following relationships:

where A ⁱ and B ^{i are} system matrices of the i-th Kalman filter;

a vehicle operator input and vehicle kinematic data, e.g. B. a brake and an accelerator pedal position, a steering wheel angle and a Fahrzeuggierrate include, represents a time index t _k ; Q ⁱ denotes a covariance matrix representing a model prediction error for the ith Kalman filter; H ^{i is} the measurement matrix for the ith Kalman filter; I represents an identity matrix;

represents a Kalman gain for the i-th Kalman filter at a time step t _k ;

and

denote the estimated mean and covariance matrix for the object at time step t _k ; and R ⁱ denotes a covariance matrix representing a measurement error for the ith Kalman filter.

wobei γⁱ ein Normalisierungskoeffizient ist, der verwendet wird, um die WVF auf 1 zu integrieren,

und

den geschätzten Mittelwert und die entsprechende Kovarianzmatrix bezeichnen, die hierin oben unter Verwendung der Beziehungen [12] und [13] ermittelt werden, und

A probability analysis is used to determine a probability that an i-th Kalman filter matches current measurement observations. The probability analysis is performed using the following relationship:

where γ ^{i is} a normalization coefficient used to integrate the WVF to 1,

and

denote the estimated mean and covariance matrix determined hereinabove using relationships [12] and [13], and

wobei

und Lⁱ in den Beziehungen [9] bzw. [17] berechnet werden.After performing the probability analysis, a weighted value for the predicted classification is updated according to the following relationship.

in which

and L ⁱ in relations [9] and [17], respectively.

wobei

After updating the weighted value, the estimated model condition state and the estimated error covariance are determined using the following relationships:

in which

The relationships [19] and [20] are the estimation output of an object position and velocity of the object tracker module 119 , The feature descriptor and the classification tag associated with the objects remain of the data mapping module 118 unchanged. The outputs are inputs to an object tracking database 113 adapted to store object information comprising object tracking files and corresponding image areas and classification marks. The object information is stored for iterative comparison with new data and for predicting relative movement of the vehicle indicative of a likely or imminent collision event. Further, a region of interest or regions of interest reflecting previously selected image regions may be applied to the module 112 for motion analysis and salient area detection that performs image area identification, to provide continuity of analysis of iterative view data. In this way, distance data or distance tracking information is superimposed on the image plane to improve the estimation.

wobei x ^ⁱ und Pⁱ hierin in den Beziehungen [15] bzw. [16] beschrieben sind und wobei die Zeitindextiefstellung t_k ignoriert wird. Diese Verteilung kann zufällig abgetastet werden, um einen Satz von Partikeln zu erhalten, der eine Wahrscheinlichkeitsverteilung des Objekts darstellt.After the data merging module 110 Pursuit estimates for objects near the vehicle 10 then the object traces will become the collision hazard evaluation module 120 where each track is evaluated for a probability of a collision. Each object tracking involves a mixture of a Gaussian distribution

where x ^ ⁱ and P ^{i are described} herein in relationships [15] and [16], respectively, and ignoring the time texture depression t _k . This distribution can be sampled randomly to obtain a set of particles representing a probability distribution of the object.

For example, for an nth object, a corresponding set of particles s = {x (ς) / n | ς = 1, ..., M}, where M denotes the number of particles that was probability distribution. The ς-th particle x (ς) / n = (x (ς) / n, y (ς) / n, vx (ς) / n, vy (ς) / n), where the longitudinal displacement of the nth object is, y (ς) / n is the lateral displacement, vx (ς) / n and vy (ς) / n are the longitudinal or lateral velocity of the object. If the object x is on the collision path and the time to collision (TTC) TTC ^(ς) = -x (ς) / n / vx (ς) / n is less than a threshold (eg 2 seconds) , the particle is called a danger. The collision hazard assessment module 120 determines the probability of a collision based on a percentage of a number of hazards per total particles associated with an object. The probability of a collision is sent to the collision countermeasure module 150 output.

The probability of a collision of an identified object is determined by a collision countermeasure module 150 analyzed. For example, this probability of a collision may be evaluated in terms of a threshold probability for a collision, and if a collision is determined to be likely, collision countermeasures may be initiated. For example, if the probability of a collision is greater than a threshold (eg 90%), the object provides an immediate threat to the host vehicle and appropriate collision countermeasures are performed. The collision countermeasures may include seat belt tightening, throttle aging, automatic braking, air bag preparation, head restraint adjustment, horn and headlight activation, pedals or steering column adjustment, estimated relative impact velocity adjustments, suspension control adjustments, and the like Adjustments to stability control systems include. A response to likely collision events may be scaled based on an increased probability. For example, a slight automatic braking may be used in the event that a low threshold probability is determined, and more drastic measures may be taken in response to the determination of a high threshold probability.

Further It should be noted that improved accuracy of judgment a probability by iteratively training the alarm models can be achieved. For example, if an alarm is issued, can contact the driver through a voice prompt and a request a screen or by any other input method a review option be provided, which requests that the driver confirms whether the alarm of imminent collision appropriate was. There are a number of methods known in the art correct alarms, false alarms or missed alarms. For example, machine learning algorithms are known in the art and can they are used to adaptively use a programming the weights and weights are dependent on alternative calculations from the nature of the feedback assigns. Furthermore, fuzzy logic can be used to input in System according to scalable Condition factors based on feedback. In this way, the accuracy of the system over the Time and based on the specific driving habits of an operator be improved.

8th FIG. 12 shows exemplary range data superimposed on a corresponding image plane in accordance with the present disclosure. FIG. The hatched bars are the radar tracks superimposed on the image of a front-facing camera. As described hereinabove, the module extracts 112 for motion analysis and salient area detection, the image areas surrounding the range sensor tracks. The feature extraction module 114 calculates the feature descriptor of the image areas. The classification module 116 takes the extracted features as input and passes them to a classifier to determine if an image area is surrounding an object. The classification determines the indicator of each screen area. For example, in 8th the boxes A and B are identified as vehicles, while the unmarked box is identified as an object on the roadside. The prediction process module 117 uses information of the object from the past (ie position, image piece and flag of a previous cycle) and predicts the current values. The data mapping module 118 connects the current measurements to the previous objects or determines if the source of a measurement (ie, position, image piece, and label) is from a specific object. The object tracker module 119 creates updated locations for the objects for storage in the object tracking database 113 ,

The Revelation has described certain preferred embodiments and modifications this. Other modifications and changes may be made for others to read and understand the description will be apparent. Therefore, the revelation not to the specific embodiment (s) which is / are the best embodiment, to the execution This revelation is considered, but the revelation should all embodiments include, which fall within the scope of the appended claims.

Claims (19)

Method for controlling a vehicle operation while a dynamic vehicle event, the method comprising that a first input image is monitored; a first monitored tracked object in the first input image in a first tracking cycle becomes; a second input image is monitored; a second one tracked object in the second input image monitored in a second tracking cycle becomes; a difference measure is determined, which is the first compares the tracked object with the second tracked object, where the degree of difference estimates whether the first tracked object and the second tracked object a single tracked object nearby represent the vehicle; the first tracked object and the second tracked object assigned based on the difference measure become; and the associated objects in a collision preparation system used to control the operation of the vehicle. The method of claim 1, further comprising determining, in the first input image, the first tracked object; determining a position in the first input image of the first tracked object; Defining features in the first input image near the first tracked object; determining a first tag that describes a likely classification of the first tracked object; in the second input image, the second tracked object is detected; determining a position in the second input image of the second tracked object; Defining features in the second input image in the vicinity of the second tracked object; determining a second tag describing a likely classification of the second tracked object; and wherein determining the measure of difference comparing the first tracked object with the second tracked object comprises comparing the positions of each of the tracked objects, the features proximate each of the tracked objects, and the tags. The method of claim 2, wherein the probability classifications a fast-moving object, a slowly moving one Object and a stationary Object include. The method of claim 2, further comprising Range sensor data, which correspond to each of the input images; and in which determining the tracked objects in the input images the range sensor data based on each of the input images correspond. The method of claim 2, wherein the second identifier based on the first license plate. The method of claim 2, wherein determining the tracked objects in the input images includes that salient Image areas and areas associated with a movement in each of the Pictures are identified. The method of claim 6, wherein the identifying associated with salient picture areas and a movement Areas in each of the pictures includes that an edge illustration is determined; a corner image is detected; a Lucan-Kanade image of an optical flow is detected; one Range of the input image indicating a ground is determined; and the salient image areas and those of a movement associated areas that are on the edge map, the corner image, Lucan-Kanade picture of an optical river and area of the input image indicating a ground are identified become. The method of claim 7, wherein identifying associated with salient picture areas and a movement Areas further comprises that an object information of a the previous input image is projected on the input image; and wherein identifying the salient image areas and the areas associated with movement further on the pro ject the object information is based on the previous input image. The method of claim 1, wherein the first object a priori for a comparison with the second object is selected. The method of claim 1, further comprising the associated tracked objects with a bank of Kalman filters to be analyzed; and wherein using the associated tracked objects in the collision preparation system comprises the analyzed associated tracked objects are used. The method of claim 10, wherein analyzing comprising associated objects with a bank of Kalman filters, that monitors a speed and a yaw rate of the vehicle become; the bank of Kalman filters on the assigned objects and the speed and yaw rate of the vehicle is applied; and a Markov decision process on the output from the Bank of Kalman filters used is assigned to the analyzed To create objects. A method of controlling a vehicle operating during a dynamic vehicle event, the method comprising monitoring range sensor information and vision system information; superimposing the range sensor information of the vision system information to produce a modified image; salient image areas in the modified image are identified using edge detection, corner detection, and optical flow detection; the prominent image areas corresponding to the floor are excluded; iteratively identifying objects in a vehicle field of view using the salient image areas, comprising identifying a first object in the vehicle field of view using the salient image areas in a first time step and a second object in the vehicle field of view using the salient image areas in a subsequent time step is identified; a position of the first object is predicted in the subsequent time step; matching the first object with the second object based on comparing the predicted position of the first object and the identified second object; Object tracking of the matched objects are determined; determining a probability of a collision between each of the matched objects and the vehicle based on the object pursuits; and the vehicle is controlled based on the probability of a collision. System for operating a collision preparation system in a vehicle, the system comprising: a vision system; one Distance sensor system; and a control module that dates monitored by the distance sensor and images from the vision system; iterative Objects in a vehicle field of view based on the range sensor data and the images identify what includes that first object in the vehicle field of view using the range sensor data and the images are identified in a first time step, and a second object in the vehicle field of view using the Distance sensor data and images in a subsequent time step is identified a position of the first object in the subsequent time step predicts the first object and the second object based on the predicted position assigns to the first object, an object tracking the associated Determines objects, a probability of a collision based on the assigned objects and the vehicle determines the object tracking, and the vehicle on the basis the probability of controls a collision. The system of claim 13, wherein determining the Object tracking of the mapped objects includes that the associated objects are processed with multiple Kalman filters; and object tracking based on a selection of a Output of the multiple Kalman filters using a Markov decision process is determined. The system of claim 13, wherein the identifying of objects in the vehicle field of view includes the distance sensor data superimposed on one of the pictures to create a modified image; image areas on the modified image, the distance sensor information are assigned to be extracted; a feature descriptor the image areas is determined; and Objects in the image areas identified on the basis of the feature descriptor. The system of claim 15, wherein the feature descriptor using a histogram of predetermined features and / or a gradient orientation histogram and / or a principal component analysis is determined. The system of claim 16, wherein the predetermined ones Features using a Haar wavelet and / or a Gabor wavelet and / or a Leung-Malik filter bank. The system of claim 13, wherein associating the first object and the second object based on the predicted Position of the first object includes that a measure of difference between the first and second object is determined; and the first and second object in accordance be brought if the difference is less than a predetermined Threshold is. The system of claim 13, wherein determining the probability of a collision between the associated objects and the vehicle based on the object tracking comprises: determining a Gaussian distribution for object tracking; randomly sampling the Gaussian distribution to obtain a set of particles; Determining a longitudinal and transverse displacement of each particle; Determining if each particle is on a collision path with the vehicle; Determining a time to collision for each particle on a collision path; Designating each particle on a collision path as a hazard if the time to collision for the respective particle is less than a threshold time period; and determining that a collision is imminent when a predetermined percentage of the set of particulates designated as a hazard is greater than a threshold percentage.