JP2019164124A - Fusion of front vehicle sensor data for detection and ranging of preceding objects - Google Patents

Abstract

To provide a method for determining the inter-vehicle distance during adaptive cruise control, which improves detection of vehicles with inconsistent rear planes (i.e., construction materials projecting from a truck bed or truck trailers at a distance above tires).SOLUTION: The method comprises: acquiring, via a camera, an image related to a preceding object; classifying the image related to the preceding object; and selecting the first sensed parameter, the second sensed parameter or a combination thereof on the basis of the classification of the image related to the preceding object.SELECTED DRAWING: Figure 2

Description

  The adaptive cruise control (ACC) system improves driver safety and reliability by utilizing a forward sensor that detects the size, speed and distance of the preceding object. In the example where the driver chooses to use ACC for longitudinal control, the ACC generates a command to change the vehicle speed in order to maintain a safe inter-vehicle driving distance. US Pat. No. 7,457,699, whose title is “Technique for detecting truck trailer for stop and go adaptive cruise control” for adaptive cruise control that repeatedly stops and progresses, is inaccurate. Targeted at a radar-based approach to the detection of near-distance track trailers that may be predicted. While this approach aims to improve the detection of vehicles with inconsistent rear surfaces (ie building materials protruding from truck bed, truck trailer at a distance above the tire), radar Remains sub-optimal in short distance situations, and therefore a new approach is needed to detect preceding objects.

  The above description of “background art” is intended to schematically represent the contents of the present disclosure. The inventor's work is expressly and implicitly as prior art to the present invention, as long as it is described in this background art section, as well as embodiments of the description that would otherwise not be suitable as prior art at the time of filing. Also not allowed.

  The present disclosure is a method, apparatus, and computer-readable medium configured to determine a distance between a vehicle and a preceding object, wherein the vehicle and the preceding object are located via a first sensing modality. Obtaining a first detection parameter associated with the distance; determining a magnitude of the first detection parameter obtained via the first detection manner via a processing circuit; and a second detection manner. Acquiring a second detection parameter related to the distance between the vehicle and the preceding object via the second detection mode and acquiring the second detection parameter via the processing circuit and the second detection mode. In order to determine the magnitude of the parameter and to determine the distance between the vehicle and the preceding object, the magnitude of the first sensing parameter and the magnitude of the second sensing parameter relative to a predetermined threshold Said decision Based on the first detection parameter, it said comprising selecting a second sensed parameter, or combination thereof, to a method, an apparatus and a computer-readable medium.

  According to an embodiment, the present disclosure obtains, via a camera, an image associated with the preceding object, classifying the image associated with the preceding object, and of the image associated with the preceding object. And further selecting the first detection parameter, the second detection parameter, or a combination thereof based on the classification.

  The above paragraphs are provided for purposes of general introduction and are not intended to limit the scope of the following claims. The described embodiments, together with further advantages, will be best understood by referring to the following detailed description in conjunction with the accompanying drawings.

FIG. 1 is a diagram illustrating a user vehicle relative to a preceding vehicle, according to an exemplary embodiment of the present disclosure. FIG. 2 is a high-level flowchart of the adaptive cruise control system of the present disclosure including a driving support system control unit and a travel speed control unit. FIG. 3 is a block diagram of a traveling speed control unit according to an exemplary embodiment of the present disclosure. FIG. 4 is a diagram illustrating identification of a preceding vehicle, according to an exemplary embodiment of the present disclosure. FIG. 5 is a high-level flowchart of an exemplary embodiment of the present disclosure. FIG. 6 is a flowchart of detection and ranging according to an exemplary embodiment of the present disclosure. FIG. 7A is a diagram illustrating detection and ranging according to an exemplary embodiment of the present disclosure. FIG. 7B is a diagram illustrating detection and ranging according to an exemplary embodiment of the present disclosure. FIG. 7C is a diagram illustrating detection and ranging according to an exemplary embodiment of the present disclosure. FIG. 8A is a diagram illustrating object recognition during detection and ranging, according to an exemplary embodiment of the present disclosure. FIG. 8B is a diagram illustrating object recognition during detection and ranging according to an exemplary embodiment of the present disclosure. FIG. 8C is a diagram illustrating object recognition during detection and ranging, according to an exemplary embodiment of the present disclosure. FIG. 9 is a block diagram of a hardware description of a computer configured to perform detection, ranging and control, according to an exemplary embodiment of the present disclosure.

  A more complete understanding of the present disclosure and many of its attendant advantages will be readily obtained as the same becomes better understood by reference to the following detailed description when considered in conjunction with the accompanying drawings. Will.

  As used herein, the term “a” (“a” or “an”) is defined as one or more than one. As used herein, the term “plurality” is defined as two or more than two. As used herein, the term “another” is defined as at least a second or more. As used herein, the terms “including” and / or “having” are defined as comprising (ie, an open language). For "one embodiment", "certain embodiments", "an embodiment", "an implementation", "an example" or similar terms Reference throughout this document means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, the appearance of such phrases or appearances at various locations throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments without limitation.

  Adaptive cruise control (ACC) is widely deployed in modern vehicles to improve passenger safety. To do this, the ACC actively measures the distance between the user's vehicle and the preceding vehicle and, in addition to other inputs, integrates this distance information with the known speed information of the user vehicle, It is determined whether the speed of the user vehicle needs to be adjusted in order to maintain the following inter-vehicle distance or to reach the target speed. Distance estimation is often performed via a radar, which is a highly efficient system for determining the distance of a vehicle in an ideal situation (eg, highway travel at a distance). However, in traffic jams or when the preceding vehicle carries a nuisance load that extends from the rear of the vehicle, the accuracy of the radar-based information may be reduced, and to control the vehicle speed properly and for collisions Affects ACC's ability to avoid.

  This drawback is a result of the technical basis of radar-based systems deployed in vehicles. Radar transmitters emit narrow conical radio waves that touch the object and are reflected or scattered in various directions. When a portion of the wave is reflected at a known time and frequency interval and received at the receiver, the distance to the preceding object can be determined. However, in order to determine a distance from the various signals, the highest reflectance centroid from the preceding object is selected. This center may cover most of the perspective behind the preceding vehicle, but may only represent a small portion of the preceding vehicle. These discrepancies result in an inaccurate determination of the true distance for the rear aspect of the preceding vehicle in the example of a flatbed truck carrying steel type I beams. For example, as the user vehicle approaches the preceding vehicle, the preceding vehicle and any rear aspects may deviate from the thin cone-shaped radio waves emitted by the user's vehicle and become untrackable out of the radar field of view. At present, effective distance estimation technology is necessary at short distances.

  For this reason, as described herein, this disclosure introduces a system for integrating cameras, radar and sonar to accurately determine the distance between vehicles over a range of distances. To do.

  FIG. 1 illustrates a user vehicle relative to a preceding vehicle, according to an exemplary embodiment of the present disclosure. The vehicle 105 is traveling at a known speed at a distance behind the preceding vehicle 110. In the embodiment, the vehicle 105 is a user vehicle 115 equipped to carry passengers, and the preceding vehicle 110 is a flat bed trailer truck 121 equipped to transport heavy equipment. In the embodiment, the aspect behind the flat bed trailer 122 of the flat bed trailer truck 121 is not within the radar field of view 111 of the user vehicle 115. However, this rear aspect of the flatbed trailer 122 of the flatbed trailer truck 121 that is not trackable by the radar due to proximity to the user vehicle 115 is within the sonar field 112 of the user vehicle 115. In the embodiment, the heavy equipment or the like of the flat bed trailer 122 of the flat bed trailer truck 121 reflects radio waves with high intensity. Although not the most rearward aspect of the flatbed trailer truck 121, this heavy equipment has a biased effect on the radar system's ability to accurately determine the distance to the flatbed trailer 122. As a result, this can obscure the true distance between the user vehicle 115 and the aspect behind the flatbed trailer truck 121. The front camera of the user vehicle 115 captures the image 113 of the preceding vehicle. These captured images 113 are used to determine the identity of the preceding vehicle, and the distance estimation techniques described above are affected by the type of vehicle (ie, shape, size, cargo). It further informs the accuracy of the estimation technique.

  While radar is known to be more effective in determining longer distances, sonar is more effective in determining shorter distances due to the technical limitations of using sound waves. Therefore, in the exemplary embodiment, when the preceding vehicle approaches the boundary between these two distance estimation approaches, it is necessary for the user vehicle to make a decision regarding the distance estimation approach to use. Based on the identification of the preceding vehicle and its predicted distance, it may be appropriate to use only radar, only sonar, or integrate the two approaches to reach the most accurate distance.

  FIG. 2 is a high level flowchart of the adaptive cruise control system of the present disclosure. FIG. 2 shows, from left to right, an overview of the inputs supplied to the driving assistance system controller 230, which is responsible for accurately determining the distance to the preceding vehicle, the value of which is then changed to change the vehicle speed. This is supplied to the traveling speed control unit 235.

  As shown in FIG. 2, the vehicle control system 200 includes, but is not limited to, a camera 203 for capturing a preceding object, a radar device 201 for determining a distance between the preceding object and the vehicle, and a preceding object. A sonar device 202 for determining the distance between the vehicle and the vehicle, a travel speed sensor 204 for detecting the travel speed of the vehicle, a driving support system control unit 230, and a travel speed control unit 235. Including. The driving system control unit 230 includes, but is not limited to, an object recognition unit 233 that uses an image captured from the camera 203, a distance determination unit 231, a radar intensity determination unit 234, and a sonar resolution determination unit 232. . The distance determining unit 233 uses the data from the radar device 201 and the sonar device 202 to determine the distance between the preceding vehicle and the present vehicle in light of the outputs from the radar intensity determining unit 234 and the sonar resolution determining unit 232. decide. When the distance is determined by the distance determining means 233, if appropriate, the value is supplied to the traveling speed control unit 235 for the adjustment of the vehicle speed.

  According to the embodiment, the camera 204 acquires at least an image of a preceding visual field including an arbitrary object within the visual field, and outputs the captured image to the driving support system control unit 230. The captured image from the camera 203 can be used for object recognition to inform the distance determining means and to record the lateral presence of the vehicle in the field of view. The presence of vehicles and their spatial location are critical in the context of environmental changes (ie, unexpected lane changes), so that vehicle speed can be controlled immediately. Data is continuously acquired from the camera 203 to provide real-time object data to the driving support system controller 230. In the embodiment, the camera 203 is a visible light camera, an infrared camera, or a combination thereof.

  According to the embodiment, the radar device 201 is a millimeter wave radar that detects a preceding object including a vehicle and measures a distance between the vehicle and the preceding vehicle. The vehicle-based millimeter wave radar device 201 mixes the transmission wave (millimeter wave) emitted from the transmitter and the reception wave reflected from the preceding vehicle, and extracts the beat frequency in the mixed signal, thereby Information about the distance between the vehicle and the preceding vehicle is acquired. This distance information is then output to the driving support system control unit 230. In the embodiment, the information on the distance to the preceding vehicle is determined through localization of the center of the reflected radio wave captured by the receiver. Similar to the heat map, a pixelated rendering of radio field strength is generated and the region with the highest strength is selected as the aspect behind the preceding vehicle. In another embodiment, the information about the distance to the preceding vehicle is determined from waveform peaks in the frequency spectrum that are correlated with objects in the field of view. The data is continuously acquired from the radar device 201 in order to provide real-time data to the driving support system control unit 230.

  According to an embodiment, the sonar device 202 uses sound waves to determine the distance between the vehicle and a preceding object such as a vehicle at close range, as is known in the art. The vehicle-based sonar device 202 measures the time from when an ultrasonic wave that is inaudible to humans is emitted to when the wave reflected by the preceding object returns to the sonar device 202. This time domain data informs the distance determination, and its output is supplied to the driving support system controller 230. Data is continuously acquired from the sonar device 202 to provide real-time data to the driving assistance system controller 230.

  According to the embodiment, the traveling speed sensor 204 detects the traveling speed of the vehicle and outputs the traveling speed to the driving support system control unit 230. This travel speed is combined with the determined distance from the drive support system control unit 230 and passed to the travel speed control unit 235 through the drive support system control unit 230, and the two values are In order to determine the travel speed adjustment, it is used by the travel speed control unit 235. The data is continuously acquired from the travel speed sensor 204 in order to provide real-time data to the driving support system control unit 230 and thus the travel speed control unit 235.

  According to the embodiment of the present disclosure, the camera 203, the millimeter wave radar device 201, the sonar device 202, and the traveling speed sensor 204 are commercial products that can be implemented in accordance with individual specification restrictions.

  According to the embodiment, the driving support system control unit 230 is a known microcomputer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and an I / O (input and output) interface. is there. A control program stored in a storage device such as a ROM enables the CPU to function as object recognition means 233, distance determination means 231, radar intensity determination means and sonar resolution determination means 232 described below. An exemplary embodiment of the driving assistance system controller 230 is described in FIG.

  According to the embodiment, the object recognition means 233 is configured to determine the identification of the preceding vehicle based on the captured image from the camera device 204. According to a standard machine learning approach, an algorithm or similar process such as a convolutional neural network is trained on a database of known vehicles that are available and can be used to classify preceding vehicles including, but not limited to, an audi, car or truck. enable. In addition, sub-classification according to manufacturer and model can be determined to allow for increased specificity and perception related to the speed, weight and shape of the preceding vehicle.

  In an exemplary embodiment of a database of object recognition means 233, the database includes a corpus of reference images, each reference image includes an identifiable reference object and is associated with a corresponding text description of the reference object in the database. In addition, the database includes a plurality of image query resolution data structures, each containing a collection of records, and each record includes an image descriptor for a reference image. Thus, each data structure can be searched using a corresponding one of a set of predetermined search processes to identify the best matching record in the data structure based on the image descriptor.

  According to the embodiment, the distance determination unit 231 determines an accurate inter-vehicle distance based on the distance data from the radar device 201 and the sonar device 202 integrated with the object data from the camera device 203 via the object recognition unit 233. Configured to determine.

  During the distance determination, the distance determination means 231 of the driving support system controller 230 must select an appropriate source for the distance estimation data. For this purpose, the data stream is either a radar device 201, a sonar device 202 or a fusion of two data streams as supplied to the driving assistance system controller 230 in order to accurately estimate the inter-vehicle distance. Selected from. In determining the proper data source, each data stream can be evaluated with respect to a predetermined threshold. For radar, the signal is generated from the center of maximum reflectivity and the threshold is set at a predetermined reflectivity intensity. Intensities higher than this predetermined reflectance intensity are considered to provide an accurate representation of the distance of the preceding object, and the threshold is set according to the specifications of the radar device 201. For sonar, the signal is generated in terms of air-based noise and the threshold is set at a predetermined resolution. A resolution higher than this predetermined resolution threshold is considered to provide an accurate estimate of the distance of the preceding object, and the threshold is set according to the specifications of the sonar device 202.

  In an exemplary embodiment of the distance determining means of the present disclosure, a predetermined radar and sonar threshold is used, and the preceding vehicle is a sedan and is traveling at a relatively slower speed than the present vehicle. Initially, the sedan is 20 meters before the vehicle, the radar intensity is higher than a predetermined threshold, and the sonar resolution is lower than the predetermined resolution. In this case, the radar is selected as the ideal modality for the distance determining means. As the inter-vehicle distance decreases, the distance is reduced (eg, 6 meters), and sonar resolution and radar intensity both exceed their individual thresholds. In light of the information from the camera equipment and the identification of the preceding vehicle, a decision can be made to transition to sonar, to maintain with radar alone, or to merge the two data streams into one distance output. As the inter-vehicle distance continues to decrease, the radar intensity may fall below a predetermined threshold, while sonar that is effective at close range where the effects of noise are minimized will continue to display resolutions above the individual thresholds. As a result, sonar is selected as the ideal style. The above-described method for sensor selection is similarly applicable when the inter-vehicle distance is increasing. In addition, at individual vehicle distances from short to long distances, the quality of the data stream affects the decision making process.

  According to exemplary embodiments of the present disclosure, as described above, object recognition by captured images from camera equipment is utilized to assist in selecting a distance estimation manner for distance determination. For example, if the vehicle is within both radar and sonar, the data from each data source can be merged to produce an improved output. However, if it is determined through object recognition that the preceding vehicle is a flatbed truck with a trailer, the radar will have the least ability to accurately detect the rear aspect of the flatbed truck, and sonar data will be given priority. It may be appropriate to do so.

  According to the embodiment, the travel speed control unit 235 controls the travel speed of the vehicle based on the travel speed set in advance by the driver and the distance and speed input from the driving support system control unit. For example, when operating the ACC, the driver selects the desired travel speed. When the desired traveling speed is recognized, the traveling speed control unit 235 adjusts the vehicle traveling speed through the throttle subsystem and the brake subsystem in order to maintain the desired traveling speed. In the embodiment, when it is determined that the preceding vehicle is within the detection distance of the vehicle via the front sensor, it is determined whether to change the vehicle traveling speed. In the exemplary embodiment, the preceding vehicle is traveling at a relatively slower speed than the present vehicle. The travel speed controller 235, when appropriate, generates a signal for operating the brake subsystem and the current travel speed passed through the drive assist system controller 230 and from the drive assist system controller 230. The determined inter-vehicle distance can be used. As the travel speeds are balanced or when the preceding vehicle is out of the field of view of the front sensor, the travel speed control unit 235 can recognize the discrepancy between the current travel speed and the target travel speed, and signal accordingly. Can be generated.

  Next, FIG. 3 is a block diagram of the connection to the travel speed control unit and the driving support system control unit according to an exemplary embodiment of the present disclosure. The traveling speed control unit 335 includes, but is not limited to, a memory subsystem 336, a throttle subsystem 337, a brake subsystem 339, and an alarm subsystem 338. Data regarding the inter-vehicle distance and the current traveling speed is supplied from the driving support system control unit 340 to the traveling speed control unit 335. The target travel speed selected by the driver during ACC operation is stored in the memory subsystem 336. The memory subsystem 336 generally includes an application-appropriate amount of volatile memory (eg, dynamic random access memory (DRAM)) and non-volatile memory (eg, flash memory, electrically erasable programmable read only memory (EEPROM)). including. In the embodiment, the code executable by the processor for determining the relative speed adjustment of the vehicle is stored in a non-volatile memory of the memory subsystem 336 of the travel speed control unit 335.

  In an embodiment, the speed adjuster processor 341 and the memory subsystem 336 are proportional--one of a variety of suitable adaptive controllers to maintain an appropriate inter-vehicle distance in light of the selected cruise control speed. Use an integral-derivative controller (PID controller). The PID controller, which is a control loop feedback mechanism, provides continuous adjustment of the output through the calculation of the error value as the difference between the measured process and the desired set point, and is proportional to reach the set point Apply corrections based on integration and differentiation. Similarly, from the perspective of this embodiment, the speed controller processor 341 may generate signals for the throttle subsystem 337 and the brake subsystem 339 as needed to adjust the vehicle speed about the set point. it can. If necessary, the speed controller processor 341 may provide a control signal to the alarm subsystem 338 to cause the alarm subsystem 338 to provide visual and / or audio feedback to the driver of the vehicle.

  In an exemplary embodiment of the travel speed controller of the present disclosure, the driver activates the ACC and sets the speed control to 80 mph. However, a preceding vehicle traveling at a speed of 75 miles per hour is detected in front of the vehicle. Continuous information regarding the distance to the preceding vehicle and the current speed of the vehicle is provided by the driving assistance system controller. In order to maintain a safe inter-vehicle distance, the speed controller processor 341 provides a signal to the brake subsystem 339 to decelerate the vehicle. When actively monitoring the current vehicle speed and the distance to the preceding vehicle as continuously provided by the driving assistance system controller 340, the speed controller processor 341 determines when the inter-vehicle distance is appropriate. Or change the vehicle speed as necessary. When the preceding vehicle goes out of sight or the inter-vehicle distance increases beyond a safe distance, the speed controller processor 341 until the set point selected by the driver is reached, as stored in the memory subsystem 336. A signal to throttle subsystem 337 can be generated to increase vehicle speed. Command generation for the throttle subsystem 337 and brake subsystem 339 is determined according to the control loop feedback mechanism of the PID controller. The use of a PID controller is understood in the art, as will be apparent from US Pat. No. 5,335,164A, which is incorporated herein by reference.

  According to the embodiment, the inter-vehicle safety distance is determined according to the traveling speed of the vehicle. The inter-vehicle safety distance may be further informed by the manufacturer and model of the preceding vehicle, and the size, shape, and weight of the preceding vehicle will affect its ability to accelerate or decelerate in the time domain. In an embodiment, information related to the identified preceding vehicle related to the safety distance between vehicles is updated via a system software update to the memory subsystem 336 of the travel speed controller 335 or via a cloud-based system software update. Also good.

  In the embodiment, the traveling speed control unit 335 is a well-known microcomputer having a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and an I / O (input and output) interface. A control program stored in a storage device such as a ROM enables the CPU to function as a traveling speed control unit 335 described below.

  FIG. 4 illustrates identification of a preceding vehicle, according to an exemplary embodiment of the present disclosure. The identification of the preceding vehicle is achieved through object recognition and machine learning. In an exemplary embodiment, a standard convolutional neural network (CNN) approach is used, as will be apparent from US Pat. No. 7,774,070 B2, as understood in the art and incorporated herein by reference. While described herein, CNNs should not be considered limiting, but are merely representative of various approaches to object recognition. In view of this disclosure, the CNN is trained (450) on a curated database of collected images of the virtual predecessor 451. In an embodiment, the curated database is actively maintained and updated via system software updates to the object recognition means or via cloud-based system software updates. In short, feature learning 452 and target classification 453 are executed in the database 451. In general, feature learning 452 comprises iterative convolution, activation via a normalized linear function, and pooling, while classification 453 includes associating learned features with known labels. The learned features (eg, edges, corners) are either manually selected or determined by the CNN via deep learning or a similar approach. After CNN training 450, a CNN test 455 is performed to ensure accuracy. Features are extracted from the test image 456 (457) and classified by the training classifier 450 (458). After confirming the effectiveness of the trained classifier, the CNN may be implemented in the object recognition unit of the driving assistance system control unit 440.

  In an embodiment, the training database 451 extracts features and develops classifiers to identify various vehicles including, but not limited to, cars, trucks, sport utility vehicles, tractor trailers, and audi dubai. Includes labeled images.

  In an embodiment, the training database 451 includes labeled images to extract features and develop classifiers to identify vehicle manufacturers and models.

  FIG. 5 is a high-level flowchart of an exemplary embodiment of the present disclosure. According to the embodiment, data from the front sensor is acquired after initialization 516 of ACC and setting of the target travel speed. These inputs including camera S518, radar and sonar S517 are transmitted to the driving assistance system controller. Within the driving support system control unit, the distance determination means S531 integrates the data from the captured image (ie, object recognition) with the estimated inter-vehicle distance as determined by radar and sonar. In an embodiment, the distance determining means S531 contextualizes the input from the radar and sonar according to the radar intensity and sonar resolution, and determines a single estimated inter-vehicle distance between the vehicle and the preceding vehicle. decide. The single estimated inter-vehicle distance is supplied to the traveling speed control unit S535 together with the vehicle traveling speed S518 determined by the mounted vehicle speed sensor. The travel speed control unit S535 then identifies these inputs (e.g., discrepancies regarding the ideal inter-vehicle distance (i.e., safe travel distance) or target travel speed (i.e., this vehicle speed vs. target travel speed) , Inter-vehicle distance, vehicle running speed). If there is a discrepancy, travel speed adjustment S542 is attempted to reach the target distance and / or target travel speed. The travel speed adjustment can be performed by the PID controller of the travel speed control unit S535 until the target is achieved. Furthermore, the loop from the front sensor input to the traveling speed control unit proceeds repeatedly until ACC is released.

  FIG. 6 is an exemplary flowchart of an embodiment of a distance determination unit of the driving support system control unit of the present disclosure. The exemplary flowchart should be considered an iterative loop after ACC initialization. According to the embodiment, the vehicle is initially under the manual control of the driver and recognizes the speed and surroundings of the vehicle including the preceding vehicle (S660). When the user activates the ACC (S661), the vehicle initializes forward sensor data acquisition including camera, radar and sonar. First, the distance determination means determines whether or not the preceding object or the preceding vehicle is within the sonar range (that is, higher than a predetermined threshold) as determined to some extent by the specifications of the sonar device (S662). In order to do so, the distance determination means examines the front sensors of the vehicle including radar, camera and sonar (S663), and in particular acquires data related to the position of the preceding vehicle. If it is determined that the preceding vehicle is not within the sonar range, the distance determination means returns to the radar and camera data to determine the inter-vehicle distance (S664). However, if it is determined that the preceding vehicle is within the sonar range with sufficient resolution to accurately estimate the vehicle, the distance determination is sufficient for the quality of the radar reflections to accurately determine the inter-vehicle distance. It is determined whether or not (S666). When the radar reflection intensity falls below a predetermined threshold, as described above, sonar is used to determine the inter-vehicle distance (S667). However, in embodiments, it may be determined that the radar reflection is at an intensity that is accurate. In this example, the distance determination can consider object recognition data from the camera to determine the reliability of the radar data (S668). For example, the preceding vehicle may be of the type that presents inherent difficulties with a radio-based approach, even though the radar intensity is higher than a predetermined threshold for intensity. In the case of a flatbed trailer truck, the radar can identify the rear face of the preceding vehicle. However, this rear surface may not be the most rearward aspect of the preceding vehicle, and inaccurate distance estimation may cause a collision. Therefore, it may be appropriate to follow sonar to delineate the rear surface and capture the rearmost aspect of the preceding vehicle. If it is determined that the radar may be unreliable, the distance determining means follows the sonar when determining the inter-vehicle distance (S667). However, in examples where the preceding vehicle has been identified as a vehicle whose distance is reliably estimated via a radar such as a sedan, the inter-vehicle distance determined by the distance determining means depends on the radar, sonar or a combination thereof be able to. After the distance is estimated through any of the three approaches, the determined inter-vehicle distance is transmitted to the travel speed control unit via the driving support system control unit in order to be used in the travel speed adjustment (S670). ).

  7A, 7B, and 7C illustrate detection and ranging of a preceding vehicle, according to an exemplary embodiment. In the embodiment, the preceding vehicle 710 is a coupe. In FIG. 7A, the preceding vehicle 710 is traveling beyond both the sonar field of view 712 and the radar field of view 711. However, the preceding vehicle 710 is traveling at a relatively slower speed than the main vehicle 705. When there is no preceding vehicle within the range of the sensor of the present vehicle 705, the present vehicle 705 continues to maintain ACC at the current traveling speed.

  In the embodiment, when the present vehicle 715 approaches the preceding vehicle, the front sensor of the present vehicle 705 starts to detect and measure the preceding vehicle 710 as shown in FIG. 7B. In an embodiment, if there is a leading vehicle 710 only in the radar field of view 711, the radar is used to determine the inter-vehicle distance. This radar-based value is then transmitted to the travel speed controller.

  However, if a sudden change in speed occurs or during traffic jams, the inter-vehicle distance reduces to a point where the preceding vehicle 710 is within the field of view of both the sonar 712 and the radar 711, as shown in FIG. 7C. There is a case. In an embodiment, data from both sources are taken into account when determining the inter-vehicle distance. If the radar reflection intensity is determined to be sufficient for ranging, as described above, a fusion of radar and sonar data will be used to provide the inter-vehicle distance to the travel speed controller. However, if the data from the radar is determined to be insufficient according to the radar reflection intensity, the sonar 712 is selected to provide the most accurate determination of the inter-vehicle distance.

  8A, 8B, and 8C illustrate object recognition during leading vehicle detection and ranging according to an exemplary embodiment of the present disclosure. In each figure, this vehicle 805 is a user vehicle 815. An algorithm or similar process involving CNN or similar approach for object recognition trained on a database of labeled images classifies the preceding vehicle accordingly.

  In the embodiment, as shown in FIG. 8A, the preceding vehicle is a car 823. The car 823 is within both the radar field of view 811 and the sonar field of view 812. The metrics for each data source indicate that the raw data is above a predetermined threshold. Since the preceding car is car 823 and the rear face of the car is roughly flat, the radar's performance limits on the mismatched surface are removed and the radar is considered reliable. As a result, sonar and radar data sources can be merged to determine the inter-vehicle distance.

  In the embodiment, as shown in FIG. 8B, the preceding vehicle is a flat bed trailer truck 821. Flatbed trailer track 821 is within both radar field of view 811 and sonar field of view 812. Further, the metrics for each data source indicate that the raw data is higher than a predetermined threshold. However, from the technical limit point of view of preceding vehicle identification, the radar is unreliable and the inter-vehicle distance should be determined solely from the sonar field of view 812. As shown in FIG. 8B, the preceding vehicle is in the radar field of view 811, but identification of the preceding vehicle as a flatbed trailer truck 821 identifies subtle nuances in the radio waves from which the center-based approach was recovered. Serious in causing things that cannot be done. The rearmost aspect of the flatbed 822 may not be accurately represented in the distance estimation data received from the radar, and therefore may not be reliable even if it is above a predetermined threshold of reflection intensity .

  According to an embodiment, the leading vehicle is a motorcycle 824, as shown in FIG. 8C. The motorcycle 824 is within both the radar field of view 811 and the sonar field of view 812. Further, the metrics for each data source indicate that the raw data is higher than a predetermined threshold. However, from the technical limit point of view of preceding vehicle identification, the radar is unreliable and the inter-vehicle distance should be determined solely from the sonar field of view 812. As shown in FIG. 8C, although the preceding vehicle is in the radar field of view 811, the identification of the preceding vehicle as a motorcycle 824 is incapable of identifying subtle differences in the radio waves collected by the center-based approach. Serious in causing. The aspect behind the motorcycle 824 may not be accurately represented in the distance estimation data from the radar, so it is recommended to estimate the distance according to the sonar data alone. In addition, because motorcycles can change speed rapidly, the identification of the preceding vehicle as the AUDUY 824 better informs the safe inter-vehicle distance and ensures driver and rider safety.

  FIG. 9 is a block diagram of a hardware description of a computer configured to perform detection, ranging and control, according to an exemplary embodiment of the present disclosure. In the embodiment, the hardware is a driving support system control unit. In FIG. 9, the driving support system control unit includes a CPU 985 that executes the processes described above / below. Process data and instructions can be stored in memory 986. These processes and instructions may be stored on a storage media disk 987 such as a hard drive (HDD) or a portable storage medium, or may be stored remotely. Further, the claimed progress is not limited by the form of computer readable media on which the instructions of the inventive process are stored. For example, the instructions are stored on a CD, DVD, FLASH memory, RAM, ROM, PROM, EPROM, EEPROM, hard disk, or any other information processing device such as a server or computer with which the driving support system controller communicates May be.

  Further, the claimed progress has been made to CPU 985 and Microsoft Windows 7®, UNIX®, Solaris®, LINUX®, Apple MAC-OS®, and those skilled in the art. It may be provided as a utility application, background daemon, or operating system component, or a combination thereof, that runs in cooperation with an operating system, such as another known system.

  In order to realize the driving support system control unit, the hardware elements may be realized by various circuit elements known to those skilled in the art. For example, CPU 985 may be a Xenon® or Core processor from Intel®, USA, an Opteron processor from AMD, US, or other processor types that may be recognized by those skilled in the art. Alternatively, CPU 985 may be implemented on an FPGA, ASIC, PLD, or using separate logic circuitry, as will be appreciated by those skilled in the art. Further, the CPU 900 may be implemented as a plurality of processors that cooperatively operate in parallel to execute the instructions of the process of the present invention described above.

  The driving support system control unit in FIG. 9 also includes a network controller 988 such as an Intel Ethernet (registered trademark) PRO network interface card from Intel Corporation in the United States for interface with the network 996. As can be appreciated, the network 996 can be a public network such as the Internet, or a private network such as a LAN or WAN network, or any combination thereof, and can also include a PSTN or ISDN subnetwork. The network 996 can also be wired such as an Ethernet network or wireless such as cellular networks including EDGE, 3G and 4G wireless cellular systems. The wireless network may also be WiFi®, Bluetooth®, or any other wireless form of known communication.

  The driving support system control unit is a display such as an NVIDIA GeForce GTX or Quadro graphics adapter from NVIDIA (USA) for interfacing with a display 990 such as a Hewlett Packard (registered trademark) HPL2445w LCD monitor. A controller 989 is further included. The general purpose I / O interface 991 interfaces with a touch screen panel 992 on or separate from the display 990.

  A sound controller 993, such as Sound Blaster X-Fi Titanium from Creative, is also provided to the driving assistance system controller to provide sound and / or music by interfacing with the speaker 994.

  The general purpose storage controller 995 connects the storage media disk 987 to a communication bus 997 which can be ISA, EISA, VESA, PCI or the like for interconnecting all the components of the driving support system controller. A description of the general features and functionality of the display 990 and display controller 989, storage controller 995, network controller 988, sound controller 993, and general purpose I / O interface 991, as these features are known, are described herein. Omitted for brevity.

  Obviously, many modifications and variations are possible in view of the above teachings. It is therefore to be understood that within the scope of the appended claims, the invention may be practiced otherwise than as specifically described herein.

  Accordingly, the foregoing discussion merely discloses and describes exemplary embodiments of the present invention. As will be appreciated by those skilled in the art, the present invention may be embodied in other specific forms without departing from their spirit or essential characteristics. Accordingly, the disclosure of the present invention is intended to be illustrative and is not intended to limit the scope of the invention and other claims. This disclosure includes some readily recognizable variations of the teachings herein and partially defines the scope of the terms of the appended claims so that the subject matter of the invention is not devoted to the public.

Claims (20)

A method for determining a distance between a vehicle and a preceding object, comprising:
Obtaining a first detection parameter related to the distance between the vehicle and the preceding object via a first detection mode;
Determining, via a processing circuit, the magnitude of the first detection parameter obtained via the first detection mode;
Obtaining a second detection parameter related to the distance between the vehicle and the preceding object via a second detection mode;
Determining the magnitude of the second detection parameter obtained via the second detection mode via a processing circuit;
Selecting the first detection parameter, the second detection parameter, or a combination thereof based on the determination of the magnitude of the first detection parameter and the magnitude of the second detection parameter for a predetermined threshold;
Including a method. Obtaining an image related to the preceding object via a camera;
Classifying the image associated with the preceding object;
Selecting the first detection parameter, the second detection parameter or a combination thereof based on the classification of the image associated with the preceding object;
The method of claim 1, further comprising: The classification is
Further comprising classifying the image based on a classifier;
The classifier is trained on a database, the database being a corpus of reference images, each reference image including an identifiable reference object and associated with a corresponding text description of the reference object in the database. A corpus and a plurality of image query resolution data structures, each comprising a set of records, each record comprising an image descriptor of one of the reference images. Each data structure is searchable using a corresponding one of a set of predetermined search processes to identify the best matching record in the data structure based on the image descriptor. The method according to claim 2.   The method of claim 1, wherein the first detection mode or the second detection mode is one of a plurality of detection modes including radar and sonar.   The method according to claim 1, wherein the first detection parameter or the second detection parameter is one of a plurality of detection parameters including radar reflection intensity and sonar resolution.   The method of claim 2, wherein the camera is a visible light camera, an infrared camera, or a combination thereof.   The method of claim 1, wherein the preceding object is a vehicle. When executed by a processing circuit, the processing circuit includes:
Obtaining a first detection parameter related to the distance between the vehicle and the preceding object via a first detection mode;
Determining the magnitude of the first detection parameter obtained via the first detection mode;
Obtaining a second detection parameter related to the distance between the vehicle and the preceding object via a second detection mode;
Determining the magnitude of the second detection parameter acquired via the second detection mode;
Selecting the first detection parameter, the second detection parameter, or a combination thereof based on the determination of the magnitude of the first detection parameter and the magnitude of the second detection parameter for a predetermined threshold;
A non-transitory computer-readable medium containing executable program instructions for implementing a method comprising: The method
Obtaining an image related to the preceding object via a camera;
Classifying the image associated with the preceding object;
Selecting the first detection parameter, the second detection parameter or a combination thereof based on the classification of the image associated with the preceding object;
The non-transitory computer readable medium of claim 8, further comprising: The classification is
Further comprising classifying the image based on a classifier;
The classifier is trained on a database, the database being a corpus of reference images, each reference image including an identifiable reference object and associated with a corresponding text description of the reference object in the database. A corpus and a plurality of image query resolution data structures, each comprising a set of records, each record comprising an image descriptor of one of the reference images. Each data structure is searchable using a corresponding one of a set of predetermined search processes to identify the best matching record in the data structure based on the image descriptor. The non-transitory computer readable medium of claim 9.   The non-transitory computer-readable medium of claim 8, wherein the first detection mode or the second detection mode is one of a plurality of detection modes including radar and sonar.   The non-transitory computer-readable medium of claim 8, wherein the first detection parameter or the second detection parameter is one of a plurality of detection parameters including radar reflection intensity and sonar resolution.   The non-transitory computer readable medium of claim 9, wherein the camera is a visible light camera, an infrared camera, or a combination thereof.   The non-transitory computer readable medium of claim 8, wherein the preceding object is a vehicle. An apparatus for determining a distance between a vehicle and a preceding object,
Via a first detection mode, obtaining a first detection parameter related to the distance between the vehicle and the preceding object;
Determining the magnitude of the first detection parameter obtained via the first detection mode;
Via a second detection mode, obtaining a second detection parameter related to the distance between the vehicle and the preceding object;
Determining the magnitude of the second detection parameter acquired via the second detection mode;
Selecting the first detection parameter, the second detection parameter, or a combination thereof based on the determination of the magnitude of the first detection parameter and the magnitude of the second detection parameter relative to a predetermined threshold;
An apparatus comprising a processing circuit configured as follows. The processing circuit includes:
Via the camera, obtaining an image related to the preceding object,
Classifying the image associated with the preceding object;
Selecting the first detection parameter, the second detection parameter or a combination thereof based on the classification of the image associated with the preceding object;
The apparatus of claim 15, further configured as follows. The processing circuit includes:
Further configured to classify the image based on a classifier;
The classifier is trained on a database, the database being a corpus of reference images, each reference image including an identifiable reference object and associated with a corresponding text description of the reference object in the database. A corpus and a plurality of image query resolution data structures, each comprising a set of records, each record comprising an image descriptor of one of the reference images. Each data structure is searchable using a corresponding one of a set of predetermined search processes to identify the best matching record in the data structure based on the image descriptor; The apparatus of claim 16.   The apparatus according to claim 15, wherein the first detection mode or the second detection mode is one of a plurality of detection modes including radar and sonar.   The apparatus according to claim 15, wherein the first detection parameter or the second detection parameter is one of a plurality of detection parameters including radar reflection intensity and sonar resolution.   The apparatus of claim 16, wherein the camera is a visible light camera, an infrared camera, or a combination thereof.