US10685447B2 - Method for identifying an object in a region surrounding a motor vehicle, driver assistance system and motor vehicle - Google Patents

Method for identifying an object in a region surrounding a motor vehicle, driver assistance system and motor vehicle Download PDF

Info

Publication number
US10685447B2
US10685447B2 US15/748,098 US201615748098A US10685447B2 US 10685447 B2 US10685447 B2 US 10685447B2 US 201615748098 A US201615748098 A US 201615748098A US 10685447 B2 US10685447 B2 US 10685447B2
Authority
US
United States
Prior art keywords
captured
image
movement sequence
motor vehicle
basis
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US15/748,098
Other languages
English (en)
Other versions
US20180211394A1 (en
Inventor
Nikolai Sergeev
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Valeo Schalter und Sensoren GmbH
Original Assignee
Valeo Schalter und Sensoren GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Valeo Schalter und Sensoren GmbH filed Critical Valeo Schalter und Sensoren GmbH
Assigned to VALEO SCHALTER UND SENSOREN GMBH reassignment VALEO SCHALTER UND SENSOREN GMBH ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SERGEEV, Nikolai
Publication of US20180211394A1 publication Critical patent/US20180211394A1/en
Application granted granted Critical
Publication of US10685447B2 publication Critical patent/US10685447B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/97—Determining parameters from multiple pictures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G—PHYSICS
    • G08—SIGNALLING
    • G08G—TRAFFIC CONTROL SYSTEMS
    • G08G1/00—Traffic control systems for road vehicles
    • G08G1/16—Anti-collision systems
    • G08G1/166—Anti-collision systems for active traffic, e.g. moving vehicles, pedestrians, bikes
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2552/00—Input parameters relating to infrastructure
    • B60W2552/05—Type of road, e.g. motorways, local streets, paved or unpaved roads
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2554/00—Input parameters relating to objects
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2555/00—Input parameters relating to exterior conditions, not covered by groups B60W2552/00, B60W2554/00
    • B60W2555/60—Traffic rules, e.g. speed limits or right of way
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle

Definitions

  • the invention relates to a method for identifying an object in a region surrounding a motor vehicle as a stationary object, wherein in the method, the surrounding region is captured in images using a vehicle-side capture device and the object is detected in at least one of the captured images using an image processing device.
  • the invention additionally relates to a driver assistance system and to a motor vehicle.
  • Said information can be provided to a driver assistance system of the motor vehicle, for example an automated full-beam regulation system.
  • the information extracted from the images can be used, for example, to regulate the full beam emitted by the motor vehicle so that, on the one hand, the region surrounding the motor vehicle, in particular a road of the motor vehicle, is illuminated as well as possible, and, on the other, so that other road users, such as oncoming vehicles, are not dazzled.
  • the captured objects in the region surrounding the motor vehicle initially need to be classified or identified so that oncoming vehicles, for example, are recognized as such in the first place.
  • a method according to the invention serves for identifying an object in a region surrounding a motor vehicle as a stationary object.
  • the surrounding region is captured in images using a vehicle-side capture device and the object is recognized in at least one of the captured images using an image processing device.
  • a first position of the object in the surrounding region relative to the motor vehicle is estimated on the basis of at least one first captured image
  • a movement sequence of the object in image coordinates is determined on the basis of the first image and at least one second captured image
  • a first movement sequence that characterizes a stationary object in the image coordinates is determined proceeding from the first estimated position in the surrounding region
  • the captured object is identified as a stationary object on the basis of a comparison of the movement sequence of the captured object to the first characterizing movement sequence.
  • the method is consequently used to differentiate between whether the objects in the surrounding region are stationary objects or non-stationary objects. If an object is identified using the method as being non-stationary, it is assumed that the object is a dynamic object.
  • the surrounding region in particular the surrounding region located in front of, in particular laterally in front of, the motor vehicle in the driving direction is captured in images using the vehicle-side capture device which comprises, for example, at least one camera.
  • the camera is in particular designed here to two-dimensionally capture the surrounding region.
  • the image processing device is designed to recognize a two-dimensional projection of the object in the at least one captured image from image data of the at least one captured image and to thus recognize the object in the surrounding region.
  • a first position of the object i.e. of the real object
  • the first image can be the image in which the object was recognized by the image processing device.
  • the first position of the real object in the surrounding region can be estimated on the basis of the two-dimensional projection of the object on the first image, for example on the basis of two-dimensional geometric measurements of the object on the image.
  • the first position is here determined in particular in a world coordinate system and describes a first, possible distance of the captured object from the motor vehicle.
  • the world coordinate system can be, for example, a vehicle coordinate system having a first axis along a vehicle lateral direction, a second axis along a vehicle longitudinal direction, and a third axis along a vehicle height direction.
  • the distance of the object from the motor vehicle is thus determined in particular only on the basis of the captured images.
  • the distance is in particular not directly measured. Consequently, the camera can have a particularly simple design and does not need to be an expensive time-of-flight camera, for example.
  • a movement sequence of the object that is to say the projection of the object, in the image coordinates is determined.
  • the image coordinates are here determined in a two-dimensional image coordinate system having a first, for example horizontal, image axis and a second, for example vertical, image axis.
  • the movement sequence in the image coordinates is determined here such that, in the first image, an image position of the object, for example the image position of a projection point of the object, in the image coordinates is determined and, in the at least one second image, an image position of the object, for example the image position of the projection point of the object in the second image, in the image coordinates is determined.
  • the change in the image positions of the object between the two images gives the movement sequence of the object.
  • the image position change of the object that is to say the movement of the projection of the object in the recorded images, is obtained by way of the motor vehicle travelling on a road. Due to the motor vehicle travelling, the image positions of the object in the image change and thus the image coordinates of the object between the at least two images change.
  • the movement sequence of the captured object in the image coordinates here corresponds to an instantaneous movement sequence, or an actual movement sequence, of the projection of the object in image coordinates.
  • the time period between two successively recorded images can be, for example, between 50 ms and 80 ms, in particular 60 ms. As a result, an image position of the object can be determined every 60 ms, for example.
  • the first movement sequence that characterizes a stationary object in the image coordinates is now determined.
  • the first characterizing movement sequence here corresponds to a first predetermined movement sequence that the projection of the object has if the object is a stationary object at the first estimated position.
  • image positions of a specified stationary reference object at the first estimated position can be determined for the at least two images.
  • the movement sequence of the captured object and the first characterizing movement sequence can be determined as respectively one trajectory and be represented, for example, in one of the captured images.
  • the instantaneous movement sequence and the first predetermined movement sequence can then be compared. To this end, for example a distance between the trajectories can be determined in the image. If, for example, the instantaneous movement sequence is congruent with the first predetermined movement sequence, or if the instantaneous movement sequence deviates from the first predetermined movement sequence by at most a specified threshold value, the captured object can be identified as a stationary object. However, if the instantaneous movement sequence deviates from the first predetermined movement sequence by more than the specified threshold value, the captured object is identified as a dynamic object.
  • the method according to the invention can be used therefore to classify or identify in a particularly simple manner the object in the surrounding region from captured images of the surrounding region.
  • a second position of the object in the surrounding region relative to the motor vehicle is estimated preferably on the basis of the at least one first captured image, and, proceeding from the second estimated position in the surrounding region, a second movement sequence that characterizes a stationary object in the image coordinates is determined, and the captured object is identified on the basis of the comparison of the movement sequence of the captured object to the second characterizing movement sequence.
  • a second, possible distance of the object from the motor vehicle is estimated. The second distance is also determined in the world coordinate system.
  • the invention is here based on the finding that it is not possible to accurately determine the distance of the object on the basis of the image that is two-dimensionally captured by the capture device, but can vary in particular along a camera axis. Therefore, two possible, plausible distances of the object from the motor vehicle are estimated.
  • the second characterizing movement sequence here corresponds to a second predetermined movement sequence which the object has if the object is a stationary object at the second estimated position.
  • the second characterizing movement sequence can likewise be determined as a trajectory and be represented in the captured image.
  • the instantaneous movement sequence is compared to the second predetermined movement sequence.
  • the captured object can then be identified as being stationary if the instantaneous movement sequence is congruent with the second predetermined movement sequence or deviates from the second predetermined movement sequence at most by a further specified threshold value.
  • the captured object is identified as a stationary object if the movement sequence of the captured object is within a corridor formed by the first and the second characterizing movement sequences.
  • the first and the second movement sequence in the image coordinates form the corridor, that is to say a movement region within which the captured object moves, if it is a stationary object.
  • a first height that is characteristic of the stationary object is specified for the captured object and the first position in the surrounding region relative to the motor vehicle is determined on the basis of the at least one first image and on the basis of the first height specified for the object
  • a second height that is characteristic of the stationary object is specified for the captured object and the second position in the surrounding region relative to the motor vehicle is determined on the basis of the at least one second image and on the basis of the second height specified for the object.
  • the first height here corresponds to a maximum height that the predetermined stationary reference object can have, for example, and the second height corresponds to a minimum height that the predetermined stationary reference object can have.
  • the minimum and the maximum heights can be stored, for example, for the image processing device, with the result that the positions of the object in world coordinates can be determined simply and quickly on the basis of the at least one captured first image using the stored heights. Consequently, the distances of the object from the motor vehicle can be plausibly and quickly determined, in particular without the need to directly measure the distance. It is thus possible to dispense with a separate sensor device for distance measurement and/or the configuration of the cameras as a time-of-flight camera.
  • a vehicle speed and/or an angular rate of the motor vehicle may be captured to determine a characterizing movement sequence.
  • the vehicle speed and/or the angular rate it is thus possible to determine the position of the object, proceeding from the first estimated position and the second estimated position for each time point at which an image is captured, in the case of a stationary object in the world coordinate system, that is to say for example the position of the reference object in the surrounding region relative to the motor vehicle, and to convert it into the image positions in the corresponding image coordinates of the image captured at that time point.
  • the characterizing movement sequences are determined under the assumption that the motor vehicle travels on an even road.
  • An even road is here a road that has neither bumps nor potholes. That means that it is assumed that the motor vehicle performs no pitch movement, i.e. no rotational movement about the vehicle transverse axis, during travelling. In other words, a pitch angle of the motor vehicle does not change during travel.
  • the characterizing movement sequences can be determined without the need to go to the effort of capturing the pitch angle of the motor vehicle.
  • only image coordinates of a predetermined direction are compared during the comparison of the movement sequence of the captured object to the characterizing movement sequence.
  • Preferably, only image coordinates in the horizontal direction are compared to one another.
  • the invention is here based on the finding that, even if the captured object is a stationary object, the vertical image coordinates of the captured object would differ significantly from the vertical image coordinates of the characterizing movement sequences if the motor vehicle travels on an uneven road having bumps. Due to the uneven road, the motor vehicle performs a pitch movement during travel, as a result of which the captured object appears to move in the vertical direction in the image.
  • This apparent vertical movement of the captured object in the image coordinates can result in the movement sequence of the captured object, determined on the basis of the image positions of the projection of the object, being outside the corridor formed by the two characterizing movement sequences.
  • the characterizing movement sequences are specifically determined in particular under the assumption that the motor vehicle moves along an even road having no bumps. This means that the apparent vertical movement of the stationary object is not reflected in the characterizing movement sequences. For this reason, during the examination as to whether the movement sequence of the captured object is within the corridor, only the image coordinates in the horizontal direction are taken into consideration. The image coordinates in the vertical direction are not taken into consideration.
  • an object can be reliably identified as a stationary object in particular without the need to separately capture the pitch angle even in the case of a road on which the motor vehicle performs a pitch movement, for example due to potholes.
  • the position of the captured object is estimated on the basis of a first image captured at a current time point, and the movement sequence of the captured object is determined on the basis of at least one second image captured at a time point before the current time point. That means that the current position of the object in the world coordinates is estimated and, proceeding from the current position, the already performed movement of the captured object is determined.
  • the movement sequence of the captured object is determined retrospectively, i.e. on the basis of the recorded history of image coordinates.
  • the characterizing movement sequence is also hereby retrospectively determined, i.e. a movement that the captured object would have performed if it were a stationary object.
  • the object can be classified particularly quickly as a stationary object or as a dynamic object.
  • the object as a stationary, light-reflecting object, in particular a delineator post or a road sign, is preferably identified as the predetermined object.
  • a light-reflecting object is here understood to mean an object that has an active element reflecting light.
  • Such an element can be, for example, a reflector which reflects the light, such as for example like a cat's-eye.
  • Such objects can be, for example, delineator posts or reflector posts, which are generally located at a side of the road of the motor vehicle and facilitate orientation for the driver of the motor vehicle in particular in the dark or at night or when visibility is poor.
  • Such objects can also be road signs that actively reflect light for improved visibility.
  • the object can be identified as a stationary light-reflecting object, in particular as a delineator post and/or as a road sign, if it has been detected, on the basis of the at least one captured first image and/or second image, that the captured object emits light in a directional or defined manner in the direction of the motor vehicle and it has been detected, on the basis of the comparison of the movement sequence of the captured light-emitting object to a movement sequence characteristic of a stationary object, that the captured object is stationary.
  • it is determined, for example on the basis of the first detected image, whether the object emits light in a directional manner in the direction of the motor vehicle. It is also possible to estimate the position of the object on the basis of this image.
  • the movement sequence of the captured light-emitting object can be determined and the object can be identified as being stationary on the basis of the comparison to the characterizing movement sequence.
  • regulation of a light emitted by a headlight of the motor vehicle for illuminating a road of the motor vehicle is blocked if the object was identified as a stationary object which actively reflects light.
  • no automatic full-beam regulation for example dipping of the light, is performed if it has been found that the captured object is not an oncoming vehicle but for example a delineator post or a road sign. It is thus possible to prevent the automatic full-beam regulation from being performed unnecessarily.
  • the invention additionally relates to a driver assistance system, in particular for full-beam regulation, for identifying an object in a region surrounding a motor vehicle as a stationary object.
  • the driver assistance system comprises a vehicle-side capture device, for example a camera, for capturing the surrounding region in images and an image processing device for recognizing the object in at least one of the captured images.
  • the image processing device is configured to estimate a first position of the object in the surrounding region relative to the motor vehicle on the basis of at least one first captured image, to determine a movement sequence of the object in image coordinates on the basis of the first image and at least one second captured image, to determine a first movement sequence that characterizes a stationary object in the image coordinates proceeding from the first estimated position in the surrounding region, and to identify the captured object as a stationary object on the basis of a comparison of the movement sequence of the captured object to the first characterizing movement sequence.
  • a motor vehicle according to the invention comprises a driver assistance system according to the invention.
  • the motor vehicle is configured in particular as a passenger vehicle.
  • the driver assistance system can here control, for example, headlights of the motor vehicle to regulate the light emitted by the headlights.
  • top”, “bottom”, “front”, “rear”, “horizontal” (a-direction), “vertical” (b-direction) etc. indicate positions and orientations based on appropriate observation of the captured images.
  • the terms “internal”, “external”, “lateral”, “right”, “left”, “top”, “bottom”, “vehicle height axis” (z-direction), “vehicle longitudinal axis” (y-direction), “vehicle transverse axis” (x-direction) etc. indicate positions and orientations based on an observer standing in front of the vehicle and looking in the direction of the vehicle longitudinal axis.
  • FIG. 1 shows a schematic illustration of an embodiment of a motor vehicle according to the invention
  • FIGS. 2 a , 2 b , 2 c show schematic illustrations of image which were recorded by a vehicle-side capture device
  • FIG. 3 shows a schematic illustration of an embodiment or determining the position of a captured object.
  • FIG. 1 shows a motor vehicle 1 having a driver assistance system 2 .
  • the driver assistance system 2 comprises a vehicle-side capture device 3 , which can comprise at least one camera.
  • the capture device 3 serves for capturing a surrounding region 4 of the motor vehicle 1 in two-dimensional images 12 , 14 (see FIGS. 2 a , 2 b , 2 c ).
  • the capture device captures in particular images 12 , 14 of a surrounding region 4 located—in the direction of travel (y-direction)—in front of and laterally in front of the motor vehicle 1 as the motor vehicle 1 travels on a road 9 .
  • the driver assistance system 2 additionally comprises an image processing device 5 , which is configured to process the images 12 , 14 of the surrounding region 4 captured by the capture device 3 and to extract for example information from the captured images 12 , 14 .
  • the driver assistance system 2 which serves in particular for full-beam regulation, can control, based on the information extracted from the images 12 , 14 , headlights 11 of the motor vehicle 1 in order to regulate or influence the light emitted by the headlights 11 .
  • the driver assistance system 2 can have a speed sensor 6 for capturing a speed of the motor vehicle 1 travelling on the road 9 and an angular rate sensor 7 for capturing an angular rate of the motor vehicle 1 about a vehicle height axis (z-direction) of the motor vehicle 1 .
  • the image processing device 5 is configured for recognizing an object 8 in at least one of the images 12 , 14 captured by the capture device 3 .
  • the object 8 has, in the surrounding region 4 , a position P(x, y, z) relative to the motor vehicle 1 .
  • the position P is here a position in a world coordinate system x, y, z, which is here defined as a vehicle coordinate system.
  • the x-axis of the world coordinate system here runs along a vehicle lateral direction
  • the y-axis runs along a vehicle longitudinal direction
  • the z-axis runs along a vehicle height direction.
  • the image processing device 5 is configured to classify the object 8 , that is to say to identify whether the object 8 is a stationary object that is positionally fixed in the surrounding region 4 , or a dynamic object. If the captured object 8 is a dynamic object, for example in the form of an oncoming vehicle, a control device 10 of the driver assistance system 2 can, for example, control the headlights 11 of the motor vehicle 1 to dip the light emitted by the headlights 11 and thus prevent dazzling a driver of the oncoming vehicle. Regulating the light emitted by the headlights 11 in this way should, however, be blocked if the image processing device 5 has detected that the object 8 is a stationary object, for example in the form of a delineator post 13 or a road sign.
  • FIG. 2 a shows an image 12 of the surrounding region 4 recorded at a first time point by the capture device 3 .
  • the first image 12 shows the surrounding region 4 ′ as a two-dimensional projection of the surrounding region 4 , the road 9 ′ as a two-dimensional projection of the road 9 , and the object 8 ′ as a two-dimensional projection of the object 8 at the first, in particular current, time point.
  • An image position P′ of the object 8 ′ is indicated using image coordinates a, b of a two-dimensional image coordinate system.
  • the object 8 ′ has in the image 12 a first image position P′(a1, b1). Based on the two-dimensional first image 12 , a first position P 1 ( x 1, y1, z1) of the real object 8 (see FIG. 3 ) relative to the motor vehicle 1 , that is to say a first, possible distance of the object 8 from the motor vehicle 1 , and a second position P 2 ( x 2, y2, z2) of the real object 8 relative to the motor vehicle 1 , that is to say a second, possible distance of the object 8 from the motor vehicle 1 , are estimated.
  • the positions P 1 , P 2 are here determined in the world coordinate system (x, y, z).
  • a first, maximum height h 1 for the object 8 is specified or assumed, as shown in FIG. 3 .
  • a second, minimum height h 2 for the object 8 is assumed.
  • the possible positions P 1 , P 2 of the real object 8 can then be determined on the basis of the image 12 and the heights h 1 , h 2 .
  • the actual position P(x, y, z) of the object 8 is here located along a camera axis 20 between the two positions P 1 , P 2 .
  • the heights h 1 , h 2 can here be, for example, a typical specified maximum height and a typical specified minimum height of a delineator post 13 .
  • a movement sequence 15 (see FIG. 2 c ) of the object 8 ′, 8 ′′, that is to say a movement sequence 15 of the projection of the captured object 8 , in the image coordinates a, b is determined.
  • a movement sequence 15 of the projection of the captured object 8 in the image coordinates a, b is determined.
  • an image position P′ of the object 8 ′ in the first image 12 and an image position P′′(a2, b2) of the object 8 ′′ in a second image 14 , recorded at a second time point can be determined.
  • the second image 14 shows the surrounding region 4 ′′ as a two-dimensional projection of the surrounding region 4 , the road 9 ′′ as a two-dimensional projection of the road 9 , and the object 8 ′′ as a two-dimensional projection of the object 8 at the second time point.
  • the second time point here occurs in particular before the first, current time point.
  • the change in the image positions P′, P′′ can be determined as a trajectory characterizing the movement sequence 15 and be represented, for example, in the first image 12 (see FIG. 2 c ).
  • movement sequences 16 , 17 which are characteristic of a stationary object, in image coordinates a, b are determined.
  • the first characterizing movement sequence 16 is here characteristic of a stationary object at the first, estimated position P 1
  • the second characteristic movement sequence 17 is characteristic of a stationary object at the second, estimated position P 2 .
  • the characterizing movement sequences 16 , 17 are determined for example by determining the image positions of a stationary reference object in the images 12 , 14 in image coordinates a, b. To this end, for the image positions of the stationary reference object in the first image 12 , the first position P 1 and the second position P 2 are converted into image coordinates a, b.
  • a further, first position of the stationary reference object in world coordinates x, y, z and a further, second position of the stationary reference object in world coordinates x, y, z, which the reference object has at the second time point, are converted into image coordinates a, b.
  • the further, first and the further, second positions can be determined, for example, on the basis of the speed and/or the angular rate of the motor vehicle 1 , which is captured using the speed sensor 6 and/or using the angular rate sensor 7 .
  • the characterizing movement sequences 16 , 17 are determined under the assumption that the road 9 , on which the motor vehicle 1 travels, is even and has no potholes or bumps, for example.
  • the movement sequences 16 , 17 which can likewise be represented as trajectories in the first image 12 (see FIG. 2 c ), form a corridor 18 .
  • the captured object 8 is then identified as a stationary object if the movement sequence 15 of the captured object 8 ′, 8 ′′ falls within the corridor 18 .
  • FIG. 2 c shows that the horizontal coordinates a of the movement sequence 15 are located within the corridor 18 , but not the vertical coordinates b of the movement sequence 15 .
  • the movement sequence 15 shows the movement sequence of the object 8 ′, 8 ′′ for the motor vehicle 1 here as it travels on a road 9 , which has bumps. Due to these bumps, the motor vehicle 1 performs a pitch movement about the vehicle transverse axis, that is to say a pitch movement along the z-axis. Due to said pitch movement, the object 8 ′, 8 ′′ appears to move in the height direction (b-direction) in the image coordinates a, b.
  • the captured object 8 can be identified as a stationary object.
  • the delineator post 13 it is additionally possible to examine whether the object 8 actively reflects light in the direction of the motor vehicle 1 .
  • the delineator post 13 has reflectors 19 .
  • the image processing device 5 can, for example on the basis of the first image 12 , detect that the object 8 ′, 8 ′′ emits light in the direction of the motor vehicle 1 .
  • the image processing device 5 can capture the object 8 as a stationary object on the basis of the movement sequences 15 , 16 , 17 and thus identify the object 8 correctly as the delineator post 13 .

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Automation & Control Theory (AREA)
  • Mathematical Physics (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)
  • Lighting Device Outwards From Vehicle And Optical Signal (AREA)
US15/748,098 2015-07-28 2016-07-26 Method for identifying an object in a region surrounding a motor vehicle, driver assistance system and motor vehicle Active 2037-03-17 US10685447B2 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
DE102015112289.8A DE102015112289A1 (de) 2015-07-28 2015-07-28 Verfahren zum Identifizieren eines Objekts in einem Umgebungsbereich eines Kraftfahrzeugs, Fahrerassistenzsystem sowie Kraftfahrzeug
DE102015112289.8 2015-07-28
DE102015112289 2015-07-28
PCT/EP2016/067715 WO2017017077A1 (de) 2015-07-28 2016-07-26 Verfahren zum identifizieren eines objekts in einem umgebungsbereich eines kraftfahrzeugs, fahrerassistenzsystem sowie kraftfahrzeug

Publications (2)

Publication Number Publication Date
US20180211394A1 US20180211394A1 (en) 2018-07-26
US10685447B2 true US10685447B2 (en) 2020-06-16

Family

ID=56511587

Family Applications (1)

Application Number Title Priority Date Filing Date
US15/748,098 Active 2037-03-17 US10685447B2 (en) 2015-07-28 2016-07-26 Method for identifying an object in a region surrounding a motor vehicle, driver assistance system and motor vehicle

Country Status (7)

Country Link
US (1) US10685447B2 (de)
EP (1) EP3329462B1 (de)
JP (1) JP6661748B2 (de)
KR (1) KR102052840B1 (de)
CN (1) CN108027971B (de)
DE (1) DE102015112289A1 (de)
WO (1) WO2017017077A1 (de)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220067400A1 (en) * 2020-08-25 2022-03-03 Hyundai Motor Company Vehicle and control method thereof
US20220089144A1 (en) * 2020-09-24 2022-03-24 Toyota Jidosha Kabushiki Kaisha Home position estimation system and home position estimation method
US11704827B2 (en) 2019-03-29 2023-07-18 Samsung Electronics Co., Ltd. Electronic apparatus and method for assisting with driving of vehicle

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102016118538A1 (de) 2016-09-29 2018-03-29 Valeo Schalter Und Sensoren Gmbh Verfahren zum Klassifizieren eines Verkehrszeichens in einem Umgebungsbereich eines Kraftfahrzeugs, Rechenvorrichtung, Fahrerassistenzsystem sowie Kraftfahrzeug
DE102017217008A1 (de) * 2017-09-26 2019-03-28 Robert Bosch Gmbh Verfahren zum Ermitteln der Steigung einer Fahrbahn
DE102018215136B4 (de) * 2018-09-06 2021-03-25 Robert Bosch Gmbh Verfahren zum Auswählen eines Bildausschnitts eines Sensors
CN112930557B (zh) * 2018-09-26 2025-09-02 相干逻辑公司 任何世界视图生成
CN110223511A (zh) * 2019-04-29 2019-09-10 合刃科技（武汉）有限公司 一种汽车路边违停智能监测方法及系统
DE102019210300B4 (de) * 2019-07-11 2024-11-28 Volkswagen Aktiengesellschaft Verfahren und Vorrichtung zum kamerabasierten Bestimmen eines Abstandes eines bewegten Objektes im Umfeld eines Kraftfahrzeugs
CN111353453B (zh) * 2020-03-06 2023-08-25 北京百度网讯科技有限公司 用于车辆的障碍物检测方法和装置
DE102020111471A1 (de) 2020-04-27 2021-10-28 Bayerische Motoren Werke Aktiengesellschaft Verfahren und System zur Bilderkennung für ein automatisiert fahrendes Fahrzeug
CN116583889B (zh) * 2020-11-27 2025-02-18 日产自动车株式会社 车辆辅助方法及车辆辅助装置
KR20240173258A (ko) * 2023-06-01 2024-12-11 현대자동차주식회사 차량 외부의 객체 위치 판단 장치 및 그 방법
CN120507118B (zh) * 2025-06-03 2025-12-09 河南永泰光电有限公司 一种车载光学镜头的性能评估方法及系统

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102007032075A1 (de) 2006-07-28 2008-01-31 Denso Corp., Kariya System, Verfahren und Programmprodukt zur Erfassung von Fahrzeugen
DE102007054048A1 (de) 2007-11-13 2009-05-14 Daimler Ag Verfahren und Vorrichtung für eine Fahrlichtsteuerung eines Fahrzeugs
EP2127945A2 (de) 2008-05-28 2009-12-02 Hella KGaA Hueck & Co. Verfahren und Vorrichtung zum Steuern der Lichtabgabe eines Fahrzeugs
DE102008063328A1 (de) 2008-12-30 2010-07-01 Hella Kgaa Hueck & Co. Verfahren und Vorrichtung zum Ermitteln einer Änderung des Nickwinkels einer Kamera eines Fahrzeugs
JP2011081613A (ja) 2009-10-07 2011-04-21 Denso It Laboratory Inc 画像処理装置
JP2011192226A (ja) 2010-03-17 2011-09-29 Hitachi Automotive Systems Ltd 車載用環境認識装置及び車載用環境認識システム
US20140104408A1 (en) * 2012-10-17 2014-04-17 Denso Corporation Vehicle driving assistance system using image information
US20150310621A1 (en) * 2012-10-29 2015-10-29 Hitachi Automotive Systems, Ltd. Image Processing Device
US20170270370A1 (en) * 2014-09-29 2017-09-21 Clarion Co., Ltd. In-vehicle image processing device
US20180118100A1 (en) * 2016-10-28 2018-05-03 Volvo Car Corporation Road vehicle turn signal assist system and method

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4052650B2 (ja) * 2004-01-23 2008-02-27 株式会社東芝 障害物検出装置、方法及びプログラム
JP4244887B2 (ja) 2004-08-31 2009-03-25 日産自動車株式会社 移動位置予測装置、移動位置予測方法、衝突判定装置、及び衝突判定方法
DE102005017422A1 (de) * 2005-04-15 2006-10-19 Robert Bosch Gmbh Fahrerassistenzsystem mit Einrichtung zur Erkennung von stehenden Objekten
JP5310235B2 (ja) * 2009-04-27 2013-10-09 株式会社豊田中央研究所 車載用照明制御装置及びプログラム
EP2418123B1 (de) * 2010-08-11 2012-10-24 Valeo Schalter und Sensoren GmbH Verfahren und System zum Unterstützen eines Fahrers eines Fahrzeugs beim Lenken des Fahrzeugs auf einer Fahrstraße und tragbares Kommunikationsgerät
DE102010034139A1 (de) * 2010-08-12 2012-02-16 Valeo Schalter Und Sensoren Gmbh Verfahren zur Unterstützung eines Parkvorgangs eines Kraftfahrzeugs, Fahrerassistenzsystem und Kraftfahrzeug

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102007032075A1 (de) 2006-07-28 2008-01-31 Denso Corp., Kariya System, Verfahren und Programmprodukt zur Erfassung von Fahrzeugen
DE102007054048A1 (de) 2007-11-13 2009-05-14 Daimler Ag Verfahren und Vorrichtung für eine Fahrlichtsteuerung eines Fahrzeugs
EP2127945A2 (de) 2008-05-28 2009-12-02 Hella KGaA Hueck & Co. Verfahren und Vorrichtung zum Steuern der Lichtabgabe eines Fahrzeugs
DE102008063328A1 (de) 2008-12-30 2010-07-01 Hella Kgaa Hueck & Co. Verfahren und Vorrichtung zum Ermitteln einer Änderung des Nickwinkels einer Kamera eines Fahrzeugs
JP2011081613A (ja) 2009-10-07 2011-04-21 Denso It Laboratory Inc 画像処理装置
JP2011192226A (ja) 2010-03-17 2011-09-29 Hitachi Automotive Systems Ltd 車載用環境認識装置及び車載用環境認識システム
EP2549457A1 (de) 2010-03-17 2013-01-23 Hitachi Automotive Systems, Ltd. Vorrichtung an bord eines fahrzeuges zur erkennung von fahrzeugumgebungen und system an bord eines fahrzeuges zur erkennung von fahrzeugumgebungen
US9064418B2 (en) * 2010-03-17 2015-06-23 Hitachi Automotive Systems, Ltd. Vehicle-mounted environment recognition apparatus and vehicle-mounted environment recognition system
US20140104408A1 (en) * 2012-10-17 2014-04-17 Denso Corporation Vehicle driving assistance system using image information
US20150310621A1 (en) * 2012-10-29 2015-10-29 Hitachi Automotive Systems, Ltd. Image Processing Device
US20170270370A1 (en) * 2014-09-29 2017-09-21 Clarion Co., Ltd. In-vehicle image processing device
US20180118100A1 (en) * 2016-10-28 2018-05-03 Volvo Car Corporation Road vehicle turn signal assist system and method

Non-Patent Citations (5)

* Cited by examiner, † Cited by third party
Title
German Search Report issued in corresponding Application No. 10 2015 112 289.8 dated Jul. 21, 2016 (10 pages).
International Search Report of the International Searching Authority issued in PCT/EP2016/067715 dated Nov. 8, 2016 (4 pages).
Notice of Preliminary Rejection issued in corresponding Korean Patent Application No. 2018-7005670, dated Apr. 17, 2019 (11 pages).
Notification of Reason for Rejection issued in corresponding Japanese Patent Application No. 2018-504280, dated Apr. 19, 2019 (5 pages).
Written Opinion of the International Searching Authority issued in PCT/EP2016/067715 dated Nov. 8, 2016 (6 pages).

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11704827B2 (en) 2019-03-29 2023-07-18 Samsung Electronics Co., Ltd. Electronic apparatus and method for assisting with driving of vehicle
US20220067400A1 (en) * 2020-08-25 2022-03-03 Hyundai Motor Company Vehicle and control method thereof
US11961309B2 (en) * 2020-08-25 2024-04-16 Hyundai Motor Company Vehicle and control method thereof
US20220089144A1 (en) * 2020-09-24 2022-03-24 Toyota Jidosha Kabushiki Kaisha Home position estimation system and home position estimation method
US11772631B2 (en) * 2020-09-24 2023-10-03 Toyota Jidosha Kabushiki Kaisha Home position estimation system and home position estimation method

Also Published As

Publication number Publication date
CN108027971B (zh) 2022-03-01
CN108027971A (zh) 2018-05-11
KR102052840B1 (ko) 2019-12-05
JP2018524741A (ja) 2018-08-30
KR20180035851A (ko) 2018-04-06
DE102015112289A1 (de) 2017-02-02
JP6661748B2 (ja) 2020-03-11
EP3329462B1 (de) 2019-06-26
US20180211394A1 (en) 2018-07-26
WO2017017077A1 (de) 2017-02-02
EP3329462A1 (de) 2018-06-06

Similar Documents

Publication Publication Date Title
US20180211394A1 (en) Method for identifying an object in a region surrounding a motor vehicle, driver assistance system and motor vehicle
JP5207146B2 (ja) 遠距離灯の自動制御方法
US20190012537A1 (en) Method for identifying an object in a surrounding region of a motor vehicle, driver assistance system and motor vehicle
US10509973B2 (en) Onboard environment recognition device
JP4993322B2 (ja) 車両周辺にある光点のカメラによる識別及び分類
JP5363085B2 (ja) 前照灯制御装置
US8310662B2 (en) Method for detecting misalignment of a vehicle headlight using a camera
JP5962850B2 (ja) 信号認識装置
US20130027511A1 (en) Onboard Environment Recognition System
US20170144585A1 (en) Vehicle exterior environment recognition apparatus
US20120275172A1 (en) Vehicular headlight apparatus
CN104732233A (zh) 用于识别对象反射的方法和装置
US9372112B2 (en) Traveling environment detection device
US9376052B2 (en) Method for estimating a roadway course and method for controlling a light emission of at least one headlight of a vehicle
JP6756507B2 (ja) 環境認識装置
US10896337B2 (en) Method for classifying a traffic sign, or road sign, in an environment region of a motor vehicle, computational apparatus, driver assistance system and motor vehicle
US8965142B2 (en) Method and device for classifying a light object located ahead of a vehicle
DE102006055906A1 (de) Verfahren zur Erkennung von Reflektoren, Schildern und Fahrzeuglichtern
KR101180676B1 (ko) 차량의 영상인식 기반 상향등 자동 제어 방법

Legal Events

Date Code Title Description
FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

AS Assignment

Owner name: VALEO SCHALTER UND SENSOREN GMBH, GERMANY

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SERGEEV, NIKOLAI;REEL/FRAME:045074/0678

Effective date: 20180209

STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4