CN112622774A - 车辆及提供车辆周围信息的方法 - Google Patents

车辆及提供车辆周围信息的方法 Download PDF

Info

Publication number
CN112622774A
CN112622774A CN202011000383.4A CN202011000383A CN112622774A CN 112622774 A CN112622774 A CN 112622774A CN 202011000383 A CN202011000383 A CN 202011000383A CN 112622774 A CN112622774 A CN 112622774A
Authority
CN
China
Prior art keywords
obstacle
sensor
vehicle
image
trajectory
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202011000383.4A
Other languages
English (en)
Inventor
朴基远
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hyundai Motor Co
Kia Corp
Original Assignee
Hyundai Motor Co
Kia Motors Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hyundai Motor Co, Kia Motors Corp filed Critical Hyundai Motor Co
Publication of CN112622774A publication Critical patent/CN112622774A/zh
Pending legal-status Critical Current

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R11/00—Arrangements for holding or mounting articles, not otherwise provided for
    • B60R11/04—Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/25—Fusion techniques
    • G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
    • G06F18/256—Fusion techniques of classification results, e.g. of results related to same input data of results relating to different input data, e.g. multimodal recognition
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/02—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to ambient conditions
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
    • B60K35/28—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
    • B60K35/29—Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/12—Mirror assemblies combined with other articles, e.g. clocks
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B60R1/27—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view providing all-round vision, e.g. using omnidirectional cameras
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R21/00—Arrangements or fittings on vehicles for protecting or preventing injuries to occupants or pedestrians in case of accidents or other traffic risks
    • B60R21/01—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents
    • B60R21/013—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents including means for detecting collisions, impending collisions or roll-over
    • B60R21/0134—Electrical circuits for triggering passive safety arrangements, e.g. airbags, safety belt tighteners, in case of vehicle accidents or impending vehicle accidents including means for detecting collisions, impending collisions or roll-over responsive to imminent contact with an obstacle, e.g. using radar systems
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/08—Active safety systems predicting or avoiding probable or impending collision or attempting to minimise its consequences
    • B60W30/095—Predicting travel path or likelihood of collision
    • B60W30/0956—Predicting travel path or likelihood of collision the prediction being responsive to traffic or environmental parameters
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/18—Propelling the vehicle
    • B60W30/18009—Propelling the vehicle related to particular drive situations
    • B60W30/18036—Reversing
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G06V10/809—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of classification results, e.g. where the classifiers operate on the same input data
    • G06V10/811—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level of classification results, e.g. where the classifiers operate on the same input data the classifiers operating on different input data, e.g. multi-modal recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/16—Type of output information
    • B60K2360/173—Reversing assist
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/16—Type of output information
    • B60K2360/176—Camera images
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/16—Type of output information
    • B60K2360/177—Augmented reality
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/16—Type of output information
    • B60K2360/178—Warnings
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/16—Type of output information
    • B60K2360/179—Distances to obstacles or vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/18—Information management
    • B60K2360/186—Displaying information according to relevancy
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/18—Information management
    • B60K2360/188—Displaying information using colour changes
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/18—Information management
    • B60K2360/191—Highlight information
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/12—Mirror assemblies combined with other articles, e.g. clocks
    • B60R2001/1253—Mirror assemblies combined with other articles, e.g. clocks with cameras, video cameras or video screens
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
    • B60R2300/105—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using multiple cameras
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/301—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing combining image information with other obstacle sensor information, e.g. using RADAR/LIDAR/SONAR sensors for estimating risk of collision
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/303—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using joined images, e.g. multiple camera images
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/307—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing virtually distinguishing relevant parts of a scene from the background of the scene
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/307—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing virtually distinguishing relevant parts of a scene from the background of the scene
    • B60R2300/308—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing virtually distinguishing relevant parts of a scene from the background of the scene by overlaying the real scene, e.g. through a head-up display on the windscreen
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/806—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for aiding parking
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/8093—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for obstacle warning
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W2050/143—Alarm means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W2050/146—Display means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/40—Photo, light or radio wave sensitive means, e.g. infrared sensors
    • B60W2420/403—Image sensing, e.g. optical camera
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2420/00—Indexing codes relating to the type of sensors based on the principle of their operation
    • B60W2420/54—Audio sensitive means, e.g. ultrasound
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2552/00—Input parameters relating to infrastructure
    • B60W2552/50—Barriers
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2554/00—Input parameters relating to objects
    • B60W2554/20—Static objects
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2554/00—Input parameters relating to objects
    • B60W2554/80—Spatial relation or speed relative to objects
    • B60W2554/801—Lateral distance
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2554/00—Input parameters relating to objects
    • B60W2554/80—Spatial relation or speed relative to objects
    • B60W2554/802—Longitudinal distance
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2556/00—Input parameters relating to data
    • B60W2556/35—Data fusion
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W30/00—Purposes of road vehicle drive control systems not related to the control of a particular sub-unit, e.g. of systems using conjoint control of vehicle sub-units
    • B60W30/06—Automatic manoeuvring for parking
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60Y—INDEXING SCHEME RELATING TO ASPECTS CROSS-CUTTING VEHICLE TECHNOLOGY
    • B60Y2302/00—Responses or measures related to driver conditions
    • B60Y2302/03—Actuating a signal or alarm device
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30261—Obstacle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30264—Parking

Landscapes

  • Engineering & Computer Science (AREA)
  • Mechanical Engineering (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Automation & Control Theory (AREA)
  • Transportation (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Databases & Information Systems (AREA)
  • Human Computer Interaction (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Chemical & Material Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • Combustion & Propulsion (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Traffic Control Systems (AREA)

Abstract

本公开涉及一种提供车辆周围信息的方法，该方法包括：利用通过周围环境传感器获取的传感器检测信息，检测至少一个基于传感器的障碍物；利用通过多个摄像头获取的图像信息，检测至少一个基于图像的障碍物；从至少一个基于传感器的障碍物和至少一个基于图像的障碍物中检测至少一个匹配的障碍物；以及利用针对至少一个匹配的障碍物的基于传感器的检测结果或基于图像的检测结果中的至少一个，校正至少一个匹配的障碍物的位置。

Description

车辆及提供车辆周围信息的方法
技术领域
本公开涉及一种车辆及提供车辆周围信息的方法，尤其涉及一种提供周围图像和障碍物信息的车辆及提供车辆周围信息的方法。
背景技术
该部分中的描述仅提供与本公开有关的背景信息，并且可以不构成现有技术。
传统上，车辆配备有后超声波传感器或后摄像头以检测周围障碍物。最近开发的车辆不仅配备有前超声波传感器，而且配备有全景式监控影像系统(Around ViewMonitor,AVM)或全景影像系统(SurroundView monitor,SVM)，以向乘员提供实时全方位图像。
在与车辆周围物体的碰撞感测变得普遍的情况下，越来越多的驾驶员将依赖于防撞警报声和全方位图像。用于感测车辆周围的障碍物的装置的基本功能是防止障碍物与车身之间的碰撞。因此，在一般情况下，障碍物感测装置不能感测从车身突出的部分(例如后视镜)的碰撞。因此，当驾驶员在没有特别注意的情况下仅利用障碍物感测装置来停车时，经常发生诸如后视镜与柱子的碰撞之类的碰撞，从而造成损害。将参照图1A至图1C对此进行描述。
图1A、1B和1C是示出普通车辆中发生后视镜碰撞的形式的视图。
参照图1A，当车辆10的挡位变为R挡时，后超声波传感器被激活。当位于车辆的侧后方的柱子20进入到后超声波传感器的感测范围30内时，产生警报声，因此驾驶员可以容易地识别柱子20。
在车辆10配备有SVM系统的情况下，如图1B所示，可以通过设置在车辆中的显示器输出车辆周围图像40。参照图1B，车辆的俯视图10'图像显示在图像的中间，并且显示包括柱子20'的车辆周围的全方位图像。另外，根据车辆的转向角的车辆的预测移动轨迹(estimated traveltrajectory)50可以进一步显示在车辆周围图像40中。
然而，我们已经发现，在这种普通车辆中，当障碍物在超声波传感器的感测范围之外时，不输出与该障碍物相关的警报声。尽管SVM系统提供了车辆的预测移动轨迹50，但是SVM系统基于车身生成预测移动轨迹50，因此没有考虑从车身突出的任何其它部分。例如，如图1C所示，没有提供从车身侧向突出的后视镜的预测轨迹60。因此，即使驾驶员利用由SVM系统提供的车辆的预测移动轨迹50驾驶车辆以避免车身与柱子20之间的碰撞，也经常发生后视镜与柱子20的碰撞。
特别地，在后视镜配备有侧中继器或用于SVM系统的摄像头的情况下，维修成本非常高。此外，在最近开发的车辆中，许多情况下难以单独地更换后视镜，需要更换包括后视镜的部件的组件，这增加了车主的经济负担。
发明内容
本公开提供一种车辆及提供车辆周围信息的方法，能够更有效地向驾驶员提供车辆周围信息。
然而，本公开要实现的目的不限于上述目的，并且以下示例性形式所属领域的技术人员将清楚地理解未提及的其它目的。
在本公开的一种形式中，一种提供车辆周围信息的方法可以包括：利用通过周围环境传感器获取的传感器检测信息，检测至少一个基于传感器的障碍物；利用通过多个摄像头获取的图像信息，检测至少一个基于图像的障碍物；从至少一个基于传感器的障碍物和至少一个基于图像的障碍物中检测至少一个匹配的障碍物；以及利用针对至少一个匹配的障碍物的基于传感器的检测结果或基于图像的检测结果中的至少一个，校正至少一个匹配的障碍物的位置。
根据本公开的一种形式的车辆可以包括：输入单元，包括周围环境传感器和多个摄像头；控制器，被配置为基于通过输入单元获取的信息，检测和跟踪至少一个周围障碍物；以及输出单元，被配置为响应于到由控制器检测和跟踪的至少一个周围障碍物的距离，输出引导信息。控制器可以包括：障碍物检测器，被配置为利用通过周围环境传感器获取的传感器检测信息，检测至少一个基于传感器的障碍物，并且利用通过多个摄像头获取的图像信息，检测至少一个基于图像的障碍物；以及障碍物匹配/校正单元，被配置为从至少一个基于传感器的障碍物和至少一个基于图像的障碍物中检测至少一个匹配的障碍物，并且利用针对至少一个匹配的障碍物的基于传感器的检测结果或基于图像的检测结果中的至少一个，校正至少一个匹配的障碍物的位置。
根据本文提供的描述，其它应用领域将变得显而易见。应该理解的是，描述和具体示例仅旨在用于说明的目的，并不旨在限制本公开的范围。
附图说明
为了使本公开易于理解，现在将通过示例的方式并参考附图描述本公开的各种形式，其中：
图1A、图1B和图1C是示出普通车辆的后视镜发生碰撞的视图；
图2是示出本公开的一种形式的用于车辆的周围信息提供装置的示例性配置的框图；
图3是示出根据本公开的一种形式的示例性周围信息提供过程的流程图；
图4A和图4B分别示出本公开的一种形式的图像引导的示例性输出形式；
图5示出本公开的另一种形式的图像引导的另一种示例性输出形式；
图6A、图6B、图6C和图6D分别示出本公开的某些形式的图像引导的示例性输出形式；以及
图7A和图7B是示出本公开的一种形式中的柱子感测原理的视图。
本文描述的附图仅出于说明目的，并且不旨在以任何方式限制本公开的范围。
具体实施方式
以下描述本质上仅是示例性的，并不旨在限制本公开、应用或用途。应当理解的是，在所有附图中，相应的附图标记表示相似或相应的部件和特征。
在下文中，将参照附图详细描述本公开的各种形式，从而本领域技术人员可以容易地实施这些形式。然而，本公开可以以许多不同的形式来实现，并且不应被解释为局限于这里阐述的形式。在附图中，为了清楚起见，将省略与本公开的描述无关的部分。
在整个说明书中，当某个部分“包含”或“包括”某个部件时，除非另外指出，否则不排除其它部件，并且可以进一步包括其它部件。在整个说明书中使用的相同附图标记指代相同的组成元件。
根据本公开的示例性形式，通过全方位图像和传感器来检测车辆周围的障碍物的位置，并且通过全方位图像检测的障碍物的位置和通过传感器检测的障碍物的位置相互匹配以进行校正。此外，当显示全方位图像时，在车辆移动期间提供考虑了从车身突出的外装部分的轨迹的图像引导。
图2是示出根据本公开的一种形式的用于车辆的周围信息提供装置的示例性配置的框图。
参照图2，周围信息提供装置可以包括：输入单元110，获取用于感测和跟踪车辆周围的障碍物的信息；控制器120，处理通过输入单元110获取的信息，以响应于障碍物的位置的感测和跟踪以及车辆接近度生成警报信息；以及输出单元130，输出各种类型的周围信息和警报信息。在下文中，将详细描述每个部件。
输入单元110可以包括摄像头111和周围环境传感器113。
摄像头111可以包括构成SVM(AVM)系统并拍摄全方位图像的多个摄像头。例如，摄像头111可以包括：前摄像头，用于拍摄车辆的前方；后摄像头，用于拍摄车辆的后方；以及两个或更多个侧摄像头，用于拍摄车辆的右侧和左侧。通常每个侧摄像头安装到左后视镜和右后视镜中的相应一个的下部。然而，本公开不限于此。
周围环境传感器113可以包括安装到车辆的前部、前侧部、后部或后侧部中至少一个区域以感测位于相应区域周围的障碍物的多个超声波传感器和多个雷达中的至少一个。
控制器120可以包括位置确定器121、障碍物检测器123、障碍物匹配/校正单元125、障碍物跟踪器127和视觉效果控制器129。
位置确定器121可以判断车辆的当前位置是否是与功能激活相对应的位置，所述功能为响应于对障碍物的感测和跟踪以及车辆的突出的外装部分的轨迹以确定是否执行警报等功能、确定是否执行一般的传感器警报或确定是否显示全方位图像。为此，位置确定器121可以从全球定位系统(GPS)接收关于车辆的当前位置的信息。对应于功能激活的位置可以以坐标的形式预先设置，或者可以以类别(例如停车场)的形式预先设置。可选地，位置确定器121可以基于驾驶员的基本设置输入、按键输入或挡位(例如，R挡)来确定功能激活，而与车辆的当前位置无关。
障碍物检测器123可以基于从输入单元110输入的信息来检测障碍物的存在与否和位置。例如，障碍物检测器123可以通过诸如通过摄像头111获取的全方位图像的边缘检测的图像处理来执行基于图像的障碍物检测，以及利用通过周围环境传感器113获取的传感器信息来执行基于传感器信息的障碍物检测。
障碍物匹配/校正单元125可以将通过障碍物检测器123通过基于图像的障碍物检测所检测到的障碍物与通过障碍物检测器123通过基于传感器信息的障碍物检测所检测到的障碍物进行匹配，并且当这些障碍物相对于车辆位于特定误差范围内时，可以确定通过基于图像的障碍物检测所检测到的障碍物和通过基于传感器信息的障碍物检测所检测到的障碍物是相同的障碍物。另外，障碍物匹配/校正单元125可以校正匹配的障碍物的位置。例如，障碍物匹配/校正单元125可以将匹配的障碍物的位置确定为基于图像的检测结果和基于传感器信息的检测结果之间的中间点、通过将预定的加权值分配给任何一个结果所获取的点，或与任何一个结果相对应的点。
在匹配的障碍物包括在全方位图像内时，即使不再通过基于传感器的检测来检测到匹配的障碍物(即，即使匹配的障碍物由于车辆移动而偏离传感器的感测范围)，障碍物跟踪器127也可以跟踪匹配的障碍物的位置。例如，在通过基于图像的障碍物检测和通过基于传感器信息的障碍物检测都检测到位于车辆的侧后方的柱子并且该柱子被设定为匹配的障碍物的情况下，即使由于车辆的移动，柱子偏离传感器的感测范围并且位于车辆的侧方，障碍物跟踪器127也可以基于车辆的移动距离和方向来跟踪柱子的位置，从而可以判断柱子与车辆之间的距离。
视觉效果控制器129可以合成在由摄像头111拍摄的各个方位图像，并且可以将其转换为俯视图形式的全方位图像。另外，视觉效果控制器129可以生成与障碍物检测器123、障碍物匹配/校正单元125和障碍物跟踪器127的计算/判断结果相对应的图像引导，并且可以将图像引导叠加在全方位图像上。例如，视觉效果控制器129可以生成与障碍物检测器123检测到的障碍物的位置相对应的图像引导，并且所生成的图像引导可以显示在全方位图像中与障碍物匹配/校正单元125的位置校正结果相对应的位置。
此外，即使由于车辆的移动，周围环境传感器113未感测到特定障碍物时，也可以在与障碍物跟踪器127的跟踪结果相对应的位置处持续显示与相应障碍物有关的图像引导。另外，视觉效果控制器129可以响应于车辆的转向角和向前/向后移动实时地生成与车辆的预测轨迹和从车身突出的外装部分的预测轨迹相对应的图像引导。因此，当任何一条预测轨迹与检测到的障碍物的位置重叠时，视觉效果控制器129可以响应于接近度而输出不同类型的警报图像引导。
输出单元130可以包括显示器131和扬声器133。显示器131可以输出包括图像引导的全方位图像，并且可以是设置在主机(head unit)或音频/视频/导航(AVN)系统中的显示器。然而，本公开不限于此。扬声器133可以响应于检测到的障碍物和车身或从车身突出的外装部分接近而输出警报声。
在下文中，将参照图3描述基于上述装置的配置的周围信息提供过程。
图3是示出本公开的一种形式中的示例性周围信息提供过程的流程图。
参照图3，可以激活全方位图像输出功能(S301)。激活全方位图像输出功能表示激活AVM或SVM系统。可以通过操纵AVM/SVM激活按钮或者换挡到R挡来激活全方位图像输出功能。
位置确定器121可以判断车辆的当前位置是否对应于预定区域(例如停车场)。在判断车辆的当前位置对应于预定区域时，位置确定器121可以执行图像引导提供功能(S302)。在某些形式中，该步骤可以省略。
障碍物检测器123可以利用通过摄像头111获取的图像以执行基于图像的障碍物检测(S304A)，并且可以利用通过周围环境传感器113获取的传感器信息以执行基于传感器信息的障碍物检测(S304B)。此时，如上所述，基于图像的障碍物检测(S304A)可以利用边缘检测(edge detection)方法。然而，本公开不限于此。如稍后将参照图7所描述的，可以通过指定根据车辆的移动而改变形状的部分来检测障碍物。另外，在基于图像的障碍物检测(S304A)中，距车辆的距离可以参考全方位图像中的像素位置与距车辆的距离匹配的信息。
障碍物匹配/校正单元125可以将基于图像检测到的障碍物的位置与基于传感器信息检测到的障碍物的位置彼此匹配，以确定检测到的障碍物是否是相同的障碍物(S305)。当相同的障碍物的位置有误差时(S306中的“是”)，校正障碍物的位置(S307)。如上所述，可以通过相互校正方法、加权值分配方法或利用与两种检测方法中的任何一种相对应的位置的校正方法中的至少一种来校正障碍物的位置。
当相同的障碍物的位置没有误差时(S306中的“否”)或者当位置校正完成时(S307)，障碍物跟踪器127判断是否存在由于车辆的移动而偏离周围环境传感器113的感测范围的障碍物(S308)。当存在偏离感测范围的障碍物时(S308中的“是”)，障碍物跟踪器127可以响应于车辆的移动而跟踪相应障碍物的位置(S309)。如上所述，可以持续跟踪相应障碍物的位置，直到相应障碍物与车辆之间的距离超过预定水平(例如，全方位图像中包括的距离)为止。
视觉效果控制器129可以根据检测到的车辆周围的障碍物、转向角和从车身突出的外装部分(例如，后视镜)生成图像引导，并且可以通过显示器131输出图像引导(S310)。视觉效果控制器129可以在由障碍物检测器123检测到的至少一个障碍物中，考虑由障碍物匹配/校正单元125未匹配的障碍物的位置、由障碍物匹配/校正单元125匹配的障碍物的校正位置和障碍物跟踪器127正在跟踪的障碍物的位置中的至少一个。
基于转向角和车辆的移动，视觉效果控制器129可以判断检测到的至少一个障碍物与车身和突出的外装部分中的每一个之间碰撞的可能性(S311)。在判断存在碰撞的可能性时(S311中的“是”)，视觉效果控制器129可以将视觉警报效果应用于图像引导(S312)。例如，视觉效果控制器129可以响应于距离来控制图像引导的颜色、形状和重复闪烁中的至少一种的形式提供视觉效果。然而，本公开不限于此。
在下文中，将参照图4至图7描述根据本公开的图像引导的具体形式。
图4A和图4B示出本公开的一些形式中的图像引导的示例性输出形式。图4A和图4B以及以下附图示出通过显示器131输出的应用图像引导的全方位图像。
参照图4A，可以在显示器的中间显示车辆图像410，并且可以在车辆图像410周围显示全方位图像和叠加在全方位图像上的图像引导。至少特定的视觉效果430可以作为图像引导的一种施加到通过周围环境传感器113检测到的障碍物420的边缘处。
如图4B所示，即使由于车辆的移动，障碍物420偏离周围环境传感器113的感测范围时，障碍物跟踪器127也可以跟踪相应障碍物420的位置，并且因此可以在与相应障碍物420的位置相对应的位置处显示视觉效果430。
图5示出了根据本公开的一种形式的图像引导的另一示例性输出形式。
参照图5，存在于车辆510周围的障碍物520的图像和与检测到的障碍物520的位置相对应的视觉效果550可以施加到全方位图像。另外，当挡位是R挡时，可以显示与响应于转向角从车辆510的两个后轮的当前位置开始的车辆510的两个后轮的预测移动轨迹相对应的图像引导530。当挡位是D挡时，图像引导530可以显示为响应于转向角从车辆510的两个前轮的当前位置向前延伸。
另外，在图5所示的全方位图像中，也可以响应于转向角和当前挡位，显示对应于从车身侧向最突出的外装部分即后视镜511的预测轨迹的图像引导540。当对应于预测轨迹的图像引导540遇到(重叠或相交)障碍物520的检测位置时，对应于后视镜511的预测轨迹的图像引导540和对应于障碍物520的检测位置的视觉效果550中的至少一个可以修改为警报的形式(例如，颜色改变、重复闪烁等)。
图6A、图6B、图6C和图6D示出本公开的某些形式中的图像引导的一些示例性输出形式。
参照图6A至图6D，不同于图5所示的图像引导的输出形式，可以响应于距离而提供不同的视觉效果。具体地，参照图6A，即使当后视镜611的预测轨迹640遇到障碍物620时，如果它们之间的间隔距离大于或等于预定距离，则预测轨迹640也可以以预定的第一颜色(例如黄色，即第一细虚线)显示。
如图6B所示，即使由于车辆的移动，障碍物620偏离周围环境传感器113的感测范围时，障碍物跟踪器127也可以持续地跟踪障碍物的位置及障碍物边缘的位置。当后视镜611和障碍物的边缘之间的距离变得等于或小于预定的第一距离时，后视镜611的预测轨迹640'可以改变为预定的第二颜色(例如橙色，即第二粗虚线)，并且可以在与障碍物620的边缘的位置相对应的位置处显示视觉效果650。
随后，如图6C所示，当车辆进一步向后移动并且因此后视镜611与障碍物的边缘之间的距离变得等于或小于比第一距离小的第二距离时，后视镜611的预测轨迹640″可以改变为预定的第三颜色(例如红色，即第三较粗虚线)，并且施加到障碍物620的边缘的视觉效果650'也可以改变为指示更高警报等级的另一颜色。
如上文参照图6A至图6C所述，响应于障碍物620与后视镜611之间的距离的实际变化，改变施加到预测轨迹的视觉效果。然而，如图6D所示，当后视镜611实际上没有接近障碍物620时，可以根据它们之间的距离将不同的视觉效果施加到预测轨迹。即，当预测轨迹与障碍物重叠时，可以将视觉效果施加到预测轨迹，使得预测轨迹的颜色或形状中的至少一种根据距离而逐步改变(例如，颜色变化、颜色渐变等)。
如图6A至图6D所示，当在车身与障碍物620之间没有碰撞的可能性时，车身的预测轨迹630保持为单一形式。然而，当在车身与障碍物620之间存在碰撞的可能性时，可以将视觉效果施加到车身的预测轨迹630，或者可以按照与施加到后视镜611的预测轨迹640类似的方式根据距离改变视觉效果。
图7A和图7B是分别示出根据本公开的一种形式的柱子感测原理的视图。
参照图7A，在诸如柱子720的三维障碍物的情况下，检测底表面723以及从底表面723延伸的一个或多个表面721和722。如图7B所示，由于柱子720相对于摄像头的位置由于车辆的移动而改变，所以各个表面721'和722'的形状改变。例如，当车辆向后移动时，表面721和722中的每一个的对角线分量与底表面723之间的角度逐渐减小。但是，底表面723的形状几乎不变。因此，当执行基于图像的障碍物检测时，障碍物检测器123可以将不变部分和改变部分的组合识别为三维障碍物，改变部分从不变部分延伸并且根据车辆的移动而角度和形状改变。因此，可以过滤不是实际障碍物(例如停车线)但是通过边缘检测可以检测的部分。
尽管基于泊车情况在上文描述了根据本公开的提供周围信息的方法，但是本公开不限于此。例如，根据本公开的提供周围信息的方法可以应用于在狭窄的小巷上行驶时保护后视镜，或者可以应用于远程泊车或自动泊车功能。
例如，当执行远程泊车或自动泊车功能时，尤其是在泊车期间，可能会发生由于周围环境传感器113检测到车辆周围存在障碍物而无法自动泊车的情况。然而，当一起应用通过摄像头111的基于图像的障碍物检测时，可以额外地确定障碍物是否足够近以致与车辆碰撞。因此，可以不管周围环境传感器113的检测结果如何都可以进行泊车。
另外，在上述形式中，主要描述了视觉警报，但是本公开不限于此。本公开还可以应用于通过扬声器133的听觉警报。例如，当车辆接近障碍物时，或者由于车辆移动而在后视镜和障碍物之间存在碰撞可能性时，可以响应于碰撞风险而输出不同类型的警报声。例如，可以响应于接近度来增加警报声的音量或输出警报声的频率。
另外，尽管基于车辆向后移动的情况在上文描述了各种形式，但是本公开不限于此，并且对于本领域技术人员而言显而易见的是，本公开也可以适用于车辆向前移动的情况。
本公开可以被实现为可以被记录在计算机可读记录介质上并因此被计算机系统读取的代码。该计算机可读记录介质包括其中存储有可以由计算机系统读取的数据的所有类型的记录装置。计算机可读记录介质的示例包括硬盘驱动器(HDD)、固态硬盘(SSD)、硅磁盘驱动器(SDD)、只读存储器(ROM)、随机存取存储器(RAM)、光盘ROM(CD-ROM)、磁带、软盘和光学数据存储装置。
此外，诸如“单元”或“模块”等的术语应理解为处理至少一个功能或操作并且可以以硬件方式(例如，处理器)、软件方式或硬件方式和软件方式的组合实现的单元。
从以上描述中显而易见的是，根据上述配置的本公开的至少一种形式的车辆及提供车辆周围信息的方法可以更有效地向驾驶员提供有关周围障碍物的信息。
特别地，考虑了从车身突出的外装部分的预测轨迹，并且通过不同的方法获取的障碍物的位置相互校正，从而表现出改进的防撞效果。
然而，通过本公开可获取的效果不限于上述效果，并且根据以上描述，本领域技术人员将清楚地理解本文中未提及的其它效果。
对于本领域技术人员而言显而易见的是，在不脱离本文阐述的本公开的思想和实质特征的情况下，可以进行形式和细节上的各种改变。

Claims (20)

1.一种提供车辆周围信息的方法，包括：
周围环境传感器利用通过周围环境传感器获取的传感器检测信息检测至少一个基于传感器的障碍物；
多个摄像头利用通过多个摄像头获取的图像信息检测至少一个基于图像的障碍物；
控制器基于所述传感器检测信息或所述图像信息中的至少一个，确定所述至少一个基于传感器的障碍物和所述至少一个基于图像的障碍物中的至少一个匹配的障碍物；以及
通过所述控制器利用针对所述至少一个匹配的障碍物的基于传感器的检测结果或基于图像的检测结果中的至少一个校正所述至少一个匹配的障碍物的位置。
2.根据权利要求1所述的方法，进一步包括：
当所述至少一个基于传感器的障碍物偏离所述周围环境传感器的感测范围时，障碍物跟踪器基于所述车辆的移动距离和方向来跟踪所述至少一个基于传感器的障碍物的位置。
3.根据权利要求2所述的方法，其中，
在所述至少一个基于传感器的障碍物的所述位置在距所述车辆预定距离之内时执行所述跟踪。
4.根据权利要求2所述的方法，进一步包括：
通过所述控制器，判断所述至少一个基于传感器的障碍物、所述至少一个基于图像的障碍物、位置被跟踪的所述至少一个基于传感器的障碍物和位置被校正的所述至少一个匹配的障碍物中的每一个的位置是否与对应于车身的预测移动轨迹的第一轨迹和对应于从所述车身突出的外装部分的预测移动轨迹的第二轨迹重叠。
5.根据权利要求4所述的方法，其中，
从所述车身突出的所述外装部分包括后视镜。
6.根据权利要求4所述的方法，进一步包括：
当障碍物与所述第一轨迹或所述第二轨迹中的至少一个重叠时，所述控制器基于从所述车辆到所述障碍物的距离输出警报信息。
7.根据权利要求6所述的方法，其中，
所述警报信息包括具有预定形式的视觉效果，
所述视觉效果叠加在由通过所述多个摄像头获取的图像构成的全方位图像上。
8.根据权利要求7所述的方法，进一步包括：
基于距与所述第一轨迹或所述第二轨迹中的至少一个重叠的所述障碍物的距离，改变所述视觉效果的颜色或形状中的至少一种。
9.根据权利要求1所述的方法，其中，
检测至少一个基于传感器的障碍物包括：
对通过所述多个摄像头获取的所述图像信息进行边缘检测；以及
响应于所述车辆相对于通过所述边缘检测而检测到的至少一个候选障碍物的移动，基于形状的变化程度和对角线角度的变化程度来检测至少一个三维障碍物。
10.一种记录有程序的非暂时性计算机可读记录介质，所述程序使处理器执行以下操作：
周围环境传感器利用通过周围环境传感器获取的传感器检测信息检测至少一个基于传感器的障碍物；
多个摄像头利用多个摄像头获取的图像信息检测至少一个基于图像的障碍物；
控制器基于所述传感器检测信息或所述图像信息中的至少一个，确定所述至少一个基于传感器的障碍物和所述至少一个基于图像的障碍物中的至少一个匹配的障碍物；以及
通过所述控制器利用针对所述至少一个匹配的障碍物的基于传感器的检测结果或基于图像的检测结果中的至少一个校正所述至少一个匹配的障碍物的位置。
11.一种车辆，包括：
输入单元，包括周围环境传感器和多个摄像头；
控制器，基于通过所述输入单元获取的信息，检测和跟踪至少一个周围障碍物；以及
输出单元，基于从所述车辆到由所述控制器检测和跟踪的所述至少一个周围障碍物的距离，输出警报信息，
其中，所述控制器包括：
障碍物检测器，被配置为：
利用通过所述周围环境传感器获取的传感器检测信息，检测至少一个基于传感器的障碍物，并且
利用通过所述多个摄像头获取的图像信息，检测至少一个基于图像的障碍物；以及
障碍物匹配/校正单元，被配置为：
基于所述传感器检测信息或所述图像信息中的至少一个，确定所述至少一个基于传感器的障碍物和所述至少一个基于图像的障碍物中的至少一个匹配的障碍物，并且
利用针对所述至少一个匹配的障碍物的基于传感器的检测结果或基于图像的检测结果中的至少一个，校正所述至少一个匹配的障碍物的位置。
12.根据权利要求11所述的车辆，其中，
所述控制器进一步包括障碍物跟踪器，当所述至少一个基于传感器的障碍物偏离所述周围环境传感器的感测范围时，所述障碍物跟踪器基于所述车辆的移动距离和方向来跟踪所述至少一个基于传感器的障碍物的位置。
13.根据权利要求12所述的车辆，其中，
在所述至少一个基于传感器的障碍物的位置在距所述车辆预定距离之内时，所述障碍物跟踪器跟踪所述至少一个基于传感器的障碍物的位置。
14.根据权利要求12所述的车辆，其中，
所述控制器被配置为：
判断所述至少一个基于传感器的障碍物、所述至少一个基于图像的障碍物、位置被跟踪的所述至少一个基于传感器的障碍物和位置被校正的所述至少一个匹配的障碍物中的每一个的位置是否与对应于车身的预测移动轨迹的第一轨迹和对应于从所述车身突出的外装部分的预测移动轨迹的第二轨迹重叠。
15.根据权利要求14所述的车辆，其中，
从所述车身突出的所述外装部分包括后视镜。
16.根据权利要求14所述的车辆，其中，
当障碍物与所述第一轨迹或所述第二轨迹中的至少一个重叠时，所述控制器响应于到所述障碍物的距离通过所述输出单元输出警报信息。
17.根据权利要求16所述的车辆，其中，
所述控制器进一步包括视觉效果控制器，所述视觉效果控制器将具有对应于所述警报信息的预定形式的视觉效果叠加在由通过所述多个摄像头获取的图像构成的全方位图像上。
18.根据权利要求17所述的车辆，其中，
所述视觉效果控制器控制所述视觉效果，并基于距与所述第一轨迹或所述第二轨迹中的至少一个重叠的所述障碍物的距离来改变所述视觉效果的颜色或形状中的至少一种。
19.根据权利要求18所述的车辆，其中，
所述视觉效果控制器控制所述视觉效果，并进一步地基于距与所述第一轨迹或所述第二轨迹中的至少一个重叠的所述障碍物的剩余距离来逐步改变所述视觉效果的颜色或形状中的至少一种。
20.根据权利要求11所述的车辆，其中，
所述障碍物检测器被配置为：
对通过所述多个摄像头获取的所述图像信息进行边缘检测，以及
响应于所述车辆相对于通过所述边缘检测而检测到的至少一个候选障碍物的移动，基于形状的变化程度和对角线角度的变化程度以检测至少一个三维障碍物。
CN202011000383.4A 2019-10-07 2020-09-22 车辆及提供车辆周围信息的方法 Pending CN112622774A (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR10-2019-0123630 2019-10-07
KR1020190123630A KR20210041224A (ko) 2019-10-07 2019-10-07 자동차 및 그를 위한 주변 정보 출력 방법

Publications (1)

Publication Number Publication Date
CN112622774A true CN112622774A (zh) 2021-04-09

Family

ID=74875998

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202011000383.4A Pending CN112622774A (zh) 2019-10-07 2020-09-22 车辆及提供车辆周围信息的方法

Country Status (4)

Country Link
US (1) US11772555B2 (zh)
KR (1) KR20210041224A (zh)
CN (1) CN112622774A (zh)
DE (1) DE102020212098A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113071418A (zh) * 2021-04-28 2021-07-06 中国第一汽车股份有限公司 车辆的后视镜控制方法、装置、电子设备及存储介质

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11940544B2 (en) * 2021-10-25 2024-03-26 Ford Global Technologies, Llc Vehicle positioning using V2X RSU messaging and vehicular sensors
CN114379547A (zh) * 2021-12-24 2022-04-22 北京百度网讯科技有限公司 制动控制方法、装置、车辆、电子设备及存储介质

Family Cites Families (32)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7663502B2 (en) * 1992-05-05 2010-02-16 Intelligent Technologies International, Inc. Asset system control arrangement and method
US11208129B2 (en) * 2002-06-04 2021-12-28 Transportation Ip Holdings, Llc Vehicle control system and method
US20130163879A1 (en) * 2010-08-30 2013-06-27 Bk-Imaging Ltd. Method and system for extracting three-dimensional information
US9123035B2 (en) * 2011-04-22 2015-09-01 Angel A. Penilla Electric vehicle (EV) range extending charge systems, distributed networks of charge kiosks, and charge locating mobile apps
JP5817843B2 (ja) * 2011-12-14 2015-11-18 トヨタ自動車株式会社 車両用情報伝達装置
WO2016027312A1 (ja) * 2014-08-19 2016-02-25 三菱電機株式会社 後方路面照射装置
US9884631B2 (en) * 2015-06-04 2018-02-06 Toyota Motor Engineering & Manufacturing North America, Inc. Transitioning between operational modes of an autonomous vehicle
US20160357187A1 (en) * 2015-06-05 2016-12-08 Arafat M.A. ANSARI Smart vehicle
US9836056B2 (en) * 2015-06-05 2017-12-05 Bao Tran Smart vehicle
US9711050B2 (en) * 2015-06-05 2017-07-18 Bao Tran Smart vehicle
US9630619B1 (en) * 2015-11-04 2017-04-25 Zoox, Inc. Robotic vehicle active safety systems and methods
US9734455B2 (en) * 2015-11-04 2017-08-15 Zoox, Inc. Automated extraction of semantic information to enhance incremental mapping modifications for robotic vehicles
US9507346B1 (en) * 2015-11-04 2016-11-29 Zoox, Inc. Teleoperation system and method for trajectory modification of autonomous vehicles
US9754490B2 (en) * 2015-11-04 2017-09-05 Zoox, Inc. Software application to request and control an autonomous vehicle service
WO2017079341A2 (en) * 2015-11-04 2017-05-11 Zoox, Inc. Automated extraction of semantic information to enhance incremental mapping modifications for robotic vehicles
US11119477B1 (en) * 2016-01-22 2021-09-14 State Farm Mutual Automobile Insurance Company Anomalous condition detection and response for autonomous vehicles
KR101827698B1 (ko) * 2016-11-01 2018-02-12 현대자동차주식회사 차량 및 그 제어방법
EP3619643A1 (en) * 2017-05-03 2020-03-11 Mobileye Vision Technologies Ltd. Detection and classification systems and methods for autonomous vehicle navigation
US10571280B2 (en) * 2017-05-09 2020-02-25 Toyota Research Institute, Inc. Systems and methods for localizing a vehicle using a roadway signature
US10372130B1 (en) * 2017-05-19 2019-08-06 Zoox, Inc. Communicating reasons for vehicle actions
US11067993B2 (en) * 2017-08-25 2021-07-20 Magna Electronics Inc. Vehicle and trailer maneuver assist system
KR20190071097A (ko) * 2017-12-14 2019-06-24 현대자동차주식회사 차량의 디스플레이 제어 장치 및 방법, 그리고 차량 시스템
GB2572006A (en) * 2018-03-16 2019-09-18 Continental Automotive Gmbh Vehicle alighting assistance device
KR102420476B1 (ko) * 2018-05-25 2022-07-13 에스케이텔레콤 주식회사 차량의 위치 추정 장치, 차량의 위치 추정 방법, 및 이러한 방법을 수행하도록 프로그램된 컴퓨터 프로그램을 저장하는 컴퓨터 판독가능한 기록매체
US11277956B2 (en) * 2018-07-26 2022-03-22 Bear Flag Robotics, Inc. Vehicle controllers for agricultural and industrial applications
KR102572784B1 (ko) * 2018-10-25 2023-09-01 주식회사 에이치엘클레무브 운전자 보조 시스템 및 그 제어방법
US10768620B1 (en) * 2019-09-17 2020-09-08 Ha Q Tran Smart vehicle
US10609148B1 (en) * 2019-09-17 2020-03-31 Ha Q Tran Smart vehicle
US11150665B2 (en) * 2019-09-17 2021-10-19 Ha Q Tran Smart vehicle
US20210089048A1 (en) * 2019-09-21 2021-03-25 Ha Q Tran Smart vehicle
US11267401B2 (en) * 2019-09-27 2022-03-08 GM Cruise Holdings, LLC Safe passenger disembarking for autonomous vehicles via audible signals
US10783559B1 (en) * 2019-10-06 2020-09-22 Bao Tran Mobile information display platforms

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113071418A (zh) * 2021-04-28 2021-07-06 中国第一汽车股份有限公司 车辆的后视镜控制方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
DE102020212098A1 (de) 2021-04-08
KR20210041224A (ko) 2021-04-15
US20210103743A1 (en) 2021-04-08
US11772555B2 (en) 2023-10-03

Similar Documents

Publication Publication Date Title
KR102648812B1 (ko) 차량 및 그의 보행자 감지 알림 방법
EP2429877B1 (en) Camera system for use in vehicle parking
EP3493179B1 (en) Driving assisting method and driving assisting device using the same
US9139135B2 (en) System and method that minimizes hazards of blind spots while driving
EP1962254B1 (en) In-vehicle apparatus for recognizing running environment of vehicle
CN112622774A (zh) 车辆及提供车辆周围信息的方法
JP4774849B2 (ja) 車両の障害物表示装置
US20070063874A1 (en) Method and device for determining the position and/or the anticipated position of a vehicle during a parking operation in relation to the oncoming lane of a multi-lane roadway
US20150035983A1 (en) Method and vehicle assistance system for active warning and/or for navigation assistance to prevent a collosion of a vehicle body part and/or of a vehicle wheel with an object
JP2016199262A (ja) 後退走行中の前輪軌道逸脱に基づく衝突の回避
US11223775B2 (en) Method and apparatus for the spatially resolved detection of an object outside a transportation vehicle with the aid of a sensor installed in a transportation vehicle
JP2007512182A (ja) 車両死角の自動視認
JP3941770B2 (ja) 衝突可能性判定装置
JP2007034988A (ja) 車両の障害物回避報知装置
JP2006123670A (ja) 移動体周辺監視装置
CN109313859B (zh) 用于自动激活机动车的障碍物识别装置的方法和障碍物辅助装置
JP2018074286A (ja) 運転支援装置
US11220213B2 (en) Rear display device
US20230242137A1 (en) Notification device and notification method
JP4345526B2 (ja) 物体監視装置
JP4397719B2 (ja) 視線検出装置
KR20210071454A (ko) 차량 및 제어방법
CN112449625A (zh) 用于在由牵引车辆和拖车组成的挂车组合的调度操作情况下进行辅助的方法、系统和挂车组合
US20230222813A1 (en) Road surface marking detection device, notification system provided with the same, and road surface marking detection
US20230037900A1 (en) Device and Method for Determining Objects Around a Vehicle

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination