US9619894B2 - System and method for estimating vehicle dynamics using feature points in images from multiple cameras - Google Patents

System and method for estimating vehicle dynamics using feature points in images from multiple cameras Download PDF

Info

Publication number
US9619894B2
US9619894B2 US14/710,364 US201514710364A US9619894B2 US 9619894 B2 US9619894 B2 US 9619894B2 US 201514710364 A US201514710364 A US 201514710364A US 9619894 B2 US9619894 B2 US 9619894B2
Authority
US
United States
Prior art keywords
camera
images
vehicle
feature points
cameras
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US14/710,364
Other languages
English (en)
Other versions
US20150332098A1 (en
Inventor
Jinsong Wang
Wende Zhang
Bakhtiar B. Litkouhi
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
GM Global Technology Operations LLC
Original Assignee
GM Global Technology Operations LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by GM Global Technology Operations LLC filed Critical GM Global Technology Operations LLC
Priority to US14/710,364 priority Critical patent/US9619894B2/en
Assigned to GM Global Technology Operations LLC reassignment GM Global Technology Operations LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: LITKOUHI, BAKHTIAR B., WANG, JINSONG, ZHANG, WENDE
Priority to DE102015107675.6A priority patent/DE102015107675A1/de
Priority to CN201510495737.XA priority patent/CN105243655B/zh
Publication of US20150332098A1 publication Critical patent/US20150332098A1/en
Application granted granted Critical
Publication of US9619894B2 publication Critical patent/US9619894B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/2033—
    • G06K9/00791—
    • G06K9/3241—
    • G06T7/0042—
    • G06T7/0071—
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/50—Depth or shape recovery
    • G06T7/55—Depth or shape recovery from multiple images
    • G06T7/579—Depth or shape recovery from multiple images from motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30252—Vehicle exterior; Vicinity of vehicle
    • G06T2207/30256—Lane; Road marking

Definitions

  • This invention relates generally to a system and method for estimating dynamics of a mobile platform and, more particularly, to a system and method for estimating vehicle dynamics by matching feature points in overlapping images from cameras in a surround-view camera system on a vehicle.
  • Modern vehicles generally include one or more cameras that provide back-up assistance, take images of the vehicle driver to determine driver drowsiness or attentiveness, provide images of the road as the vehicle is traveling for collision avoidance purposes, provide structure recognition, such as roadway signs, etc.
  • Other vehicle vision applications include vehicle lane sensing systems to sense the vehicle travel lane and drive the vehicle in the lane-center. Many of these known lane sensing systems detect lane-markers on the road for various applications, such as lane departure warning (LDW), lane keeping (LK), lane centering (LC), etc., and have typically employed a single camera, either at the front or rear of the vehicle, to provide the images that are used to detect the lane-markers.
  • LDW lane departure warning
  • LK lane keeping
  • LC lane centering
  • a surround-view camera system on a vehicle that includes a front camera, a rear camera and left and right side cameras, where the camera system generates a top-down view of the vehicle and surrounding areas using the images from the cameras, and where the images overlap each other at the corners of the vehicle.
  • the top-down view can be displayed for the vehicle driver to see what is surrounding the vehicle for back-up, parking, etc.
  • Future vehicles may not employ rearview mirrors, but may instead include digital images provided by the surround view cameras.
  • U.S. Patent Application Publication No. 2013/0293717 to Zhang et al., filed Apr. 9, 2013, titled, Full Speed Lane Sensing With A Surrounding View System discloses a system and method for providing lane sensing on a vehicle by detecting roadway lane-markers, where the system employs a surround-view camera system providing a top-down view image around the vehicle.
  • the method includes detecting left-side and right-side lane boundary lines in the top-down view image, and then determining whether the lane boundary lines in the image are aligned from image frame to a next image frame and are aligned from image to image in the top-down view image.
  • Camera calibration generally refers to estimating a number of camera parameters including both intrinsic and extrinsic parameters, where the intrinsic parameters include focal length, optical center, radial distortion parameters, etc., and the extrinsic parameters include camera location, camera orientation, etc.
  • Camera extrinsic parameters calibration typically involves determining a set of parameters that relate camera image coordinates to vehicle coordinates and vice versa. Some camera parameters, such as camera focal length, optical center, etc., are stable, while other parameters, such as camera orientation and position, are not. For example, the height of the camera depends on the load of the vehicle, which will change from time to time.
  • the images from the cameras overlap at the corners of the vehicle, where the camera calibration process “stitches” the adjacent images together so that common elements in the separate images directly overlap with each other to provide the desired top-down view.
  • these camera images are stitched together to provide this image using any of a number of calibration techniques so that when the vehicle is first put into service, the cameras are properly calibrated.
  • One calibration technique employed is to position the vehicle on a checker-board pattern of alternating light and dark squares where each point of the squares is suitably identified. Using these points in the developed images allows the camera calibration software to correct the position of the images so that overlapping points in adjacent images are identified at the same location.
  • VPM video processing modules
  • the present disclosure describes a system and method for estimating dynamics of a mobile platform by matching feature points in overlapping images from cameras on the platform, such as cameras in a surround-view camera system on a vehicle.
  • the method includes identifying overlap image areas for any two cameras in the surround-view camera system, identifying common feature points in the overlap image areas, and determining that common feature points in the overlap image areas are not at the same location.
  • the method also includes estimating three-degree of freedom vehicle dynamic parameters from the matching between the common feature points, and estimating vehicle dynamics of one or more of pitch, roll and height variation using the vehicle dynamic parameters.
  • FIG. 1 is an illustration of a vehicle including a surround-view camera system having multiple cameras
  • FIG. 2 is an isometric view of a vehicle showing a coordinate system for world coordinates, vehicle coordinates and camera coordinates;
  • FIG. 3 is a coordinate system showing a surround-view camera system in a stationary position
  • FIG. 4 is the coordinate system shown in FIG. 3 where the coordinate system has been changed as a result of a change in vehicle dynamics
  • FIG. 5 is a representation of four raw images from four cameras for the surround-view camera system showing matched feature pairs
  • FIG. 6 is a block diagram of a system showing a process for matching feature points
  • FIG. 7 is a representation of an image from a forward or rearward looking camera on a vehicle showing a horizon line
  • FIG. 8 is an image from a forward or rearward looking camera on a vehicle showing movement of the horizon line when the vehicle pitches down;
  • FIG. 9 is a representation of an image from a forward or rearward looking camera on a vehicle where the vehicle is in a roll configuration.
  • FIG. 10 is a representation of an image from a forward or rearward looking camera on a vehicle showing a drift situation.
  • the system and method has particular application for estimating vehicle dynamics on a vehicle.
  • the system and method may have application for other mobile platforms, such as on trains, machines, tractors, boats, recreation vehicles, etc.
  • FIG. 1 is a top illustration of a vehicle 10 including a surround-view camera system having a front-view camera 12 , a rear-view camera 14 , a left-side driver view camera 16 and a right-side passenger view camera 18 .
  • the cameras 12 - 18 can be any camera suitable for the purposes described herein, many of which are known in the automotive art, that are capable of receiving light, or other radiation, and converting the light energy to electrical signals in a pixel format using, for example, charged coupled devices (CCD).
  • CCD charged coupled devices
  • the cameras 12 - 18 generate frames of image data at a certain data frame rate that can be stored for subsequent image processing in a video processing module (VPM) 20 .
  • VPM video processing module
  • the cameras 12 - 18 can be mounted within or on any suitable structure that is part of the vehicle 10 , such as bumpers, facie, grill, side-view mirrors, door panels, etc., as would be well understood and appreciated by those skilled in the art.
  • the side cameras 16 and 18 are mounted under the side view mirrors and are pointed downwards.
  • the cameras 12 - 18 generate images of certain limited areas around the vehicle 10 that partially overlap. Particularly, area 24 is the image area for the camera 12 , area 26 is the image area for the camera 14 , area 28 is the image area for the camera 16 , and area 30 is the image area for the camera 18 , where area 32 is an overlap area of the images 24 and 28 , area 34 is an overlap area of the images 24 and 30 , area 36 is an overlap area of the images 28 and 26 , and area 38 is an overlap area of the images 30 and 26 .
  • Image data from the cameras 12 - 18 is sent to the VPM 20 that processes the image data to stitch the images together that can then be displayed on a vehicle display as a single top-down view image around the vehicle 10 .
  • Software algorithms are known that employ rotation matrices R and translation vectors t to orient and reconfigure the images from adjacent cameras so that the images properly overlap.
  • Vehicle dynamics i.e., pitch, roll and height variation, influence the orientation of the images 24 - 30 from the cameras 12 - 18 .
  • the vehicle 10 may pitch down during vehicle braking, may pitch up during hard accelerations, may roll sideways during turns, may pitch up or down during vehicle loading, etc., which may cause the images from the cameras 12 - 18 to change relative to each other even though the cameras are properly calibrated.
  • the present invention proposes a system and method for integrating information available from sensors on the vehicle about the vehicle dynamics into the algorithm in the VPM 20 that determines the surround-view image using the cameras 12 - 18 so the image can be corrected based on those changes to the vehicle dynamics.
  • the proposed camera correction technique defines three coordinate systems, namely, a world coordinate system, a vehicle coordinate system and a camera coordinate system each defined in an X-Y-Z positional orientation.
  • FIG. 2 is an illustration of a vehicle 50 including a camera 52 showing these coordinates systems, where the subscript V represents the vehicle coordinates, the subscript W represents the world coordinates and the subscript C represents the camera coordinates.
  • the vehicle 50 includes a sensor 54 intended to represent all of the available sensors on any particular vehicle that can sense vehicle pitch, height variation and roll and provide signals on the vehicle bus identifying vehicle dynamics.
  • Equations (1)-(3) below represent the stationary orientation of the surround-view camera system and equations (4)-(6) below identify the orientation of a dynamic position of the vehicle 50 , where R dyn is the rotation dynamics matrix of the vehicle 50 in all of pitch, roll and height variation.
  • R dyn is the rotation dynamics matrix of the vehicle 50 in all of pitch, roll and height variation.
  • FIG. 3 is an illustration 150 of an X, Y and Z coordinate system including four cameras 152 that are part of a surround-view camera system showing such a combined reference frame.
  • FIG. 4 shows the illustration 150 being rotated in pitch and roll, where the rotation matrix and translation vector change accordingly.
  • the change in the relative orientation of the images from the cameras 12 - 18 in the surround-view camera system from the calibration of the cameras 12 - 18 provided at vehicle manufacturer or at the dealer can be used to estimate the vehicle dynamics parameter R dyn , namely, rotation dynamics in two-degrees of freedom for pitch ⁇ and roll ⁇ of the vehicle 50 , and translation dynamics in one-degree of freedom, namely, the height offset ⁇ z of the vehicle 50 .
  • the present invention also proposes a system and method to estimate vehicle dynamics in this manner that uses the overlap image area for any two of the cameras 12 - 18 in the surround-view camera system to determine that common points in the overlap image area are not at the same location, assuming the cameras are properly calibrated.
  • the algorithm detects matching feature points (u,v) in the two images, and estimates three-degree of freedom vehicle dynamic parameters from the difference between the feature points (u,v).
  • Providing the matching feature points (u,v) in the images from two cameras are the projection of the same location in world coordinates and solving dynamic equations with the unknowns gives an estimate of the pitch ⁇ , roll ⁇ and/or height variation of the vehicle 50 based on the distance between the points.
  • FIG. 5 is an illustration 80 of the vehicle 10 in a parking lot adjacent to other vehicles, where parking lot lines 78 can be used to identify the matching feature points (u,v) in two overlapping camera images.
  • the matching feature points (u,v) do not have to be on the ground, but can be on any suitable object above ground or otherwise, such as on the vehicle body.
  • points 82 and 84 represent the same or common point in the overlap area 32 for the images provided by the front camera 12 and the left-side camera 16 , respectively.
  • Points 86 and 88 represent the same or common point in the overlap area 34 for the images from the camera 12 and the right-side camera 18 , respectively.
  • Points 90 and 92 represent the same or common point the overlap area 36 between the left-side camera 16 and the rear camera 14 , respectively.
  • Points 94 and 96 represent the same or common point in the overlap area 38 between the right side camera 18 and the rear camera 14 , respectively.
  • the distance between the points 82 and 84 , or the points 86 and 88 , or the points 90 and 92 , or the points 94 and 96 caused by the change in vehicle dynamics provides the mechanism for determining that change in the vehicle dynamics.
  • feature point detection and matching techniques and algorithms such as SIFT, SURF, ORB, etc.
  • FIG. 6 is a block diagram of a system 100 showing a general representation of the process for identifying the matching feature points (u,v) in the overlap areas 32 , 34 , 36 and 38 .
  • box 102 represents the image from the front camera 12
  • box 104 represents the image from the left-side camera 16
  • box 106 represents the image from the right-side camera 18
  • box 108 represents the image from the rear camera 14 .
  • a synchronization block 110 synchronizes the timing of the images 102 - 108 from the cameras 12 - 18 so that all of the images 32 , 34 , 36 and 38 are aligned in time before being aligned in space from the calibration process.
  • the images 102 and 104 from the cameras 12 and 16 respectively, generate a region of interest 112 in the overlap area 32
  • the images 102 and 106 from the cameras 12 and 18 respectively, generate a region of interest 114 in the overlap area 34
  • the images 104 and 108 from the cameras 16 and 14 respectively, generate a region of interest 116 in the overlap area 36
  • the images 106 and 108 from the cameras 14 and 18 respectively, generate a region of interest 118 in the overlap area 38 .
  • the regions of interest 112 , 114 , 116 and 118 are then provided to a processor 120 that identifies the several matching feature points (u,v) in the regions of interest 112 , 114 , 116 and 118 in the manner as discussed herein.
  • Equations (8)-(12) below show the process for determining the dynamic rotation matrix R dyn and the translation vector t′ W2V that identify a change in the vehicle dynamics using the common matching feature points, which can then be used as an input to other vehicle systems, where equation (8) shows the two feature points in the overlap image area that should be at the same location X when the cameras are calibrated.
  • Suitable algorithms can be employed for this process, such as the Levenberg-Marquardt algorithm, gradient descent algorithms, etc.
  • FIG. 7 is a representation of an image 160 from a camera on the vehicle 10 traveling along a roadway 166 in front of the vehicle 10 and defining the non-dynamic horizon between air and ground by dotted line 162 , where the vehicle 10 is not exhibiting roll, pitch or height variation.
  • FIG. 8 is a representation of an image 168 including the non-dynamic horizon line 162 , but where the actual horizon is now at line 164 showing that the vehicle 10 has pitched down.
  • FIG. 9 is a representation of an image 170 of the roadway 166 where the horizon line 162 is shown angled in a roll direction.
  • FIG. 10 is a representation an image 172 where the vehicle 10 has drifted on the roadway 166 in a left direction.
  • the process of determining the vehicle dynamics based on changes in the surround-view image can include temporal tracking and smoothing.
  • the temporal tracking and smoothing can be provided for non-transient vehicle change detection, such as a flat tire, bad suspension, towing and heavy load. Further, the tracking can be performed to detect dangerous vehicle dynamic changes, such as roll over detection and prevention, zigzag driving, etc. If the roll or pitch dynamics change aggressively anti-roll systems or other vehicle system can be notified to take corrective action.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
  • Closed-Circuit Television Systems (AREA)
US14/710,364 2014-05-16 2015-05-12 System and method for estimating vehicle dynamics using feature points in images from multiple cameras Active 2035-06-09 US9619894B2 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US14/710,364 US9619894B2 (en) 2014-05-16 2015-05-12 System and method for estimating vehicle dynamics using feature points in images from multiple cameras
DE102015107675.6A DE102015107675A1 (de) 2014-05-16 2015-05-15 System und Verfahren zur Schätzung einer Fahrzeugdynamik unter Verwendung von Merkmalpunkten in Bildern von mehreren Kameras
CN201510495737.XA CN105243655B (zh) 2014-05-16 2015-05-16 使用图像中的特征点估算车辆动态的系统和方法

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201461994686P 2014-05-16 2014-05-16
US14/710,364 US9619894B2 (en) 2014-05-16 2015-05-12 System and method for estimating vehicle dynamics using feature points in images from multiple cameras

Publications (2)

Publication Number Publication Date
US20150332098A1 US20150332098A1 (en) 2015-11-19
US9619894B2 true US9619894B2 (en) 2017-04-11

Family

ID=54361851

Family Applications (1)

Application Number Title Priority Date Filing Date
US14/710,364 Active 2035-06-09 US9619894B2 (en) 2014-05-16 2015-05-12 System and method for estimating vehicle dynamics using feature points in images from multiple cameras

Country Status (3)

Country Link
US (1) US9619894B2 (zh)
CN (1) CN105243655B (zh)
DE (1) DE102015107675A1 (zh)

Cited By (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180307915A1 (en) * 2017-04-24 2018-10-25 Toyota Research Institute, Inc. Systems and methods for identifying changes within a mapped environment
WO2019072451A1 (de) * 2017-10-13 2019-04-18 Robert Bosch Gmbh Verfahren zum verarbeiten von bildern
US11226200B2 (en) 2017-06-28 2022-01-18 Boe Technology Group Co., Ltd. Method and apparatus for measuring distance using vehicle-mounted camera, storage medium, and electronic device

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11370422B2 (en) * 2015-02-12 2022-06-28 Honda Research Institute Europe Gmbh Method and system in a vehicle for improving prediction results of an advantageous driver assistant system
DE102016103819A1 (de) 2016-03-03 2017-09-07 Heraeus Deutschland GmbH & Co. KG Anschlussträger, optoelektronisches Bauteil und Verfahren zur Herstellung eines Anschlussträgers oder eines optoelektronischen Bauteils
US10259390B2 (en) 2016-05-27 2019-04-16 GM Global Technology Operations LLC Systems and methods for towing vehicle and trailer with surround view imaging devices
MX371375B (es) * 2016-08-15 2020-01-28 Trackmobile Llc Asistencia visual para impulsor de vagon.
JP6601352B2 (ja) * 2016-09-15 2019-11-06 株式会社デンソー 車両姿勢推定装置
JP6789767B2 (ja) * 2016-11-11 2020-11-25 スタンレー電気株式会社 監視システム
US10466027B2 (en) 2017-06-21 2019-11-05 Fujitsu Ten Corp. Of America System and method for marker placement
US10878709B2 (en) * 2018-07-19 2020-12-29 The Boeing Company System, method, and computer readable medium for autonomous airport runway navigation
WO2020068960A1 (en) * 2018-09-26 2020-04-02 Coherent Logix, Inc. Any world view generation
DE102018132256A1 (de) 2018-12-14 2020-06-18 Valeo Schalter Und Sensoren Gmbh Verfahren zum Bestimmen eines Aufstellungsfehlers eines ersten optoelektronischen Sensors durch Vergleich mit einem zweiten optoelektronischen Sensor, Computerprogrammprodukt, elektronische Recheneinrichtung, Fahrerassistenzsystem sowie Kraftfahrzeug
WO2022204953A1 (zh) * 2021-03-30 2022-10-06 深圳市锐明技术股份有限公司 确定俯仰角的方法、装置及终端设备
DE102022121670A1 (de) 2022-08-26 2024-02-29 Connaught Electronics Ltd. Fahrspurerkennung und Führen eines Fahrzeugs

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130293717A1 (en) 2012-05-02 2013-11-07 GM Global Technology Operations LLC Full speed lane sensing with a surrounding view system
US9247214B2 (en) * 2012-11-21 2016-01-26 Fujitsu Limited Image processing apparatus and image processing method with projective transform of multiple cameras

Family Cites Families (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2008187566A (ja) * 2007-01-31 2008-08-14 Sanyo Electric Co Ltd カメラ校正装置及び方法並びに車両
JP5299231B2 (ja) * 2009-11-17 2013-09-25 富士通株式会社 キャリブレーション装置
JP5491235B2 (ja) * 2010-03-02 2014-05-14 東芝アルパイン・オートモティブテクノロジー株式会社 カメラキャリブレーション装置
CN102142138A (zh) * 2011-03-23 2011-08-03 深圳市汉华安道科技有限责任公司 一种车辆辅助系统中的图像处理方法以及子系统
DE102011076795A1 (de) * 2011-05-31 2012-09-20 Robert Bosch Gmbh Verfahren zum Bestimmen einer Nickbewegung einer in einem Fahrzeug verbauten Kamera und Verfahren zur Steuerung einer Lichtaussendung zumindest eines Frontscheinwerfers eines Fahrzeugs
CN102402855A (zh) * 2011-08-29 2012-04-04 深圳市蓝盾科技有限公司 用于智能交通的双摄像机实时全景视频融合方法及系统

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130293717A1 (en) 2012-05-02 2013-11-07 GM Global Technology Operations LLC Full speed lane sensing with a surrounding view system
US9247214B2 (en) * 2012-11-21 2016-01-26 Fujitsu Limited Image processing apparatus and image processing method with projective transform of multiple cameras

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Jeong, Jaeheon et al. "Trinocular Visual Odometry for Divergent Views with Minimal Overlap" IEEE, 2013, pp. 229-236, 2013.
Knorr, Moritz, et al. "Online Extrinsic Multi-Camera Calibration Using Ground Plane Induced Homographies" IEEE Intelligent Vehicles Symposium (IV), Gold Coast, Australia, Jun. 23-26, 2013, pp. 236-241.

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180307915A1 (en) * 2017-04-24 2018-10-25 Toyota Research Institute, Inc. Systems and methods for identifying changes within a mapped environment
US10489663B2 (en) * 2017-04-24 2019-11-26 Toyota Research Institute, Inc. Systems and methods for identifying changes within a mapped environment
US11226200B2 (en) 2017-06-28 2022-01-18 Boe Technology Group Co., Ltd. Method and apparatus for measuring distance using vehicle-mounted camera, storage medium, and electronic device
WO2019072451A1 (de) * 2017-10-13 2019-04-18 Robert Bosch Gmbh Verfahren zum verarbeiten von bildern

Also Published As

Publication number Publication date
CN105243655A (zh) 2016-01-13
DE102015107675A1 (de) 2015-11-19
DE102015107675A9 (de) 2016-01-14
CN105243655B (zh) 2018-09-14
US20150332098A1 (en) 2015-11-19

Similar Documents

Publication Publication Date Title
US9981605B2 (en) Surround-view camera system (VPM) and vehicle dynamic
US9619894B2 (en) System and method for estimating vehicle dynamics using feature points in images from multiple cameras
US9986173B2 (en) Surround-view camera system (VPM) online calibration
US9516277B2 (en) Full speed lane sensing with a surrounding view system
US20190248290A1 (en) Method for determining misalignment of a vehicular camera
JP5172314B2 (ja) ステレオカメラ装置
US20160291145A1 (en) Wheel detection and its application in object tracking and sensor registration
JP6137081B2 (ja) 車載機器
US11288833B2 (en) Distance estimation apparatus and operating method thereof
US10946799B2 (en) Vehicle vision system with overlay calibration
EP3358295A1 (en) Image processing device, stereo camera device, vehicle, and image processing method
EP3418122B1 (en) Position change determination device, overhead view image generation device, overhead view image generation system, position change determination method, and program
US9892519B2 (en) Method for detecting an object in an environmental region of a motor vehicle, driver assistance system and motor vehicle
US20220196395A1 (en) Method for ascertaining an operating angle between a tractor and a trailer of the tractor
JP2012166705A (ja) 車載カメラレンズ用異物付着判定装置
CN111160070A (zh) 车辆全景图像盲区消除方法、装置、存储介质及终端设备
JP2004310522A (ja) 車両用画像処理装置
US11377027B2 (en) Image processing apparatus, imaging apparatus, driving assistance apparatus, mobile body, and image processing method
KR20160050439A (ko) 차량용 후방 카메라의 영상 보정 방법
WO2017122688A1 (ja) 車載カメラのレンズ異常検出装置
JP2018152014A (ja) 車両用映像表示装置
US20240103525A1 (en) Vehicle and control method thereof
US11433810B2 (en) Image processing apparatus, camera, moveable body, and image processing method
WO2023026626A1 (ja) 信号処理装置、信号処理システム、信号処理方法及びプログラム
US20200065987A1 (en) Signal processing apparatus, moving body, and stereo camera

Legal Events

Date Code Title Description
AS Assignment

Owner name: GM GLOBAL TECHNOLOGY OPERATIONS LLC, MICHIGAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:WANG, JINSONG;ZHANG, WENDE;LITKOUHI, BAKHTIAR B.;SIGNING DATES FROM 20150511 TO 20150512;REEL/FRAME:035622/0120

FEPP Fee payment procedure

Free format text: PAYOR NUMBER ASSIGNED (ORIGINAL EVENT CODE: ASPN); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCF Information on status: patent grant

Free format text: PATENTED CASE

MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4