US20180215313A1 - Vehicle vision system using at least two cameras - Google Patents
Vehicle vision system using at least two cameras Download PDFInfo
- Publication number
- US20180215313A1 US20180215313A1 US15/886,871 US201815886871A US2018215313A1 US 20180215313 A1 US20180215313 A1 US 20180215313A1 US 201815886871 A US201815886871 A US 201815886871A US 2018215313 A1 US2018215313 A1 US 2018215313A1
- Authority
- US
- United States
- Prior art keywords
- vehicle
- location
- disposed
- cameras
- view
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000012545 processing Methods 0.000 claims abstract description 17
- 238000000034 method Methods 0.000 claims abstract description 9
- 238000000926 separation method Methods 0.000 claims abstract 12
- 238000001514 detection method Methods 0.000 claims description 26
- 238000003384 imaging method Methods 0.000 description 17
- 230000008901 benefit Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 230000003068 static effect Effects 0.000 description 3
- 235000004522 Pentaglottis sempervirens Nutrition 0.000 description 2
- 238000004364 calculation method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 240000004050 Pentaglottis sempervirens Species 0.000 description 1
- 241000905137 Veronica schmidtiana Species 0.000 description 1
- 230000000295 complement effect Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 231100001261 hazardous Toxicity 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012552 review Methods 0.000 description 1
- 230000003595 spectral effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
- B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
- B60R1/26—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view to the rear of the vehicle
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/31—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles providing stereoscopic vision
-
- G—PHYSICS
- G05—CONTROLLING; REGULATING
- G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
- G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
- G05D1/02—Control of position or course in two dimensions
- G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
- G05D1/0231—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means
- G05D1/0246—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means
- G05D1/0251—Control of position or course in two dimensions specially adapted to land vehicles using optical position detecting means using a video camera in combination with image processing means extracting 3D information from a plurality of images taken from different locations, e.g. stereo vision
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/10—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used
- B60R2300/105—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of camera system used using multiple cameras
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/20—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of display used
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
- B60R2300/303—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using joined images, e.g. multiple camera images
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/802—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring and displaying vehicle exterior blind spot views
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/802—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring and displaying vehicle exterior blind spot views
- B60R2300/8026—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring and displaying vehicle exterior blind spot views in addition to a rear-view mirror system
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/806—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for aiding parking
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/8066—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring rearward traffic
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/808—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for facilitating docking to a trailer
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
- G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
Definitions
- the present invention relates generally to a vehicle vision system for a vehicle and, more particularly, to a vehicle vision system that utilizes one or more cameras at a vehicle.
- the present invention provides a driver assistance system or vision system or autonomous vehicle control system for a vehicle that utilizes one or more cameras (preferably one or more CMOS cameras) to capture image data representative of images exterior of the vehicle, and processes image data captured from multiple cameras viewing a common region exterior the vehicles, whereby the image data captured by additional cameras at the vehicle can be used to detect objects and to determine the location of detected objects in three dimensional space.
- one or more cameras preferably one or more CMOS cameras
- FIG. 1 is a plan view of a vehicle with a vision system that incorporates cameras in accordance with the present invention
- FIG. 2 is a side elevation of a vehicle showing a first rearward viewing camera at a rear bumper of the vehicle and a second rearward viewing camera at a spoiler of the vehicle;
- FIG. 3 is a side elevation of a vehicle showing a first rearward viewing camera at a rear bumper or tailgate of the vehicle and a second rearward viewing camera at a CHMSL location of the vehicle;
- FIG. 4 is a block diagram showing that image data captured by a single camera and preconditions are used to provide three dimensional (3D) output data;
- FIG. 5 is a block diagram showing that image data captured by two cameras are used to provide three dimensional (3D) output data.
- a vehicle vision system and/or driver assist system and/or object detection system and/or alert system operates to capture images exterior of the vehicle and may process the captured image data to display images and to detect objects at or near the vehicle and in the predicted path of the vehicle, such as to assist a driver of the vehicle in maneuvering the vehicle in a rearward direction.
- the vision system includes an image processor or image processing system that is operable to receive image data from one or more cameras and provide an output to a display device for displaying images representative of the captured image data.
- the vision system may provide display, such as a rearview display or a top down or bird's eye or surround view display or the like.
- a vehicle 10 includes an imaging system or vision system 12 that includes at least one exterior viewing imaging sensor or camera, such as a rearward viewing imaging sensor or camera 14 a (and the system may optionally include multiple exterior viewing imaging sensors or cameras, such as a forward viewing camera 14 b at the front (or at the windshield) of the vehicle, and a sideward/rearward viewing camera 14 c , 14 d at respective sides of the vehicle), which captures images exterior of the vehicle, with the camera having a lens for focusing images at or onto an imaging array or imaging plane or imager of the camera ( FIG. 1 ).
- an imaging system or vision system 12 that includes at least one exterior viewing imaging sensor or camera, such as a rearward viewing imaging sensor or camera 14 a (and the system may optionally include multiple exterior viewing imaging sensors or cameras, such as a forward viewing camera 14 b at the front (or at the windshield) of the vehicle, and a sideward/rearward viewing camera 14 c , 14 d at respective sides of the vehicle), which captures images exterior of the vehicle, with
- a forward viewing camera may be disposed at the windshield of the vehicle and view through the windshield and forward of the vehicle, such as for a machine vision system (such as for traffic sign recognition, headlamp control, pedestrian detection, collision avoidance, lane marker detection and/or the like).
- the system includes a second rearward viewing camera 14 e that is disposed at a center high mounted stop lamp (CHMSL) location at the vehicle.
- the vision system 12 includes a control or electronic control unit (ECU) or processor 18 that is operable to process image data captured by the camera or cameras and may detect objects or the like and/or provide displayed images at a display device 16 for viewing by the driver of the vehicle (although shown in FIG.
- the data transfer or signal communication from the camera to the ECU may comprise any suitable data or communication link, such as a vehicle network bus or the like of the equipped vehicle.
- the amount of exterior cameras is increasing in vehicles, with many vehicles having four or more surround view cameras 14 a - d .
- Additional cameras in the CHMSL or rear spoiler area are included in some vehicles (such as camera 14 e in FIGS. 1-3 ).
- Machine vision algorithms used in object detection functions utilize one camera and require therefore additional data in order to perform three dimensional detection. These additional inputs stipulate pre-conditions for the machine vision algorithm. For example, for a trailer angle detection (TAD) function, a target sticker and user measurements may be required, while for an object detection function, vehicle motion may be required for accurate object location calculation.
- the pre-conditions are used to augment the two dimensional image data in order to be able to calculate three dimensional output data utilizing the machine vision algorithm (see FIG. 4 ).
- the vision system of the present invention uses image data captured by two (or more cameras) to detect object data in three dimensions.
- the image data of additional cameras at the vehicle can be used to detect object data in three dimensions.
- the locations of the cameras at the vehicle in 3D space is known and/or can be derived from the design mounting location and extrinsic camera location calibration algorithms.
- the location of the cameras in 3D space allows the system to determine the optical base between the cameras.
- the optical base and the image data captured by the two cameras can be utilized with stereoscopic detection algorithms as long as the objects of interest are present in the fields of view of both cameras and visible in both camera captured images.
- the stereoscopic detection principle that uses two cameras can be extended to multiple cameras as long as the object of interest is present in the fields of views of each of the cameras and visible in the camera images. Increasing the number of cameras will increase the detection confidence and accuracy.
- the system of the present invention may be used in various applications/features that would benefit from this technology.
- the system may be used for trailer angle detection, where detection and calculation of the angle between a tow vehicle and a trailer can be made by processing image data captured by two rearward viewing cameras.
- the system may be used for trailer hitch assist, which is a feature that autonomously maneuvers a vehicle in reverse and aligns the tow ball and trailer hitch automatically for trailer hitching (such as a system that utilizes aspects of the systems described in U.S. patent application Ser. No. 15/885,855, filed Feb. 1, 2018 (Attorney Docket MAG04 P-3253), which is hereby incorporated herein by reference in its entirety).
- the system may be used to enhance object detection to provide detection of objects and their location relative to the equipped vehicle in 3D space.
- This feature is commonly used for autopark and reverse automated emergency braking systems and the like.
- the system may also be used enhance pedestrian and/or vehicle detection and classification, which provides detection and classification of pedestrians and vehicles and their location relative to the equipped vehicle in 3D space. This feature is commonly used for autopark and reverse automated emergency braking systems and the like.
- the system may be used to enhance blind spot detection to detect vehicles located in a blind spot area at the side and rear of the equipped vehicle.
- the system may also be used to enhance autonomous vehicle control features, with all features that allow autonomous maneuvering of a vehicle utilizing the detection of objects in the vehicle environment.
- the vision system of the present invention utilizes two separate spaced apart cameras that have overlapping fields of view.
- the system may utilize two rearward viewing cameras, both disposed at a rear center region of the vehicle, but spaced vertically apart, such as one at the vehicle bumper and one at a spoiler of the vehicle or at a center high mounted stop lamp of the vehicle or the like.
- the system may include more rearward viewing cameras, such as two additional rearward viewing cameras spaced laterally apart and towards opposite sides of the vehicle, and at a known height relative to the two center cameras, to further enhance three dimensional sensing.
- Modern vehicles such as autonomous vehicles, may be equipped with up to eight or more cameras viewing exterior of the vehicle, such as a tri-camera/tri-focal camera disposed at the windshield of the vehicle and viewing forward through the windshield, two cameras at each side, and one at the rear (in addition to a rear backup camera).
- a tri-camera/tri-focal camera disposed at the windshield of the vehicle and viewing forward through the windshield, two cameras at each side, and one at the rear (in addition to a rear backup camera).
- Such cameras when disposed at a particular vehicle model are disposed at a set, determined location (such as at respective coordinates of a vehicle coordinate system) and thus the distance between any two cameras is known and set for any given vehicle.
- the system can use stereo vision processing of image data captured by those cameras to determine the location of the object or other vehicle in three dimensional space relative to the equipped vehicle.
- the system of the present invention can process image data captured by the cameras at the overlapping fields of view to detect objects present in the overlapping fields of view with enhanced three dimensional sensing and enhanced distance sensing and the like.
- the camera or sensor may comprise any suitable camera or sensor.
- the camera may comprise a “smart camera” that includes the imaging sensor array and associated circuitry and image processing circuitry and electrical connectors and the like as part of a camera module, such as by utilizing aspects of the vision systems described in International Publication Nos. WO 2013/081984 and/or WO 2013/081985, which are hereby incorporated herein by reference in their entireties.
- the system includes an image processor operable to process image data captured by the camera or cameras, such as for detecting objects or other vehicles or pedestrians or the like in the field of view of one or more of the cameras.
- the image processor may comprise an image processing chip selected from the EyeQ family of image processing chips available from Mobileye Vision Technologies Ltd. of Jerusalem, Israel, and may include object detection software (such as the types described in U.S. Pat. Nos. 7,855,755; 7,720,580 and/or 7,038,577, which are hereby incorporated herein by reference in their entireties), and may analyze image data to detect vehicles and/or other objects.
- the system may generate an alert to the driver of the vehicle and/or may generate an overlay at the displayed image to highlight or enhance display of the detected object or vehicle, in order to enhance the driver's awareness of the detected object or vehicle or hazardous condition during a driving maneuver of the equipped vehicle.
- the vehicle may include any type of sensor or sensors, such as imaging sensors or radar sensors or lidar sensors or ladar sensors or ultrasonic sensors or the like.
- the imaging sensor or camera may capture image data for image processing and may comprise any suitable camera or sensing device, such as, for example, a two dimensional array of a plurality of photosensor elements arranged in at least 640 columns and 480 rows (at least a 640 ⁇ 480 imaging array, such as a megapixel imaging array or the like), with a respective lens focusing images onto respective portions of the array.
- the photosensor array may comprise a plurality of photosensor elements arranged in a photosensor array having rows and columns.
- the imaging array has at least 300,000 photosensor elements or pixels, more preferably at least 500,000 photosensor elements or pixels and more preferably at least 1 million photosensor elements or pixels.
- the imaging array may capture color image data, such as via spectral filtering at the array, such as via an RGB (red, green and blue) filter or via a red/red complement filter or such as via an RCC (red, clear, clear) filter or the like.
- the logic and control circuit of the imaging sensor may function in any known manner, and the image processing and algorithmic processing may comprise any suitable means for processing the images and/or image data.
- the vision system and/or processing and/or camera and/or circuitry may utilize aspects described in U.S. Pat. Nos. 9,233,641; 9,146,898; 9,174,574; 9,090,234; 9,077,098; 8,818,042; 8,886,401; 9,077,962; 9,068,390; 9,140,789; 9,092,986; 9,205,776; 8,917,169; 8,694,224; 7,005,974; 5,760,962; 5,877,897; 5,796,094; 5,949,331; 6,222,447; 6,302,545; 6,396,397; 6,498,620; 6,523,964; 6,611,202; 6,201,642; 6,690,268; 6,717,610; 6,757,109; 6,802,617; 6,806,452; 6,822,563; 6,891,563; 6,946,978; 7,859,565; 5,550,677; 5,670,935
- the system may communicate with other communication systems via any suitable means, such as by utilizing aspects of the systems described in International Publication Nos. WO/2010/144900; WO 2013/043661 and/or WO 2013/081985, and/or U.S. Pat. No. 9,126,525, which are hereby incorporated herein by reference in their entireties.
- the system may utilize aspects of the parking assist systems described in U.S. Pat. No. 8,874,317 and/or U.S. Publication Nos. US-2017-0329346; US-2017-0317748; US-2017-0253237; US-2017-0050672; US-2017-0017847; US-2017-0015312 and/or US-2015-0344028, which are hereby incorporated herein by reference in their entireties.
- the system may utilize aspects of the trailering or trailer angle detection systems described in U.S. Pat. No. 9,085,261 and/or U.S. Pat. No. 6,690,268, and/or U.S. Publication Nos. US-2017-0254873; US-2017-0217372; US-2017-0050672; US-2015-0217693; US-2014-0160276; US-2014-0085472 and/or US-2015-0002670, and/or U.S. provisional applications, Ser. No. 62/533,694, filed Jul. 18, 2017, Ser. No. 62/518,765, filed Jun. 13, 2017, Ser. No. 62/474,646, filed Mar. 22, 2017, Ser. No. 62/474,645, filed Mar. 22, 2017, Ser. No. 62/474,644, filed Mar. 22, 2017, Ser. No. 62/466,449, filed Mar. 3, 2017, which are hereby incorporated herein by reference in their entireties.
- the system may utilize sensors, such as radar or lidar sensors or the like.
- the sensing system may utilize aspects of the systems described in U.S. Pat. Nos. 9,753,121; 9,689,967; 9,599,702; 9,575,160; 9,146,898; 9,036,026; 8,027,029; 8,013,780; 6,825,455; 7,053,357; 7,408,627; 7,405,812; 7,379,163; 7,379,100; 7,375,803; 7,352,454; 7,340,077; 7,321,111; 7,310,431; 7,283,213; 7,212,663; 7,203,356; 7,176,438; 7,157,685; 6,919,549; 6,906,793; 6,876,775; 6,710,770; 6,690,354; 6,678,039; 6,674,895 and/or 6,587,186, and/or International Publication Nos.
- the vision system may include a display for displaying images captured by one or more of the imaging sensors for viewing by the driver of the vehicle while the driver is normally operating the vehicle.
- the vision system may include a video display device, such as by utilizing aspects of the video display systems described in U.S. Pat. Nos.
- the vision system (utilizing the forward viewing camera and a rearward viewing camera and other cameras disposed at the vehicle with exterior fields of view) may be part of or may provide a display of a top-down view or bird's-eye view system of the vehicle or a surround view at the vehicle, such as by utilizing aspects of the vision systems described in International Publication Nos.
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Mechanical Engineering (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Aviation & Aerospace Engineering (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- General Physics & Mathematics (AREA)
- Automation & Control Theory (AREA)
- Electromagnetism (AREA)
- Traffic Control Systems (AREA)
- Image Analysis (AREA)
- Closed-Circuit Television Systems (AREA)
Abstract
Description
- The present application claims the filing benefits of U.S. provisional application Ser. No. 62/453,761, filed Feb. 2, 2017, which is hereby incorporated herein by reference in its entirety.
- The present invention relates generally to a vehicle vision system for a vehicle and, more particularly, to a vehicle vision system that utilizes one or more cameras at a vehicle.
- Use of imaging sensors in vehicle imaging systems is common and known. Examples of such known systems are described in U.S. Pat. Nos. 5,949,331; 5,670,935 and/or 5,550,677, which are hereby incorporated herein by reference in their entireties.
- The present invention provides a driver assistance system or vision system or autonomous vehicle control system for a vehicle that utilizes one or more cameras (preferably one or more CMOS cameras) to capture image data representative of images exterior of the vehicle, and processes image data captured from multiple cameras viewing a common region exterior the vehicles, whereby the image data captured by additional cameras at the vehicle can be used to detect objects and to determine the location of detected objects in three dimensional space.
- These and other objects, advantages, purposes and features of the present invention will become apparent upon review of the following specification in conjunction with the drawings.
-
FIG. 1 is a plan view of a vehicle with a vision system that incorporates cameras in accordance with the present invention; -
FIG. 2 is a side elevation of a vehicle showing a first rearward viewing camera at a rear bumper of the vehicle and a second rearward viewing camera at a spoiler of the vehicle; -
FIG. 3 is a side elevation of a vehicle showing a first rearward viewing camera at a rear bumper or tailgate of the vehicle and a second rearward viewing camera at a CHMSL location of the vehicle; -
FIG. 4 is a block diagram showing that image data captured by a single camera and preconditions are used to provide three dimensional (3D) output data; and -
FIG. 5 is a block diagram showing that image data captured by two cameras are used to provide three dimensional (3D) output data. - A vehicle vision system and/or driver assist system and/or object detection system and/or alert system operates to capture images exterior of the vehicle and may process the captured image data to display images and to detect objects at or near the vehicle and in the predicted path of the vehicle, such as to assist a driver of the vehicle in maneuvering the vehicle in a rearward direction. The vision system includes an image processor or image processing system that is operable to receive image data from one or more cameras and provide an output to a display device for displaying images representative of the captured image data. Optionally, the vision system may provide display, such as a rearview display or a top down or bird's eye or surround view display or the like.
- Referring now to the drawings and the illustrative embodiments depicted therein, a
vehicle 10 includes an imaging system orvision system 12 that includes at least one exterior viewing imaging sensor or camera, such as a rearward viewing imaging sensor orcamera 14 a (and the system may optionally include multiple exterior viewing imaging sensors or cameras, such as aforward viewing camera 14 b at the front (or at the windshield) of the vehicle, and a sideward/rearward viewing camera FIG. 1 ). Optionally, a forward viewing camera may be disposed at the windshield of the vehicle and view through the windshield and forward of the vehicle, such as for a machine vision system (such as for traffic sign recognition, headlamp control, pedestrian detection, collision avoidance, lane marker detection and/or the like). In the illustrated embodiment, the system includes a secondrearward viewing camera 14 e that is disposed at a center high mounted stop lamp (CHMSL) location at the vehicle. Thevision system 12 includes a control or electronic control unit (ECU) orprocessor 18 that is operable to process image data captured by the camera or cameras and may detect objects or the like and/or provide displayed images at adisplay device 16 for viewing by the driver of the vehicle (although shown inFIG. 1 as being part of or incorporated in or at an interiorrearview mirror assembly 20 of the vehicle, the control and/or the display device may be disposed elsewhere at or in the vehicle). The data transfer or signal communication from the camera to the ECU may comprise any suitable data or communication link, such as a vehicle network bus or the like of the equipped vehicle. - The amount of exterior cameras is increasing in vehicles, with many vehicles having four or more surround view cameras 14 a-d. Additional cameras in the CHMSL or rear spoiler area are included in some vehicles (such as
camera 14 e inFIGS. 1-3 ). Machine vision algorithms used in object detection functions utilize one camera and require therefore additional data in order to perform three dimensional detection. These additional inputs stipulate pre-conditions for the machine vision algorithm. For example, for a trailer angle detection (TAD) function, a target sticker and user measurements may be required, while for an object detection function, vehicle motion may be required for accurate object location calculation. The pre-conditions are used to augment the two dimensional image data in order to be able to calculate three dimensional output data utilizing the machine vision algorithm (seeFIG. 4 ). - In order to provide better algorithm performance and user friendliness, such preconditions should be removed. This will increase customer satisfaction and improve better algorithm performance. For example, removal of user measurements and target sticker requirements for trailer angle detection will improve customer satisfaction and is a cost saving (no target sticker). Similarly, removal of a vehicle motion requirement will allow for 3D detection of objects in static conditions (where the vehicle is static or non-moving and the object is static or non-moving), resulting in enhanced object detection performance compared to current systems.
- The vision system of the present invention uses image data captured by two (or more cameras) to detect object data in three dimensions. Thus, instead of using additional information associated with pre-conditions, the image data of additional cameras at the vehicle can be used to detect object data in three dimensions. The locations of the cameras at the vehicle in 3D space is known and/or can be derived from the design mounting location and extrinsic camera location calibration algorithms.
- The location of the cameras in 3D space allows the system to determine the optical base between the cameras. The optical base and the image data captured by the two cameras can be utilized with stereoscopic detection algorithms as long as the objects of interest are present in the fields of view of both cameras and visible in both camera captured images. The stereoscopic detection principle that uses two cameras can be extended to multiple cameras as long as the object of interest is present in the fields of views of each of the cameras and visible in the camera images. Increasing the number of cameras will increase the detection confidence and accuracy.
- The system of the present invention may be used in various applications/features that would benefit from this technology. For example, the system may be used for trailer angle detection, where detection and calculation of the angle between a tow vehicle and a trailer can be made by processing image data captured by two rearward viewing cameras. Also, for example, the system may be used for trailer hitch assist, which is a feature that autonomously maneuvers a vehicle in reverse and aligns the tow ball and trailer hitch automatically for trailer hitching (such as a system that utilizes aspects of the systems described in U.S. patent application Ser. No. 15/885,855, filed Feb. 1, 2018 (Attorney Docket MAG04 P-3253), which is hereby incorporated herein by reference in its entirety). Also, for example, the system may be used to enhance object detection to provide detection of objects and their location relative to the equipped vehicle in 3D space. This feature is commonly used for autopark and reverse automated emergency braking systems and the like. The system may also be used enhance pedestrian and/or vehicle detection and classification, which provides detection and classification of pedestrians and vehicles and their location relative to the equipped vehicle in 3D space. This feature is commonly used for autopark and reverse automated emergency braking systems and the like. Also, the system may be used to enhance blind spot detection to detect vehicles located in a blind spot area at the side and rear of the equipped vehicle. The system may also be used to enhance autonomous vehicle control features, with all features that allow autonomous maneuvering of a vehicle utilizing the detection of objects in the vehicle environment.
- Therefore, the vision system of the present invention utilizes two separate spaced apart cameras that have overlapping fields of view. For example, the system may utilize two rearward viewing cameras, both disposed at a rear center region of the vehicle, but spaced vertically apart, such as one at the vehicle bumper and one at a spoiler of the vehicle or at a center high mounted stop lamp of the vehicle or the like. The system may include more rearward viewing cameras, such as two additional rearward viewing cameras spaced laterally apart and towards opposite sides of the vehicle, and at a known height relative to the two center cameras, to further enhance three dimensional sensing.
- Modern vehicles, such as autonomous vehicles, may be equipped with up to eight or more cameras viewing exterior of the vehicle, such as a tri-camera/tri-focal camera disposed at the windshield of the vehicle and viewing forward through the windshield, two cameras at each side, and one at the rear (in addition to a rear backup camera). Such cameras, when disposed at a particular vehicle model are disposed at a set, determined location (such as at respective coordinates of a vehicle coordinate system) and thus the distance between any two cameras is known and set for any given vehicle.
- When the equipped vehicle is driven on a road (or when the equipped vehicle is parked), an object or other vehicle present in the fields of view of two (or more) of the cameras is seen by the two (or more) cameras. Because the distance (in a lateral direction and vertical direction or height) between the spaced apart cameras is known, the system can use stereo vision processing of image data captured by those cameras to determine the location of the object or other vehicle in three dimensional space relative to the equipped vehicle. By using two or more cameras that have fields of view encompassing a common exterior region (such as rearward or sideward or forward of the vehicle), the system of the present invention can process image data captured by the cameras at the overlapping fields of view to detect objects present in the overlapping fields of view with enhanced three dimensional sensing and enhanced distance sensing and the like.
- The camera or sensor may comprise any suitable camera or sensor. Optionally, the camera may comprise a “smart camera” that includes the imaging sensor array and associated circuitry and image processing circuitry and electrical connectors and the like as part of a camera module, such as by utilizing aspects of the vision systems described in International Publication Nos. WO 2013/081984 and/or WO 2013/081985, which are hereby incorporated herein by reference in their entireties.
- The system includes an image processor operable to process image data captured by the camera or cameras, such as for detecting objects or other vehicles or pedestrians or the like in the field of view of one or more of the cameras. For example, the image processor may comprise an image processing chip selected from the EyeQ family of image processing chips available from Mobileye Vision Technologies Ltd. of Jerusalem, Israel, and may include object detection software (such as the types described in U.S. Pat. Nos. 7,855,755; 7,720,580 and/or 7,038,577, which are hereby incorporated herein by reference in their entireties), and may analyze image data to detect vehicles and/or other objects. Responsive to such image processing, and when an object or other vehicle is detected, the system may generate an alert to the driver of the vehicle and/or may generate an overlay at the displayed image to highlight or enhance display of the detected object or vehicle, in order to enhance the driver's awareness of the detected object or vehicle or hazardous condition during a driving maneuver of the equipped vehicle.
- The vehicle may include any type of sensor or sensors, such as imaging sensors or radar sensors or lidar sensors or ladar sensors or ultrasonic sensors or the like. The imaging sensor or camera may capture image data for image processing and may comprise any suitable camera or sensing device, such as, for example, a two dimensional array of a plurality of photosensor elements arranged in at least 640 columns and 480 rows (at least a 640×480 imaging array, such as a megapixel imaging array or the like), with a respective lens focusing images onto respective portions of the array. The photosensor array may comprise a plurality of photosensor elements arranged in a photosensor array having rows and columns. Preferably, the imaging array has at least 300,000 photosensor elements or pixels, more preferably at least 500,000 photosensor elements or pixels and more preferably at least 1 million photosensor elements or pixels. The imaging array may capture color image data, such as via spectral filtering at the array, such as via an RGB (red, green and blue) filter or via a red/red complement filter or such as via an RCC (red, clear, clear) filter or the like. The logic and control circuit of the imaging sensor may function in any known manner, and the image processing and algorithmic processing may comprise any suitable means for processing the images and/or image data.
- For example, the vision system and/or processing and/or camera and/or circuitry may utilize aspects described in U.S. Pat. Nos. 9,233,641; 9,146,898; 9,174,574; 9,090,234; 9,077,098; 8,818,042; 8,886,401; 9,077,962; 9,068,390; 9,140,789; 9,092,986; 9,205,776; 8,917,169; 8,694,224; 7,005,974; 5,760,962; 5,877,897; 5,796,094; 5,949,331; 6,222,447; 6,302,545; 6,396,397; 6,498,620; 6,523,964; 6,611,202; 6,201,642; 6,690,268; 6,717,610; 6,757,109; 6,802,617; 6,806,452; 6,822,563; 6,891,563; 6,946,978; 7,859,565; 5,550,677; 5,670,935; 6,636,258; 7,145,519; 7,161,616; 7,230,640; 7,248,283; 7,295,229; 7,301,466; 7,592,928; 7,881,496; 7,720,580; 7,038,577; 6,882,287; 5,929,786 and/or 5,786,772, and/or U.S. Publication Nos. US-2014-0340510; US-2014-0313339; US-2014-0347486; US-2014-0320658; US-2014-0336876; US-2014-0307095; US-2014-0327774; US-2014-0327772; US-2014-0320636; US-2014-0293057; US-2014-0309884; US-2014-0226012; US-2014-0293042; US-2014-0218535; US-2014-0218535; US-2014-0247354; US-2014-0247355; US-2014-0247352; US-2014-0232869; US-2014-0211009; US-2014-0160276; US-2014-0168437; US-2014-0168415; US-2014-0160291; US-2014-0152825; US-2014-0139676; US-2014-0138140; US-2014-0104426; US-2014-0098229; US-2014-0085472; US-2014-0067206; US-2014-0049646; US-2014-0052340; US-2014-0025240; US-2014-0028852; US-2014-005907; US-2013-0314503; US-2013-0298866; US-2013-0222593; US-2013-0300869; US-2013-0278769; US-2013-0258077; US-2013-0258077; US-2013-0242099; US-2013-0215271; US-2013-0141578 and/or US-2013-0002873, which are all hereby incorporated herein by reference in their entireties. The system may communicate with other communication systems via any suitable means, such as by utilizing aspects of the systems described in International Publication Nos. WO/2010/144900; WO 2013/043661 and/or WO 2013/081985, and/or U.S. Pat. No. 9,126,525, which are hereby incorporated herein by reference in their entireties.
- The system may utilize aspects of the parking assist systems described in U.S. Pat. No. 8,874,317 and/or U.S. Publication Nos. US-2017-0329346; US-2017-0317748; US-2017-0253237; US-2017-0050672; US-2017-0017847; US-2017-0015312 and/or US-2015-0344028, which are hereby incorporated herein by reference in their entireties.
- The system may utilize aspects of the trailering or trailer angle detection systems described in U.S. Pat. No. 9,085,261 and/or U.S. Pat. No. 6,690,268, and/or U.S. Publication Nos. US-2017-0254873; US-2017-0217372; US-2017-0050672; US-2015-0217693; US-2014-0160276; US-2014-0085472 and/or US-2015-0002670, and/or U.S. provisional applications, Ser. No. 62/533,694, filed Jul. 18, 2017, Ser. No. 62/518,765, filed Jun. 13, 2017, Ser. No. 62/474,646, filed Mar. 22, 2017, Ser. No. 62/474,645, filed Mar. 22, 2017, Ser. No. 62/474,644, filed Mar. 22, 2017, Ser. No. 62/466,449, filed Mar. 3, 2017, which are hereby incorporated herein by reference in their entireties.
- The system may utilize sensors, such as radar or lidar sensors or the like. The sensing system may utilize aspects of the systems described in U.S. Pat. Nos. 9,753,121; 9,689,967; 9,599,702; 9,575,160; 9,146,898; 9,036,026; 8,027,029; 8,013,780; 6,825,455; 7,053,357; 7,408,627; 7,405,812; 7,379,163; 7,379,100; 7,375,803; 7,352,454; 7,340,077; 7,321,111; 7,310,431; 7,283,213; 7,212,663; 7,203,356; 7,176,438; 7,157,685; 6,919,549; 6,906,793; 6,876,775; 6,710,770; 6,690,354; 6,678,039; 6,674,895 and/or 6,587,186, and/or International Publication Nos. WO 2018/007995 and/or WO 2011/090484 and/or U.S. Publication Nos. US-2018-0015875; US-2017-0356994; US-2017-0315231; US-2017-0276788; US-2017-0254873; US-2017-0222311 and/or US-2010-0245066, and/or U.S. patent application Ser. No. 15/675,919, filed Aug. 14, 2017 (Attorney Docket MAG04 P-3092R), which are hereby incorporated herein by reference in their entireties.
- Optionally, the vision system may include a display for displaying images captured by one or more of the imaging sensors for viewing by the driver of the vehicle while the driver is normally operating the vehicle. Optionally, for example, the vision system may include a video display device, such as by utilizing aspects of the video display systems described in U.S. Pat. Nos. 5,530,240; 6,329,925; 7,855,755; 7,626,749; 7,581,859; 7,446,650; 7,338,177; 7,274,501; 7,255,451; 7,195,381; 7,184,190; 5,668,663; 5,724,187; 6,690,268; 7,370,983; 7,329,013; 7,308,341; 7,289,037; 7,249,860; 7,004,593; 4,546,551; 5,699,044; 4,953,305; 5,576,687; 5,632,092; 5,708,410; 5,737,226; 5,802,727; 5,878,370; 6,087,953; 6,173,501; 6,222,460; 6,513,252 and/or 6,642,851, and/or U.S. Publication Nos. US-2014-0022390; US-2012-0162427; US-2006-0050018 and/or US-2006-0061008, which are all hereby incorporated herein by reference in their entireties. Optionally, the vision system (utilizing the forward viewing camera and a rearward viewing camera and other cameras disposed at the vehicle with exterior fields of view) may be part of or may provide a display of a top-down view or bird's-eye view system of the vehicle or a surround view at the vehicle, such as by utilizing aspects of the vision systems described in International Publication Nos. WO 2010/099416; WO 2011/028686; WO 2012/075250; WO 2013/019795; WO 2012/075250; WO 2012/145822; WO 2013/081985; WO 2013/086249 and/or WO 2013/109869, and/or U.S. Publication No. US-2012-0162427, which are hereby incorporated herein by reference in their entireties.
- Changes and modifications in the specifically described embodiments can be carried out without departing from the principles of the invention, which is intended to be limited only by the scope of the appended claims, as interpreted according to the principles of patent law including the doctrine of equivalents.
Claims (20)
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US15/886,871 US11034295B2 (en) | 2017-02-02 | 2018-02-02 | Vehicle vision system using at least two cameras |
US17/304,029 US11648877B2 (en) | 2017-02-02 | 2021-06-14 | Method for detecting an object via a vehicular vision system |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201762453761P | 2017-02-02 | 2017-02-02 | |
US15/886,871 US11034295B2 (en) | 2017-02-02 | 2018-02-02 | Vehicle vision system using at least two cameras |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/304,029 Continuation US11648877B2 (en) | 2017-02-02 | 2021-06-14 | Method for detecting an object via a vehicular vision system |
Publications (2)
Publication Number | Publication Date |
---|---|
US20180215313A1 true US20180215313A1 (en) | 2018-08-02 |
US11034295B2 US11034295B2 (en) | 2021-06-15 |
Family
ID=62977071
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US15/886,871 Active 2039-06-14 US11034295B2 (en) | 2017-02-02 | 2018-02-02 | Vehicle vision system using at least two cameras |
US17/304,029 Active 2038-04-09 US11648877B2 (en) | 2017-02-02 | 2021-06-14 | Method for detecting an object via a vehicular vision system |
Family Applications After (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/304,029 Active 2038-04-09 US11648877B2 (en) | 2017-02-02 | 2021-06-14 | Method for detecting an object via a vehicular vision system |
Country Status (1)
Country | Link |
---|---|
US (2) | US11034295B2 (en) |
Cited By (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180362026A1 (en) * | 2017-06-20 | 2018-12-20 | Valeo Schalter Und Sensoren Gmbh | Method for operating a parking assistance apparatus of a motor vehicle with a combined view comprising a transparent trailer and surroundings |
US20200079164A1 (en) * | 2018-09-11 | 2020-03-12 | Connaught Electronics Ltd. | Method and system for determining a state of a trailer being towed by a vehicle |
US20200108877A1 (en) * | 2018-10-05 | 2020-04-09 | Toyota Motor Engineering & Manufacturing North America, Inc. | Spoiler apparatus for use with vehicles |
CN111174765A (en) * | 2020-02-24 | 2020-05-19 | 北京航天飞行控制中心 | Planet vehicle target detection control method and device based on visual guidance |
US11091212B2 (en) * | 2018-05-09 | 2021-08-17 | Toyota Jidosha Kabushiki Kaisha | Vehicular aerodynamic device |
US20220212668A1 (en) * | 2021-01-07 | 2022-07-07 | Magna Electronics Inc. | Vehicular trailer hitching assist system |
US20220212603A1 (en) * | 2019-05-15 | 2022-07-07 | Continental Automotive Systems, Inc. | Rear-view camera for pick-up trucks with open tailgates |
US20230256985A1 (en) * | 2022-02-14 | 2023-08-17 | Continental Advanced Lidar Solutions Us, Llc | Method and system for avoiding vehicle undercarriage collisions |
Families Citing this family (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US11034295B2 (en) | 2017-02-02 | 2021-06-15 | Magna Electronics Inc. | Vehicle vision system using at least two cameras |
US11620522B2 (en) * | 2019-12-31 | 2023-04-04 | Magna Electronics Inc. | Vehicular system for testing performance of headlamp detection systems |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5289321A (en) * | 1993-02-12 | 1994-02-22 | Secor James O | Consolidated rear view camera and display system for motor vehicle |
US20130286193A1 (en) * | 2012-03-21 | 2013-10-31 | Magna Electronics Inc. | Vehicle vision system with object detection via top view superposition |
US20140247352A1 (en) * | 2013-02-27 | 2014-09-04 | Magna Electronics Inc. | Multi-camera dynamic top view vision system |
US20150201120A1 (en) * | 2012-07-27 | 2015-07-16 | Clarion Co., Ltd. | In-vehicle image recognizer |
US20150210312A1 (en) * | 2014-01-30 | 2015-07-30 | Mobileye Vision Technologies Ltd. | Systems and methods for detecting low-height objects in a roadway |
US20160119587A1 (en) * | 2014-10-28 | 2016-04-28 | Nissan North America, Inc. | Vehicle object detection system |
US9558409B2 (en) * | 2012-09-26 | 2017-01-31 | Magna Electronics Inc. | Vehicle vision system with trailer angle detection |
US20170280091A1 (en) * | 2014-08-18 | 2017-09-28 | Jaguar Land Rover Limited | Display system and method |
Family Cites Families (34)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5550677A (en) | 1993-02-26 | 1996-08-27 | Donnelly Corporation | Automatic rearview mirror system using a photosensor array |
US5670935A (en) | 1993-02-26 | 1997-09-23 | Donnelly Corporation | Rearview vision system for vehicle including panoramic view |
WO2007053710A2 (en) | 2005-11-01 | 2007-05-10 | Donnelly Corporation | Interior rearview mirror with display |
WO2001064481A2 (en) | 2000-03-02 | 2001-09-07 | Donnelly Corporation | Video mirror systems incorporating an accessory module |
ES2391556T3 (en) | 2002-05-03 | 2012-11-27 | Donnelly Corporation | Object detection system for vehicles |
JP2005311698A (en) * | 2004-04-21 | 2005-11-04 | Auto Network Gijutsu Kenkyusho:Kk | Vehicle periphery visually recognizing apparatus |
US7720580B2 (en) | 2004-12-23 | 2010-05-18 | Donnelly Corporation | Object detection system for vehicle |
KR101735134B1 (en) | 2009-07-27 | 2017-05-24 | 마그나 일렉트로닉스 인크. | Parking assist system |
WO2012103193A1 (en) | 2011-01-26 | 2012-08-02 | Magna Electronics Inc. | Rear vision system with trailer angle detection |
US10099614B2 (en) | 2011-11-28 | 2018-10-16 | Magna Electronics Inc. | Vision system for vehicle |
US9242602B2 (en) * | 2012-08-27 | 2016-01-26 | Fotonation Limited | Rearview imaging systems for vehicle |
US9288446B2 (en) * | 2012-09-10 | 2016-03-15 | Nissan North America, Inc. | Vehicle video system |
US9446713B2 (en) | 2012-09-26 | 2016-09-20 | Magna Electronics Inc. | Trailer angle detection system |
DE102012025322B4 (en) * | 2012-12-22 | 2014-08-21 | Audi Ag | Motor vehicle with camera monitor system |
US10755110B2 (en) | 2013-06-28 | 2020-08-25 | Magna Electronics Inc. | Trailering assist system for vehicle |
US10160382B2 (en) | 2014-02-04 | 2018-12-25 | Magna Electronics Inc. | Trailer backup assist system |
JP6337269B2 (en) | 2014-04-09 | 2018-06-06 | パナソニックIpマネジメント株式会社 | Vehicle evaluation device |
US10328932B2 (en) | 2014-06-02 | 2019-06-25 | Magna Electronics Inc. | Parking assist system with annotated map generation |
US10525883B2 (en) * | 2014-06-13 | 2020-01-07 | Magna Electronics Inc. | Vehicle vision system with panoramic view |
US10214206B2 (en) | 2015-07-13 | 2019-02-26 | Magna Electronics Inc. | Parking assist system for vehicle |
US10078789B2 (en) | 2015-07-17 | 2018-09-18 | Magna Electronics Inc. | Vehicle parking assist system with vision-based parking space detection |
US10086870B2 (en) | 2015-08-18 | 2018-10-02 | Magna Electronics Inc. | Trailer parking assist system for vehicle |
US20170217372A1 (en) | 2016-02-02 | 2017-08-03 | Magna Electronics Inc. | Wireless camera system for vehicle and trailer |
US20170253237A1 (en) | 2016-03-02 | 2017-09-07 | Magna Electronics Inc. | Vehicle vision system with automatic parking function |
US10863335B2 (en) | 2016-03-04 | 2020-12-08 | Magna Electronics Inc. | Vehicle trailer angle detection system using short range communication devices |
US20170317748A1 (en) | 2016-05-02 | 2017-11-02 | Magna Electronics Inc. | Vehicle positioning by visible light communication |
US20170329346A1 (en) | 2016-05-12 | 2017-11-16 | Magna Electronics Inc. | Vehicle autonomous parking system |
US20180154831A1 (en) * | 2016-07-29 | 2018-06-07 | Faraday&Future Inc. | Composite display from one or more cameras |
US11014561B2 (en) | 2017-02-01 | 2021-05-25 | Magna Electronics Inc. | Vehicle trailer hitch assist system |
US11034295B2 (en) | 2017-02-02 | 2021-06-15 | Magna Electronics Inc. | Vehicle vision system using at least two cameras |
US10706291B2 (en) | 2017-03-03 | 2020-07-07 | Magna Electronics Inc. | Trailer angle detection system for vehicle |
US10885652B2 (en) | 2017-03-22 | 2021-01-05 | Magna Electronics Inc. | Trailer angle detection system for vehicle |
US10552976B2 (en) | 2017-03-22 | 2020-02-04 | Magna Electronics Inc. | Trailer angle detection system for vehicle |
US10532698B2 (en) | 2017-07-14 | 2020-01-14 | Magna Electronics Inc. | Trailer angle detection using rear backup camera |
-
2018
- 2018-02-02 US US15/886,871 patent/US11034295B2/en active Active
-
2021
- 2021-06-14 US US17/304,029 patent/US11648877B2/en active Active
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US5289321A (en) * | 1993-02-12 | 1994-02-22 | Secor James O | Consolidated rear view camera and display system for motor vehicle |
US20130286193A1 (en) * | 2012-03-21 | 2013-10-31 | Magna Electronics Inc. | Vehicle vision system with object detection via top view superposition |
US20150201120A1 (en) * | 2012-07-27 | 2015-07-16 | Clarion Co., Ltd. | In-vehicle image recognizer |
US9558409B2 (en) * | 2012-09-26 | 2017-01-31 | Magna Electronics Inc. | Vehicle vision system with trailer angle detection |
US20140247352A1 (en) * | 2013-02-27 | 2014-09-04 | Magna Electronics Inc. | Multi-camera dynamic top view vision system |
US20150210312A1 (en) * | 2014-01-30 | 2015-07-30 | Mobileye Vision Technologies Ltd. | Systems and methods for detecting low-height objects in a roadway |
US20170280091A1 (en) * | 2014-08-18 | 2017-09-28 | Jaguar Land Rover Limited | Display system and method |
US20160119587A1 (en) * | 2014-10-28 | 2016-04-28 | Nissan North America, Inc. | Vehicle object detection system |
Cited By (11)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20180362026A1 (en) * | 2017-06-20 | 2018-12-20 | Valeo Schalter Und Sensoren Gmbh | Method for operating a parking assistance apparatus of a motor vehicle with a combined view comprising a transparent trailer and surroundings |
US10464555B2 (en) * | 2017-06-20 | 2019-11-05 | Valeo Schalter Und Sensoren Gmbh | Method for operating a parking assistance apparatus of a motor vehicle with a combined view comprising a transparent trailer and surroundings |
US11091212B2 (en) * | 2018-05-09 | 2021-08-17 | Toyota Jidosha Kabushiki Kaisha | Vehicular aerodynamic device |
US20200079164A1 (en) * | 2018-09-11 | 2020-03-12 | Connaught Electronics Ltd. | Method and system for determining a state of a trailer being towed by a vehicle |
US20200108877A1 (en) * | 2018-10-05 | 2020-04-09 | Toyota Motor Engineering & Manufacturing North America, Inc. | Spoiler apparatus for use with vehicles |
US10780927B2 (en) * | 2018-10-05 | 2020-09-22 | Toyota Motor Engineering & Manufacturing North America, Inc. | Spoiler apparatus for use with vehicles |
US20220212603A1 (en) * | 2019-05-15 | 2022-07-07 | Continental Automotive Systems, Inc. | Rear-view camera for pick-up trucks with open tailgates |
US11884215B2 (en) * | 2019-05-15 | 2024-01-30 | Continental Automotive Systems, Inc. | Rear-view camera for pick-up trucks with open tailgates |
CN111174765A (en) * | 2020-02-24 | 2020-05-19 | 北京航天飞行控制中心 | Planet vehicle target detection control method and device based on visual guidance |
US20220212668A1 (en) * | 2021-01-07 | 2022-07-07 | Magna Electronics Inc. | Vehicular trailer hitching assist system |
US20230256985A1 (en) * | 2022-02-14 | 2023-08-17 | Continental Advanced Lidar Solutions Us, Llc | Method and system for avoiding vehicle undercarriage collisions |
Also Published As
Publication number | Publication date |
---|---|
US11034295B2 (en) | 2021-06-15 |
US11648877B2 (en) | 2023-05-16 |
US20210300245A1 (en) | 2021-09-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11648877B2 (en) | Method for detecting an object via a vehicular vision system | |
US11794647B2 (en) | Vehicular vision system having a plurality of cameras | |
US11756230B2 (en) | Vehicular trailer hitching assist system with determination of location of hitch | |
US11836989B2 (en) | Vehicular vision system that determines distance to an object | |
US10078789B2 (en) | Vehicle parking assist system with vision-based parking space detection | |
US10647282B2 (en) | Vehicle vision system with undercarriage cameras | |
US10040481B2 (en) | Vehicle trailer angle detection system using ultrasonic sensors | |
US11875575B2 (en) | Vehicular trailering assist system with trailer collision angle detection | |
US11618383B2 (en) | Vehicular vision system with display of combined images | |
US10773707B2 (en) | Vehicle control system with reverse assist | |
US11787339B2 (en) | Trailer hitching assist system with trailer coupler detection | |
US10300859B2 (en) | Multi-sensor interior mirror device with image adjustment | |
US20220134951A1 (en) | Vehicular trailer hitching assist system with hitch ball location estimation | |
US10647266B2 (en) | Vehicle vision system with forward viewing camera |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
AS | Assignment |
Owner name: MAGNA ELECTRONICS INC., MICHIGAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:DIESSNER, HORST D.;REEL/FRAME:048752/0832 Effective date: 20190330 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT RECEIVED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |