US20230077247A1 - Vehicular camera monitoring system - Google Patents

Vehicular camera monitoring system Download PDF

Info

Publication number
US20230077247A1
US20230077247A1 US18/054,963 US202218054963A US2023077247A1 US 20230077247 A1 US20230077247 A1 US 20230077247A1 US 202218054963 A US202218054963 A US 202218054963A US 2023077247 A1 US2023077247 A1 US 2023077247A1
Authority
US
United States
Prior art keywords
driver
vehicle
passenger
display screen
monitoring system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
US18/054,963
Inventor
Niall R. Lynam
Yuesheng Lu
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Magna Electronics Inc
Original Assignee
Magna Electronics Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Magna Electronics Inc filed Critical Magna Electronics Inc
Priority to US18/054,963 priority Critical patent/US20230077247A1/en
Publication of US20230077247A1 publication Critical patent/US20230077247A1/en
Pending legal-status Critical Current

Links

Images

Classifications

    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/02Rear-view mirror arrangements
    • B60R1/06Rear-view mirror arrangements mounted on vehicle exterior
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R11/00Arrangements for holding or mounting articles, not otherwise provided for
    • B60R11/04Mounting of cameras operative during drive; Arrangement of controls thereof relative to the vehicle
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60KARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00Arrangement of adaptations of instruments
    • B60K35/22
    • B60K35/28
    • B60K35/60
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/02Rear-view mirror arrangements
    • B60R1/04Rear-view mirror arrangements mounted inside vehicle
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/12Mirror assemblies combined with other articles, e.g. clocks
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B60R1/26Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view to the rear of the vehicle
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/28Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with an adjustable field of view
    • GPHYSICS
    • G08SIGNALLING
    • G08GTRAFFIC CONTROL SYSTEMS
    • G08G1/00Traffic control systems for road vehicles
    • G08G1/16Anti-collision systems
    • G08G1/168Driving aids for parking, e.g. acoustic or visual feedback on parking space
    • B60K2360/1526
    • B60K2360/165
    • B60K2360/173
    • B60K2360/21
    • B60K2360/779
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60KARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2370/00Details of arrangements or adaptations of instruments specially adapted for vehicles, not covered by groups B60K35/00, B60K37/00
    • B60K2370/16Type of information
    • B60K2370/165Videos and animations
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/30Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles providing vision in the non-visible spectrum, e.g. night or infrared vision
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/12Mirror assemblies combined with other articles, e.g. clocks
    • B60R2001/1253Mirror assemblies combined with other articles, e.g. clocks with cameras, video cameras or video screens
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B60R2300/303Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing using joined images, e.g. multiple camera images
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/802Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring and displaying vehicle exterior blind spot views
    • B60R2300/8026Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring and displaying vehicle exterior blind spot views in addition to a rear-view mirror system
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B60VEHICLES IN GENERAL
    • B60RVEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/8066Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for monitoring rearward traffic
    • BPERFORMING OPERATIONS; TRANSPORTING
    • B62LAND VEHICLES FOR TRAVELLING OTHERWISE THAN ON RAILS
    • B62DMOTOR VEHICLES; TRAILERS
    • B62D15/00Steering not otherwise provided for
    • B62D15/02Steering position indicators ; Steering position determination; Steering aids
    • B62D15/027Parking aids, e.g. instruction means
    • B62D15/028Guided parking by providing commands to the driver, e.g. acoustically or optically

Definitions

  • the present invention relates to vehicular vision systems that display video images derived from image data captured by one or more cameras of the vehicle.
  • the present invention provides a camera monitoring system that has a driver-side video display screen that displays video images derived from image data captured by a driver-side camera and a passenger-side video display screen that displays video images derived from image data captured by a passenger-side camera.
  • the displayed video images (derived from image data captured by the driver-side and passenger-side cameras) allow the driver to see vehicles approaching and/or overtaking the equipped vehicle in the respective side traffic lanes adjacent to the traffic lane of the road along which the equipped vehicle is traveling.
  • the driver-side video display screen displays parking assist video images derived from image data captured by the driver-side camera and the passenger-side video display screen displays parking assist video images derived from image data captured by the passenger-side camera.
  • the parking assist images may comprise video images of the ground adjacent to and forward and/or rearward of the vehicle to assist the driver in maneuvering the vehicle into or out from a parking space.
  • the driver-side video display screen Upon completion of the parking-space exit maneuver when the vehicle has fully exited a parking space and commences normal forward driving along the road, the driver-side video display screen automatically reverts to display of forward-driving driving assist video images derived from image data captured by the driver-side camera, and the passenger-side video display screen automatically reverts to display of forward-driving driving assist video images derived from image data captured by the passenger-side camera, with the displayed forward-driving driving assist video images (that are viewable by a driver of the equipped vehicle when normally driving the vehicle forward in traffic and when not parallel or perpendicular parking into a vacant parking space or when not exiting from a parking space) being different from the displayed parking assist video images (that are viewable by the driver of the equipped vehicle when parallel or perpendicular parking into a vacant parking space or when exiting from a parking space).
  • Differences between forward-driving driving assist video images and parking assist video images can include difference in the vertical field of view as seen on the display (preferably, the vertical field of view is larger when parking so that ground area adjacent to and along the side of the vehicle is viewable to the driver on the display screen). Differences between forward-driving driving assist video images and parking assist video images can include difference in the horizontal field of view as seen on the display (preferably, the horizontal field of view is narrower when parking so that the driver can focus viewing on obstacles or other vehicles close to the side of the equipped vehicle). Differences between forward-driving driving assist video images and parking assist video images can include zoomed images so the driver can better view what is close to and adjacent to the being parked vehicle.
  • Differences between forward-driving driving assist video images and parking assist video images can include electronically generated graphic overlays, icons, indicia and/or the like, that overlay in real-time on the parking assist video images to allow the driver to gauge distance to, separation from and approach to obstacles, such as curbs, posts, walls and/or other vehicles, as the equipped vehicle is being parked.
  • such electronically generated graphic overlays are generated responsive to sensor data captured by the likes of ultrasonic or radar or lidar or camera sensors of the equipped vehicle (including image data sensor output from one or more of the driver-side camera, the passenger-side camera and the rear viewing camera) being provided to a data processor where such sensor data is processed to determine the likes of curbs, posts, walls and/or other vehicles, as the equipped vehicle is being parked.
  • FIG. 1 is a perspective view of a driver-side camera an driver-side display screen in accordance with the present invention
  • FIG. 2 is a plan view of a vehicle, showing the driver-side display and the passenger-side display in the vehicle;
  • FIG. 3 is a plan view of the vehicle of FIG. 2 , showing the vehicle during a parallel parking maneuver;
  • FIG. 4 is a perspective view of an exemplary camera and video display for a camera monitoring system
  • FIG. 5 is a perspective view of another exemplary camera and video display for a camera monitoring system
  • FIG. 6 is a plan view of a vehicle, showing the respective fields of view of a driver-side sideview camera, a rearward viewing camera and a passenger-side sideview camera;
  • FIGS. 7 A-C are plan views of a dual-mode interior rearview mirror of the vehicle of FIG. 6 that displays video images derived from the rearward viewing camera ( FIG. 7 A ) or all three cameras ( FIGS. 7 B and 7 C ); and
  • FIG. 8 is a list of preferred display screen parameters and/or characteristics for the camera monitoring system.
  • an exterior rearview mirror system 10 for a vehicle 11 includes an exterior rearview mirror assembly 12 and an in-cabin display device or module 14 disposed at or mounted at the interior of the vehicle door 11 a or otherwise near to the exterior rearview mirror assembly 12 ( FIG. 1 ).
  • the exterior rearview mirror assembly 12 is mounted at the side 11 b of the host or subject or equipped vehicle 11 and includes a mirror reflective element 16 and a mirror shell or casing 18 , and a camera or imaging sensor 20 that is disposed at the mirror casing 18 and has a generally rearward and sideward field of view at the side of the equipped vehicle.
  • An interior rearview mirror assembly 40 is mounted in the interior cabin of the equipped vehicle.
  • Interior mirror assembly 40 may be attached at an in-cabin side of a windshield of the equipped vehicle.
  • interior mirror assembly 40 comprises a dual-mode interior rearview video mirror that can switch from a traditional reflection mode to a panoramic live-video display mode, such as is by utilizing aspects of the mirror assemblies and systems described in U.S. Pat. Nos. 10,442,360; 10,421,404; 10,166,924 and/or 10,046,706, and/or U.S. Publication Nos. US-2019-0146297; US-2019-0118717 and/or US-2017-0355312, which are all hereby incorporated herein by reference in their entireties.
  • Dual-mode inter dual-mode interior rearview video mirror 40 receive video feed from either or both of side cameras 20 , 21 and/or may receive video feed from rear-mounted camera 50 .
  • the driver-side exterior rearview mirror assembly 12 may comprise an FMVSS-111 compliant planar reflective element 16 that meets FMVSS-111 field of view requirements but does not substantially exceed them (for example, the reflective element may have a field of view of at least about 8 degrees from the side of the vehicle, but not more than about 12 degrees).
  • the camera 20 may be disposed at any suitable location at mirror assembly 12 , such as at an output portion of the mirror casing 18 , such as shown in FIG. 1 . Alternatively, and as can be seen in FIG.
  • the side camera can be behind the mirror reflective element and can view via a light transmitting aperture created (preferably via the likes of laser ablation) in the mirror reflector used for the reflective element (or the side camera can view via a transflective mirror reflector used for the reflective element). Presence of the camera can be masked by using a jail bar or dot pattern of the mirror reflector in front of where the camera is disposed.
  • the display device or module may utilize aspects of the modules described in U.S. Publication Nos. US-2018-0134217 and/or US-2014-0285666, which are hereby incorporated herein by reference in their entireties.
  • the small or reduced size mirror reflective element provides a reduced size rearward field of view to the driver of the vehicle (as compared to larger size mirror reflective elements), and the smaller field of view may be supplemented or enhanced via the video display device 14 disposed at the interior of the vehicle cabin and in an area at or near the exterior rearview mirror assembly.
  • display device 14 may be disposed at an inboard portion of the vehicle door 11 a and may be disposed immediately opposite exterior rearview mirror assembly 12 .
  • the exterior rearview mirror assembly 12 and display device 14 may be incorporated into an imaging module that is mountable to or incorporated in a vehicle door, such that all of the wiring and communication links between the camera and display are provided within the module.
  • the module may be mounted at or installed in the door assembly and may be electrically connected to a vehicle wiring harness via an electrical connector at the door, so that power and/or control of the module and/or system is provided via the vehicle wiring harness.
  • the display device 14 includes a video display screen 22 supported at or housed by a housing or casing or structure 24 so that the display screen is at a location near the door that is readily viewable by a driver of the vehicle and is an area that the driver of the vehicle readily cognitively associates with where to look to see what is to the side and rear of the vehicle (because of the proximity of the display screen to the exterior rearview mirror assembly).
  • the display screen and the exterior mirror move with the vehicle door when the vehicle door is opened and closed.
  • Video display screen 22 may comprise any suitable video display screen, such as discussed below, and may comprise a touch screen to allow for user control of the video display and/or other vehicle functions via touching of appropriate locations at the display screen by the driver of the vehicle.
  • the video display screen may comprise directional arrow inputs to allow the user or driver to zoom in or out or to pan the video display up or down or to either side to set the rearward field of view at the display screen to a preferred setting for that individual.
  • the system may include a memory system that also allows the user to set the rearward field of view to a desired setting and to store that setting such as in a similar manner as is done for memory mirror settings and/or memory seat settings and the like.
  • CMS Camera monitoring systems
  • the advantages include reduced wind drag, reduced vehicle weight, and improved field of view.
  • reduced depth perception is a concern. Because a typical camera only provides monocular view, the ability to judge depth due to parallax is lost. Additionally, as a trade-off to gaining additional field of view on a given display size, the image may become distorted to the point where judgment of distance using relative object size is also hindered.
  • the video display displays an invariant field of view (i.e., the driver is unable to vary what he or she sees when viewing the video display, as can be done with an exterior mirror by moving the driver's head to a different position).
  • the present invention provides the following concepts to help the driver visualize where other vehicles are relative to his or her own vehicle.
  • CMS offers opportunity for overlays to be superimposed on the real-time video being displayed on the in-cabin CMS monitor. Such overlays can enhance the ease and efficacy of CMS-aided parking.
  • CMS may use overlays and different types of camera views shown on the in-cabin monitor during a parking maneuver.
  • the camera views seen by the driver automatically revert to what is suitable for normal driving either immediately or once a threshold condition is met (such as, for example, when the vehicle's speed reaches a threshold speed such as 10 miles per hour after exiting a parking space or when the vehicle has travelled a threshold distance such as 100 feet after exiting the parking space or when a time period such as 10 seconds has elapsed after the vehicle has exited the parking space).
  • a threshold condition such as, for example, when the vehicle's speed reaches a threshold speed such as 10 miles per hour after exiting a parking space or when the vehicle has travelled a threshold distance such as 100 feet after exiting the parking space or when a time period such as 10 seconds has elapsed after the vehicle has exited the parking space).
  • the camera view seen by the driver viewing the driver-side video monitor continues to enable the driver to see any cyclist or vehicle approaching the vehicle even while the camera view seen by the driver viewing the passenger-side video monitor may have changed during a later stage of the parallel parking maneuver (as the vehicle is maneuvered further into the parking space so as to be alongside the curb) to allow the driver see alignment of the vehicle with the curb.
  • the camera view seen by the driver viewing the driver-side video monitor enables the driver to see any cyclist or vehicle approaching on the left side (or should the vehicle be parked with the driver-side camera curbside, the camera view seen by the driver viewing the passenger-side video monitor needs to enable the driver to see any cyclist or vehicle approaching on the right side).
  • the system automatically displays parking assist video images at the parking-side video display screen (typically the passenger side) and automatically displays driving assist video images at the road-side video display screen (typically the driver side), such as responsive to the driver turning the ignition on after the vehicle is parked or responsive to the driver shifting the vehicle into forward gear (or reverse gear if the vehicle has to be backed up a little to achieve sufficient clearance from the vehicle parked ahead of the equipped vehicle for exiting the parking space).
  • parking assist video images at the parking-side video display screen typically the passenger side
  • driving assist video images at the road-side video display screen typically the driver side
  • the driver-side video monitor and the passenger-side video monitor display driving assist video images representative of the sideward and rearward or rearward-view portion of the field of view of the respective driver-side camera and passenger-side camera.
  • the driver of the vehicle may drive past a parking space between two spaced apart parked vehicles in which the driver intends to parallel park the vehicle.
  • the driver shifts the vehicle into reverse gear to commence reversing of the vehicle into the parking space.
  • the driver-side video monitor Upon shifting into reverse gear and/or commencing the parking maneuver into the parking space (at the passenger side of the road), the driver-side video monitor continues display of the driving assist video images and the passenger-side video monitor automatically ceases display of the driving assist video images and displays parking assist video images that are different than the driving assist video images.
  • the parking assist video images may be directed more downward (as compared to the driving assist video images) so as to include the ground region at the passenger side of the vehicle and at and around the parking space so that the driver can more readily see the ground and objects or curb at or in the parking space.
  • the passenger-side video monitor may display downward viewing parking assist video images to show the driver the curb or objects in or adjacent to the target parking space, while the driver-side video monitor may display rearward viewing parking assist video images to show the driver any rearward approaching vehicles or bicycles or the like that may be approaching along the road.
  • the driver-side video monitor may provide video images of other traffic or the like, while the passenger-side video monitor may provide video images of the curb or parking space.
  • the driver-side parking assist video images thus may be different than the passenger-side parking assist video images (with the different video images derived from different portions of the fields of view of the respective cameras).
  • the driver-side video display screen displays the driving assist video images while the passenger-side video display screen displays the parking assist video images.
  • the driver-side video monitor continues display of the driving assist video images and the passenger-side video monitor automatically ceases display of the parking assist video images and displays the driving assist video images that are different than the driving assist video images.
  • the system may determine the vehicle has exited the parking space (and thus may switch the passenger-side video monitor to display the driving assist video images) responsive to a threshold event or condition, such as responsive to the vehicle reaching a threshold forward speed (e.g., 5 mph or 10 mph or 15 mph), or the vehicle traveling a threshold forward distance (e.g., 20 ft or 50 ft or 100 ft), or elapsing of a threshold time period (e.g., 5 seconds or 10 seconds or 15 seconds) after the vehicle has exited the parking space.
  • a threshold forward speed e.g., 5 mph or 10 mph or 15 mph
  • a threshold forward distance e.g., 20 ft or 50 ft or 100 ft
  • a threshold time period e.g., 5 seconds or 10 seconds or 15 seconds
  • Rear backup systems with FMVSS 111-required field of view and in-cabin video display performance are now mandatory on all new light vehicles made in or imported into the USA.
  • a CMS-equipped vehicle utilizing a driver-side video monitor (receiving its video feed from a driver-side CMS camera) and a passenger-side video monitor (receiving its video feed from a passenger-side CMS camera), rear backup video images compliant with FMVSS 111 likely will continue to be displayed on a separate centrally-located video screen when rear gear is selected by the driver.
  • NHTSA in 2008 undertook an On-Road Study of Drivers' Use of Rearview Video Systems.
  • a driver's use of rearview video (RV) systems was observed during staged and naturalistic backing maneuvers to determine whether drivers look at the rearview display during reversing maneuvers and whether use of the system affects the drivers' reversing behavior.
  • the vehicles used were equipped with an interior rearview mirror, a planar driver-side sideview mirror and a convex passenger-side sideview mirror. Overall, results of this study revealed that drivers look at rearview video displays during reversing maneuvers at least some of the time.
  • a CMS-equipped vehicle such as, for example, a 2019 Lexus ES or 2019 Audi e-tron
  • a CMS-equipped vehicle such as, for example, a 2019 Lexus ES or 2019 Audi e-tron
  • a center display may be provided.
  • a video display device may be provided in the interior rearview mirror assembly, such as to display video images at the mirror reflective element for viewing by the driver of the vehicle.
  • dual-mode interior rearview mirror assemblies provide a reflected view to the driver of the vehicle (such as is provided by a conventional interior rearview mirror when adjusted to the driver's preferences) and, if selected by the driver, provide a video display view to the driver, whereby a video display screen disposed behind the reflective element operates to display video images derived from image data captured by a rearward viewing camera of the vehicle.
  • Such dual-mode interior rearview mirror assemblies may utilize aspects of the mirror assemblies and systems described in U.S. Pat.
  • Vehicles equipped with such dual-mode interior mirror assemblies use a rear-mounted and rearward viewing camera that is separate from the rear backup camera required under FMVSS 111.
  • a dual-mode interior mirror is currently usually an optional feature whereas installation of a rear backup camera is mandated on all vehicles in the USA.
  • rear backup cameras have a field of vision of at least 130 degrees (and many have a field of vision of at least 180 degrees) whereas rear-mounted cameras providing video feed to dual-mode interior mirrors typically have a field of vision of 50 degree or thereabouts.
  • CMOS imagers used for rear backup cameras typically range from VGA resolution to 1.28 megapixel or thereabouts.
  • CMOS imagers providing video feed to dual-mode interior mirrors are 1.6 megapixel resolution. If CMS and the rear backup function share a common rear camera, the imager resolution should be at least 8 megapixels (and more preferably at least 15 megapixels), and the system should use a wide angle lens and sophisticated de-warping and image distortion-reducing video processing.
  • a dual-mode interior mirror can operate either using a single video feed from a rear camera or using video feeds from a driver-side camera, a rear camera and a passenger-side camera.
  • the interior mirror can display video images from the rearward camera only ( FIG. 7 A ), or video images from the side cameras can be displayed (i) in a manner that distinctly distinguishes/demarcates video images monitoring the side lanes from the main video images provided by the rear camera ( FIG. 7 B ) or (ii) with the video feeds from all three cameras seamlessly stitched together to form a single panoramic video image ( FIG. 7 C ).
  • two side cameras and two respective driver-side and passenger-side interior monitors with the left or driver-side video monitor that is associated with the driver-side side camera mounted left of the steering column and preferably at or where an exterior rearview mirror is located or would have been located, and with the right passenger-side video monitor that is associated with the passenger-side side camera mounted right of center of the vehicle and optionally at or where an exterior rearview mirror is located or would have been located
  • a dual-mode interior rearview mirror assembly typically receiving video feed from a rear-mounted camera or potentially receiving video feeds, and such as shown in FIG.
  • the video feeds from all three cameras are provided to an electronic control unit (ECU) that includes an image processor/video processor.
  • ECU electronice control unit
  • image processor/video processor Taking as exemplary the driver-side camera video feed and the video feed to the dual-mode interior mirror (with the processing of the passenger-side camera video feed being similar to the processing of the driver-side camera video feed), both such video feeds are processed at the ECU and features commonly viewed or displayed in both the driver-side monitor and the dual-mode interior mirror are identified and tracked.
  • the respective field of view of the driver-side sideview camera as displayed on the driver-side monitor and the field of view of the rear-mounted camera provided to the dual-mode interior mirror are panned, adjusted or cropped so that confusing duplication of video image display seen by the driver using the driver-side monitor and the dual-mode interior mirror is mitigated or reduced.
  • the ECU thus dynamically adjusts or balances what is displayed to be seen on the driver-side monitor and on the dual-mode interior mirror so that duplication is minimized or mitigated while providing to the driver wide angle and readily discernible video images of traffic behind, approaching and/or overtaking the equipped vehicle.
  • the field of view of the dual-mode interior mirror narrows or reduces (from, for example, around 50 degrees to around 30 degrees or lower) when either the driver selects a turn signal indicating intent to make a lane change, or when a lane keeping system automatically detects that the vehicle is moving to make a lane change (such a lane keeping system that may utilize aspects of the systems described in U.S. Pat. Nos.
  • the CMS continues to be operable after a driver turns off the ignition and prepares to exit the vehicle.
  • a driver Before opening a vehicular door to exit, a driver can and should check in the driver sideview mirror to see if the likes of an approaching cyclist is not in hazard of colliding with the opened door.
  • the CMS remains active after the ignition is turned off so that the driver can check what may be approach at the side before opening the door. Lighting and other accessories remain powered and active in many vehicles today for a time period (typically 30 seconds to a five minutes or ten minutes or more) after the ignition is turned off by a driver.
  • CMS can similarly remain temporarily powered when the driver turns off the ignition.
  • the CMS (at least the driver-side camera and the driver-side video display screen) remains operational after ignition shut off and until a period of time following a detection or determination that the driver has left the driver seat (e.g., a period of time following determination, such as by a driver-seat monitoring or occupancy detection system, that the driver seat is unoccupied).
  • the period of time does not start until the driver has left the seat/vehicle, so the CMS (at least the driver-side camera and the driver-side video display screen, but optionally the passenger-side camera and the passenger-side video display screen as well) remains operating while the driver is in the driver seat so that the driver can use the CMS to determine if a pedestrian or bicycle or other vehicle is approaching the subject vehicle at the driver side of the subject vehicle before the driver opens the vehicle door.
  • the driver-seat occupancy determination may only be made (i.e., the driver-seat occupancy detecting system may only operate) responsive to the ignition of the vehicle being turned off when the vehicle is parked.
  • Driver-seat occupancy can be detected by likes of an in-seat weight sensor, or by a camera-based driver monitoring systems such as the types described in U.S. Pat. No. 9,405,120 and/or U.S. Publication Nos. US-2017-0274906; US-2016-0209647; US-2016-0137126; US-2015-0352953; US-2015-0296135; US-2015-0294169; US-2015-0232030; US-2015-0092042; US-2015-0022664; US-2015-0015710; US-2015-0009010 and/or US-2014-0336876, which are hereby incorporated herein by reference in their entireties.
  • Automobiles today rely on electronics for nearly everything from engine management to chassis control to braking to driver assistance. Automotive electronics and the components used therefor are built to automotive specifications and standards far exceeding those applicable to consumer products.
  • ASILs Automotive Safety Integrity Levels
  • ASIL-D grade because the risks associated with their failure are the highest.
  • Components like rear lights require only an ASIL-A grade.
  • Headlights, brake lights, rear backup camera (if involved in automatic braking), radar cruise control, vision ADAS (if involved in likes of automatic braking) are generally ASIL-B.
  • Cruise control would generally be ASIL-C.
  • Aspects of engine management could warrant ASIL-C (or even ASIL-D).
  • Active suspension could warrant ASIL-C or ASIL-B depending on its particular context and configuration on a vehicle.
  • Quality Management means that risk associated with a hazardous event is not unreasonable and does not therefore require safety measures in accordance with ISO 26262.
  • Rear backup cameras enhance safety during reversing maneuvers whereas CMS is relied on by the driver continually during forward driving. Thus, a failure by CMS that impacts safety arguably could have a more immediate potential impact than a failure of a rear backup camera.
  • Display monitors for CMS use either TFT-LCD (Thin Film Transistor Liquid Crystal Display) screens or OLED (Organic Light Emitting Display) screens.
  • the screens usually are hard-coated to resist scratching and are surface-treated to be anti-glare.
  • a high screen brightness of at least around 1,000 cd/m 2 must be attainable throughout the use lifetime of the vehicle.
  • LCD screens use backlighting whereas OLED screens are self-illuminating. At a given level of brightness, LCD screens tend to consume less power than OLED screens.
  • OLED screens surpass LCD screens in color uniformity, color gamut, viewing angle, contrast ratio and black level.
  • OLED screens can be made thinner than LCD screens.
  • active methods e.g. vibration, electrostatic repelling, wind curtain, lens wiper
  • Glare/flare/ghosting caused by strong light sources can obscure important objects being imaged from otherwise being seen by a driver viewing the CMS display.
  • Glare/flare can be reduced/eliminated by a properly designed optical system (lens and imager) used by CMS camera.
  • Standards and test procedures like ISO-9358 and ISO-18844 (which are hereby incorporated herein by reference in their entireties) can be used to measure these image aberrations.
  • ISO 16505 (which is hereby incorporated herein by reference in its entirety) provides performance requirements and test procedures to evaluate image quality.
  • Rearview mirror systems are inherently failsafe. CMS is not inherently failsafe. UNECE R46 at 6.2.2.3.2. (which is hereby incorporated herein by reference in its entirety) states that if the system is not operational (e.g. CMS failure), it shall be indicated to the driver by, for example, warning indication, display information or absence of status indicator, and the operator's manual shall explain the information indicated.
  • CMS is not inherently failsafe.
  • UNECE R46 at 6.2.2.3.2. (which is hereby incorporated herein by reference in its entirety) states that if the system is not operational (e.g. CMS failure), it shall be indicated to the driver by, for example, warning indication, display information or absence of status indicator, and the operator's manual shall explain the information indicated.
  • a CMS-equipped vehicle can (i) have two exterior sideview cameras and two associated in-cabin video monitors, and also have an FMVSS 111-compliant interior mirror assembly (this can be a dual-mode interior mirror assembly receiving video feed from a third rear-mounted camera), or (ii) have two exterior sideview cameras and two associated in-cabin video monitors, and also have a rear-mounted third camera and an associated third in-cabin video monitor but with no FMVSS 111-compliant interior mirror assembly present in the vehicle.
  • video data captured by CMS cameras can be processed by a central ECU or by multiple ECUs.
  • failure mode (iv) above could leave the driver wholly dependent on an interior rearview mirror (if present) whereas failure modes (i) to (iii) would still allow the driver to drive with partial CMS utility (for example, if the failure is on the passenger side, the driver can still use the driver-side CMS and can use an interior rearview mirror assembly, if present).
  • the driver can choose to use the dual-mode interior mirror to at least partially monitor what traffic may be approaching (for example) on the driver-side should any of a driver-side CMS camera or a driver-side monitor or a driver-side ECU fail.
  • UNECE R46 at 6.2.2.3.2. does not elaborate on what type or degree of alert or warning the driver should receive in the event of CMS failure, nor does UNECE R46 address sub-system or component redundancy in CMS.
  • CMS can provide to a driver “clear and reasonably unobstructed view” when all the potential use cases, including corner cases, are covered.
  • Some corner cases include rain drops or dust that obstruct a significant portion of a camera's lens surface, glare/flare caused by a dirty lens surface or poor optical design, lack of camera dynamic range which causes under-exposure at dark areas or saturation in bright areas, causing problems for visibility of objects displayed on a display screen.
  • the above corner cases can be addressed in system requirement and product design.
  • a driver loses depth perception in CMS. This can be remedied using two identical cameras positioned apart by a certain distance and with the cameras sending images separately to a stereographic display screen. The images from a driver-side left CMS camera are projected to driver's left eye only while the images from a passenger-side right camera are projected to the driver's right eye only. This mimics a human eye's stereoscopic depth perception.
  • Such a system may utilize aspects of the systems described in U.S. provisional application Ser. No. 62/942,305, filed Dec. 2, 2019, which is hereby incorporated herein by reference in its entirety.
  • an in-cabin camera that views the driver's head can be used to detect driver head position change. Then the CMS image being displayed on an in-cabin CMS display can be adjusted or panned accordingly to mimic what the driver would see looking at different angles at an exterior side view mirror.
  • the video feeds from the three cameras can be provided to a central electronics control unit (ECU) and be processed thereat so that (taking the left side/driver side as an example) left-side/driver-side video image data captured by the rear-viewing camera to be displayed at the left side of the video display screen of the dual-mode interior mirror assembly and left side lane/driver-side video image data captured by the driver-side CMS camera to be displayed at the left side/driver-side CMS in-cabin video monitor are modified via data processing at a data processor of the ECU so that a driver viewing both the video display screen of the dual-mode interior mirror assembly and the left-side/driver-side CMS monitor is not confused seeing an overtaking vehicle on both screens.
  • ECU central electronics control unit
  • Such data processing can include reduction/elimination of redundant or duplicitous image information.
  • Such data processing can include scaling of images so that scale of what the driver sees viewing the video screen of the dual-mode interior video mirror is in-scale with what the driver sees viewing the video screen of the in-cabin CMS monitor.
  • display of video images at the video screen of the dual-mode interior video mirror of vehicle approach in a side lane is episodic and occurs only when a lane change maneuver of the equipped vehicle is anticipated (such as by the driver activating a turn signal indicator or such as by a camera vision-based lane departure warning system of the equipped vehicle detecting a lane change maneuver).
  • a lane change maneuver of the equipped vehicle is anticipated (such as by the driver activating a turn signal indicator or such as by a camera vision-based lane departure warning system of the equipped vehicle detecting a lane change maneuver).
  • the video feeds from the three cameras can be provided to a central electronics control unit (ECU) and be processed thereat, and left-side/driver-side video image data captured by the rear-viewing camera is displayed at the left side/driver side of the video display screen of the dual-mode interior mirror assembly or right-side/passenger-side video image data captured by the rear-viewing camera is displayed at the right side/passenger of the video display screen of the dual-mode interior mirror assembly only when no approaching or overtaking other vehicle is detected by a vehicle detection system of the equipped vehicle to be present in the respective driver-side or passenger-side side lane.
  • ECU central electronics control unit
  • the camera or cameras may comprise any suitable imaging sensor or camera, such as a pixelated imaging array or the like, such as a CMOS imaging array sensor, a CCD sensor or other sensors or the like, such as a camera or sensor of the types disclosed in commonly assigned, U.S. Pat. Nos. 7,965,336; 5,550,677; 5,760,962; 6,097,023 and 5,796,094, which are hereby incorporated herein by reference in their entireties.
  • the camera may comprise a stereo imaging camera or the like, such as by utilizing aspects of the imaging systems described in U.S. Pat. Nos. 6,396,397 and/or 5,796,094, which are hereby incorporated herein by reference in their entireties.
  • the camera may comprise an infrared or near infrared light sensitive camera and may be suitable for capturing images in low lighting conditions, and/or the camera may include or be associated with an illumination source (such as an infrared or near-infrared light emitting illumination source that, when actuated to emit infrared or near-infrared light at the side of the vehicle, enhances the camera's performance but is not visible or discernible to the driver of the vehicle), such as by utilizing aspects of the cameras described in U.S. Pat. Nos. 7,965,336; 5,550,677; 5,760,962; 6,097,023 and 5,796,094, which are hereby incorporated herein by reference in their entireties.
  • an illumination source such as an infrared or near-infrared light emitting illumination source that, when actuated to emit infrared or near-infrared light at the side of the vehicle, enhances the camera's performance but is not visible or discernible to the driver of the vehicle
  • the sideward and rearward viewing camera may be incorporated at the exterior rearview mirror assembly or elsewhere at the vehicle, such as at a side portion of the vehicle, and having a sideward and rearward field of view.
  • the camera may have a wide angle field of view at the side of the vehicle and/or may have an adjustable field of view and/or may capture images for use in other vision systems, such as for use in a top-down view or bird's-eye view vision system of the vehicle or a surround view vision system at the vehicle, such as by utilizing aspects of the vision systems described in International Publication Nos.
  • the display may be mounted elsewhere in the vehicle and/or at the vehicle door or vehicle door opening, such as at the A-pillar of the vehicle, or at the top of the instrument panel of the vehicle, or at the steering column of the vehicle, or at or in the headliner of the vehicle and/or the like.
  • the mirror and camera and display system of the present invention may provide a docking station (such as at or near the vehicle door and/or door opening of the vehicle) for a cell phone (or tablet or other personal portable display capable device), where the cell phone or personal display device may be docked or connected so that the display screen of the personal display device may function as the display and may display the images captured by the camera.
  • the display screen preferably displays the captured video images at the entire viewable portion of the display screen, in order to provide relatively large displayed images for viewing by the driver of the vehicle while the driver is normally operating the vehicle.
  • the user inputs are touch or proximity sensors disposed at a portion of the display screen.
  • the video display screen when normally operating to display video images captured by the camera, may display the video images over the entire display portion or active portion of the display screen and, responsive to a user input (such as a user or driver touching a portion of the display screen or touch screen), may display icons or indicia at a portion of the display screen to indicate where the user can touch to actuate or control the display settings or the like.
  • the user inputs or touch sensors may comprise any suitable sensors or inputs, and may utilize aspects of the inputs and sensors described in U.S. Pat. Nos. 7,224,324; 7,253,723; 7,255,451 and/or 8,154,418, and/or International Publication Nos. WO 2012/051500; WO 2010/124064; WO 2011/044312; WO 2013/071070 and/or WO 2013/126719, which are hereby incorporated herein by reference in their entireties.
  • the display may utilize aspects of the displays of the types disclosed in U.S. Pat. Nos. 9,264,672; 9,041,806; 7,855,755; 7,777,611; 7,626,749; 7,581,859; 7,446,924; 7,446,650; 7,370,983; 7,338,177; 7,274,501; 7,255,451; 7,195,381; 7,184,190; 6,690,268; 6,329,925; 5,668,663; 5,530,240 and/or 5,724,187, and/or in U.S. Publication No. US-2006-0050018, and/or International Publication No. WO 2004/058540, which are all hereby incorporated herein by reference in their entireties.
  • the display may be viewable through the reflective element when the display is activated to display information.
  • the display element may be any type of display element, such as a vacuum fluorescent (VF) display element, a light emitting diode (LED) display element, such as an organic light emitting diode (OLED) or an inorganic light emitting diode, an electroluminescent (EL) display element, a liquid crystal display (LCD) element, a video screen display element or backlit thin film transistor (TFT) display element or the like, and may be operable to display various information (as discrete characters, icons or the like, or in a multi-pixel manner) to the driver of the vehicle, such as passenger side inflatable restraint (PSIR) information, tire pressure status, and/or the like.
  • PSIR passenger side inflatable restraint

Abstract

A vehicular camera monitoring system a driver-side camera, a driver-side video display screen disposed at a driver-side cabin region of an interior cabin of the vehicle, a passenger-side camera and a passenger-side video display screen disposed at a passenger-side cabin region of the interior cabin of the vehicle. Upon completion of a parking maneuver of the vehicle, and until a period of time elapses following powering down of the vehicle upon completion of the parking maneuver of the vehicle, the driver-side video display screen displays driver-side video images of a driver-side region sideward of the driver side of the vehicle and rearward of the vehicle and the passenger-side video display screen displays passenger-side video images of a passenger-side region sideward of the passenger side of the vehicle and rearward of the vehicle.

Description

    CROSS REFERENCE TO RELATED APPLICATIONS
  • The present application is a continuation of U.S. patent application Ser. No. 16/949,976, filed Nov. 23, 2020, now U.S. Pat. No. 11,498,494, which claims the filing benefits of U.S. provisional application Ser. No. 62/942,351, filed Dec. 2, 2019, and U.S. provisional application Ser. No. 62/941,089, filed Nov. 27, 2019, which are hereby incorporated herein by reference in their entireties.
  • FIELD OF THE INVENTION
  • The present invention relates to vehicular vision systems that display video images derived from image data captured by one or more cameras of the vehicle.
  • BACKGROUND OF THE INVENTION
  • It is known to provide a video display at the exterior rearview mirror assembly, such as described in U.S. Pat. No. 7,777,611, which is hereby incorporated herein by reference in its entirety, or to provide a video display at an interior rearview mirror assembly to display sideward and/or rearward images, such as described in U.S. Pat. No. 5,670,935, which is hereby incorporated herein by reference in its entirety. A variety of interior and exterior mirror assemblies with indicators are known in the art, such as U.S. Pat. Nos. 5,668,663; 5,355,284; 5,788,357; 6,257,746; 6,005,724; 5,481,409; 6,111,683; 6,045,243; 6,264,353; 6,512,624; 6,356,376; 2,263,382; 2,580,014; 3,266,016; 4,499,451; 4,588,267; 4,630,904; 4,623,222; 4,721,364; 4,906,085; 5,313,335; 5,587,699; 5,575,552; 5,436,741; 5,587,699; 5,938,320; 6,700,692 and 5,786,772, which are all hereby incorporated herein by reference in their entireties.
  • SUMMARY OF THE INVENTION
  • The present invention provides a camera monitoring system that has a driver-side video display screen that displays video images derived from image data captured by a driver-side camera and a passenger-side video display screen that displays video images derived from image data captured by a passenger-side camera. During normal forward driving of the vehicle along a road, the displayed video images (derived from image data captured by the driver-side and passenger-side cameras) allow the driver to see vehicles approaching and/or overtaking the equipped vehicle in the respective side traffic lanes adjacent to the traffic lane of the road along which the equipped vehicle is traveling. During a parking maneuver or a parking-space exit maneuver of the vehicle, the driver-side video display screen displays parking assist video images derived from image data captured by the driver-side camera and the passenger-side video display screen displays parking assist video images derived from image data captured by the passenger-side camera. The parking assist images may comprise video images of the ground adjacent to and forward and/or rearward of the vehicle to assist the driver in maneuvering the vehicle into or out from a parking space. Upon completion of the parking-space exit maneuver when the vehicle has fully exited a parking space and commences normal forward driving along the road, the driver-side video display screen automatically reverts to display of forward-driving driving assist video images derived from image data captured by the driver-side camera, and the passenger-side video display screen automatically reverts to display of forward-driving driving assist video images derived from image data captured by the passenger-side camera, with the displayed forward-driving driving assist video images (that are viewable by a driver of the equipped vehicle when normally driving the vehicle forward in traffic and when not parallel or perpendicular parking into a vacant parking space or when not exiting from a parking space) being different from the displayed parking assist video images (that are viewable by the driver of the equipped vehicle when parallel or perpendicular parking into a vacant parking space or when exiting from a parking space).
  • Differences between forward-driving driving assist video images and parking assist video images can include difference in the vertical field of view as seen on the display (preferably, the vertical field of view is larger when parking so that ground area adjacent to and along the side of the vehicle is viewable to the driver on the display screen). Differences between forward-driving driving assist video images and parking assist video images can include difference in the horizontal field of view as seen on the display (preferably, the horizontal field of view is narrower when parking so that the driver can focus viewing on obstacles or other vehicles close to the side of the equipped vehicle). Differences between forward-driving driving assist video images and parking assist video images can include zoomed images so the driver can better view what is close to and adjacent to the being parked vehicle. Differences between forward-driving driving assist video images and parking assist video images can include electronically generated graphic overlays, icons, indicia and/or the like, that overlay in real-time on the parking assist video images to allow the driver to gauge distance to, separation from and approach to obstacles, such as curbs, posts, walls and/or other vehicles, as the equipped vehicle is being parked. Preferably, such electronically generated graphic overlays are generated responsive to sensor data captured by the likes of ultrasonic or radar or lidar or camera sensors of the equipped vehicle (including image data sensor output from one or more of the driver-side camera, the passenger-side camera and the rear viewing camera) being provided to a data processor where such sensor data is processed to determine the likes of curbs, posts, walls and/or other vehicles, as the equipped vehicle is being parked.
  • These and other objects, advantages, purposes and features of the present invention will become apparent upon review of the following specification in conjunction with the drawings.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • FIG. 1 is a perspective view of a driver-side camera an driver-side display screen in accordance with the present invention;
  • FIG. 2 is a plan view of a vehicle, showing the driver-side display and the passenger-side display in the vehicle;
  • FIG. 3 is a plan view of the vehicle of FIG. 2 , showing the vehicle during a parallel parking maneuver;
  • FIG. 4 is a perspective view of an exemplary camera and video display for a camera monitoring system;
  • FIG. 5 is a perspective view of another exemplary camera and video display for a camera monitoring system;
  • FIG. 6 is a plan view of a vehicle, showing the respective fields of view of a driver-side sideview camera, a rearward viewing camera and a passenger-side sideview camera;
  • FIGS. 7A-C are plan views of a dual-mode interior rearview mirror of the vehicle of FIG. 6 that displays video images derived from the rearward viewing camera (FIG. 7A) or all three cameras (FIGS. 7B and 7C); and
  • FIG. 8 is a list of preferred display screen parameters and/or characteristics for the camera monitoring system.
  • DESCRIPTION OF THE PREFERRED EMBODIMENTS
  • Referring now to the drawings and the illustrative embodiments depicted therein, an exterior rearview mirror system 10 for a vehicle 11 includes an exterior rearview mirror assembly 12 and an in-cabin display device or module 14 disposed at or mounted at the interior of the vehicle door 11 a or otherwise near to the exterior rearview mirror assembly 12 (FIG. 1 ). The exterior rearview mirror assembly 12 is mounted at the side 11 b of the host or subject or equipped vehicle 11 and includes a mirror reflective element 16 and a mirror shell or casing 18, and a camera or imaging sensor 20 that is disposed at the mirror casing 18 and has a generally rearward and sideward field of view at the side of the equipped vehicle. An interior rearview mirror assembly 40 is mounted in the interior cabin of the equipped vehicle. Interior mirror assembly 40 may be attached at an in-cabin side of a windshield of the equipped vehicle. Preferably, interior mirror assembly 40 comprises a dual-mode interior rearview video mirror that can switch from a traditional reflection mode to a panoramic live-video display mode, such as is by utilizing aspects of the mirror assemblies and systems described in U.S. Pat. Nos. 10,442,360; 10,421,404; 10,166,924 and/or 10,046,706, and/or U.S. Publication Nos. US-2019-0146297; US-2019-0118717 and/or US-2017-0355312, which are all hereby incorporated herein by reference in their entireties. Dual-mode inter dual-mode interior rearview video mirror 40 receive video feed from either or both of side cameras 20, 21 and/or may receive video feed from rear-mounted camera 50.
  • In the illustrated embodiment, the driver-side exterior rearview mirror assembly 12 may comprise an FMVSS-111 compliant planar reflective element 16 that meets FMVSS-111 field of view requirements but does not substantially exceed them (for example, the reflective element may have a field of view of at least about 8 degrees from the side of the vehicle, but not more than about 12 degrees). The camera 20 may be disposed at any suitable location at mirror assembly 12, such as at an output portion of the mirror casing 18, such as shown in FIG. 1 . Alternatively, and as can be seen in FIG. 4 , the side camera can be behind the mirror reflective element and can view via a light transmitting aperture created (preferably via the likes of laser ablation) in the mirror reflector used for the reflective element (or the side camera can view via a transflective mirror reflector used for the reflective element). Presence of the camera can be masked by using a jail bar or dot pattern of the mirror reflector in front of where the camera is disposed. The display device or module may utilize aspects of the modules described in U.S. Publication Nos. US-2018-0134217 and/or US-2014-0285666, which are hereby incorporated herein by reference in their entireties.
  • The small or reduced size mirror reflective element provides a reduced size rearward field of view to the driver of the vehicle (as compared to larger size mirror reflective elements), and the smaller field of view may be supplemented or enhanced via the video display device 14 disposed at the interior of the vehicle cabin and in an area at or near the exterior rearview mirror assembly. For example, and as shown in FIG. 1 , display device 14 may be disposed at an inboard portion of the vehicle door 11 a and may be disposed immediately opposite exterior rearview mirror assembly 12. Optionally, the exterior rearview mirror assembly 12 and display device 14 may be incorporated into an imaging module that is mountable to or incorporated in a vehicle door, such that all of the wiring and communication links between the camera and display are provided within the module. The module may be mounted at or installed in the door assembly and may be electrically connected to a vehicle wiring harness via an electrical connector at the door, so that power and/or control of the module and/or system is provided via the vehicle wiring harness.
  • In the illustrated embodiment, the display device 14 includes a video display screen 22 supported at or housed by a housing or casing or structure 24 so that the display screen is at a location near the door that is readily viewable by a driver of the vehicle and is an area that the driver of the vehicle readily cognitively associates with where to look to see what is to the side and rear of the vehicle (because of the proximity of the display screen to the exterior rearview mirror assembly). When the mirror and display device are mounted at the vehicle door, the display screen and the exterior mirror move with the vehicle door when the vehicle door is opened and closed. Video display screen 22 may comprise any suitable video display screen, such as discussed below, and may comprise a touch screen to allow for user control of the video display and/or other vehicle functions via touching of appropriate locations at the display screen by the driver of the vehicle. For example, the video display screen may comprise directional arrow inputs to allow the user or driver to zoom in or out or to pan the video display up or down or to either side to set the rearward field of view at the display screen to a preferred setting for that individual. The system may include a memory system that also allows the user to set the rearward field of view to a desired setting and to store that setting such as in a similar manner as is done for memory mirror settings and/or memory seat settings and the like.
  • Camera monitoring systems (CMS) have been proposed as replacements for conventional rear-view mirrors. The advantages include reduced wind drag, reduced vehicle weight, and improved field of view. Despite these advantages, there are a number of reasons such systems are not yet in widespread use in vehicles. For example, reduced depth perception is a concern. Because a typical camera only provides monocular view, the ability to judge depth due to parallax is lost. Additionally, as a trade-off to gaining additional field of view on a given display size, the image may become distorted to the point where judgment of distance using relative object size is also hindered. Also, the video display displays an invariant field of view (i.e., the driver is unable to vary what he or she sees when viewing the video display, as can be done with an exterior mirror by moving the driver's head to a different position). The present invention provides the following concepts to help the driver visualize where other vehicles are relative to his or her own vehicle.
  • Parking a vehicle (whether perpendicular parking or the more challenging parallel parking) is an acquired skill. Drivers who are used to parking aided by sideview mirrors may need to relearn parking aided by CMS. Modern vehicles come replete with the likes of ultrasonic sensors and other obstacle sensors and thus CMS offers opportunity for overlays to be superimposed on the real-time video being displayed on the in-cabin CMS monitor. Such overlays can enhance the ease and efficacy of CMS-aided parking. Thus CMS may use overlays and different types of camera views shown on the in-cabin monitor during a parking maneuver.
  • Upon completion of parking (or upon exiting a parking space and entering traffic), the camera views seen by the driver automatically revert to what is suitable for normal driving either immediately or once a threshold condition is met (such as, for example, when the vehicle's speed reaches a threshold speed such as 10 miles per hour after exiting a parking space or when the vehicle has travelled a threshold distance such as 100 feet after exiting the parking space or when a time period such as 10 seconds has elapsed after the vehicle has exited the parking space).
  • Also, when parallel parking (such as shown in FIG. 3 ), the camera view seen by the driver viewing the driver-side video monitor continues to enable the driver to see any cyclist or vehicle approaching the vehicle even while the camera view seen by the driver viewing the passenger-side video monitor may have changed during a later stage of the parallel parking maneuver (as the vehicle is maneuvered further into the parking space so as to be alongside the curb) to allow the driver see alignment of the vehicle with the curb.
  • Upon start-up and while maneuvering to exit a parking space, the camera view seen by the driver viewing the driver-side video monitor enables the driver to see any cyclist or vehicle approaching on the left side (or should the vehicle be parked with the driver-side camera curbside, the camera view seen by the driver viewing the passenger-side video monitor needs to enable the driver to see any cyclist or vehicle approaching on the right side). The system automatically displays parking assist video images at the parking-side video display screen (typically the passenger side) and automatically displays driving assist video images at the road-side video display screen (typically the driver side), such as responsive to the driver turning the ignition on after the vehicle is parked or responsive to the driver shifting the vehicle into forward gear (or reverse gear if the vehicle has to be backed up a little to achieve sufficient clearance from the vehicle parked ahead of the equipped vehicle for exiting the parking space).
  • Thus, as the vehicle drives along a road, the driver-side video monitor and the passenger-side video monitor display driving assist video images representative of the sideward and rearward or rearward-view portion of the field of view of the respective driver-side camera and passenger-side camera. When the driver of the vehicle wishes to park the vehicle along a side of the road (a parking-side of the road, typically the passenger side, but could be the driver side, such as on one-way streets), the driver of the vehicle may drive past a parking space between two spaced apart parked vehicles in which the driver intends to parallel park the vehicle. When the vehicle is positioned at least partially forward of the parking space, the driver shifts the vehicle into reverse gear to commence reversing of the vehicle into the parking space. Upon shifting into reverse gear and/or commencing the parking maneuver into the parking space (at the passenger side of the road), the driver-side video monitor continues display of the driving assist video images and the passenger-side video monitor automatically ceases display of the driving assist video images and displays parking assist video images that are different than the driving assist video images. The parking assist video images may be directed more downward (as compared to the driving assist video images) so as to include the ground region at the passenger side of the vehicle and at and around the parking space so that the driver can more readily see the ground and objects or curb at or in the parking space.
  • For situations where the parking space is at the passenger side of the vehicle (as shown in FIG. 3 ), the passenger-side video monitor may display downward viewing parking assist video images to show the driver the curb or objects in or adjacent to the target parking space, while the driver-side video monitor may display rearward viewing parking assist video images to show the driver any rearward approaching vehicles or bicycles or the like that may be approaching along the road. Thus, the driver-side video monitor may provide video images of other traffic or the like, while the passenger-side video monitor may provide video images of the curb or parking space. The driver-side parking assist video images thus may be different than the passenger-side parking assist video images (with the different video images derived from different portions of the fields of view of the respective cameras).
  • After the vehicle is parked, and when started again to exit the parking space, the driver-side video display screen displays the driving assist video images while the passenger-side video display screen displays the parking assist video images. After the vehicle has exited the parking space, the driver-side video monitor continues display of the driving assist video images and the passenger-side video monitor automatically ceases display of the parking assist video images and displays the driving assist video images that are different than the driving assist video images. The system may determine the vehicle has exited the parking space (and thus may switch the passenger-side video monitor to display the driving assist video images) responsive to a threshold event or condition, such as responsive to the vehicle reaching a threshold forward speed (e.g., 5 mph or 10 mph or 15 mph), or the vehicle traveling a threshold forward distance (e.g., 20 ft or 50 ft or 100 ft), or elapsing of a threshold time period (e.g., 5 seconds or 10 seconds or 15 seconds) after the vehicle has exited the parking space.
  • Rear backup systems with FMVSS 111-required field of view and in-cabin video display performance are now mandatory on all new light vehicles made in or imported into the USA. Today, most new vehicles meet this mandate by using a single video display (with diagonal dimension at least 5 inches) located at or about the center region of the front instrument console. In a CMS-equipped vehicle utilizing a driver-side video monitor (receiving its video feed from a driver-side CMS camera) and a passenger-side video monitor (receiving its video feed from a passenger-side CMS camera), rear backup video images compliant with FMVSS 111 likely will continue to be displayed on a separate centrally-located video screen when rear gear is selected by the driver.
  • NHTSA in 2008 (see NHTSA DOT HS 811 024, September 2008, which is hereby incorporated herein by reference in its entirety) undertook an On-Road Study of Drivers' Use of Rearview Video Systems. A driver's use of rearview video (RV) systems was observed during staged and naturalistic backing maneuvers to determine whether drivers look at the rearview display during reversing maneuvers and whether use of the system affects the drivers' reversing behavior. The vehicles used were equipped with an interior rearview mirror, a planar driver-side sideview mirror and a convex passenger-side sideview mirror. Overall, results of this study revealed that drivers look at rearview video displays during reversing maneuvers at least some of the time.
  • Participants' responses regarding where they look primarily to determine whether the area behind the vehicle is clear when reversing showed a fairly equal distribution of reported glances to the side mirrors, center rearview mirror, over their shoulders, and at a rearview video display (when present).
  • Since 2008, a significant proportion of the American driving population has grown accustomed to reversing using mirrors and a rear backup display. Thus, a further study should be able to establish how the now standard-availability of rear backup systems in vehicles has impacted (and hopefully increased) the amount of time during a backup maneuver that the driver looks at the rear backup display.
  • In a CMS-equipped vehicle (such as, for example, a 2019 Lexus ES or 2019 Audi e-tron), there may be three video screens for the driver to look at/scan when executing a backup maneuver. This may impact the amount of time during a backup maneuver that a driver looks at a rear backup display and this may impact the efficacy of rear backup systems in terms of making the driver aware of children or other objects present in the blind zone behind the vehicle.
  • In addition to CMS having driver-side display and a passenger-side display, a center display may be provided. Also, or otherwise, a video display device may be provided in the interior rearview mirror assembly, such as to display video images at the mirror reflective element for viewing by the driver of the vehicle. For example, dual-mode interior rearview mirror assemblies provide a reflected view to the driver of the vehicle (such as is provided by a conventional interior rearview mirror when adjusted to the driver's preferences) and, if selected by the driver, provide a video display view to the driver, whereby a video display screen disposed behind the reflective element operates to display video images derived from image data captured by a rearward viewing camera of the vehicle. Such dual-mode interior rearview mirror assemblies may utilize aspects of the mirror assemblies and systems described in U.S. Pat. Nos. 10,442,360; 10,421,404; 10,166,924; 10,046,706 and/or 10,029,614, and/or U.S. Publication Nos. US-2019-0258131; US-2019-0146297; US-2019-0118717 and/or US-2017-0355312, and/or U.S. provisional application Ser. No. 62/942,305, filed Dec. 2, 2019, which are all hereby incorporated herein by reference in their entireties.
  • Vehicles equipped with such dual-mode interior mirror assemblies use a rear-mounted and rearward viewing camera that is separate from the rear backup camera required under FMVSS 111. One reason for this is that a dual-mode interior mirror is currently usually an optional feature whereas installation of a rear backup camera is mandated on all vehicles in the USA. Furthermore, rear backup cameras have a field of vision of at least 130 degrees (and many have a field of vision of at least 180 degrees) whereas rear-mounted cameras providing video feed to dual-mode interior mirrors typically have a field of vision of 50 degree or thereabouts. CMOS imagers used for rear backup cameras typically range from VGA resolution to 1.28 megapixel or thereabouts. CMOS imagers providing video feed to dual-mode interior mirrors are 1.6 megapixel resolution. If CMS and the rear backup function share a common rear camera, the imager resolution should be at least 8 megapixels (and more preferably at least 15 megapixels), and the system should use a wide angle lens and sophisticated de-warping and image distortion-reducing video processing.
  • A dual-mode interior mirror can operate either using a single video feed from a rear camera or using video feeds from a driver-side camera, a rear camera and a passenger-side camera. As shown in FIGS. 7A-C, the interior mirror can display video images from the rearward camera only (FIG. 7A), or video images from the side cameras can be displayed (i) in a manner that distinctly distinguishes/demarcates video images monitoring the side lanes from the main video images provided by the rear camera (FIG. 7B) or (ii) with the video feeds from all three cameras seamlessly stitched together to form a single panoramic video image (FIG. 7C).
  • With two side cameras and two respective driver-side and passenger-side interior monitors (with the left or driver-side video monitor that is associated with the driver-side side camera mounted left of the steering column and preferably at or where an exterior rearview mirror is located or would have been located, and with the right passenger-side video monitor that is associated with the passenger-side side camera mounted right of center of the vehicle and optionally at or where an exterior rearview mirror is located or would have been located) and with a dual-mode interior rearview mirror assembly (typically receiving video feed from a rear-mounted camera or potentially receiving video feeds, and such as shown in FIG. 6 , from a driver-side sideview camera, a passenger-side sideview camera and the rearward viewing camera), it is important that a driver utilizing such a three video monitor CMS is not overloaded or confused by, for example, seeing a view of traffic approaching or overtaking in the left-side lane displayed both at the driver-side monitor and at the dual-mode interior mirror.
  • To mitigate or avoid such confusion, the video feeds from all three cameras are provided to an electronic control unit (ECU) that includes an image processor/video processor. Taking as exemplary the driver-side camera video feed and the video feed to the dual-mode interior mirror (with the processing of the passenger-side camera video feed being similar to the processing of the driver-side camera video feed), both such video feeds are processed at the ECU and features commonly viewed or displayed in both the driver-side monitor and the dual-mode interior mirror are identified and tracked. Based at least on such image/video processing, the respective field of view of the driver-side sideview camera as displayed on the driver-side monitor and the field of view of the rear-mounted camera provided to the dual-mode interior mirror are panned, adjusted or cropped so that confusing duplication of video image display seen by the driver using the driver-side monitor and the dual-mode interior mirror is mitigated or reduced. The ECU thus dynamically adjusts or balances what is displayed to be seen on the driver-side monitor and on the dual-mode interior mirror so that duplication is minimized or mitigated while providing to the driver wide angle and readily discernible video images of traffic behind, approaching and/or overtaking the equipped vehicle.
  • Alternatively, in a vehicle equipped with a driver-side sideview camera and driver-side monitor, a passenger-side sideview camera and passenger-side monitor, and a rearward-viewing camera and dual-mode interior mirror, the field of view of the dual-mode interior mirror narrows or reduces (from, for example, around 50 degrees to around 30 degrees or lower) when either the driver selects a turn signal indicating intent to make a lane change, or when a lane keeping system automatically detects that the vehicle is moving to make a lane change (such a lane keeping system that may utilize aspects of the systems described in U.S. Pat. Nos. 6,882,287; 7,581,859; 9,126,525; 9,180,908; 9,340,227; 9,946,940; 10,268,904 and/or 10,315,651, which are hereby incorporated herein by reference in their entireties). Such dynamic and automatic reduction of the CMS field of view can help mitigate confusion of what is to be seen at the left side or right side monitor. Thus, what is displayed at the dual-mode interior mirror can episodically change based on a prevailing traffic condition. Likewise, and when appropriate, what is shown on the driver-side monitor and/or passenger-side monitor can also and correspondingly episodically change based on the prevailing traffic condition.
  • Preferably, the CMS continues to be operable after a driver turns off the ignition and prepares to exit the vehicle. Before opening a vehicular door to exit, a driver can and should check in the driver sideview mirror to see if the likes of an approaching cyclist is not in hazard of colliding with the opened door. Plainly, it is desirable that the CMS remains active after the ignition is turned off so that the driver can check what may be approach at the side before opening the door. Lighting and other accessories remain powered and active in many vehicles today for a time period (typically 30 seconds to a five minutes or ten minutes or more) after the ignition is turned off by a driver. Thus, CMS can similarly remain temporarily powered when the driver turns off the ignition.
  • There are of course scenarios (such as a parent parked to pick up a child at a school or a taxi driver parked waiting to be hailed) where more than a few minutes may pass between turning the ignition off and opening a car door (either for the driver to exit or to allow a passenger to enter). Should it be required to have CMS operational in such circumstances, the CMS can remain powered and operational (even after the driver turns off the ignition) for as long as the driver seat is occupied. Passenger seat occupancy detection is now ubiquitous in modern vehicles to allow the passenger-side airbag to be disabled when the front passenger-side seat is unoccupied. Driver seat occupancy detection can use similar means and technologies.
  • Thus, for example, the CMS (at least the driver-side camera and the driver-side video display screen) remains operational after ignition shut off and until a period of time following a detection or determination that the driver has left the driver seat (e.g., a period of time following determination, such as by a driver-seat monitoring or occupancy detection system, that the driver seat is unoccupied). The period of time does not start until the driver has left the seat/vehicle, so the CMS (at least the driver-side camera and the driver-side video display screen, but optionally the passenger-side camera and the passenger-side video display screen as well) remains operating while the driver is in the driver seat so that the driver can use the CMS to determine if a pedestrian or bicycle or other vehicle is approaching the subject vehicle at the driver side of the subject vehicle before the driver opens the vehicle door. The driver-seat occupancy determination may only be made (i.e., the driver-seat occupancy detecting system may only operate) responsive to the ignition of the vehicle being turned off when the vehicle is parked.
  • Driver-seat occupancy can be detected by likes of an in-seat weight sensor, or by a camera-based driver monitoring systems such as the types described in U.S. Pat. No. 9,405,120 and/or U.S. Publication Nos. US-2017-0274906; US-2016-0209647; US-2016-0137126; US-2015-0352953; US-2015-0296135; US-2015-0294169; US-2015-0232030; US-2015-0092042; US-2015-0022664; US-2015-0015710; US-2015-0009010 and/or US-2014-0336876, which are hereby incorporated herein by reference in their entireties.
  • Automobiles today rely on electronics for nearly everything from engine management to chassis control to braking to driver assistance. Automotive electronics and the components used therefor are built to automotive specifications and standards far exceeding those applicable to consumer products.
  • The IEC 61508 standard (which is hereby incorporated herein by reference in its entirety) applicable to electrical/electronic/programmable safety-related products is replaced in the automotive industry by the ISO 26262 Functional Safety standard (which is hereby incorporated herein by reference in its entirety) that covers the entire product life-cycle of safety-related electrical/electronic systems for road vehicles. There are four Automotive Safety Integrity Levels (“ASILs”) identified by ISO 26262: A, B, C, and D. ASILs establish safety requirements based on the probability and acceptability of harm for automotive components, with ASIL-A representing the lowest degree and ASIL-D represents the highest degree of automotive hazard. Systems like airbags, anti-lock brakes and electric power steering are ASIL-D grade because the risks associated with their failure are the highest. Components like rear lights require only an ASIL-A grade. Headlights, brake lights, rear backup camera (if involved in automatic braking), radar cruise control, vision ADAS (if involved in likes of automatic braking) are generally ASIL-B. Cruise control would generally be ASIL-C. Aspects of engine management could warrant ASIL-C (or even ASIL-D). Active suspension could warrant ASIL-C or ASIL-B depending on its particular context and configuration on a vehicle.
  • Rear backup cameras today rank between QM and ASIL-B (if involved in automatic emergency braking) in accordance with ISO 26262. Quality Management (“QM”) means that risk associated with a hazardous event is not unreasonable and does not therefore require safety measures in accordance with ISO 26262. Rear backup cameras enhance safety during reversing maneuvers whereas CMS is relied on by the driver continually during forward driving. Thus, a failure by CMS that impacts safety arguably could have a more immediate potential impact than a failure of a rear backup camera.
  • Modern cars are built to last, and indeed they do with the average age of vehicles on US roads today being 11.8 years.
  • Display monitors for CMS use either TFT-LCD (Thin Film Transistor Liquid Crystal Display) screens or OLED (Organic Light Emitting Display) screens. The screens usually are hard-coated to resist scratching and are surface-treated to be anti-glare. To be discernable in bright sunlight, a high screen brightness of at least around 1,000 cd/m2 must be attainable throughout the use lifetime of the vehicle. LCD screens use backlighting whereas OLED screens are self-illuminating. At a given level of brightness, LCD screens tend to consume less power than OLED screens. OLED screens surpass LCD screens in color uniformity, color gamut, viewing angle, contrast ratio and black level. OLED screens can be made thinner than LCD screens. By appropriately specifying screen brightness to remain above a minimum brightness even when aged, CMS monitors can be used throughout the lifetime of the vehicle.
  • Copious means exist to prevent or mitigate lens obstruction from rain, ice, dirt etc. These include using a hydrophobic coating on the camera's outermost lens surface, using a lens heater, using active methods (e.g. vibration, electrostatic repelling, wind curtain, lens wiper) and using a washer spray.
  • Glare/flare/ghosting caused by strong light sources (sun, headlights) can obscure important objects being imaged from otherwise being seen by a driver viewing the CMS display. Glare/flare can be reduced/eliminated by a properly designed optical system (lens and imager) used by CMS camera. Standards and test procedures like ISO-9358 and ISO-18844 (which are hereby incorporated herein by reference in their entireties) can be used to measure these image aberrations.
  • Under nighttime driving conditions, such as on a dark country road, electrical noise from a CMS camera as amplified by the control electronics can overshadow display of important objects in the exterior scene. It is thus important to design the camera to have low imager noise. Standards and test procedures like ISO-15739 (which is hereby incorporated herein by reference in its entirety) can be used to measure imager noise.
  • With regard to image quality of the image presented on a CMS electronic visual display, ISO 16505 (which is hereby incorporated herein by reference in its entirety) provides performance requirements and test procedures to evaluate image quality.
  • Rearview mirror systems are inherently failsafe. CMS is not inherently failsafe. UNECE R46 at 6.2.2.3.2. (which is hereby incorporated herein by reference in its entirety) states that if the system is not operational (e.g. CMS failure), it shall be indicated to the driver by, for example, warning indication, display information or absence of status indicator, and the operator's manual shall explain the information indicated.
  • A CMS-equipped vehicle can (i) have two exterior sideview cameras and two associated in-cabin video monitors, and also have an FMVSS 111-compliant interior mirror assembly (this can be a dual-mode interior mirror assembly receiving video feed from a third rear-mounted camera), or (ii) have two exterior sideview cameras and two associated in-cabin video monitors, and also have a rear-mounted third camera and an associated third in-cabin video monitor but with no FMVSS 111-compliant interior mirror assembly present in the vehicle. In either of (i) or (ii), video data captured by CMS cameras can be processed by a central ECU or by multiple ECUs.
  • There are thus multiple failure modes possible with CMS, including:
      • i. Failure of one but not all of the CMS cameras (for example, should a driver-side sideview camera be damaged/broken but the rest of the system still functions);
      • ii. Failure of one but not all of the CMS monitors (for example, should a passenger-side monitor be damaged/broken but the driver-side camera and monitor still function);
      • iii. Assuming multiple ECUs are present, failure of one but not all of the CMS ECUs (for example, should a driver-side ECU be damaged/broken but the passenger side of the system still functions);
      • iv. Should a single central ECU be used for all cameras in the CMS, failure of that central ECU.
  • A driver's ability to drive safely with a faulty CMS would be impaired by a failure mode effecting either or both of the sideview cameras/monitors, and this would be exacerbated by absence of an FMVSS 111-compliant interior mirror assembly. For example, failure mode (iv) above could leave the driver wholly dependent on an interior rearview mirror (if present) whereas failure modes (i) to (iii) would still allow the driver to drive with partial CMS utility (for example, if the failure is on the passenger side, the driver can still use the driver-side CMS and can use an interior rearview mirror assembly, if present).
  • If an FMVSS 111-compliant dual-mode interior mirror assembly is present and assuming the rear-mounted camera and ECU associated therewith remain operational, the driver can choose to use the dual-mode interior mirror to at least partially monitor what traffic may be approaching (for example) on the driver-side should any of a driver-side CMS camera or a driver-side monitor or a driver-side ECU fail.
  • UNECE R46 at 6.2.2.3.2. does not elaborate on what type or degree of alert or warning the driver should receive in the event of CMS failure, nor does UNECE R46 address sub-system or component redundancy in CMS.
  • Note that CMS can provide to a driver “clear and reasonably unobstructed view” when all the potential use cases, including corner cases, are covered. Some corner cases include rain drops or dust that obstruct a significant portion of a camera's lens surface, glare/flare caused by a dirty lens surface or poor optical design, lack of camera dynamic range which causes under-exposure at dark areas or saturation in bright areas, causing problems for visibility of objects displayed on a display screen. The above corner cases can be addressed in system requirement and product design.
  • A driver loses depth perception in CMS. This can be remedied using two identical cameras positioned apart by a certain distance and with the cameras sending images separately to a stereographic display screen. The images from a driver-side left CMS camera are projected to driver's left eye only while the images from a passenger-side right camera are projected to the driver's right eye only. This mimics a human eye's stereoscopic depth perception. Such a system may utilize aspects of the systems described in U.S. provisional application Ser. No. 62/942,305, filed Dec. 2, 2019, which is hereby incorporated herein by reference in its entirety.
  • Also, an in-cabin camera that views the driver's head can be used to detect driver head position change. Then the CMS image being displayed on an in-cabin CMS display can be adjusted or panned accordingly to mimic what the driver would see looking at different angles at an exterior side view mirror.
  • Also, when a dual-mode interior video mirror assembly (receiving its video feed from a rear viewing camera) is used with a left-side/driver-side CMS camera/monitor and right-side/passenger-side CMS camera/monitor configuration, the video feeds from the three cameras can be provided to a central electronics control unit (ECU) and be processed thereat so that (taking the left side/driver side as an example) left-side/driver-side video image data captured by the rear-viewing camera to be displayed at the left side of the video display screen of the dual-mode interior mirror assembly and left side lane/driver-side video image data captured by the driver-side CMS camera to be displayed at the left side/driver-side CMS in-cabin video monitor are modified via data processing at a data processor of the ECU so that a driver viewing both the video display screen of the dual-mode interior mirror assembly and the left-side/driver-side CMS monitor is not confused seeing an overtaking vehicle on both screens. Such data processing can include reduction/elimination of redundant or duplicitous image information. Such data processing can include scaling of images so that scale of what the driver sees viewing the video screen of the dual-mode interior video mirror is in-scale with what the driver sees viewing the video screen of the in-cabin CMS monitor.
  • Optionally, display of video images at the video screen of the dual-mode interior video mirror of vehicle approach in a side lane is episodic and occurs only when a lane change maneuver of the equipped vehicle is anticipated (such as by the driver activating a turn signal indicator or such as by a camera vision-based lane departure warning system of the equipped vehicle detecting a lane change maneuver). In this regard, use can be made of the systems described in U.S. Pat. No. 10,300,856, which is hereby incorporated herein by reference in its entirety.
  • Also, when a dual-mode interior mirror assembly (receiving its video feed from a rear viewing camera) is used with a left-side/driver-side CMS camera/monitor and right-side/passenger-side CMS camera/monitor configuration, the video feeds from the three cameras can be provided to a central electronics control unit (ECU) and be processed thereat, and left-side/driver-side video image data captured by the rear-viewing camera is displayed at the left side/driver side of the video display screen of the dual-mode interior mirror assembly or right-side/passenger-side video image data captured by the rear-viewing camera is displayed at the right side/passenger of the video display screen of the dual-mode interior mirror assembly only when no approaching or overtaking other vehicle is detected by a vehicle detection system of the equipped vehicle to be present in the respective driver-side or passenger-side side lane.
  • The camera or cameras may comprise any suitable imaging sensor or camera, such as a pixelated imaging array or the like, such as a CMOS imaging array sensor, a CCD sensor or other sensors or the like, such as a camera or sensor of the types disclosed in commonly assigned, U.S. Pat. Nos. 7,965,336; 5,550,677; 5,760,962; 6,097,023 and 5,796,094, which are hereby incorporated herein by reference in their entireties. Optionally, the camera may comprise a stereo imaging camera or the like, such as by utilizing aspects of the imaging systems described in U.S. Pat. Nos. 6,396,397 and/or 5,796,094, which are hereby incorporated herein by reference in their entireties. Optionally, the camera may comprise an infrared or near infrared light sensitive camera and may be suitable for capturing images in low lighting conditions, and/or the camera may include or be associated with an illumination source (such as an infrared or near-infrared light emitting illumination source that, when actuated to emit infrared or near-infrared light at the side of the vehicle, enhances the camera's performance but is not visible or discernible to the driver of the vehicle), such as by utilizing aspects of the cameras described in U.S. Pat. Nos. 7,965,336; 5,550,677; 5,760,962; 6,097,023 and 5,796,094, which are hereby incorporated herein by reference in their entireties.
  • The sideward and rearward viewing camera may be incorporated at the exterior rearview mirror assembly or elsewhere at the vehicle, such as at a side portion of the vehicle, and having a sideward and rearward field of view. Optionally, the camera may have a wide angle field of view at the side of the vehicle and/or may have an adjustable field of view and/or may capture images for use in other vision systems, such as for use in a top-down view or bird's-eye view vision system of the vehicle or a surround view vision system at the vehicle, such as by utilizing aspects of the vision systems described in International Publication Nos. WO 2010/099416; WO 2011/028686; WO 2012/075250; WO 2013/019795; WO 2012/075250; WO 2012/145822; WO 2013/081985; WO 2013/086249 and/or WO 2013/109869, and/or U.S. Publication No. US-2012-0162427, which are hereby incorporated herein by reference in their entireties.
  • Optionally, the display may be mounted elsewhere in the vehicle and/or at the vehicle door or vehicle door opening, such as at the A-pillar of the vehicle, or at the top of the instrument panel of the vehicle, or at the steering column of the vehicle, or at or in the headliner of the vehicle and/or the like. Optionally, as an alternative to a self-contained display monitor, the mirror and camera and display system of the present invention may provide a docking station (such as at or near the vehicle door and/or door opening of the vehicle) for a cell phone (or tablet or other personal portable display capable device), where the cell phone or personal display device may be docked or connected so that the display screen of the personal display device may function as the display and may display the images captured by the camera.
  • The display screen preferably displays the captured video images at the entire viewable portion of the display screen, in order to provide relatively large displayed images for viewing by the driver of the vehicle while the driver is normally operating the vehicle. In the illustrated embodiment, the user inputs are touch or proximity sensors disposed at a portion of the display screen. The video display screen, when normally operating to display video images captured by the camera, may display the video images over the entire display portion or active portion of the display screen and, responsive to a user input (such as a user or driver touching a portion of the display screen or touch screen), may display icons or indicia at a portion of the display screen to indicate where the user can touch to actuate or control the display settings or the like. The user inputs or touch sensors may comprise any suitable sensors or inputs, and may utilize aspects of the inputs and sensors described in U.S. Pat. Nos. 7,224,324; 7,253,723; 7,255,451 and/or 8,154,418, and/or International Publication Nos. WO 2012/051500; WO 2010/124064; WO 2011/044312; WO 2013/071070 and/or WO 2013/126719, which are hereby incorporated herein by reference in their entireties.
  • Optionally, the display may utilize aspects of the displays of the types disclosed in U.S. Pat. Nos. 9,264,672; 9,041,806; 7,855,755; 7,777,611; 7,626,749; 7,581,859; 7,446,924; 7,446,650; 7,370,983; 7,338,177; 7,274,501; 7,255,451; 7,195,381; 7,184,190; 6,690,268; 6,329,925; 5,668,663; 5,530,240 and/or 5,724,187, and/or in U.S. Publication No. US-2006-0050018, and/or International Publication No. WO 2004/058540, which are all hereby incorporated herein by reference in their entireties. The display may be viewable through the reflective element when the display is activated to display information. The display element may be any type of display element, such as a vacuum fluorescent (VF) display element, a light emitting diode (LED) display element, such as an organic light emitting diode (OLED) or an inorganic light emitting diode, an electroluminescent (EL) display element, a liquid crystal display (LCD) element, a video screen display element or backlit thin film transistor (TFT) display element or the like, and may be operable to display various information (as discrete characters, icons or the like, or in a multi-pixel manner) to the driver of the vehicle, such as passenger side inflatable restraint (PSIR) information, tire pressure status, and/or the like.
  • Changes and modifications to the specifically described embodiments may be carried out without departing from the principles of the present invention, which is intended to be limited only by the scope of the appended claims as interpreted according to the principles of patent law.

Claims (35)

1. A vehicular camera monitoring system, the vehicular camera monitoring system comprising:
a driver-side camera disposed at a driver side of a vehicle equipped with the vehicular camera monitoring system and viewing at least sideward and rearward at the driver side of the vehicle, and wherein the driver-side camera views at least a portion of a driver-side traffic lane adjacent to a traffic lane in which the vehicle is traveling along a road;
the driver-side camera capturing image data;
a driver-side video display screen disposed at a driver-side cabin region of an interior cabin of the vehicle and viewable by a driver of the vehicle, wherein the driver-side video display screen is operable to display driver-side video images derived at least in part from image data captured by the driver-side camera;
a passenger-side camera disposed at a passenger side of the vehicle and viewing at least sideward and rearward at the passenger side of the vehicle, and wherein the passenger-side camera views at least a portion of a passenger-side traffic lane adjacent to the traffic lane in which the vehicle is traveling;
the passenger-side camera capturing image data;
a passenger-side video display screen disposed at a passenger-side cabin region of the interior cabin of the vehicle and viewable by the driver of the vehicle, wherein the passenger-side video display screen is operable to display passenger-side video images derived at least in part from image data captured by the passenger-side camera;
wherein, with the vehicle traveling along the road, the driver-side video display screen displays driver-side video images of a driver-side region sideward of the driver side of the vehicle and rearward of the vehicle to assist the driver when driving the vehicle along the road and the passenger-side video display screen displays passenger-side video images of a passenger-side region sideward of the passenger side of the vehicle and rearward of the vehicle to assist the driver when driving the vehicle along the road; and
wherein, upon completion of a parking maneuver of the vehicle, and until a period of time elapses following powering down of the vehicle upon completion of the parking maneuver of the vehicle, the driver-side video display screen displays driver-side video images of the driver-side region sideward of the driver side of the vehicle and rearward of the vehicle and the passenger-side video display screen displays passenger-side video images of the passenger-side region sideward of the passenger side of the vehicle and rearward of the vehicle.
2. The vehicular camera monitoring system of claim 1, wherein the driver-side video display screen is operable to display a graphic overlay at the displayed driver-side video images to alert the driver of relative locations of other vehicles viewed by the driver-side camera.
3. The vehicular camera monitoring system of claim 1, wherein the passenger-side video display screen is operable to display a graphic overlay at the displayed passenger-side video images to alert the driver of relative locations of other vehicles viewed by the passenger-side camera.
4. The vehicular camera monitoring system of claim 1, wherein the driver-side camera is disposed at a driver-side exterior rearview mirror assembly mounted at an exterior portion of a driver-side door of the vehicle, and wherein the passenger-side camera is disposed at a passenger-side exterior rearview mirror assembly mounted at an exterior portion of a passenger-side door of the vehicle.
5. The vehicular camera monitoring system of claim 1, wherein the vehicular camera monitoring system is operable responsive to a driver-seat occupancy detection system, and wherein the vehicular camera monitoring system remains operating until a period of time elapses following determination by the driver-seat occupancy detection system that a driver seat of the vehicle is unoccupied.
6. The vehicular camera monitoring system of claim 5, wherein the driver-seat occupancy detection system operates to determine occupancy of the driver seat responsive to the vehicle being powered down.
7. The vehicular camera monitoring system of claim 5, wherein at least the driver-side camera and the driver-side video display screen of the vehicular camera monitoring system remain operating until the period of time elapses following determination that the driver seat is unoccupied.
8. The vehicular camera monitoring system of claim 7, wherein the passenger-side camera and the passenger-side video display screen of the vehicular camera monitoring system remain operating until the period of time elapses following determination that the driver seat is unoccupied.
9. The vehicular camera monitoring system of claim 1, wherein, during the parking maneuver of the vehicle, the passenger-side video display screen displays passenger-side video images that provide to the driver of the vehicle viewing the passenger-side video display screen a view of a passenger-side ground region at the passenger side of the vehicle.
10. The vehicular camera monitoring system of claim 1, wherein, responsive to the driver driving the vehicle past a parallel parking space between two parked vehicles at the passenger side of the road and shifting the vehicle into reverse gear to commence the parking maneuver to maneuver the vehicle into the parallel parking space, (i) the driver-side video display screen continues to display the driver-side video images of the driver-side region sideward of the driver side of the vehicle and rearward of the vehicle and (ii) the passenger-side video display screen displays passenger-side video images that provide to the driver of the vehicle viewing the passenger-side video display screen a view of a passenger-side ground region at the passenger side of the vehicle.
11. The vehicular camera monitoring system of claim 10, wherein the passenger-side video images displayed by the passenger-side video display screen during the parking maneuver of the vehicle comprise passenger-side video images of the ground adjacent to and forward and/or rearward of the vehicle to assist the driver in maneuvering the vehicle into the parallel parking space.
12. The vehicular camera monitoring system of claim 10, wherein, after the vehicle is parallel parked in the parallel parking space at the passenger side of the road, and responsive to the driver executing a parallel parking space exit maneuver of the vehicle out of the parallel parking space onto the road, the driver-side video display screen displays driver-side video images and the passenger-side video display screen displays the passenger-side video images that provide to the driver of the vehicle viewing the passenger-side video display screen the view of the passenger-side ground region at the passenger side of the vehicle.
13. The vehicular camera monitoring system of claim 12, wherein, upon completion of the parallel parking space exit maneuver, the passenger-side video display screen displays passenger-side video images of the passenger-side region sideward of the passenger side of the vehicle and rearward of the vehicle.
14. The vehicular camera monitoring system of claim 13, wherein the passenger-side video display screen commences display of the passenger-side region sideward of the passenger side of the vehicle and rearward of the vehicle responsive to a threshold condition being met following completion of the parallel parking space exit maneuver.
15. The vehicular camera monitoring system of claim 14, wherein the threshold condition comprises at least one selected from the group consisting of (i) the vehicle reaching a threshold speed after exiting the parallel parking space, (ii) the vehicle traveling a threshold distance after exiting the parallel parking space, and (iii) a threshold period of time elapsing after the vehicle has exited the parallel parking space.
16. A vehicular camera monitoring system, the vehicular camera monitoring system comprising:
a driver-side camera disposed at a driver side of a vehicle equipped with the vehicular camera monitoring system and viewing at least sideward and rearward at the driver side of the vehicle, and wherein the driver-side camera views at least a portion of a driver-side traffic lane adjacent to a traffic lane in which the vehicle is traveling along a road;
the driver-side camera capturing image data;
a driver-side video display screen disposed at a driver-side cabin region of an interior cabin of the vehicle and viewable by a driver of the vehicle, wherein the driver-side video display screen is operable to display driver-side video images derived at least in part from image data captured by the driver-side camera;
a passenger-side camera disposed at a passenger side of the vehicle and viewing at least sideward and rearward at the passenger side of the vehicle, and wherein the passenger-side camera views at least a portion of a passenger-side traffic lane adjacent to the traffic lane in which the vehicle is traveling;
the passenger-side camera capturing image data;
a passenger-side video display screen disposed at a passenger-side cabin region of the interior cabin of the vehicle and viewable by the driver of the vehicle, wherein the passenger-side video display screen is operable to display passenger-side video images derived at least in part from image data captured by the passenger-side camera;
wherein the driver-side camera is disposed at a driver-side exterior rearview mirror assembly mounted at an exterior portion of a driver-side door of the vehicle, and wherein the passenger-side camera is disposed at a passenger-side exterior rearview mirror assembly mounted at an exterior portion of a passenger-side door of the vehicle;
wherein, with the vehicle traveling along the road, the driver-side video display screen displays driver-side video images of a driver-side region sideward of the driver side of the vehicle and rearward of the vehicle to assist the driver when driving the vehicle along the road and the passenger-side video display screen displays passenger-side video images of a passenger-side region sideward of the passenger side of the vehicle and rearward of the vehicle to assist the driver when driving the vehicle along the road;
wherein, responsive to the driver driving the vehicle past a parallel parking space between two parked vehicles at the passenger side of the road and shifting the vehicle into reverse gear to commence a parking maneuver to maneuver the vehicle into the parallel parking space, (i) the driver-side video display screen continues to display the driver-side video images of the driver-side region sideward of the driver side of the vehicle and rearward of the vehicle and (ii) the passenger-side video display screen displays passenger-side video images that provide to the driver of the vehicle viewing the passenger-side video display screen a view of a passenger-side ground region at the passenger side of the vehicle; and
wherein, upon completion of the parking maneuver of the vehicle, and until a period of time elapses following powering down of the vehicle upon completion of the parking maneuver of the vehicle, the driver-side video display screen displays driver-side video images of the driver-side region sideward of the driver side of the vehicle and rearward of the vehicle and the passenger-side video display screen displays passenger-side video images of the passenger-side region sideward of the passenger side of the vehicle and rearward of the vehicle.
17. The vehicular camera monitoring system of claim 16, wherein the driver-side video display screen is operable to display a graphic overlay at the displayed driver-side video images to alert the driver of relative locations of other vehicles viewed by the driver-side camera.
18. The vehicular camera monitoring system of claim 16, wherein the passenger-side video display screen is operable to display a graphic overlay at the displayed passenger-side video images to alert the driver of relative locations of other vehicles viewed by the passenger-side camera.
19. The vehicular camera monitoring system of claim 16, wherein the vehicular camera monitoring system is operable responsive to a driver-seat occupancy detection system, and wherein the vehicular camera monitoring system remains operating until a period of time elapses following determination by the driver-seat occupancy detection system that a driver seat of the vehicle is unoccupied.
20. The vehicular camera monitoring system of claim 19, wherein the driver-seat occupancy detection system operates to determine occupancy of the driver seat responsive to the vehicle being powered down.
21. The vehicular camera monitoring system of claim 19, wherein at least the driver-side camera and the driver-side video display screen of the vehicular camera monitoring system remain operating until the period of time elapses following determination that the driver seat is unoccupied.
22. The vehicular camera monitoring system of claim 21, wherein the passenger-side camera and the passenger-side video display screen of the vehicular camera monitoring system remain operating until the period of time elapses following determination that the driver seat is unoccupied.
23. The vehicular camera monitoring system of claim 16, wherein the passenger-side video images displayed by the passenger-side video display screen during the parking maneuver of the vehicle comprise passenger-side video images of the ground adjacent to and forward and/or rearward of the vehicle to assist the driver in maneuvering the vehicle into the parallel parking space.
24. The vehicular camera monitoring system of claim 16, wherein, after the vehicle is parallel parked in the parallel parking space at the passenger side of the road, and responsive to the driver executing a parallel parking space exit maneuver of the vehicle out of the parallel parking space onto the road, the driver-side video display screen displays driver-side video images and the passenger-side video display screen displays the passenger-side video images that provide to the driver of the vehicle viewing the passenger-side video display screen the view of the passenger-side ground region at the passenger side of the vehicle.
25. The vehicular camera monitoring system of claim 24, wherein, upon completion of the parallel parking space exit maneuver, the passenger-side video display screen displays passenger-side video images of the passenger-side region sideward of the passenger side of the vehicle and rearward of the vehicle.
26. The vehicular camera monitoring system of claim 25, wherein the passenger-side video display screen commences display of the passenger-side region sideward of the passenger side of the vehicle and rearward of the vehicle responsive to a threshold condition being met following completion of the parallel parking space exit maneuver.
27. The vehicular camera monitoring system of claim 26, wherein the threshold condition comprises at least one selected from the group consisting of (i) the vehicle reaching a threshold speed after exiting the parallel parking space, (ii) the vehicle traveling a threshold distance after exiting the parallel parking space, and (iii) a threshold period of time elapsing after the vehicle has exited the parallel parking space.
28. A vehicular camera monitoring system, the vehicular camera monitoring system comprising:
a driver-side camera disposed at a driver side of a vehicle equipped with the vehicular camera monitoring system and viewing at least sideward and rearward at the driver side of the vehicle, and wherein the driver-side camera views at least a portion of a driver-side traffic lane adjacent to a traffic lane in which the vehicle is traveling along a road;
the driver-side camera capturing image data;
a driver-side video display screen disposed at a driver-side cabin region of an interior cabin of the vehicle and viewable by a driver of the vehicle, wherein the driver-side video display screen is operable to display driver-side video images derived at least in part from image data captured by the driver-side camera;
a passenger-side camera disposed at a passenger side of the vehicle and viewing at least sideward and rearward at the passenger side of the vehicle, and wherein the passenger-side camera views at least a portion of a passenger-side traffic lane adjacent to the traffic lane in which the vehicle is traveling;
the passenger-side camera capturing image data;
a passenger-side video display screen disposed at a passenger-side cabin region of the interior cabin of the vehicle and viewable by the driver of the vehicle, wherein the passenger-side video display screen is operable to display passenger-side video images derived at least in part from image data captured by the passenger-side camera;
wherein, with the vehicle traveling along the road, the driver-side video display screen displays driver-side video images of a driver-side region sideward of the driver side of the vehicle and rearward of the vehicle to assist the driver when driving the vehicle along the road and the passenger-side video display screen displays passenger-side video images of a passenger-side region sideward of the passenger side of the vehicle and rearward of the vehicle to assist the driver when driving the vehicle along the road;
wherein the vehicular camera monitoring system is operable responsive to a driver-seat occupancy detection system of the vehicle; and
wherein, upon powering down of the vehicle and until a period of time elapses following determination by the driver-seat occupancy detection system that a driver seat of the vehicle is unoccupied, the driver-side video display screen displays driver-side video images of the driver-side region sideward of the driver side of the vehicle and rearward of the vehicle.
29. The vehicular camera monitoring system of claim 28, wherein the driver-side video display screen is operable to display a graphic overlay at the displayed driver-side video images to alert the driver of relative locations of other vehicles viewed by the driver-side camera.
30. The vehicular camera monitoring system of claim 28, wherein the passenger-side video display screen is operable to display a graphic overlay at the displayed passenger-side video images to alert the driver of relative locations of other vehicles viewed by the passenger-side camera.
31. The vehicular camera monitoring system of claim 28, wherein the driver-side camera is disposed at a driver-side exterior rearview mirror assembly mounted at an exterior portion of a driver-side door of the vehicle, and wherein the passenger-side camera is disposed at a passenger-side exterior rearview mirror assembly mounted at an exterior portion of a passenger-side door of the vehicle.
32. The vehicular camera monitoring system of claim 28, wherein the driver-seat occupancy detection system operates to determine occupancy of the driver seat responsive to the vehicle being powered down.
33. The vehicular camera monitoring system of claim 28, wherein the passenger-side camera and the passenger-side video display screen of the vehicular camera monitoring system remain operating until the period of time elapses following determination that the driver seat is unoccupied.
34. The vehicular camera monitoring system of claim 28, wherein the passenger-side camera and the passenger-side video display screen of the vehicular camera monitoring system remain operating until a period of time elapses following determination that a passenger sitting in a passenger seat at the passenger side of the vehicle exits the vehicle.
35. The vehicular camera monitoring system of claim 28, wherein, during a parking maneuver of the vehicle, the passenger-side video display screen displays passenger-side video images that provide to the driver of the vehicle viewing the passenger-side video display screen a view of a passenger-side ground region at the passenger side of the vehicle.
US18/054,963 2019-11-27 2022-11-14 Vehicular camera monitoring system Pending US20230077247A1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US18/054,963 US20230077247A1 (en) 2019-11-27 2022-11-14 Vehicular camera monitoring system

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201962941089P 2019-11-27 2019-11-27
US201962942351P 2019-12-02 2019-12-02
US16/949,976 US11498494B2 (en) 2019-11-27 2020-11-23 Vehicular camera monitoring system
US18/054,963 US20230077247A1 (en) 2019-11-27 2022-11-14 Vehicular camera monitoring system

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
US16/949,976 Continuation US11498494B2 (en) 2019-11-27 2020-11-23 Vehicular camera monitoring system

Publications (1)

Publication Number Publication Date
US20230077247A1 true US20230077247A1 (en) 2023-03-09

Family

ID=75973687

Family Applications (2)

Application Number Title Priority Date Filing Date
US16/949,976 Active 2041-08-03 US11498494B2 (en) 2019-11-27 2020-11-23 Vehicular camera monitoring system
US18/054,963 Pending US20230077247A1 (en) 2019-11-27 2022-11-14 Vehicular camera monitoring system

Family Applications Before (1)

Application Number Title Priority Date Filing Date
US16/949,976 Active 2041-08-03 US11498494B2 (en) 2019-11-27 2020-11-23 Vehicular camera monitoring system

Country Status (1)

Country Link
US (2) US11498494B2 (en)

Families Citing this family (31)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7147255B2 (en) * 2018-05-11 2022-10-05 トヨタ自動車株式会社 image display device
US11142126B2 (en) 2019-01-25 2021-10-12 Magna Mirrors Of America, Inc. Interior rearview mirror assembly
US11205083B2 (en) 2019-04-02 2021-12-21 Magna Electronics Inc. Vehicular driver monitoring system
US11214199B2 (en) 2019-06-03 2022-01-04 Magna Mirrors Of America, Inc. Interior rearview mirror assembly with display and tilt mechanism
DE102019123740B4 (en) * 2019-09-04 2023-01-26 Motherson Innovations Company Limited Method for providing a rear and/or side view of a vehicle, virtual mirror device and vehicle
US11498494B2 (en) * 2019-11-27 2022-11-15 Magna Electronics Inc. Vehicular camera monitoring system
US11505123B2 (en) 2019-12-02 2022-11-22 Magna Electronics Inc. Vehicular camera monitoring system with stereographic display
US11388354B2 (en) * 2019-12-06 2022-07-12 Razmik Karabed Backup-camera-system-based, on-demand video player
US11242008B2 (en) 2020-02-07 2022-02-08 Magna Mirrors Of America, Inc. Vehicular vision system with center stack display and mirror display for surround view and CMS cameras
US11807169B2 (en) 2020-02-24 2023-11-07 Magna Mirrors Of America, Inc. Vehicular vision system with enhanced forward and sideward views
US20210300263A1 (en) * 2020-03-25 2021-09-30 Magna Mirrors Of America, Inc. Vehicular display system with adjustable mounting of rear camera
US11465561B2 (en) 2020-04-17 2022-10-11 Magna Mirrors Of America, Inc. Interior rearview mirror assembly with driver monitoring system
US11560143B2 (en) 2020-05-26 2023-01-24 Magna Electronics Inc. Vehicular autonomous parking system using short range communication protocols
US11582425B2 (en) 2020-06-10 2023-02-14 Magna Electronics Inc. Driver monitoring system using camera with adjustable mirror
US11912204B2 (en) 2020-06-24 2024-02-27 Magna Mirrors Of America, Inc. Low-profile actuator for extendable camera
CN116096605A (en) 2020-06-24 2023-05-09 麦格纳镜片美国有限公司 Vehicle imaging system with expandable camera
KR20220005778A (en) * 2020-07-07 2022-01-14 현대자동차주식회사 Outside mirror type display apparatus use for a car
US11708035B2 (en) 2020-07-31 2023-07-25 Magna Mirrors Of America, Inc. Vehicular imaging system with extendable camera
US11884216B2 (en) * 2020-10-16 2024-01-30 Magna Mirrors Of America, Inc. Vehicular full mirror display system with auxiliary forward and rearward views
US20220203898A1 (en) * 2020-12-29 2022-06-30 Larry Warren Vehicle side mirror with video display and automatic camera system
USD1015342S1 (en) * 2021-01-08 2024-02-20 Sony Group Corporation Display screen or portion thereof with animated graphical user interface
US11851080B2 (en) 2021-02-03 2023-12-26 Magna Mirrors Of America, Inc. Vehicular driver monitoring system with posture detection and alert
EP4301626A1 (en) 2021-03-01 2024-01-10 Magna Mirrors Of America, Inc. Interior rearview mirror assembly with driver monitoring system
US11780372B2 (en) 2021-03-01 2023-10-10 Magna Mirrors Of America, Inc. Vehicular driver monitoring system with driver monitoring camera and near IR light emitter at interior rearview mirror assembly
US20220311974A1 (en) * 2021-03-24 2022-09-29 Deere & Company Work vehicle fleet with intuitive vision system
US11930264B2 (en) 2021-05-18 2024-03-12 Magna Electronics Inc. Vehicular driver monitoring system with camera view optimization
US11766968B2 (en) 2021-05-18 2023-09-26 Magna Mirrors Of America, Inc. Vehicular interior rearview mirror assembly with video mirror display and VRLC stack
WO2023034956A1 (en) 2021-09-03 2023-03-09 Magna Electronics Inc. Vehicular interior cabin monitoring system with light emitter that is selectively operable for dms and oms functions
WO2023062132A1 (en) 2021-10-13 2023-04-20 Magna Mirrors Holding Gmbh Vehicular overhead console with light transmissive panel
CN114162337B (en) * 2021-11-27 2024-01-26 西北工业大学 Windshield system for reducing oil injection lubrication windage loss of high-speed spiral bevel gear
CN114821531B (en) * 2022-04-25 2023-03-28 广州优创电子有限公司 Lane line recognition image display system based on electronic exterior rearview mirror ADAS

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030236622A1 (en) * 2002-04-25 2003-12-25 Kenneth Schofield Imaging system for vehicle
US20110181728A1 (en) * 2008-12-19 2011-07-28 Delphi Technologies, Inc. Electronic side view display system
US20120013742A1 (en) * 2010-07-16 2012-01-19 Delphi Technologies, Inc. Vision system and method for displaying a field of view dependent upon detecting an object
US20180312114A1 (en) * 2017-04-28 2018-11-01 Toyota Jidosha Kabushiki Kaisha Image display apparatus
US20190344828A1 (en) * 2018-05-08 2019-11-14 Toyota Jidosha Kabushiki Kaisha Parking assist apparatus
US20210155167A1 (en) * 2019-11-27 2021-05-27 Magna Electronics Inc. Vehicular camera monitoring system

Family Cites Families (68)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US2263382A (en) 1939-06-19 1941-11-18 Gotzinger George Mirror signal
US2580014A (en) 1949-09-13 1951-12-25 Gazda Antoine Combined rearview mirror and direction indicating device
US3266016A (en) 1964-08-06 1966-08-09 Maru Sho Outside signal for automobiles
GB2099628B (en) 1981-04-07 1984-11-14 Nippon Denso Co Electroluminescent device combined with mirror
US4623222A (en) 1983-11-14 1986-11-18 Nippondenso Co., Ltd. Liquid crystal type dazzle-free transmissive-reflective mirror
JPS60143301A (en) 1983-12-29 1985-07-29 Nippon Denso Co Ltd Nonglaring type reflection mirror and its manufacture
US4588267A (en) 1984-01-18 1986-05-13 Ronald Pastore Combination rear view mirror and digital clock
US4630904A (en) 1985-01-22 1986-12-23 Ronald Pastore Combination rear view mirror and digital displays
US4906085A (en) 1986-06-30 1990-03-06 Kabushiki Kaisha Tokai Rika Denki Seisakusho Sideview mirror assembly with regulating filter using sunlight for automobiles
US5355284A (en) 1990-02-20 1994-10-11 K. W. Muth Company, Inc. Mirror assembly
US5481409A (en) 1990-02-20 1996-01-02 K. W. Muth Company, Inc. Mirror assembly
US5313335A (en) 1992-06-05 1994-05-17 Delco Electronics Corporation Blindzone signal indicator
US5436741A (en) 1993-12-28 1995-07-25 Harman Automotive, Inc. Holographic signaling mirror
US5668663A (en) 1994-05-05 1997-09-16 Donnelly Corporation Electrochromic mirrors and devices
US5587699A (en) 1994-11-03 1996-12-24 United Technologies Automotive Systems Inc. Exterior mirror with information display
US5575552A (en) 1994-12-09 1996-11-19 United Technologies Automotive Systems, Inc. Lighted mirror apparatus
US5786772A (en) 1996-03-22 1998-07-28 Donnelly Corporation Vehicle blind spot detection display system
US6045243A (en) 1996-08-28 2000-04-04 K.W. Muth Company, Inc. Mirror assembly
US5788357A (en) 1996-08-28 1998-08-04 K. W. Muth Company, Inc. Mirror assembly
US5938320A (en) 1997-03-19 1999-08-17 Harman Automotive, Inc. Enhanced illuminated polymeric indicator employed in a mirror housing of an automotive vehicle
US6111683A (en) 1997-04-02 2000-08-29 Gentex Corporation Electrochromic mirrors having a signal light
US6700692B2 (en) 1997-04-02 2004-03-02 Gentex Corporation Electrochromic rearview mirror assembly incorporating a display/signal light
US6356376B1 (en) 1997-04-02 2002-03-12 Gentex Corporation Electrochromic rearview mirror incorporating a third surface metal reflector and a display/signal light
US6005724A (en) 1998-10-05 1999-12-21 K. W. Muth Company, Inc. Mirror coating, mirror utilizing same, and a mirror assembly
US6257746B1 (en) 1998-11-03 2001-07-10 K. W. Muth Company, Inc. Signalling assembly
US6264353B1 (en) 1998-11-23 2001-07-24 Lear Automotive Dearborn, Inc. Exterior mirror with supplement turn signal
US7581859B2 (en) 2005-09-14 2009-09-01 Donnelly Corp. Display device for exterior rearview mirror
US6882287B2 (en) 2001-07-31 2005-04-19 Donnelly Corporation Automotive lane change aid
US7777611B2 (en) 2006-11-06 2010-08-17 Donnelly Corporation Display device for exterior rearview mirror
WO2010099416A1 (en) 2009-02-27 2010-09-02 Magna Electronics Alert system for vehicle
WO2011014482A1 (en) 2009-07-27 2011-02-03 Magna Electronics Inc. Parking assist system
ES2538827T3 (en) 2009-09-01 2015-06-24 Magna Mirrors Of America, Inc. Imaging and display system for a vehicle
WO2012068331A1 (en) 2010-11-19 2012-05-24 Magna Electronics Inc. Lane keeping system and lane centering system
WO2013067082A1 (en) 2011-11-01 2013-05-10 Magna Mirrors Of America, Inc. Vision system with door mounted exterior mirror and display
WO2013109869A1 (en) 2012-01-20 2013-07-25 Magna Electronics, Inc. Vehicle vision system with free positional virtual panoramic view
US9340227B2 (en) 2012-08-14 2016-05-17 Magna Electronics Inc. Vehicle lane keep assist system
US9280202B2 (en) 2013-05-10 2016-03-08 Magna Electronics Inc. Vehicle vision system
WO2014204794A1 (en) 2013-06-21 2014-12-24 Magna Electronics Inc. Vehicle vision system
US20140375476A1 (en) 2013-06-24 2014-12-25 Magna Electronics Inc. Vehicle alert system
US20150009010A1 (en) 2013-07-03 2015-01-08 Magna Electronics Inc. Vehicle vision system with driver detection
US9701258B2 (en) 2013-07-09 2017-07-11 Magna Electronics Inc. Vehicle vision system
US10908417B2 (en) 2013-09-19 2021-02-02 Magna Electronics Inc. Vehicle vision system with virtual retinal display
DE102015202846B4 (en) 2014-02-19 2020-06-25 Magna Electronics, Inc. Vehicle vision system with display
US20150294169A1 (en) 2014-04-10 2015-10-15 Magna Electronics Inc. Vehicle vision system with driver monitoring
US20150296135A1 (en) 2014-04-10 2015-10-15 Magna Electronics Inc. Vehicle vision system with driver monitoring
US20150352953A1 (en) 2014-06-04 2015-12-10 Magna Electronics Inc. Vehicle control system with mobile device interface
US9405120B2 (en) 2014-11-19 2016-08-02 Magna Electronics Solutions Gmbh Head-up display and vehicle using the same
US9946940B2 (en) 2014-12-18 2018-04-17 Magna Electronics Inc. Vehicle vision system with adaptive lane marker detection
US10247941B2 (en) 2015-01-19 2019-04-02 Magna Electronics Inc. Vehicle vision system with light field monitor
US20180134217A1 (en) 2015-05-06 2018-05-17 Magna Mirrors Of America, Inc. Vehicle vision system with blind zone display and alert system
US10421404B2 (en) 2015-06-26 2019-09-24 Magna Mirrors Of America, Inc. Interior rearview mirror assembly with full screen video display
US10046706B2 (en) 2015-06-26 2018-08-14 Magna Mirrors Of America, Inc. Interior rearview mirror assembly with full screen video display
US10214206B2 (en) 2015-07-13 2019-02-26 Magna Electronics Inc. Parking assist system for vehicle
US10078789B2 (en) 2015-07-17 2018-09-18 Magna Electronics Inc. Vehicle parking assist system with vision-based parking space detection
US10268904B2 (en) 2015-07-31 2019-04-23 Magna Electronics Inc. Vehicle vision system with object and lane fusion
US10086870B2 (en) 2015-08-18 2018-10-02 Magna Electronics Inc. Trailer parking assist system for vehicle
US10315651B2 (en) 2015-10-13 2019-06-11 Magna Electronics Inc. Vehicle vision system with lateral control algorithm for lane keeping
US10029614B2 (en) 2015-11-17 2018-07-24 Magna Mirrors Of America, Inc. Interior rearview mirror assembly
US10703204B2 (en) 2016-03-23 2020-07-07 Magna Electronics Inc. Vehicle driver monitoring system
ES2834890T3 (en) 2016-05-02 2021-06-21 Magna Mirrors Of America Inc Rear view mirror assembly without housing
US10166924B2 (en) 2016-05-11 2019-01-01 Magna Mirrors Of America, Inc. Vehicle vision system with display by a mirror
US20170329346A1 (en) 2016-05-12 2017-11-16 Magna Electronics Inc. Vehicle autonomous parking system
US20170355312A1 (en) 2016-06-08 2017-12-14 Magna Mirrors Of America, Inc. Interior rearview mirror assembly with full screen video display
US10442360B2 (en) 2017-03-02 2019-10-15 Magna Mirrors Of America, Inc. Interior rearview mirror assembly with display and tilt mechanism
US10583779B2 (en) 2017-10-02 2020-03-10 Magna Electronics Inc. Parking assist system using backup camera
US20190118717A1 (en) 2017-10-25 2019-04-25 Magna Mirrors Of America, Inc. Full mirror display utilizing a trailer camera
US20200130676A1 (en) 2018-10-25 2020-04-30 Magna Electronics Inc. Autonomous vehicle parking system
US11505123B2 (en) 2019-12-02 2022-11-22 Magna Electronics Inc. Vehicular camera monitoring system with stereographic display

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030236622A1 (en) * 2002-04-25 2003-12-25 Kenneth Schofield Imaging system for vehicle
US20110181728A1 (en) * 2008-12-19 2011-07-28 Delphi Technologies, Inc. Electronic side view display system
US20120013742A1 (en) * 2010-07-16 2012-01-19 Delphi Technologies, Inc. Vision system and method for displaying a field of view dependent upon detecting an object
US20180312114A1 (en) * 2017-04-28 2018-11-01 Toyota Jidosha Kabushiki Kaisha Image display apparatus
US20190344828A1 (en) * 2018-05-08 2019-11-14 Toyota Jidosha Kabushiki Kaisha Parking assist apparatus
US20210155167A1 (en) * 2019-11-27 2021-05-27 Magna Electronics Inc. Vehicular camera monitoring system

Also Published As

Publication number Publication date
US20210155167A1 (en) 2021-05-27
US11498494B2 (en) 2022-11-15

Similar Documents

Publication Publication Date Title
US11498494B2 (en) Vehicular camera monitoring system
US11679719B2 (en) Vehicular vision system with console video display and mirror video display for exterior viewing cameras
US11034299B2 (en) Vehicular vision system with episodic display of video images showing approaching other vehicle
US11718291B2 (en) Vehicular collision avoidance system
US11288888B2 (en) Vehicular control system
US10515279B2 (en) Vehicle vision system with front and rear camera integration
US20050146607A1 (en) Object Approaching Detection Anti Blind e-Mirrors System
US20150294169A1 (en) Vehicle vision system with driver monitoring
US20140313335A1 (en) Vision system for vehicle with adjustable cameras
US20140340516A1 (en) Rear view camera system using rear view mirror location
US11884216B2 (en) Vehicular full mirror display system with auxiliary forward and rearward views
US20220363194A1 (en) Vehicular display system with a-pillar display
WO2023094370A1 (en) Lateral boundary warning method in assisted driving
CN115489432A (en) Monitoring system for vehicle

Legal Events

Date Code Title Description
STPP Information on status: patent application and granting procedure in general

Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED