WO2018129310A1 - Système et procédé d'affichage d'image sur un élément intérieur de véhicule - Google Patents

Système et procédé d'affichage d'image sur un élément intérieur de véhicule Download PDF

Info

Publication number
WO2018129310A1
WO2018129310A1 PCT/US2018/012574 US2018012574W WO2018129310A1 WO 2018129310 A1 WO2018129310 A1 WO 2018129310A1 US 2018012574 W US2018012574 W US 2018012574W WO 2018129310 A1 WO2018129310 A1 WO 2018129310A1
Authority
WO
WIPO (PCT)
Prior art keywords
camera
image
display
vehicle
operator
Prior art date
Application number
PCT/US2018/012574
Other languages
English (en)
Inventor
Mario Javier ENRIQUEZ ORTIZ
Christopher Kring
Original Assignee
Shanghai Yanfeng Jinqiao Automotive Trim Systems Co. Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Yanfeng Jinqiao Automotive Trim Systems Co. Ltd filed Critical Shanghai Yanfeng Jinqiao Automotive Trim Systems Co. Ltd
Publication of WO2018129310A1 publication Critical patent/WO2018129310A1/fr

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B60R1/25—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view to the sides of the vehicle
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/18—Image warping, e.g. rearranging pixels individually
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/765—Interface circuits between an apparatus for recording and another apparatus
    • H04N5/77—Interface circuits between an apparatus for recording and another apparatus between a recording apparatus and a television camera
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/20—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of display used
    • B60R2300/202—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of display used displaying a blind spot scene on the vehicle part responsible for the blind spot

Definitions

  • the present invention relates to a system and method for image display on a vehicle interior component.
  • the present invention also relates to a display system for a vehicle to facilitate observation of an object outside of the vehicle by an operator of the vehicle.
  • the present invention further relates to a display system for a vehicle interior configured to use data from data sources to present an image of objects outside the vehicle for an operator of the vehicle.
  • the present invention further relates to a vehicle interior component providing a display system for a vehicle interior to facilitate observation of an object outside of the vehicle by an operator of the vehicle.
  • the present invention further relates to a system and method of displaying an image of an object outside of a vehicle on a vehicle interior component inside the vehicle for the operator of the vehicle.
  • the present invention further relates to an improved system and method for displaying objects outside of the vehicle that would be obstructed to an operator of a vehicle by a structure or component on a display panel on the structure or component so that the operator has a generally continuous unobstructed view of objects outside the vehicle (and in a consistent scale/form with other visible objects) while operating the vehicle.
  • the present invention relates to a method of operating a display system for a vehicle configured to be operated from a field of view of an operator in a position in a vehicle interior comprising a structure in a position in the vehicle interior presenting an at least partial obstruction to the field of view of the operator and with a display screen on the structure in the field of view of the operator configured to use data from data sources including a camera image from outside of the vehicle from a camera system connected to a computing system configured to process data representative of the camera image into a display image on the display screen.
  • the method may comprise the steps of: (a) defining an object plane at a distance outside of the vehicle; (b) obtaining data comprising the camera image from the camera system; (c) processing of the camera image into the display image; (d) displaying the display image on the display screen.
  • Processing of the camera image into the display image may comprise transforming the camera image so that the display image is at least partially representative of the field of view of the operator at the object plane in the absence of the obstruction from the structure.
  • Processing of the camera image may comprise using data from data sources and at least one of (a) transforming the camera image to the display image based on data representative of position of the operator of the vehicle; (b) geometric transformation of the camera image into the display image; (c) mapping of the camera image; (d) remapping of the camera image into the display image.
  • Processing of the camera image may comprise use of data comprising at least one of data representative of (a) location of the object plane external to the vehicle; (b) position of the operator in the vehicle interior; (c) position of the structure presenting the obstruction in the vehicle interior; (d) position of the camera; (e) position of the display screen; (f) form of the display screen; (g) a reference position for the vehicle.
  • Data used for processing of the camera image may comprise data representative of (a) location of the object plane; (b) position of the operator in the vehicle.
  • Data used for processing of the camera image may comprise data representative of location of the object plane; data representative of the location of the object plane may comprise data based on at least one of (a) distance from a vehicle reference point; (b) distance of an object detected external to the vehicle.
  • Processing of the camera image may comprise correction of the camera image to the display image based on data representative of position of the camera and position of the operator.
  • Data from data sources may comprise data representative of at least one of camera parameters, display parameters, operator parameters and object parameters.
  • Camera parameters comprise data representative of at least one of camera position, camera orientation, camera angle, camera field of view, camera type, camera image resolution, camera image size, camera image array size.
  • Display parameters comprise data representative of at least one of display screen position, display screen form, display screen orientation, display screen type, display screen resolution, display screen size, display image array size.
  • Operator parameters comprise data representative of operator position, operator head position, operator eye position, operator field of view, operator seat position.
  • Object parameters may comprise data representative of object plane position, object detection, object position detection; data representative of object detection may be used to determine the object plane position.
  • the method may comprise the step of determining a transformation for transforming the camera image to the display image; the transformation may be used for processing of the camera image into the display image.
  • the transformation may be based on data representative of location of the object plane; the transformation may be based on data representative of (a) position of the display screen; (b) position of the operator; (c) position of the camera.
  • the transformation may be based on data representative of at least one of camera parameters, display parameters, operator parameters, object parameters.
  • the transformation may comprise at least one of (a) transformation of the camera image to the display image based on data representative of position of the operator of the vehicle; (b) geometric transformation of the camera image into the display image; (c) mapping of the camera image; (d) remapping of the camera image into the display image; (e) correction of the camera image to the display image based on data
  • the transformation may be based on a function comprising at least one of (a) remapping of a data array from the camera image to a data array for the display screen; (b) correcting the camera image from the object plane into the display image presented on the display screen; (c) transforming a data array from the camera image into and a data array for the display image.
  • the method may comprise the step of monitoring data sources; monitoring data sources may comprise monitoring of at least one of camera parameters, display parameters, operator parameters, object parameters. Monitoring data sources may comprise monitoring adjustments; adjustments may comprise at least one of system adjustments and operation adjustments.
  • the method may comprise the step of initialization; initialization may comprise initial settings of parameters for processing of the camera image into the display image comprising data representative of at least one of (a) location of the object plane external to the vehicle; (b) position of the operator in the vehicle interior; (c) position of the camera; (d) position of the display screen; (e) a reference position for the vehicle.
  • the present invention relates to a vehicle interior component comprising a display system for a vehicle configured to be operated from a field of view of an operator in a position in a vehicle interior comprising a structure in a position in the vehicle interior presenting an at least partial obstruction to the field of view of the operator.
  • the component may comprise (a) a camera system comprising a camera configured to obtain a camera image; (b) a computing system configured for processing of the camera image using data; (c) a display screen on the structure within the vehicle configured to display a display image for the operator of the vehicle. Processing of the camera image may comprise transformation of the camera image into the display image so that the display image is at least partially representative of the field of view of the operator in the absence of the obstruction from the structure.
  • Processing of the camera image may comprise at least one of (a) transformation of the camera image to the display image based on data representative of position of the operator of the vehicle; (b) geometric transformation of the camera image into the display image; (c) mapping of the camera image; (d) remapping of the camera image into the display image.
  • Processing of the camera image into the display image may comprise correction of the camera image into the display image based on data representative of position of the camera and position of the operator.
  • Processing of the camera image into the display image may comprise use of data representative of location of an object plane external to the vehicle.
  • Data representative of location of the object plane may comprise data based on at least one of (a) distance from a vehicle reference point; (b) distance of an object detected external to the vehicle. Data may comprise data representative of a relative distance between position of the camera and position of the operator. Data may comprise data representative of (a) an image in a field of view of the camera system; (b) a relative distance between the camera and the operator; (c) location of an object plane.
  • Data may comprise data representative of at least one of (a) location of an object plane external to the vehicle; (b) position of the operator in the vehicle interior; (c) position of the structure presenting the obstruction in the vehicle interior; (d) position of the camera; (e) position of the display screen; (f) shape of the display screen; (g) a reference position for the vehicle.
  • Data may comprise data representative of adjustment of at least one of (a) position of the operator in the vehicle; (b) location of an object plane.
  • Data may comprise data representative of at least one of camera parameters, display parameters, operator parameters and object parameters;
  • camera parameters may comprise data representative of at least one of camera position, camera orientation, camera angle, camera field of view, camera type, camera image resolution, camera image size, camera image array size;
  • display parameters may comprise data representative of at least one of display screen position, display screen form, display screen orientation, display screen type, display screen resolution, display screen size, display image array size;
  • operator parameters may comprise data representative of operator position, operator head position, operator eye position, operator field of view, operator seat position;
  • object parameters may comprise data representative of object plane position, object detection, object position detection.
  • Data may comprise at least one of system parameters, operation parameters, adjustment data, correction data.
  • the computing system may comprise an image processing system.
  • Data sources may be configured to provide data; data sources may comprise at least one of the camera system; a vehicle system; a driver position detection system; an object position detector; a network.
  • the display screen may comprise a form; processing of the camera image into the display image may comprise correction of the camera image into the display image based on data representative of the form of the display screen.
  • the form of the display screen may comprise a three-dimensional shape.
  • the present invention relates to a method of operating a display system for a vehicle configured to be operated from a field of view of an operator in a position in a vehicle interior comprising a structure in a position in the vehicle interior presenting an at least partial obstruction to the field of view of the operator and with a display screen having a form on the structure in the field of view of the operator configured to use data from data sources including a camera image from outside of the vehicle from a camera system connected to a computing system configured to process data representative of the camera image into a display image on the display screen.
  • the method may comprise the steps of (a) obtaining data representative of the form of the display screen on the structure; (b) obtaining data from data sources including the camera image from the camera system; (c) processing of the camera image into the display image; (d) displaying the display image on the display screen.
  • Processing of the camera image into the display image may comprise transforming the camera image so that the display image is at least partially representative of the field of view of the operator in the absence of the obstruction from the structure.
  • Processing of the camera image into the display image may comprise at least one of (1) remapping the image; (2) adjusting the image; (3) correcting the image.
  • Processing of the camera image into the display image may comprise using data representative of the form of the display screen for transforming the camera image into the display image for the display screen.
  • Data representative of the form of the display screen may comprise data from a data source.
  • the form of the display screen may comprise a three- dimensional shape on the structure in the vehicle.
  • the display screen may comprise a flexible panel configured to be installed in the form on
  • Data used for processing of the camera image to the display image may comprise data representative of (a) an image in a field of view of the camera system; (b) a relative distance between position of the camera of the camera system and position of the operator in the vehicle; (c) position of the operator in the vehicle; (d) position of the display screen in the vehicle; (e) a vehicle reference point.
  • Processing of the camera image into the display image may comprise use of data representative of at least one of camera parameters, display parameters, operator parameters and object parameters.
  • Display parameters may comprise data representative of at least one of display screen position, display screen form, display screen orientation, display screen type, display screen resolution, display screen size, display image array size.
  • the method may comprise the step of determining a transformation for transforming the camera image to the display image; the transformation may be used for processing of the camera image into the display image.
  • the transformation may comprise transforming a data array from the camera image into a data array for the display image based on data representative of the form of the display screen.
  • the transformation may comprise at least one of (a) remapping of a data array from the camera image into a data array for the display image; (b) correcting the camera image into the display image.
  • the transformation may comprise transforming a data array from the camera image into a data array for the display image based on data representative of (a) the form of the display screen; (b) position of the display screen; (c) position of the operator; (d) position of the camera.
  • the transformation may comprise use of data representative of at least one of camera parameters, display parameters, operator parameters and object parameters.
  • the transformation may comprise use of display parameters; display parameters may comprise data representative of at least one of display screen position, display screen form, display screen orientation, display screen type, display screen resolution, display screen size, display image array size.
  • the vehicle may comprise at least one vehicle system; and data sources may comprise at least one vehicle system.
  • the method may comprise the step of monitoring data sources;
  • monitoring data sources may comprise monitoring of at least one of (a) camera parameters, (b) display parameters, (c) operator parameters, (d) object parameters, (e) adjustments; adjustments may comprise at least one of system adjustments and operation adjustments.
  • the method may comprise the step of initialization; the step of initialization may comprise at least one (a) initializing the computing system; (b) verification of connectivity to data sources; (c) obtaining data representative of initial settings of parameters for processing of the camera image into the display image comprising at least one of position of the display screen, position of the operator, position of the camera.
  • the operator may comprise a driver of the vehicle.
  • the system and method for image display on vehicle interior panel/component may comprise one or more of the inventive concepts and features shown and described in the FIGURES and specification.
  • FIGURES
  • FIGURE 1 A is a schematic perspective view of a vehicle with systems including a display system according to an exemplary embodiment.
  • FIGURE IB is a schematic partial perspective view of an interior of a vehicle with the display system according to an exemplary embodiment.
  • FIGURE 1C is a schematic partial perspective view of an interior of a vehicle with the display system according to an exemplary embodiment.
  • FIGURE ID is a schematic partial perspective view of an interior of a vehicle in operation according to an exemplary embodiment.
  • FIGURE IE is a schematic partial perspective view of an interior of a vehicle in operation with the display system according to an exemplary embodiment.
  • FIGURE 2A is a schematic partial side view of the display system in use in a vehicle according to an exemplary embodiment.
  • FIGURE 2B is a schematic cut-away side view of the display system installed in a vehicle according to an exemplary embodiment.
  • FIGURE 2C is a schematic partial perspective view of an exterior of a vehicle with the display system installed according to an exemplary embodiment.
  • FIGURE 2D is a schematic cut-away perspective view of the display system installed in a vehicle according to an exemplary embodiment.
  • FIGURES 2E through 2G are schematic perspective views of a display panel of the display system on a structure in a vehicle according to an exemplary embodiment.
  • FIGURES 3A through 3C are schematic views of a display panel according to an exemplary embodiment.
  • FIGURE 4A is a schematic perspective view of a structure in a vehicle interior according to an exemplary embodiment.
  • FIGURE 4B is a schematic perspective view of a display panel installed on a structure in a vehicle interior according to an exemplary embodiment.
  • FIGURES 5 A and 5B are schematic perspective views of a vehicle in operation in an environment according to an exemplary embodiment.
  • FIGURE 6A is a schematic partial perspective view of an interior of a vehicle with the display system in operation according to an exemplary embodiment.
  • FIGURES 6B and 6C are schematic top views of the display system in use in a vehicle according to an exemplary embodiment.
  • FIGURE 7A is a schematic partial perspective view of environment external to a vehicle from the interior of the vehicle according to an exemplary embodiment.
  • FIGURE 7B is a schematic partial perspective view of environment external to a vehicle from the interior of the vehicle according to an exemplary embodiment.
  • FIGURE 7C is a schematic partial perspective view of environment external to a vehicle according to an exemplary embodiment.
  • FIGURES 8A through 8C are schematic partial perspective views of environment external to a vehicle showing a field of view according to an exemplary embodiment.
  • FIGURES 8D and 8E are schematic partial perspective views of an image from a field of view in environment external to a vehicle according to an exemplary embodiment.
  • FIGURE 8F is a schematic partial perspective view of an image from environment external to a vehicle on a structure in the interior of a vehicle with a display system in operation according to an exemplary embodiment.
  • FIGURE 9A is a schematic partial perspective view of an interior of a vehicle with the display system in operation according to an exemplary embodiment.
  • FIGURE 9B is a schematic side view of the display system in use in a vehicle according to an exemplary embodiment.
  • FIGURE 9C is a schematic top view of the display system in use in a vehicle according to an exemplary embodiment.
  • FIGURES 10A and 10B are schematic diagrams of an image for a display panel of the display system in a vehicle according to an exemplary embodiment.
  • FIGURES IOC and 10D schematic partial perspective views of an image on a display panel of a display system in a vehicle according to an exemplary embodiment.
  • FIGURES 11 A and 1 IB are schematic diagrams of transformation of an image for display with the display system according to an exemplary embodiment.
  • FIGURE 11C is a schematic system diagram of transformation of an image for display with the display system according to an exemplary embodiment.
  • FIGURES 12A to 12F are schematic diagrams of a partial view for an operator of a vehicle without a display system according to an exemplary embodiment.
  • FIGURES 13 A to 13F are schematic diagrams of a partial view for an operator of a vehicle without a display system according to an exemplary embodiment.
  • FIGURES 14A to 14F are schematic diagrams of a partial view for an operator of a vehicle with a display system according to an exemplary embodiment.
  • FIGURES 15A and 15B are schematic diagrams of a partial view for an operator of a vehicle with a display system according to an exemplary embodiment.
  • FIGURES 15C and 15D are a schematic perspective views of a vehicle in operation with a display system in an environment according to an exemplary embodiment.
  • FIGURES 16A, 16C and 16E are schematic partial side views of the display system in use in a vehicle according to an exemplary embodiment.
  • FIGURES 16B, 16D and 16F are schematic partial perspective views of an interior of a vehicle in operation with the display system according to an exemplary embodiment.
  • FIGURES 17A and 17B are schematic system block diagrams of implementation of a display system for a vehicle with vehicle systems according to an exemplary embodiment.
  • FIGURES 18A and 18B are schematic system block diagrams of functionality of a display system for a vehicle with vehicle systems according to an exemplary embodiment.
  • FIGURES 19 and 20 are schematic flow diagrams of a system and method of operation of a display system for a vehicle according to an exemplary embodiment.
  • FIGURE 21 is a schematic flow diagram of a system and method of operation of a display system for a vehicle according to an exemplary embodiment.
  • FIGURES 22A to 22H are schematic flow diagrams of a system and method of operation of the display system for a vehicle according to an exemplary embodiment.
  • FIGURES 23A to 23C are schematic representational diagrams of a transformation function for implementation of a display system for a vehicle according to an exemplary embodiment. DESCRIPTION
  • a vehicle V is shown schematically according to an exemplary embodiment providing an interior I; as indicated schematically in FIGURES 1 A-1C, the vehicle may be configured to provide a cockpit for an operator/driver of the vehicle and seating for the driver and other vehicle occupants; the vehicle may provide interior structures such as pillars P (e.g. for the roof of the vehicle interior/passenger compartment) and a windshield W and side windows SW.
  • pillars P e.g. for the roof of the vehicle interior/passenger compartment
  • windshield W and side windows SW e.g. for the roof of the vehicle interior/passenger compartment
  • objects and/or persons PO in the environment ENV external to the vehicle may be visible in the field of view of the operator/driver DR of the vehicle through windshield W and side windows SW to the extent that visibility of the objects/persons (e.g. otherwise in the field of view of the
  • FIGURE ID visibility of a person PO and other objects in the environment ENV outside of the vehicle to the operator/driver DR of the vehicle may be obstructed at least partially by structures such as pillar P as the vehicle travels through the environment ENV (e.g. on road/street, etc. under operation of the driver). See also FIGURES 1 A-1C and 13A-13F (indicating obstruction of pillar P in relation to windshield W and side windows SW).
  • a display system may be provided for the vehicle with at least one display panel DP on a structure shown as pillar P configured to provide at least partial visibility to the
  • a camera system CS of the display system DS provides at least one camera C with a field of view outside of the vehicle (e.g.
  • camera system CS is configured to obtain image/data from the field of view CFOV to include persons/objects PO outside of the vehicle that would be obstructed from the field of view DFOV of the operator DR of the vehicle by a structure such as pillar P in the vehicle interior; image/data from the camera system CS is processed and presented for display on a display panel DP on the structure shown as pillar P. See also FIGURES 6A, 9A, 17A-17B, 18A-18B, 19-21, 22A-22H and 23A-23C.
  • the display panel DP may be positioned on the obstruction/structure shown as pillar P so that the operator/driver DR will have a generally continuous unobstructed view of objects (e.g. such as person/object PO) within the field of view CFOV of the camera system in the environment ENV outside the vehicle but obstructed from the field of view DFOV of the operator DR (e.g. by structure such as pillar P between the windshield W and side windows SW) during operation of the vehicle.
  • objects e.g. such as person/object PO
  • FIGURES ID, 7B and 13A-13F indicating obstructed view. See also FIGURES 1A-1C and 15C-15D.
  • the display system for the vehicle interior may be configured to transform the image from the camera system for display as an image on the display panel so that persons/objects appear to the operator/driver on the image on the display panel (indirectly) generally represented more accurately in the form/size the persons/objects would have appeared to the operator/driver (directly) in an unobstructed view.
  • FIGURES 9A and 10A-10D See also FIGURES 2A, 6B-6C, 9B-9C, 17A-17B and 18A-18B (use of vehicle reference point and data from data sources available for a transformation function/image processing for the mapping/display of an image from the camera system on the display panel).
  • the vehicle V provides a display system DS with camera C of a camera system CS on the structure shown as pillar P of the vehicle V (e.g. on the exterior of the vehicle/pillar); display system DS also provides a display panel DP visible to the operator/driver DR of the vehicle V (e.g.
  • the structure shown as pillar P is an obstruction in the field of view DFOV of the vehicle operator/driver DR; as indicated schematically according to an exemplary embodiment, image/data obtained from the field of view CFOV of the camera system CS corresponding to the field of view DFOV of the vehicle operator/driver DR obstructed by the pillar P is transformed/presented as image/data on the display panel DP on the pillar P for the vehicle operator/driver DR.
  • FIGURES 1 A-1E, 9A- 9C, 10A-10D, 11A-11C, 12A-12F, 13A-13F, 14A-14F, 17A-17B and 18A-18B (schematic diagrams indicating functionality/features of display system and system block diagram of implementation according to exemplary embodiment).
  • the system and method is configured to obtain a camera image from a remote/separate camera and transform/map the camera image as a display image on a display panel to provide a more accurate representation of size/scale of objects for the operator/driver viewpoint (e.g. to produce a simulated "invisible" pillar/structure effect for the vehicle operator/driver).
  • FIGURES IE, 6A, 9A, 14A-14F and 16A-16F (indicating schematically a dynamic "invisible pillar" effect generally provided by the display system and method).
  • the display system may be configured (e.g. with input/output, data/processing capability, etc.), so that the image presented on the display system for the operator/driver (e.g. with transformation, remapping, correction based on data, settings, adjustments, etc.) will be generally continuously updated (e.g. in real time) based on an image obtained from the camera system that is generally continuously updated (e.g. in real time) as the vehicle is in use/operation (e.g. in the environment in real time). See also FIGURES 5A-5B, 8A-8G, 16A-16F, 17A-17B, 18A-18B, 22A-22H and 23A-23C.
  • the display system DS may be configured for installation and use in a vehicle V to be operated by an operator/driver DR.
  • the display system DS may comprise systems and components such as a camera system CS with at least one camera C and a driver position detection device shown as eye position tracker ET and a display screen shown as display panel DP configured for installation/integration in the interior of the vehicle with a structure shown as pillar P; as indicated schematically according to an exemplary embodiment, the display system DS may comprise a computing system (e.g.
  • the position of the vehicle operator/driver with respect to a vehicle reference point VC may be configured to be fixed or adjusted/positioned for the display system (e.g. set, programmed, etc.) at manufacture (e.g. at the component/vehicle factory), at installation (e.g. for calibration/validation), during operation (e.g.
  • a vehicle operator/driver for maintenance/customization (e.g. at a service center), based on a control program (e.g. varied using an algorithm, data sources, etc.), using data/information from vehicle systems (e.g. driver seat position, etc.), using data/information from a sensor/detector device such as an position detector/eye tracker, etc. See FIGURES 2 A, 6B-6C, 9B-9C, 16A-16F, 17A-17B and 18A-18B (position of vehicle operator/driver with respect to vehicle reference point may comprise data in a data file/data source available for use in a transformation function/image processing for the mapping/display of an image from the camera system on the display panel).
  • systems and components for the display system DS such as a camera system CS with at least one camera C and sensor/detector devices SD (e.g. configured to function as data sources such as object distance detectors, etc.) may be configured for installation/integration at the exterior of the vehicle with a structure shown as pillar P; as indicated schematically according to an exemplary embodiment, the at least one camera C of the camera system CS and sensor/detector devices SD (e.g. object distance detectors, etc.) may be positioned at various/multiple locations at and on the vehicle (e.g.
  • FIGURES 15C-15D e.g. schematic representation of vehicle indicating various potential locations for camera C, sensor/detector devices SD, etc. as data sources for the display system.
  • FIGURES 2A-2D and 15C-15D the camera system with camera and
  • sensor/detector devices if any of the display system may be configured to be fixed or adjusted/positioned for use at manufacture (e.g. at the component/vehicle factory), at installation (e.g. for calibration/validation), during operation (e.g. by a vehicle operator/driver), for maintenance/customization (e.g. at a service center), etc.
  • a display screen shown as display panel DP may be provided and configured in a variety of forms/orientations for installation/integration in the interior of the vehicle with a structure shown as pillar P; as shown schematically in FIGURE 2E, the display panel DP may comprise a display screen provided in a generally flat form mounted on the structure; as shown
  • the display panel DP may comprise a flexible display screen provided in a generally horizontal orientation on/around the structure; as shown schematically in FIGURE 2G, the display panel DP may comprise a flexible display screen provided in a rotated orientation on and around the structure (with adjacent driver position detector ET).
  • a display panel DP for the display system may comprise a display screen/array configured to operate with a computing/control system to display an image (e.g. addressable as an x, y array of display elements pixels); as shown schematically according to an exemplary embodiment, the display panel DP may comprise a flat/flexible screen configured to be oriented/formed and mounted/shaped on the structure shown as pillar P. See also FIGURES 2A-2B and 2E-2G (installed display panel positioned and/or formed on structure).
  • the flat two- dimensional display panel DP (e.g. addressable as an x, y array of display elements/pixels) when conformed for installation on pillar P provides a three-dimensional form (X, Y, Z position) generally conforming to the shape of pillar P (with each x, y pixel of the display array located at a corresponding X, Y, Z position with reference to a vehicle reference point VC).
  • position of display panel/structure with respect to vehicle reference point and form/shape of display panel on the structure may comprise data in a data file/data source available for use in a transformation function/image processing for the mapping/display of an image from the camera system on the display panel).
  • the vehicle V operates in an environment ENV; as shown schematically, the environment ENV outside of the vehicle V may comprise objects such as terrain, trees/plants, roadways/pathways, etc.; according to an exemplary embodiment shown schematically in FIGURES 5A-5B, display system DS may be configured to establish or determine a focal/object plane FP at a distance from the vehicle (e.g. from vehicle reference point VC). See also
  • FIGURES 2A, 6B-6C, 9B-9C and 15A-15D are identical to FIGURES 2A, 6B-6C, 9B-9C and 15A-15D.
  • a focal/object plane FP may be established (e.g. in the environment ENV) at a distance/position where a person/object PO is located or is expected to be or appear (e.g. with respect to vehicle reference point VC). See also FIGURES 15A-15D.
  • the position of the focal/object plane FP with respect to the vehicle reference point VC may be configured to be fixed or adjusted/positioned for the display system (e.g. set, programmed, etc.) at manufacture (e.g. at the component/vehicle factory), at installation (e.g.
  • FIGURES 5 A and 6C Compare FIGURES 5 A and 6C with FIGURES 5B and 6B (indicating variable location of focal/object plane based on location of person/object of interest, e.g. from data sources, sensor/detector, manual adjustment, etc.).
  • the display system may use the distance of the focal/object plane FP as a reference point (e.g. relative to the vehicle reference point VC) in a transformation function to transform the image from the camera system (e.g. in camera field of view CFOV) for display as an image on the display panel DP so that persons/objects (e.g. shown as person PO) appear to the operator/driver DR on the image on the display panel (indirectly) generally represented more accurately in the form/size the persons/objects would have appeared to the operator/driver (directly) in an unobstructed view (e.g.
  • a reference point e.g. relative to the vehicle reference point VC
  • CFOV camera field of view
  • the transformation function applied on an image from the camera system by the display system may be configured so that accuracy in representation in the transformed image for the display panel of the form/size of persons/objects at the focal/object plane will be enhanced (with relative accuracy generally being reduced in the transformed image of objects located apart/further from the focal/object plane).
  • FIGURES 2A, 6B-6C, 9B-9C, 17A-17B and 8A-18B position of focal/object plane with respect to vehicle reference point may comprise data available for use in a transformation function/image processing for the mapping/display of an image from the camera system on the display panel).
  • an object such as a person PO
  • an object obtained in an image/data in the field of view CFOV of a camera C of the camera system CS of the display system DS that otherwise would be obstructed from visibility in the field of view DFOV of the operator/driver DR of the vehicle may be transformed and presented in an image/data on the display panel DP in the vehicle interior I to the operator/driver DR of the vehicle in a consistent scale/size/form with other objects visible such as through the windshield W and side windows SW of the vehicle.
  • FIGURES 12A-12F (obstructed objects indicated in broken lines) and FIGURES 13A-13F (obstructed view indicated) and FIGURE IOC (inconsistent scale image indicated) with FIGURES 14A-14D (unobstructed objects visible generally continuously from windshield across display panel on pillar to side window) and FIGURES 10D (consistent/corrected scale image).
  • FIGURES 19 20, 21, 22 A- 22H and 23A-23C (e.g. method of operation/functionality of the display system using parameter values and data sources according to an exemplary embodiment).
  • the operator/driver DR of the vehicle would have in the absence of obstructions an unobstructed field of view of the environment ENV including various objects external to the vehicle such as person PO; as indicated schematically in FIGURE 7B, the interior of the vehicle contains structures such as pillar P that become an obstruction to the field of view of the operator/driver DR and obstruct at least partially the visibility the environment ENV such as of person PO. See also FIGURES 2A, 5A-5B, 6B-6C and 9B-9C (indicating field of view of driver/operator DFOV).
  • the camera system from a position at the exterior of the vehicle may be configured to capture from a field of view an image FMC of the environment ENV that may include the field of view of the operator/driver that is obstructed by the structure/pillar. See FIGURES 2A, 6B-6C and 9B-9C (indicating field of view of camera system CFOV for the display system).
  • the image IMC captured in the field of view of the camera of the camera system in the environment ENV is not identical in representation (e.g.
  • the camera of the camera system and the eye position of the operator/driver of the vehicle and the object shown as person PO in the environment are each at a different location/position (e.g. relative to the reference point of the vehicle VC).
  • the image IMC from the camera of the camera system indicates that person PO is relatively closer to the camera (see FIGURE 7C) and relatively more distant to the operator/driver of the vehicle (see FIGURE 7A); person PO in the environment ENV is indicated relatively in a larger size/form in the image IMC from the camera system and relatively in a smaller size/form in the view of the driver/operator DR. See also FIGURES 2A, 5A-5B, 6B-6C and 9B-9C. [0076] Referring to FIGURES 8A-8F, the system/method of image processing/transformation function of the display system for images obtained by the camera system from the environment ENV for display on the display panel DP is shown schematically.
  • the environment ENV for operation of the vehicle may comprises objects such as person PO.
  • the camera system of the display system is configured to obtain an image IMC (indicated as of person PO at focal/object plane FP); as indicated schematically in FIGURES 8D-8F, the display system is configured to transform/remap data/image FMC from the camera system (e.g. using a
  • the display system DS may be configured to transform the image FMC from the field of view CFOV of the camera system CS for display as an image IMD/IMDC on the display panel DP so that objects in the environment ENV such as person PO that would otherwise have been obstructed from view (e.g.
  • the vehicle operator/driver DR appear in the field of view DFOV of the vehicle operator/driver DR as an image IMD on the display panel DP; by use of an image transformation/remapping function objects from the environment ENV at the focal/object plane FP such as person PO from the camera field of view CFOV are presented as an image IMD on the display panel DP for the operator/driver DR (e.g. indirectly transformed from an image FMC captured by the camera system) in generally a more accurate form/size to represent as the objects would have appeared in the field of view DFOV of the operator/driver DR in the absence of an obstruction (e.g.
  • FIGURES 7A-7C, 10A-10D and 15A-15B See also FIGURES 2A, 6B-6C, 9B-9C, 17A-17B and 18A-18B (use of vehicle reference point VC and data from data sources available for a transformation function/image processing for the mapping/display of an image from the camera system on the display panel).
  • the display system comprising a camera system CS and a display screen shown as display panel DP on a structure shown as pillar P in a vehicle interior may be configured (1) to obtain an image IMC from the field of view CFOV of a camera system CS from the environment ENV of the vehicle including objects such as person PO; and (2) to transform/convert to the image IMC from the camera system to an image IMDC for presentation on the display panel DP in the vehicle interior; and (3) to display the image IMDC of the environment ENV that includes objects such as person PO that are obstructed from the field of view DFOV of the operator/driver DR of the vehicle by the pillar P in the vehicle interior; the display system may be configured to (1) obtain data from data sources (e.g. such as component positions, settings, adjustments, parameters, values, constants, etc.) and (2) use a data sources (e.g. such as component positions, settings, adjustments, parameters, values, constants, etc.) and (2) use a data sources (e.g. such as component positions, settings,
  • transformation/mapping function e.g. geometric transformation, scaling, fitting, etc.
  • transform/map e.g. remap, correct, etc.
  • the camera C of the camera system CS e.g. mounted and directed externally from a vehicle structure such as pillar P
  • the operator/driver DR of the vehicle e.g. seated inside the vehicle
  • the focal/object plane FP e.g. from the person/object PO at the distance of the focal/object plane FP
  • vehicle reference point VC in the absence of transformation/correction (e.g. correction for object
  • an image FMDU on display panel DP would be uncorrected and at least partially incorrect in scale/form to what the operator/driver DR of the vehicle V would perceive in the absence of the obstruction.
  • FIGURE IOC an image FMDU on display panel DP would be uncorrected and at least partially incorrect in scale/form to what the operator/driver DR of the vehicle V would perceive in the absence of the obstruction.
  • the display system may transform/remap (e.g. map, correct, adjust, etc.) an image FMC from the environment ENV including an object shown as person PO from the camera system CS (see FIGURE 10A) using data from data sources into an image IMD/IMDC for display panel DP (see FIGURE 10D) that is generally correct in scale/form to what the operator/driver DR would perceive through the windshield W and/or side window SW of the vehicle (see FIGURES IB and IE). See also FIGURES 23A-23C.
  • transform/remap e.g. map, correct, adjust, etc.
  • FIGURES 11 A-l 1C a data/array of image IMC (e.g. arrangement of pixels PX of an image/file) from field of view CFOV of the camera system (e.g. from environment external to the vehicle) is provided to the image processing system for transformation/mapping into a data/array of remapped image IMDC (e.g. transformed/corrected arrangement of pixels PX) for presentation on the display screen shown as curved/flexible screen display panel DP on the pillar P in the vehicle interior.
  • image IMC e.g. arrangement of pixels PX of an image/file
  • CFOV of the camera system e.g. from environment external to the vehicle
  • a data/array of remapped image IMDC e.g. transformed/corrected arrangement of pixels PX
  • FIGURES 5A-5B and 9A-9C (indicating the person/object PO at the focal/object plane FP of the display system).
  • the camera image IMC is as a data array (e.g. of pixels PX arranged in an x,y array as data from the camera) transformed into a data array for the display image EVIDC for presentation on the display panel (e.g. converted/remapped to the proper number and arrangement of pixels PX in an x,y array as data for addressing the display panel);
  • the remapping/transformation function may be configured for image processing of a form of image from the camera system (e.g. number of pixels, size, resolution, aspect, orientation,
  • FIGURE 1 IB indicating transformation of a rotated camera image IMCR to a corrected/remapped image IMDC for display panel.
  • FIGURES 15 A the image from the camera with the object PO that is obstructed by the structure is transformed/presented as a corrected image IMDC on the display panel DP on the structure with the object generally representative in scale and form to simulate/duplicate the unobstructed view of the operator/driver of the object through the windshield W and side windows SW if not obstructed by the structure.
  • FIGURES 12A-12F, 13A-13F, 14A-14F See also FIGURE 15B (presentation of an uncorrected image IMDU on the display panel).
  • the form/format of the camera image IMC from the camera system used by the display system for image transformation/correction may be different (e.g. in size, resolution, type, orientation, etc.) than the format used for the display panel DP to display the display image IMDC (e.g.
  • the camera image IMC may have a different pixel resolution/size than the compatible display image FMDC to be remapped such as by using a remapping routine provided with a transformation/mapping function as indicated in FIGURES 23A-23C); as indicated schematically in FIGURES 8C-8F and 11C, the image processing system may be configured to convert/remap the camera image IMC into a corrected display image EVIDC (e.g. with adjustment for form, size, resolution, type, orientation, etc.) for compatible presentation on the display panel DP. See e.g. FIGURE 11 A (e.g. mapping of camera image of a pixel resolution/format to a display image of a different pixel
  • the display system (with image processing system) is configured/implemented in operation of the vehicle to use data from data sources with the image processing system (e.g. using computation methods/algorithm and/or a look-up table/data map configured for the system from a database) to transform the image/data from the camera system to a generally accurate representative corrected/transformed image/data presented on the display panel.
  • data sources e.g. using computation methods/algorithm and/or a look-up table/data map configured for the system from a database
  • the display system (with computing system, operating program, input/output devices, etc.) is configured to perform the image capture/transformation/display functions in a generally continuous operation during use of the vehicle (e.g. in real-time operation with efficient undelayed processing and data/process flow).
  • FIGURES 12A-12F and 13A-13F in operation of a conventional vehicle from time to time objects external to the vehicle such as person PO may be wholly or partially obstructed from view to the operator/driver of the vehicle by a structure such as pillar P.
  • a person/object combination PO shown as a child playing with a ball near the vehicle may enter the field of view of the driver briefly and intermittently be obstructed partially (FIGURES 13A, 13C and 13F) and obstructed wholly (FIGURES 13B and 13E) (e.g.
  • FIGURES ID, 7B and 6A-6C and 9A-9C indicating structure/pillar as "blind spot” to field of view DFOV of vehicle operator.
  • the vehicle operator may be deprived of a continuous unobstructed field of view as the vehicle is in operation as a person/object is obstructed by structures of the vehicle (such as a pillar between windshield and side window) notwithstanding the proximity of the person/object PO to the vehicle.
  • FIGURES 12A-12F showing obstructed objects in broken lines on pillar/structure
  • FIGURES 13A-13F obstructed objects behind pillar
  • the display system DS for the vehicle V is configured to facilitate the operator/driver DR of the vehicle by use of camera C (e.g. of camera system) the observation of persons/objects PO outside of the vehicle. See also FIGURES 13A-13F and 14A-14F.
  • the operator/driver DR of the vehicle has a field of view DFOV; the field of view DFOV of the vehicle operator/driver DR may be obstructed by pillar P as indicated in FIGURES 13A-13F.
  • the display system DP provides by camera/camera system C/CS located on the exterior of the vehicle (e.g. on the exterior of pillar P) a field of view CFOV; the field of view CFOV of the camera/camera system C/CS is not obstructed by structure/pillar P as indicated in FIGURES 14A-14F (e.g.
  • FIGURES 23A-23C e.g. mapping/remapping function for image transformation.
  • a person/object combination PO shown as a child playing with a ball near the vehicle may enter the field of view of the driver from the windshield or side window and then will remain in the field of view of the camera and be partially displayed on the display panel (FIGURES 14A, 14C and 14F) and wholly displayed on the display panel (FIGURES 14B and 14E) (e.g.
  • the vehicle operator/driver is provided a continuous generally unobstructed field of view as the vehicle is in operation during the time of proximity of the person/object PO to the vehicle; the display system and/or other vehicle systems may also be configured to provide supplemental feedback to the operator/driver (e.g.
  • FIGURES 15A and 15C-15D indicating outside view of implementation of display system on vehicle V with camera system CS and detector/sensor system SD and with person/object PO in proximity).
  • the distance of the person/object PO to the camera C may vary during operation of the vehicle in the environment ENV; persons/objects may enter and exit the field of view as the vehicle is in operation/transit in the environment ENV; as indicated schematically, adjustment may be made to the location of the focal/object plane FP for the display system (e.g. distance/position relative to the vehicle reference point RC).
  • FIGURES 6A-6C and 9A-9C indicating expanded/unobstructed field of view of camera CFOV and with object at focal/object plane FP in image on display panel DP on structure/pillar P that otherwise would obstruct field of view DFOV of vehicle operator/driver DR). See also FIGURES 19, 20, 21, 22A-22H and 23A-23C (system and method for image transformation based on data from data sources such as position of focal/object plane, etc.).
  • the display system may be configured to use data from data sources (such as detector/sensor devices C/SD and SD shown in FIGURES 15C-15D) for the image
  • the display system may use data from a detector/sensor SD to determine the distance of an object outside of the vehicle relative to the camera (and/or relative to the vehicle operator/driver) and use the data representative of relative
  • adjustment may be made to the location of the focal/object plane FP for the display system (e.g. distance/position relative to the vehicle reference point RC). See FIGURES 2A, 5A-5B, 19, 20, 21, 22A-22H and 23A-23C (e.g. data as to distance can be used with other data to improve scale/scope and proportion/dimension of objects appearing as an image on the display panel).
  • the vehicle provides a vehicle reference point VC at a generally fixed position; the camera/camera system C/CS is in a generally fixed position (e.g. on pillar P of vehicle); the display panel DP is in a generally fixed position (e.g. on pillar of vehicle); the operator/driver DR of the vehicle is in a generally consistent position (e.g. in driver seat subject to adjustment and movement); the
  • selected/established focal/object plane FP for the display system may be set at a specific distance from the vehicle reference point VC (e.g. 5 to 15 meters) according to an exemplary embodiment (e.g. subject to adjustment or modification such as manually, by adaptive control, based on object distance detection, based on traffic pattern/driving environment, etc.); as indicated, values of parameters such as distances/positions of components and features may be used in the transformation/correction function of the image processing system of the display system. See FIGURES 17A-17B, 18A-18B and 23A-23C. As indicated schematically according to an exemplary embodiment in FIGURES 16A-16F, variation of the relative position of the vehicle operator/driver (e.g.
  • the head/eye position of driver will alter the relative position of objects such as person PO in the environment ENV with respect to obstruction such a structure/pillar P (e.g. partial or entire visibility and/or partial or entire obstruction of objects); variation of the relative position of the vehicle operator/driver will in operation of the transformation/mapping function of the image processing system alter the accurate representation (e.g. size, scale, position, content, etc.) of the image on the display panel.
  • the operator/driver DR is in a generally stable intermediate position as to be comfortably seated for driving under standard conditions; the field of view CFOV of the camera
  • the parameters of the display system may be altered along with the corresponding transformation/correction function of the image processing system; for example, if the operator/driver moves forward as shown schematically in FIGURES 16A-16B, the relative position of the operator/driver DR to the display panel DP and camera system is altered and the corresponding transformation/correction function of the image processing system is altered (e.g.
  • the display system may comprise connectivity with a sensor/detector for driver position (shown as eye tracking system ET). See also FIGURE 2G (showing schematically location of eye tracking sensor/system ET on pillar P).
  • the sensor/detector for driver position will provide data representative of the position of the driver (e.g. the eyes of the driver, head of the driver, body of the driver, seat of the driver, etc.) to the display system; such data from sensor ET is available for use by the image processing system for data/image from the camera to enhance the scale/accuracy of the image displayed by the display system on the display panel.
  • FIGURES 10A-10B, 16A-16F, 21 and 23A-23C See e.g. FIGURES 10A-10B, 16A-16F, 21 and 23A-23C.
  • the eye tracking system ET is configured to provide data as to changes in driver position to the display system (e.g. for the image processing/transformation system). See FIGURES 11C, 17A-17B and 18A- 18B. See also FIGURES 21 and 23A-23C
  • the display system may be initialized/calibrated for operation (e.g. including relative spatial position of the vehicle operator to camera system/array) with program data, data from data sources (e.g.
  • the operator of the vehicle may be provided with input/controls to facilitate adjustment of the display system (e.g. a control such as a joystick or as used in a conventional system to adjust vehicle systems such as exterior side mirrors, etc.); when adjustment of the display system/image has been completed for the vehicle operator, the vehicle operator may set and/or save and store adjustment settings for present and future use of the display system/vehicle.
  • a control such as a joystick or as used in a conventional system to adjust vehicle systems such as exterior side mirrors, etc.
  • the vehicle operator may set and/or save and store adjustment settings for present and future use of the display system/vehicle.
  • the display system is configured using image processing and data (such as driver position) from data sources to provide on the display panel for the vehicle operator DR an image that is generally continuous and consistent in scope/scale with objects otherwise visible through the windshield and side windows. See e.g. FIGURES 2C, 14A-14F and 16A-16F.
  • the display system DS may be connected to vehicle systems, instrumentation, sensors/detectors, data sources, etc. (see FIGURES 17A-17B and 18A-18B.
  • the display system DS with camera/camera system C/CS and display panel P may be installed in a structure shown as pillar P in the vehicle/vehicle interior. See also FIGURES 1 A-IB and 1C and 2C-2D (showing schematically components of display system DS).
  • the display panel DP may be integrated with a vehicle trim component as indicated on pillar P (e.g. front pillars and rear pillars) and provided with a non-planar/curved form (e.g. flexible screen and/or panel conforming generally to the shape/form of the vehicle trim/pillar P). See also FIGURES 2A-2G.
  • the system may comprise a computing system (e.g. providing an image processing system) and a camera system and display (e.g. display panel and display form/shape and orientation) configured with an operating program and program data/storage;
  • the display system may be configured to use vehicle system data and initial data input settings (e.g. input from operator/driver, default settings/values, adjustments, etc.) and data sources (such as detector/sensors, internal/external networks, the internet, etc.). See also FIGURES 2A-2D, 11C, 15C-15D, 18A-18B and 23A-23C.
  • the components and systems/subsystems may be connected with a computing system (e.g. with data sources/storage and/or network, sensors/detectors, etc.) configured to provide processing capability for the display system (e.g. connection of implementation of standard/conventional computing/processing capability such using a personal computer system with standard operating programs and device connectivity such as USB camera, display controller, input/output and communications, etc.);
  • a computing system e.g. with data sources/storage and/or network, sensors/detectors, etc.
  • processing capability for the display system e.g. connection of implementation of standard/conventional computing/processing capability such using a personal computer system with standard operating programs and device connectivity such as USB camera, display controller, input/output and communications, etc.
  • the display system may be implemented with a special purpose system (e.g.
  • the display system may be implemented as a generally stand-alone system and/or with use (e.g. integration and/or interconnectivity) of vehicle systems and subsystems (including computing systems, data sources/storage, networking, cameras, detectors and sensors, settings/controls, display devices, etc.). See FIGURES 2A-2D, 11C, 15C-15D and 17A-17B.
  • FIGURES 18A and 18B a functional block diagram for the display system is shown schematically according to an exemplary embodiment; the system may comprise an operating program (with implementation of a transformation function, etc.), camera system, display screen, display screen form (e.g. pillar/structure form), camera position data, driver position data, focal/object plane distance, adjustments such as correction/modification of data, settings and parameters. See also FIGURES 2A-2D, 3A-3C, 4B, 6B-6C, 8C-8F, 9A, 11 A-11C, 15A-15D, 17A-17B, 22A-22H and 23A-23C.
  • functional systems/subsystems may be configured (e.g. implemented, programmed, etc.) to perform (e.g. with a computing system, data from data sources/storage and/or network, sensors/detectors, etc.) the functions of data management (e.g. settings, adjustments, acquisition, storage, etc.), image processing (e.g. obtaining, transformation, correction, mapping, displaying, etc.), detection (e.g. object detection, etc.), computation (e.g. determination of mapping, correction, etc.), monitoring, communications, etc. to implement the display system. See also FIGURES IE and 6A-6C.
  • the display panel DP is provided on the structure shown as pillar P at a position between the operator/driver DR and the camera C of the camera system CS; as indicated schematically in FIGURES 3A-3C, 4B, 8C-8F and 11 A-l 1C, the display panel DP comprises a two-dimensional array of display elements (e.g. pixels) that may be formed into a three-dimensional form (e.g.
  • the display panel DP comprises a position where a portion of the field of view DFOV of the operator/driver is obstructed and a position where an image from the field of view CFOV of the camera system will be presented for display for the operator/driver DR.
  • the camera of the camera system obtains an image comprising data (e.g. display elements/pixels in an x,y array) from the environment to be used for display on the display panel; as indicated
  • the image from the camera system (in an x,y array of a determined size from the specific device/settings) is transformed/corrected into the image for the display panel (in an x,y array of a determined size for the specific device/settings) (e.g. image from camera system may be corrected, mapped, etc. including to resolve variations/differences in aspect/ration, resolution, pixel arrangement, size, format, array dimensions, etc. with the display panel, for example remapping as to fit a 1920x1080 image from the camera to a 1080x720 image for the display panel).
  • the form and position of the display panel may be used for determining the
  • the position (e.g. in an ⁇ , ⁇ , ⁇ reference) of the three-dimensional shape of the display panel corresponding to each display element (e.g. pixel in an x,y array) of the two-dimensional array of the display panel can be determined (see FIGURES 4B and 9A); the position (e.g. in an ⁇ , ⁇ , ⁇ reference) of the vehicle operator/driver can be determined.
  • a projection from the position of the operator/driver through each display element (e.g. on the x,y array) of the display panel and corresponding position e.g.
  • the focal/object plane e.g. a system parameter or operating parameter
  • the focal/object plane can be used to determine the portion of the image from the field of view CFOV of the camera (e.g. in an x,y array) for transformation to the image for display on the display panel (see also FIGURES 23A-23C);
  • the position (e.g. in an ⁇ , ⁇ , ⁇ reference) of the camera of the camera system can be used with the position of the display panel (e.g. pillar/structure of obstruction) and the position of the operator/driver (e.g. driver head/eye position) to determine the transformation/correction function for image processing (e.g.
  • FIGURES 23A-23C using algorithm/formula, computation as indicated in FIGURES 23A-23C); application of the transformation/correction function to the portion of the camera image (e.g. an x,y data array/file) produces a corrected/remapped display image for presentation at the display panel (e.g. an x,y data array/file) (see FIGURES 18A-18B).
  • the display system is configured to apply a transformation function to the camera image EVIC to provide a corrected display image IMDC for the display panel DP to represent objects from the environment at the focal/object plane more accurately in size/scale to the viewpoint of the operator/driver (e.g. a display image in the proper/compatible x,y array/file for the display device and with the proper/compatible portion of content from the camera image to provide a more accurate representation of the environment for the viewpoint of the operator/driver). See also FIGURES 21 and 22A-22H (e.g. display system in operation may provide for adjustments of settings such as camera resolution and display device resolution and parameters such as the focal/object plane position, operator/driver position, etc. for the image transformation function).
  • a transformation function to the camera image EVIC to provide a corrected display image IMDC for the display panel DP to represent objects from the environment at the focal/object plane more accurately in size/scale to the viewpoint of the operator/driver (e.g. a display image in the proper/compatible x,y array/
  • the operation of the system and method for the vehicle/operation e.g. functions after installation and initial calibration of the system in the vehicle
  • flow diagrams e.g. indicating initial setup/initialization of the system followed by steps comprising continuing operation of the system during use of the vehicle
  • the system may comprise the steps of initialization (e.g. system, computing system, camera system, display system clearance, driver initialization), setting the mapping/transformation function, recording the image at the camera system (e.g.
  • the method may comprise the steps of determining parameters (e.g. system, vehicle, distance to focal/object plane, display screen form, driver position, etc.), initialization (e.g. of system/settings), initial data from data source, data/settings verification, determination of the image transformation function, obtaining the image from the camera system, transformation/correction of the image, presentation of the corrected image on the display screen, monitoring of data (e.g. data sources, settings/adjustments).
  • parameters e.g. system, vehicle, distance to focal/object plane, display screen form, driver position, etc.
  • initialization e.g. of system/settings
  • initial data from data source e.g. of initial data from data source
  • data/settings verification e.g. of system/settings
  • determination of the image transformation function e.g. of the image from the camera system, transformation/correction of the image, presentation of the corrected image on the display screen, monitoring of data (e.g. data sources, settings/adjustments).
  • the method of the display system may generally comprise a set of steps (e.g. after initialization/setup at start with combinations of operating steps configured to continue in a process loop during operation of the vehicle) including initialization/setup of the system and parameters, determination of the image transformation function (using data from data sources), obtaining an image from the camera system, image transformation (using the transformation function), display of image on the display panel (see FIGURES 6A-9C and 22A); the method may include the stems of obtaining data from data sources, adjustments and corrections (see FIGURES 11C and 22B); the method may generally comprise the step of determination of the focal/object plane distance/position (see FIGURES 5A-5B and 22C); the method generally may comprise the step of determination of the display screen form (see FIGURES 3A-3C, 4A-4B and 22D); the method generally may comprise adjustment/correction (e.g.
  • the method generally may comprise a combination of steps including determination of the focal/object plane distance/position and adjustments/corrections to system parameters (see FIGURE 9A-9C and 22F).
  • the step of monitoring of data sources and settings may comprise driver position tracking (e.g. from data sources) and object monitoring/detection (e.g. using data sources such as position detection to adjust the focal/object plane); adjustments/corrections are used to modify parameters for the display system including the image transformation function.
  • system/device settings e.g.
  • camera type/resolution, camera position as installed, display screen type/resolution, etc. may be provided for initialization; data sources may be connected/initialized (e.g. with initial values set or obtained); initial parameters may be established (e.g. default/preset values for focal/object plane distance, driver position, etc.); determination may be made of parameters for display screen mapping (e.g. data file comprising three-dimensional form/position); monitoring data sources/settings may comprise monitoring of operation settings (e.g. manual controls from operator such as seat position, calibration, etc.) and operation/parameter adjustments (e.g. driver position tracking, object detection, etc.) during operation for making adjustments/corrections for image transformation/display.
  • operation settings e.g. manual controls from operator such as seat position, calibration, etc.
  • operation/parameter adjustments e.g. driver position tracking, object detection, etc.
  • FIGURES 23 A and 23B the system and method of image processing (e.g. image mapping/remapping) is indicated schematically according to an exemplary embodiment. See also FIGURES IE, 2A-2G, 3A-3B, 6A-6C, 7A-7C, 8A-8F, 9A-9C, 10A-10D, 11A-11C, and 15A-15B, 17A-17B, 18A-18B and 22A-22H.
  • the system/method is configured to obtain a camera image in a field of view DFOV of the vehicle operator/driver DR that is obstructed (e.g.
  • the pillar/structure with display panel presents a display image on the display panel DP intended to represent the field of view that the vehicle operator/driver DR would have (e.g. at least partially at the position of the display panel/obstruction of the pillar/structure) in the absence of the obstruction. See generally
  • the system and method is configured to adapt for the relative distances (and angular/field of view alignment) between the
  • operator/driver DR e.g. seated in the vehicle
  • display panel DP e.g. mounted on the interior of the pillar/structure
  • focal/object plane e.g. location of a detected object, object distance, distance where objects would be anticipated, distance from vehicle reference point, etc.
  • operator/driver e.g. seated in the vehicle
  • focal/object plane e.g. location of a detected object, object distance, distance where objects would be anticipated, distance from vehicle reference point, etc.
  • the image transformation/remapping function calculates a parameter of eye angle (horizontal and vertical) representative of the field of view DFOV of the operator/driver (e.g.
  • a scaling factor parameter labeled horizon based on the distance to the focal/object plane
  • a scaling factor parameter labeled horizon based on the distance to the focal/object plane
  • a parameter of camera angle horizontal and vertical representative of the image from the field of view of the camera system to mapped into the display image
  • the size of an object in the display image will be matched to what the operator/driver would expect to see at the in the absence of an obstruction, namely an object represented in a reduced size in the display image from the camera image to reflect the relative distance between the camera and the eye/head position of the operator/driver). See also
  • FIGURES 11A-11C and 14A-14F are identical to FIGURES 11A-11C and 14A-14F.
  • the transformation/remapping function for image processing may use parameters (including size/resolution and form/shape of the display panel and calculated parameters as indicated in FIGURES 23A-23C) and data from data sources to determine mapping for the data/image from the camera system (e.g. pixels in x,y array) to the data/image fit to present to the display panel (e.g. pixels in an x,y array) (as indicated in FIGURES 9A and 11 A-l 1C).
  • the system may be configured to provide image processing/transformation for the display screen/panel as the vehicle is in operation in the environment (e.g. driving, etc.) to provide the vehicle operator with a generally continuous display image (e.g.
  • FIGURES 14A-14F and 16A-16F substantially real time updated) at the display screen (see FIGURES 14A-14F and 16A-16F) using data from data sources including from the camera system (see FIGURES 17A-17B and 18A-18B) with determination and/or application of the transformation function and remapping function based on parameters/data from data sources (see FIGURES 11 A-l 1C and 23A-23C). See also FIGURES 8A-8F.
  • data for display element/pixel in an x,y array/file may be mapped from the camera image to the display image, corrected for camera/image array/file size/type differences, pixel/resolution differences, etc.
  • the image mapping/remapping function implementation may use known/conventional display mapping routines such as available in commercial or open source software libraries (e.g. remap routine from OpenCV.org.).
  • a transformed/corrected image from the camera system e.g. data/file in an x,y array
  • the display panel e.g. a three-dimensional form/shape on the structure/pillar.
  • the system and method is configured to obtain (e.g. for image processing/transformation to a transformed/corrected image) at least a portion of camera image (e.g. from the field of view of the camera at the focal/object plane) in a data array representative of a projection of the display panel (e.g.
  • the image/data obtained from the camera system e.g. data for display
  • FIGURE 11 A data for display element/pixel in an x,y array/file, mapping from the camera image to the display image, corrected for camera/image array/file size/type differences, pixel/resolution differences, etc. as indicated schematically in FIGURE 11 A) (see FIGURES 8C- 8F and 11 A-l 1C). See also FIGURES 4B, 9A and 23A-23B (e.g. indicating projection from driver position through three-dimensional form/shape of display panel on structure/pillar onto focal/object plane to obtain portion of image/data array for remapping/transformation function).
  • the transformation of the camera image (e.g. data array/file) to the display image (e.g. data array/file) comprises a mathematical transformation/correction (e.g. to improve the accuracy of the image representation corrected for the relative positions of the operator/driver and the camera) and a geometric transformation/projection (e.g. remapping to fit the data/image from the camera to the size/shape of the display panel).
  • a mathematical transformation/correction e.g. to improve the accuracy of the image representation corrected for the relative positions of the operator/driver and the camera
  • a geometric transformation/projection e.g. remapping to fit the data/image from the camera to the size/shape of the display panel.
  • processing/computing system uses for the system/method implementation).
  • the position (e.g. values of a position) of an item as used by the system for the image processing/transformation may comprise data representative of the position of the item (e.g. as may be set or may vary during operation); for example, the position of the vehicle operator for the system may be a data value based on data representative of a vehicle seat position, sensor data associated with the operator, initial setting, default value/setting, data from a data source (such as driver preference/identity), monitoring/detecting systems, vehicle operator size, movement/motion detection, etc. (see also FIGURES 16A-16F); as shown schematically, any of a wide variety of data sources may be used and/or calibrated to provide data
  • data representative of variations in position of an item may be detected and/or adjusted in operation of the system; for example, the location/position of object plane relative to reference point such as vehicle reference point in vehicle operation) may vary during operation of the vehicle (e.g. position of object plane relative to reference point such as vehicle reference point in vehicle operation); location of the object plane may be a fixed value (e.g. fixed or pre-set as a system parameter) or a value that can be adjusted (e.g.
  • a vehicle reference point VC may provide a reference/datum for any position (e.g. for data value representative of a position any item such operator position, camera position, display position, object position, etc.) as used by the system.
  • FIGURES IE, 6A, 9A and 14A-14F indicating schematically use of display system/method to produce a simulated "invisible" pillar/structure effect for the vehicle operator/driver).
  • the display system may be implemented to perform a wide variety of functions and features with corresponding variations of the system configuration (e.g. components, subsystems, data sources, etc.) and operations (e.g. process/method steps, data, sequencing, adjustments, modifications, etc.).
  • system configuration e.g. components, subsystems, data sources, etc.
  • operations e.g. process/method steps, data, sequencing, adjustments, modifications, etc.
  • the vehicle V provides a display system DS with a camera C of a camera system CS on the structure shown as pillar P of the vehicle V (e.g. on the exterior of the vehicle/pillar); display system DS also provides a display panel DP visible to the operator/driver DR of the vehicle V. See also FIGURES 1A-1E, 9A-9C, 10A-10D, 11A-11C, 12A-12F, 13A-13F, 14A-14F, 17A-17B and 18A-18B (schematic diagrams indicating functionality/features of display system and system block diagram of implementation according to exemplary embodiment).
  • FIGURES 2A-2D, 11A-11B, 17A-17B, 18A-18B, 19, 20, 21 and 22A-22H implementation of the display system with image processing/transformation system and data sources is shown schematically according to an exemplary embodiment. See also FIGURES 23A-23C.
  • a system/functional diagram of the display system according to an exemplary embodiment (including interconnection to other systems/subsystems and data sources) is shown in FIGURES 17A-17B and 18A-18B.
  • the method of operation of the display system according to an exemplary embodiment is shown schematically in FIGURES 19, 20, 21, and 22A-22H. See also FIGURES 23A-23C (indicating computational methods for image processing/transformation).
  • processing of the camera image may comprise using data from data sources and at least one of (a) transformation of the camera image to the display image based on position of the operator of the vehicle; (b) geometric transformation of the camera image into the display image; (c) mapping of the camera image; (d) remapping of the camera image into the display image. See for example FIGURES 6A-6C, 9A-9C, 22A-22H and 23A-23C.
  • processing of the camera image may comprise correction of the camera image to the display image based on position of the camera and position of the operator. See for example FIGURES
  • data used for processing of the camera image may comprise data representative of (a) the position of the operator in the vehicle; (b) the location of the object plane. See for example FIGURES 6A-6C, 9A-9C, 22A-22H and 23A-23C.
  • processing of the camera image may comprise use of data comprising at least one of data representative of (a) location of the object plane external to the vehicle; (b) position of the operator in the vehicle interior; (c) position of the structure presenting the obstruction in the vehicle interior; (d) position of the camera; (e) position of the display screen; (f) form of the display screen; (g) a reference position for the vehicle. See for example FIGURES 6A-6C, 9A-9C, 22A-22H and 23 A-23C.
  • the display screen may comprise a non-planar display panel. See for example FIGURES IE, 2E-2G, 4B, 8C-8F and 9A- 9C.
  • transformation for image processing may comprise at least one of (a) transformation of the camera image to the display image based on position of the operator of the vehicle; (b) geometric transformation of the camera image into the display image; (c) mapping of the camera image; (d) remapping of the camera image into the display image; (e) correction of the camera image to the display image based on position of the camera and position of the operator; (f) a form of the display screen on the structure. See for example FIGURES 6A-6C, 9A-9C, 22A-22H and 23A-23C.
  • the data from data sources may comprise data representative of at least one of camera parameters, display parameters, operator parameters and object parameters;
  • camera parameters comprise data representative of at least one of camera position, camera orientation, camera angle, camera field of view, camera type, camera image resolution, camera image size, camera image array size;
  • display screen parameters comprise data representative of at least one of display screen position, display screen form, display screen orientation, display screen type, display screen resolution, display screen size, display image array size;
  • operator parameters comprise data representative of operator position, operator head position, operator eye position, operator field of view, operator seat position;
  • object parameters comprise data representative of object plane position, object detection, object position detection; data representative of object detection may be used to determine the object plane position. See for example FIGURES 2A, 3A-3C, 4B, 5A-5B, 6A-6C, 9A-9C, 15C-15D, 16A-16F, 17A-17B, 18A-18B, 22A-22H and 23A-23C.
  • the display screen on the structure may comprise a form; and method of operation of the display system may comprise the step of determining the form of the display screen using display screen parameters. See for example FIGURES IE, 2E-2G, 4B, 8C-8F, 9A-9C and 22A-22H.
  • the transformation for processing the image from the camera system may be based on data representative of at least one of camera parameters, display screen parameters, operator parameters, object parameters; the transformation may be based on data representative of (a) form of the display panel; (b) position of the display panel; (c) position of the operator; (d) position of the camera; (e) location of object plane; (f) a data array from the camera image; and (g) a data array for the display panel. See for example FIGURES 2A, 4B, 6A-6C, 9A-9C, 15C-15D, 16A-16F, 17A-17B, 18A-18B, 22A-22H and 23A-23C.
  • the transformation for processing the image from the camera system may be based on data representative of at least one of camera parameters, display screen parameters, operator parameters, object parameters; the transformation may be based on data representative of (a) form of the display panel; (b) position of the display panel; (c) position of the operator; (d) position of the camera; (e) location of object plane;
  • transformation may be based on a function comprising at least one of (a) remapping of a data array from the camera image to a data array for the display panel; (b) correcting by
  • the method of operation of the display system may comprise the step of monitoring data sources; monitoring data sources may comprise monitoring of at least one of camera parameters, display screen parameters, operator parameters, object parameters; monitoring data sources may comprise monitoring adjustments; adjustments comprise at least one of system adjustments and operation adjustments. See for example FIGURES 2C-2D, 6A-6C, 9A-9C, 15C-15D, 16A-16F, 17A-17B, 18A-18B, 22A-22H and 23A-23C.
  • a vehicle interior component may comprise a display system for a vehicle configured to be operated from a field of view of an operator in a position in a vehicle interior comprising a structure in a position in the vehicle interior presenting an at least partial obstruction to the field of view of the operator;
  • the vehicle interior component may a camera system comprising a camera configured to obtain a camera image;
  • a computing system configured to provide image processing on the camera image using data;
  • processing of the camera image may comprise at least one of (a) transformation of the camera image to the display image based on position of the operator of the vehicle; (b) geometric transformation of the camera image into the display image; (c) mapping of the camera image; (d) remapping of the camera image into the display image. See for example FIGURES 11 A-l 1C, 18A-18B, 22A-22H and 23A-23C.
  • processing of the camera image may comprise correction of the camera image to the display image based on position of the camera and position of the operator; processing of the camera image may comprise use of data representative of location of an object plane external to the vehicle.
  • data may comprise data representative of location of an object plane external to the vehicle. See for example FIGURES 5A-5C, 6A-6C, 9A-9C, 17A-17B, 18A-18B, 22A-22H and 23A-23C.
  • data may comprise data representative of at least one of (a) location of an object plane external to the vehicle; (b) position of the operator in the vehicle interior; (c) position of the structure presenting the obstruction in the vehicle interior; (d) position of the camera; (e) position of the display panel; (f) shape of the display panel; (g) a reference position for the vehicle. See for example FIGURES 2A, 4B, 6A-6C, 9A-9C, 15C-15D, 17A-17B, 18A-18B, 22A-22H and 23A-23C.
  • data sources comprise adjustment of data representative of at least one of (a) the position of the operator in the vehicle; (b) location of the object plane. See for example FIGURES 2A, 5A-5C, 6A-6C, 9A-9C, 15C- 15D, 16A-16F, 17A-17B, 18A-18B, 22A-22H and 23A-23C.
  • data may comprise data representative of a relative distance between position of camera and position of the operator. See for example FIGURES 6A-6C, 9A-9C, 16A-16F, 17A-17B, 18A-18B, 22A-22H and 23A-23C.
  • data may comprise at least data representative of (a) an image in a field of view of the camera system; (b) a relative distance between the camera and the operator; (c) location of an object plane. See for example FIGURES 6A-6C, 9A-9C, 15C-15D, 16A-16F, 17A-17B, 18A-18B, 22A-22H and 23A-23C.
  • the display system may further comprise a sensor for determining the distance of an object from the camera system; data for processing of the camera image may comprise the distance of the object from the camera system. See for example FIGURES 6A-6C, 9A-9C, 15C-15D, 17A-17B, 18A-18B, 22A-22H and 23A-23C.
  • the computing system may comprise an image processing system. See for example FIGURES 17A-17B, 18A-18B, 22A-22H and 23A-23C.
  • data sources comprise at least one vehicle system. See for example FIGURES 1A-1C, 2A-2D, 15C-15D, 17A-17B, 18A-18B, 22A-22H and 23A-23C.
  • the display panel may comprise a curved display screen. See for example FIGURES 2A-2F, 4B, 8C-8F and 11C.
  • data from data sources may comprise data representative of at least one of camera parameters, display parameters, operator parameters and object parameters;
  • camera parameters comprise data representative of at least one of camera position, camera orientation, camera angle, camera field of view, camera type, camera image resolution, camera image size, camera image array size;
  • display screen parameters comprise data representative of at least one of display screen position, display screen form, display screen orientation, display screen type, display screen resolution, display screen size, display image array size;
  • operator parameters comprise data representative of operator position, operator head position, operator eye position, operator field of view, operator seat position;
  • object parameters comprise data representative of object plane position, object detection, object position detection.
  • the method for the display system may comprise the step of defining an object plane at a distance outside of the vehicle; data representative of object detection may be used to determine the object plane. See for example FIGURES 5A-5B, 6A-6B, 9A-9C, 15C-15D, 17A-17B, 18A- 18B, 22A-22H and 23A-23C.
  • data may comprise at least one of system parameters, operation parameters, adjustment data, correction data;
  • adjustment data may comprise at least one of operator parameters and object parameters. See for example FIGURES 6A-6C, 9A-9C, 15C-15D, 16A-16F, 17A-17B, 18A-18B, 22A-22H and 23A-23C.
  • determining a form of the display screen may comprise use of display screen parameters.
  • the display screen may comprise a display panel configured to be formed into a three-dimensional shape; and the form of the display screen may comprise the three-dimensional shape of the display panel. See for example FIGURES 2A, 3A- 3C, 4B, 8C-8F, 11A-11C, 17A-17B, 18A-18B, 22A-22H and 23A-23C.
  • the display screen may comprise a flexible panel; the display screen may comprise a curved display panel. See for example FIGURES 3A- 3C, 4B and 8C-8F.
  • the transformation for image processing may be based on data representative of at least one of camera parameters, display screen parameters, operator parameters, object parameters; the transformation may be based on data representative of (a) form of the display screen; (b) position of the display screen; (c) position of the operator; (d) position of the camera; (e) location of object plane; (f) a data array from the camera image; and (g) a data array for the display screen.
  • FIGURES IE 2A, 3A-3C, 4B, 5A-5B, 6A-6C, 9A-9C, 15C-15D, 16A-16F, 17A-17B, 18A-18B, 22A-22H and 23A-23C.
  • the transformation may be based on a function comprising remapping of a data array from the camera image to a data array for the display screen; the transformation may be based on a function comprising correction based on proportion/scale (e.g. relative distance) of an image at the object plane to a display image presented on the display screen. See for example FIGURES 2A, 3A-3C, 4B, 6A-6C, 9A-9C, 15C-15D, 16A-16F, 17A-17B, 18A-18B, 22A-22H and 23A- 23C.
  • proportion/scale e.g. relative distance
  • the vehicle may comprise at least one vehicle system; and wherein transformation may be based on data representative of detected parameters from at least one vehicle system. See for example
  • the method for the display system may comprise the step of monitoring data sources; monitoring data sources may comprise monitoring of at least one of camera parameters, display screen parameters, operator parameters, object parameters; monitoring data sources may comprise monitoring adjustments; adjustments comprise at least one of system adjustments and operation adjustments. See for example FIGURES IE, 2A, 3A-3C, 4B, 5A-5B, 6A-6C, 9A-9C, 15C-15D, 16A-16F, 17A-17B, 18A-18B, 22A-22H and 23A-23C.
  • processing of the image from the camera may comprise at least one of (1) remapping the image; (2) adjusting the image; or (3) correcting the image. See for example FIGURES IE, 2A, 6A-6C, 9A-9C, 15C-15D, 16A- 16F, 17A-17B, 18A-18B, 22A-22H and 23A-23C.
  • data sources may comprise a parameter from at least one detector; data sources may comprise detected parameters from at least one vehicle system; data sources providing data to the computing system may comprise at least the camera system; data may comprise at least data representative of (a) an image in a field of view of the camera system; (b) a relative distance between the camera system and the position of the operator; (c) position of the operator of the vehicle. See for example FIGURES IE, 2A, 6A-6C, 9A-9C, 15C-15D, 16A-16F, 17A-17B, 18A-18B, 22A-22H and 23 A- 23C.
  • the step of initialization may comprise initializing the computing system; the step of initializing the computing system may be performed at start of operation of the vehicle; the step of initializing the computing system may comprise verification of data and settings.
  • the operator may comprise a driver of the vehicle. See for example FIGURES IE, 2A, 6A-6C, 9A-9C, 15C-15D, 16A-16F, 17A-17B, 18A-18B, 22A-22H and 23 A- 23C.
  • data sources for the display system may comprise and provide such data/data operations as: (a) system initial/data calibration (e.g. camera position, driver position, sensors/detectors, settings, constants, data files, etc.); (b) display system/camera system calibration (image map, display panel form/curvature correction, design parameters, dimensional data, etc.); (c) field of view (for camera/camera system, operator/driver, sensor/detector devices, etc.); (d) person/object distance to camera (data representative of object distance) from vehicle systems, sensor/detectors (sonar, ultrasonic, laser, radar, stereo camera system); (e) object recognition/form (from data/database, data sources including network data); (f) driver position (data representative of driver eye position) from default/driver data file (system calibration/initialization, driver profile, database/records, settings, network data, etc.), sensor/detector (e.g. body/head/eye tracking
  • system initial/data calibration e.g. camera position
  • the image processing/transformation operations and functions of the display system for an image data/array from the camera system may comprise such functions as: (a) initialization (e.g. verify data/array from camera system and other data sources); (b) remap/resize - remap image/data from camera system to display panel including adjustment for driver-camera position (e.g. scale and size of objects) and adjust to field of view (e.g. pixel/resolution, width/depth, angle, aspect, orientation/rotation, etc.); (c) Adjust/Scale - apply input/data (including user adjustment if provided); apply position/distance information (if provided) to display objects accurately (e.g. at driver view on display panel); (d)
  • initialization e.g. verify data/array from camera system and other data sources
  • remap/resize - remap image/data from camera system to display panel including adjustment for driver-camera position (e.g. scale and size of objects) and adjust to field of view (e.g.
  • the display system DS may comprise a camera system CS (e.g.
  • a sensor/detector/instrumentation system e.g. with one or a set of multiple sensor/detectors SD
  • vehicle systems e.g. including from vehicle network/data
  • subsystems/components shown schematically in FIGURES 2D and 15C- 15D
  • the system and method may comprise a display system for a vehicle providing a windshield and a structure such as a set of pillars and configured to be operated by an operator in a position in the vehicle interior to facilitate observation of an object outside of the vehicle and configured to use data from data sources.
  • the display system may comprise a camera system configured to obtain an image; a computing system configured to provide image processing on the image from the camera system using data; and a display panel on the structure within the vehicle configured to display an image for the operator of the vehicle.
  • Image processing on the image from the camera may comprise at least one of (1) remapping the image; (2) adjusting the image; or (3) correcting the image; data sources providing data to the computing system comprise at least the camera system.
  • the display panel may comprise a curved display panel.
  • the display panel may comprise a display panel in a vehicle trim component.
  • the vehicle trim component is installed on a pillar of the set of pillars.
  • Data may comprise at least data representative of (a) an image in a field of view of the camera system; (b) a relative distance between the camera system and the position of the operator; (c) eye position of the operator of the vehicle.
  • the camera system may comprise a camera, a set of cameras or a stereo camera.
  • the display system may comprise a sensor for determining the distance of the object from the camera system; image processing may comprise using the distance of the object from the camera to transform the image for display on the display panel.
  • the computing system may comprise an image processing system.
  • the display panel may comprise a flexible screen.
  • Data sources comprise at least one vehicle system; the vehicle system may comprise a distance monitor for an object in proximity to the vehicle.
  • the system and method may comprise a vehicle interior component having a display system for a vehicle providing a windshield and a structure such as a set of pillars and configured to be operated by an operator in a position in the vehicle interior to facilitate observation of an object outside of the vehicle and configured to use data from data sources.
  • the vehicle interior component may comprise a camera system configured to obtain an image; a computing system configured to provide image processing on the image from the camera system using data; and a display panel on the structure within the vehicle to display an image for the operator of the vehicle.
  • Image processing on the image from the camera may comprise at least one of (1) remapping the image; (2) adjusting the image; or (3) correcting the image; data sources providing data to the computing system comprise at least the camera system; when the object is obstructed by the structure an image is presented on the display panel on the structure containing the object generally representative in scale and form to scale and form that the operator would perceive for the object if not obstructed by the structure. See for example FIGURES 2A, 3A-3C, 4B, 5A-5B, 6A-6C, 9A-9C, 15C-15D, 16A-16F, 17A-17B, 18A-18B, 22A-22H and 23A-23C.
  • the system and method may comprise a vehicle providing a windshield and set of pillars and configured to be operated by an operator in a position in the vehicle interior to facilitate observation of an object outside of the vehicle and configured to use data from data sources.
  • the vehicle may comprise a display system comprising a camera system configured to obtain an image; a computing system configured to provide image processing on the image from the camera system using data; and a display panel within the vehicle to display an image for the operator of the vehicle.
  • the vehicle also may comprise at least one vehicle system of the vehicle configured to provide data as a data source.
  • Image processing on the image from the camera may comprise at least one of (1) remapping the image; (2) adjusting the image; or (3) correcting the image; data sources providing data to the computing system comprise at least the camera system; when the object is obstructed by the structure an image is presented on the display panel on the structure containing the object generally representative in scale and form to scale and form that the operator would perceive for the object if not obstructed by the structure.
  • the vehicle may comprise a sensor for determining the distance of the object from the camera system; wherein image processing may comprise using the distance of the object from the camera to transform the image for display on the display panel.
  • the computing system may comprise an image processing system.
  • the display panel may comprise a flexible screen. At least one vehicle system may comprise a distance monitor for an object in proximity to the vehicle.
  • the system and method may comprise a method of operating a display system for a vehicle providing a windshield and set of pillars and configured to be operated by an operator in a position in the vehicle interior to facilitate observation of an object outside of the vehicle from a camera system and configured to use data from data sources connected to a computing system configured to display an image for the operator of the vehicle on a display panel.
  • the method may comprise the steps of: obtaining data from the camera system; transforming the data from the camera system in the computing system using data from data sources; displaying the image on the display panel within the vehicle for the operator of the vehicle.
  • Image processing on the image from the camera may comprise at least one of (1) remapping the image; (2) adjusting the image; or (3) correcting the image; data sources providing data to the computing system comprise at least the camera system; when the object is obstructed by the structure an image is presented on the display panel on the structure containing the object generally representative in scale and form to scale and form that the operator would perceive for the object if not obstructed by the structure.
  • Data may comprise at least data representative of (a) an image in a field of view of the camera system; (b) a relative distance between the camera system and the position of the operator; (c) eye position of the operator of the vehicle.
  • the system and method may comprise an improved system and method for displaying objects outside of the vehicle that would be obstructed to an operator of a vehicle by a structure or component on a display panel on the structure or component that otherwise would present the obstruction so that the operator has a generally continuous unobstructed view of objects outside the vehicle from the windshield and side windows and on the structure or component while operating the vehicle; the objects may be visible on the display panel in the vehicle interior to the operator of the vehicle in a consistent scale/form with other objects visible such as through the windshield and side windows of the vehicle.
  • the system and method for image display on vehicle interior panel/component may comprise one or more of the inventive concepts and features shown and described in the
  • the system and method may integrate any of a wide variety of components/subsystems from existing technology to perform the general functions; for example, the display system may use any of a wide variety of cameras/camera systems, computing systems, sensors/detectors (including
  • the display system in other implementations according to an exemplary embodiment may comprise future-developed technology.
  • the present inventions may comprise conventional technology (e.g. as implemented and/or integrated in exemplary embodiments, modifications, variations, combinations, equivalents, etc.) or may comprise any other applicable technology (present and/or future) with suitability and/or capability to perform the functions and processes/operations described in the specification and/or illustrated in the FIGURES. All such technology (e.g. as implemented in embodiments, modifications, variations, combinations, equivalents, etc.) is considered to be within the scope of the present inventions of the present patent document.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Mechanical Engineering (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)

Abstract

L'invention concerne un système et un procédé d'affichage d'image sur un élément intérieur de véhicule. Le système et le procédé sont configurés de sorte à afficher des objets cachés du champ de vision de l'opérateur du véhicule sur un panneau d'affichage ; les objets peuvent être visibles par l'opérateur visualisant le panneau d'affichage dans une échelle/une forme cohérente avec des objets visibles par l'opérateur à travers le pare-brise et les fenêtres latérales du véhicule. Le système peut être configuré de sorte à obtenir une image d'un appareil de prise de vues distant/séparé et à transformer/à cartographier en une image d'affichage sur un panneau d'affichage pour fournir une représentation plus précise de la taille/de l'échelle des objets pour le point de vue de l'opérateur/du conducteur (par exemple un effet de pilier "invisible" simulé). Le système peut être configuré de sorte à utiliser des données provenant de sources de données et de paramètres ; le système et le procédé d'affichage d'image sur un élément d'intérieur de véhicule peuvent comprendre un ou plusieurs concepts et caractéristiques tels que décrits.
PCT/US2018/012574 2017-01-08 2018-01-05 Système et procédé d'affichage d'image sur un élément intérieur de véhicule WO2018129310A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201762443753P 2017-01-08 2017-01-08
US62/443,753 2017-01-08

Publications (1)

Publication Number Publication Date
WO2018129310A1 true WO2018129310A1 (fr) 2018-07-12

Family

ID=62791244

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2018/012574 WO2018129310A1 (fr) 2017-01-08 2018-01-05 Système et procédé d'affichage d'image sur un élément intérieur de véhicule

Country Status (1)

Country Link
WO (1) WO2018129310A1 (fr)

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10394322B1 (en) 2018-10-22 2019-08-27 Evolution Optiks Limited Light field display, adjusted pixel rendering method therefor, and vision correction system and method using same
US10564831B2 (en) 2015-08-25 2020-02-18 Evolution Optiks Limited Vision correction system, method and graphical user interface for implementation on electronic devices having a graphical display
US10636116B1 (en) 2018-10-22 2020-04-28 Evolution Optiks Limited Light field display, adjusted pixel rendering method therefor, and vision correction system and method using same
US10761604B2 (en) 2018-10-22 2020-09-01 Evolution Optiks Limited Light field vision testing device, adjusted pixel rendering method therefor, and vision testing system and method using same
US10860099B2 (en) 2018-10-22 2020-12-08 Evolution Optiks Limited Light field display, adjusted pixel rendering method therefor, and adjusted vision perception system and method using same addressing astigmatism or similar conditions
US10936064B2 (en) 2018-10-22 2021-03-02 Evolution Optiks Limited Light field display, adjusted pixel rendering method therefor, and adjusted vision perception system and method using same addressing astigmatism or similar conditions
WO2022037926A1 (fr) * 2020-08-19 2022-02-24 Valeo Schalter Und Sensoren Gmbh Caméra, ensemble montant de pare-brise et véhicule
US11287883B2 (en) 2018-10-22 2022-03-29 Evolution Optiks Limited Light field device, pixel rendering method therefor, and adjusted vision perception system and method using same
US11327563B2 (en) 2018-10-22 2022-05-10 Evolution Optiks Limited Light field vision-based testing device, adjusted pixel rendering method therefor, and online vision-based testing management system and method using same
US11353699B2 (en) 2018-03-09 2022-06-07 Evolution Optiks Limited Vision correction system and method, light field display and light field shaping layer and alignment therefor
WO2022128559A1 (fr) * 2020-12-15 2022-06-23 Ams International Ag Agencement de surveillance, dispositif activé par affichage et procédé de fonctionnement d'un agencement de surveillance
US11487361B1 (en) 2019-11-01 2022-11-01 Evolution Optiks Limited Light field device and vision testing system using same
US11500460B2 (en) 2018-10-22 2022-11-15 Evolution Optiks Limited Light field device, optical aberration compensation or simulation rendering
US11500461B2 (en) 2019-11-01 2022-11-15 Evolution Optiks Limited Light field vision-based testing device, system and method
US11635617B2 (en) 2019-04-23 2023-04-25 Evolution Optiks Limited Digital display device comprising a complementary light field display or display portion, and vision correction system and method using same
US11693239B2 (en) 2018-03-09 2023-07-04 Evolution Optiks Limited Vision correction system and method, light field display and light field shaping layer and alignment therefor
EP4219210A1 (fr) * 2022-01-27 2023-08-02 Faurecia Clarion Electronics Co., Ltd. Dispositif de commande d'affichage et procédé de commande d'affichage
US11789531B2 (en) 2019-01-28 2023-10-17 Evolution Optiks Limited Light field vision-based testing device, system and method
US11823598B2 (en) 2019-11-01 2023-11-21 Evolution Optiks Limited Light field device, variable perception pixel rendering method therefor, and variable perception system and method using same
US11902498B2 (en) 2019-08-26 2024-02-13 Evolution Optiks Limited Binocular light field display, adjusted pixel rendering method therefor, and vision correction system and method using same
US12019799B2 (en) 2018-10-22 2024-06-25 Evolution Optiks Limited Light field device, pixel rendering method therefor, and adjusted vision perception system and method using same

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080151054A1 (en) * 2006-12-20 2008-06-26 Aisin Aw Co., Ltd. Driving support method and driving support apparatus
US20080309764A1 (en) * 2007-06-13 2008-12-18 Aisin Aw Co., Ltd. Driving assist apparatuses and methods
US8089512B2 (en) * 2007-10-02 2012-01-03 Aisin Aw Co., Ltd. Driving support device, driving support method and computer program
US20130096820A1 (en) * 2011-10-14 2013-04-18 Continental Automotive Systems, Inc. Virtual display system for a vehicle
US20150336511A1 (en) * 2014-05-20 2015-11-26 Panasonic Intellectual Property Management Co., Ltd. Image display system and display used in image display system
US20160134815A1 (en) * 2013-07-30 2016-05-12 Toyota Jidosha Kabushiki Kaisha Driving assist device

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080151054A1 (en) * 2006-12-20 2008-06-26 Aisin Aw Co., Ltd. Driving support method and driving support apparatus
US20080309764A1 (en) * 2007-06-13 2008-12-18 Aisin Aw Co., Ltd. Driving assist apparatuses and methods
US8089512B2 (en) * 2007-10-02 2012-01-03 Aisin Aw Co., Ltd. Driving support device, driving support method and computer program
US20130096820A1 (en) * 2011-10-14 2013-04-18 Continental Automotive Systems, Inc. Virtual display system for a vehicle
US20160134815A1 (en) * 2013-07-30 2016-05-12 Toyota Jidosha Kabushiki Kaisha Driving assist device
US20150336511A1 (en) * 2014-05-20 2015-11-26 Panasonic Intellectual Property Management Co., Ltd. Image display system and display used in image display system

Cited By (34)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11262901B2 (en) 2015-08-25 2022-03-01 Evolution Optiks Limited Electronic device, method and computer-readable medium for a user having reduced visual acuity
US10564831B2 (en) 2015-08-25 2020-02-18 Evolution Optiks Limited Vision correction system, method and graphical user interface for implementation on electronic devices having a graphical display
US11693239B2 (en) 2018-03-09 2023-07-04 Evolution Optiks Limited Vision correction system and method, light field display and light field shaping layer and alignment therefor
US11353699B2 (en) 2018-03-09 2022-06-07 Evolution Optiks Limited Vision correction system and method, light field display and light field shaping layer and alignment therefor
US11726563B2 (en) 2018-10-22 2023-08-15 Evolution Optiks Limited Light field device, pixel rendering method therefor, and adjusted vision perception system and method using same
US11287883B2 (en) 2018-10-22 2022-03-29 Evolution Optiks Limited Light field device, pixel rendering method therefor, and adjusted vision perception system and method using same
US10761604B2 (en) 2018-10-22 2020-09-01 Evolution Optiks Limited Light field vision testing device, adjusted pixel rendering method therefor, and vision testing system and method using same
US10860099B2 (en) 2018-10-22 2020-12-08 Evolution Optiks Limited Light field display, adjusted pixel rendering method therefor, and adjusted vision perception system and method using same addressing astigmatism or similar conditions
US10884495B2 (en) 2018-10-22 2021-01-05 Evolution Optiks Limited Light field display, adjusted pixel rendering method therefor, and vision correction system and method using same
US10936064B2 (en) 2018-10-22 2021-03-02 Evolution Optiks Limited Light field display, adjusted pixel rendering method therefor, and adjusted vision perception system and method using same addressing astigmatism or similar conditions
US12019799B2 (en) 2018-10-22 2024-06-25 Evolution Optiks Limited Light field device, pixel rendering method therefor, and adjusted vision perception system and method using same
US11966507B2 (en) 2018-10-22 2024-04-23 Evolution Optiks Limited Light field vision testing device, adjusted pixel rendering method therefor, and vision testing system and method using same
US10642355B1 (en) 2018-10-22 2020-05-05 Evolution Optiks Limited Light field display, adjusted pixel rendering method therefor, and vision correction system and method using same
US10474235B1 (en) 2018-10-22 2019-11-12 Evolution Optiks Limited Light field display, adjusted pixel rendering method therefor, and vision correction system and method using same
US11327563B2 (en) 2018-10-22 2022-05-10 Evolution Optiks Limited Light field vision-based testing device, adjusted pixel rendering method therefor, and online vision-based testing management system and method using same
US10636116B1 (en) 2018-10-22 2020-04-28 Evolution Optiks Limited Light field display, adjusted pixel rendering method therefor, and vision correction system and method using same
US11841988B2 (en) 2018-10-22 2023-12-12 Evolution Optiks Limited Light field vision-based testing device, adjusted pixel rendering method therefor, and online vision-based testing management system and method using same
US10699373B1 (en) 2018-10-22 2020-06-30 Evolution Optiks Limited Light field display, adjusted pixel rendering method therefor, and vision correction system and method using same
US11500460B2 (en) 2018-10-22 2022-11-15 Evolution Optiks Limited Light field device, optical aberration compensation or simulation rendering
US11762463B2 (en) 2018-10-22 2023-09-19 Evolution Optiks Limited Light field device, optical aberration compensation or simulation rendering method and vision testing system using same
US11619995B2 (en) 2018-10-22 2023-04-04 Evolution Optiks Limited Light field vision-based testing device, adjusted pixel rendering method therefor, and online vision-based testing management system and method using same
US10394322B1 (en) 2018-10-22 2019-08-27 Evolution Optiks Limited Light field display, adjusted pixel rendering method therefor, and vision correction system and method using same
US11789531B2 (en) 2019-01-28 2023-10-17 Evolution Optiks Limited Light field vision-based testing device, system and method
US11635617B2 (en) 2019-04-23 2023-04-25 Evolution Optiks Limited Digital display device comprising a complementary light field display or display portion, and vision correction system and method using same
US11899205B2 (en) 2019-04-23 2024-02-13 Evolution Optiks Limited Digital display device comprising a complementary light field display or display portion, and vision correction system and method using same
US11902498B2 (en) 2019-08-26 2024-02-13 Evolution Optiks Limited Binocular light field display, adjusted pixel rendering method therefor, and vision correction system and method using same
US11500461B2 (en) 2019-11-01 2022-11-15 Evolution Optiks Limited Light field vision-based testing device, system and method
US11487361B1 (en) 2019-11-01 2022-11-01 Evolution Optiks Limited Light field device and vision testing system using same
US11823598B2 (en) 2019-11-01 2023-11-21 Evolution Optiks Limited Light field device, variable perception pixel rendering method therefor, and variable perception system and method using same
DE102020121700A1 (de) 2020-08-19 2022-02-24 Valeo Schalter Und Sensoren Gmbh Kamera, a-säulen-anordnung und fahrzeug
WO2022037926A1 (fr) * 2020-08-19 2022-02-24 Valeo Schalter Und Sensoren Gmbh Caméra, ensemble montant de pare-brise et véhicule
WO2022128559A1 (fr) * 2020-12-15 2022-06-23 Ams International Ag Agencement de surveillance, dispositif activé par affichage et procédé de fonctionnement d'un agencement de surveillance
EP4219210A1 (fr) * 2022-01-27 2023-08-02 Faurecia Clarion Electronics Co., Ltd. Dispositif de commande d'affichage et procédé de commande d'affichage
US11803235B2 (en) 2022-01-27 2023-10-31 Faurecia Clarion Electronics Co., Ltd. Display controlling device and display controlling method

Similar Documents

Publication Publication Date Title
WO2018129310A1 (fr) Système et procédé d'affichage d'image sur un élément intérieur de véhicule
JP7161410B2 (ja) シーンにおけるカメラの姿勢を特定するシステム及び方法
KR101790407B1 (ko) 동적으로 교정된 헤드-업 디스플레이
CN107554430B (zh) 车辆盲区可视化方法、装置、终端、系统及车辆
EP1878618B1 (fr) Procédé et appareil de support de conduite
US20070003162A1 (en) Image generation device, image generation method, and image generation program
US8502860B2 (en) Electronic control system, electronic control unit and associated methodology of adapting 3D panoramic views of vehicle surroundings by predicting driver intent
US9126533B2 (en) Driving support method and driving support device
EP2045132A2 (fr) Dispositif de support de commande, procédé de support de commande, et programme informatique
EP1939040A2 (fr) Procédé d'assistance à la conduite et appareil d'assistance à la conduite
CN111739101B (zh) 一种消除车辆a柱盲区的装置和方法
EP2079053A1 (fr) Procédé et appareil d'étalonnage de superposition d'affichage vidéo
JP2017078923A (ja) キャリブレーション装置、キャリブレーション方法、及び、キャリブレーションプログラム
US20120236287A1 (en) External environment visualization apparatus and method
US11794667B2 (en) Image processing apparatus, image processing method, and image processing system
JP2011217318A (ja) 車両用後方視認システム
JP7341149B2 (ja) 画像内のバーチャル細長距離目印と、コンピュータプログラム製品と、ディスプレイシステムとによって自動車の環境領域を表示する方法
EP3326145A1 (fr) Transformation de tableau
US11987182B2 (en) Image processing apparatus, image processing method, and image processing system
KR20180021822A (ko) 후방 교차 교통-퀵 룩스
JP2008037118A (ja) 車両用表示装置
KR20190026507A (ko) 차량용 영상 생성 장치 및 방법
EP3184364B1 (fr) Dispositif de vision arrière pour un véhicule
KR100757596B1 (ko) 영상처리 기술을 이용한 전측방 거리 감지 장치 및 방법
WO2010007960A1 (fr) Système d'image vidéo de conversion de point de vue pour une caméra montée sur un véhicule et procédé d'acquisition d'image vidéo de conversion de point de vue

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 18736176

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 18736176

Country of ref document: EP

Kind code of ref document: A1