US10353658B2 - Human machine interface (HMI) control unit for multiple vehicle display devices - Google Patents

Human machine interface (HMI) control unit for multiple vehicle display devices Download PDF

Info

Publication number
US10353658B2
US10353658B2 US15/272,699 US201615272699A US10353658B2 US 10353658 B2 US10353658 B2 US 10353658B2 US 201615272699 A US201615272699 A US 201615272699A US 10353658 B2 US10353658 B2 US 10353658B2
Authority
US
United States
Prior art keywords
display device
vehicular display
operational mode
application
graphic user
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US15/272,699
Other languages
English (en)
Other versions
US20180081614A1 (en
Inventor
David Tsai
Steven Basra
John McLaughlin
Shady Shahid
Greg McNamara
Per Selvaag
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Toyota Motor Sales USA Inc
Original Assignee
Toyota Motor Sales USA Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Toyota Motor Sales USA Inc filed Critical Toyota Motor Sales USA Inc
Priority to US15/272,699 priority Critical patent/US10353658B2/en
Assigned to TOYOTA MOTOR SALES, U.S.A., INC. reassignment TOYOTA MOTOR SALES, U.S.A., INC. ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: Selvaag, Per, MCNAMARA, GREG, MCLAUGHLIN, JOHN, BASRA, STEVE, TSAI, DAVID, SHAHID, SHADY
Priority to JP2017179809A priority patent/JP6791822B2/ja
Priority to EP17192204.0A priority patent/EP3299208B1/en
Publication of US20180081614A1 publication Critical patent/US20180081614A1/en
Application granted granted Critical
Publication of US10353658B2 publication Critical patent/US10353658B2/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B60K35/10—Input arrangements, i.e. from user to vehicle, associated with vehicle functions or specially adapted therefor
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
    • B60K35/21—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor using visual output, e.g. blinking lights or matrix displays
    • B60K35/22—Display screens
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
    • B60K35/21—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor using visual output, e.g. blinking lights or matrix displays
    • B60K35/23—Head-up displays [HUD]
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
    • B60K35/29—Instruments characterised by the way in which information is handled, e.g. showing information on plural displays or prioritising information according to driving conditions
    • B60K37/06—
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/34—Route searching; Route guidance
    • G01C21/36—Input/output arrangements for on-board computers
    • G01C21/3688—Systems comprising multiple parts or multiple output devices (not client-server), e.g. detachable faceplates, key fobs or multiple output screens
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/0482—Interaction with lists of selectable items, e.g. menus
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
    • G06F3/1454—Digital output to display device ; Cooperation and interconnection of the display device with other functional units involving copying of the display data of a local workstation or window to a remote workstation or window so that an actual copy of the data is displayed simultaneously on two or more displays, e.g. teledisplay
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09G—ARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00—Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/14—Display of multiple viewports
    • B60K2350/1004—
    • B60K2350/1024—
    • B60K2350/1028—
    • B60K2350/1032—
    • B60K2350/104—
    • B60K2350/2052—
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/11—Instrument graphical user interfaces or menu aspects
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/11—Instrument graphical user interfaces or menu aspects
    • B60K2360/113—Scrolling through menu items
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/143—Touch sensitive instrument input devices
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/143—Touch sensitive instrument input devices
    • B60K2360/1438—Touch screens
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/143—Touch sensitive instrument input devices
    • B60K2360/1438—Touch screens
    • B60K2360/1442—Emulation of input devices
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/145—Instrument input by combination of touch screen and hardware input devices
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/18—Information management
    • B60K2360/182—Distributing information between displays
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K2360/00—Indexing scheme associated with groups B60K35/00 or B60K37/00 relating to details of instruments or dashboards
    • B60K2360/20—Optical features of instruments
    • B60K2360/33—Illumination features
    • B60K2360/334—Projection means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048—Indexing scheme relating to G06F3/048
    • G06F2203/04803—Split screen, i.e. subdividing the display area or the window area into separate subareas

Definitions

  • Human machine interfaces with multiple vehicle display units provide challenges to the vehicle operator with respect to minimizing distractions, and focusing on the vehicle path, or course, ahead of the vehicle. It is desirable that a vehicle user have the capability of providing user input to applications displayed on a selected one of several vehicle displays with input feedback from non-selected ones of the vehicle displays, and to also present user input controls that provides different graphic user interfaces based upon the active applications.
  • HMI human machine interface
  • a method in a human machine interface (HMI) control unit for a plurality of vehicular display devices includes receiving action input data from a configurable application input module in a first operational mode.
  • a first vehicular display device of the plurality of vehicular display devices operates as a master vehicular display device relative to a second vehicular display device of the plurality of vehicular displays, and the second vehicular display operates to provide mirrored feedback corresponding to the first vehicular display device.
  • the method continues by generating a control message based on the action input data, the control message including a first identifier for the first vehicular display device and a second identifier for a second vehicular display device.
  • the control message indicates that the second vehicular display device operates as the master vehicular display device and that the first vehicular display device operates to provide the mirrored feedback relating to a subsequent action input data affecting a corresponding graphic user element of the second vehicular display device.
  • the method continues by transmitting the control message to place the first and the second vehicular display devices in the second operational mode.
  • a human machine interface (HMI) control unit in another implementation, includes a communication interface to service communication with vehicular devices, one or more processors coupled to the communication interface, the one or more processors for controlling operations of the human machine interface control unit, and a memory coupled to the one or more processors, the memory for storing data and program instructions used by the one or more processors.
  • the one or more processors are configured to execute instructions stored in the memory to receive action input data from a configurable application input module in a first operational mode.
  • a first vehicular display device of the plurality of vehicular display devices operates as a master vehicular display device relative to a second vehicular display device of the plurality of vehicular displays, and the second vehicular display operates to provide mirrored feedback corresponding to the first vehicular display device.
  • the one or more processors are configured to execute instructions stored in the memory to generate a control message based on the action input data, the control message including a first identifier for the first vehicular display device and a second identifier for a second vehicular display device.
  • the control message indicates that the second vehicular display device operates as the master vehicular display device and that the first vehicular display device operates to provide the mirrored feedback relating to a subsequent action input data affecting a corresponding graphic user element of the second vehicular display device.
  • the one or more processors are configured to execute instructions stored in the memory to transmit the control message to place the first and the second vehicular display devices in the second operational mode.
  • a human machine interface (HMI) device for use in a vehicle.
  • the HMI device includes a configurable application input module configured to be accessible through a control surface of the vehicle, and a HMI control unit in communication with the configurable application input module.
  • the configurable application input module includes a touch screen, and a graphic user interface (GUI) data generator coupled to the touch screen display and operable to present a graphic user interface for display by the touch screen.
  • GUI graphic user interface
  • the HMI control unit includes a communication interface to service communication with vehicular devices, one or more processors coupled to the wireless communication interface, the one or more processors for controlling operations of the human machine interface (HMI) control unit, and memory coupled to the one or more processors, the memory for storing data and program instructions used by the one or more processors.
  • HMI human machine interface
  • the one or more processors are configured to execute instructions stored in the memory to receive an action input data, via a touch screen of the configurable application input module, in a first operational mode, wherein in the first operational mode a first vehicular display device of the plurality of vehicular display devices operates as a master vehicular display device relative to a second vehicular display device of the plurality of vehicular displays, the second vehicular display operates to provide mirrored feedback corresponding to the first vehicular display device.
  • the one or more processors are further configured to execute instructions stored in the memory to generate a control message based on the action input data, the control message including a first identifier for the first vehicular display device and a second identifier for a second vehicular display device, wherein the control message indicates that the second vehicular display device of the plurality of vehicular display devices operates as the master vehicular display device and that the first vehicular display device operates to provide the mirrored feedback relating to a subsequent action input data affecting a corresponding graphic user element of the second vehicular display device.
  • the one or more processors are further configured to execute instructions stored in the memory to configure the touch screen by providing mode configuration data to the GUI data generator of the configurable application input module.
  • FIG. 1 is a plan view of a vehicle dash panel including a plurality of display devices and human machine interface (HMI) control unit;
  • HMI human machine interface
  • FIG. 2 is a plan view illustrating a steering assembly with a HMI control unit as shown in FIG. 1 ;
  • FIG. 3 is a functional diagram illustrating a first operational mode with regard to the plurality of display devices of FIG. 1 ;
  • FIG. 4 is a functional diagram illustrating a second operational mode with regard to the plurality of display devices of FIG. 1 ;
  • FIG. 5 a block diagram of a vehicle network that includes the HMI control unit, an upper display control unit, and a lower display control unit;
  • FIG. 6 is an illustration of a message format that may be used in the transmission and/or receipt of messages in the vehicle network of FIG. 5 ;
  • FIG. 7 is a block diagram of a HMI control unit of FIGS. 1-6 ;
  • FIG. 8 is a block diagram of a configurable application input module of FIGS. 1-7 ;
  • FIG. 9 is an example process for configuring a configurable application input module in a HMI control unit for a plurality of vehicular display devices.
  • FIG. 10 is an example process in a human machine interface (HMI) device for configuring associated display for a configurable application input module's touch screen.
  • HMI human machine interface
  • a configurable human machine interface (HMI) apparatus and method is provided with a configurable application input module, such as, for example, a touch screen device that may provide input by a vehicle user.
  • the configurable application input module may be modified and/or changed based upon an application selected by the vehicle user.
  • the selected application is displayed within a display or a zone of a display of at least first and second vehicle display screens.
  • the displays for the selected application may be visually modified based on first and second operational modes, selectable by the vehicle user, for the operation of the configurable HMI apparatus and method disclosed herein.
  • the first operational mode may be referred to as a locked mode, which may also be a default mode of operation.
  • a first display device may operate as a master relative to another display device.
  • Vehicle user input provided though the configurable application input module, is reflected in the master device while another display device operates to provide mirrored feedback back to the user.
  • a second display device may operate as the master relative to the first display device, where the second display device reflects the vehicle user's input while the another, or first display device in this example, provides the mirrored feedback.
  • the vehicle display devices may include an upper display device, such as a heads-up vehicle display to provide a line-of-sight display information, and a lower display device, such as a console display device mounted on a vehicle console.
  • FIG. 1 is a plan view of a vehicle dash panel 100 .
  • the dash panel 100 includes a steering assembly 102 , a lower display device 104 , and an upper display device 106 .
  • the steering assembly 102 includes a human machine interface (HMI) control unit 210 .
  • HMI human machine interface
  • the HMI control unit 210 operates in at least two operation modes in relation with the lower display device 104 and the upper display device 106 . The first being a locked mode and the second being an unlocked mode.
  • the vehicle operator manipulates graphic user elements of the lower display device 104 while the upper display device 106 mirrors activity with the lower display device 104 .
  • the vehicle operator may manipulate graphic user elements of the upper display device 106 while the lower display device 104 mirrors activity with the upper display device 106 .
  • the HMI control unit 210 returns to the first operational mode—the locked state.
  • the lower display device 104 may be provided as a touch screen or other larger display device having a sufficient tactile feel, passive safety, and air bag accommodation with respect to the operation of the respective vehicle.
  • the lower display device 104 may operate as a functional control system, providing several display zones within the screen display associated with various vehicles systems, such as audio visual, telephony, mapping, automotive operational conditions (e.g., tire pressures, engine temperature, and automotive warnings), etc.
  • the lower display device 104 may also function as vehicle controls, the vehicle operation information presentation, and the telephonic functional displays.
  • the lower display device 104 can operate to provide information from vehicle cameras and other inputs directed where needed by the vehicle user (as such information can temporarily, as in time of stress or crisis, replace some or all of the functional displays, for example of speed or engine RPM).
  • the lower display device 104 may provide visual information with operational modes including a standard screen, a highway screen, an audio enjoyment screen, a night vision and fog screen, and/or a difficult driving or crisis screen (e.g., very heavy traffic, ice conditions, etc.).
  • the lower display device may also provide a malfunction and alarm screen, a telephonic and Email screen, a climate control optimization screen, an Internet and communication screen (passenger-side), an Internet and communication screen (driver-side moving/stopped), a “preflight” checkout screen to allow the driver to ascertain correct functions and settings of various subsystems, a preventive maintenance screen, with data, predicted service times, and even camera images of key parts of vehicle (e.g., wheels, engine, etc.), navigational screens, including GPS based MAP data, a local navigational screen for parking (such as use of camera and other sensory data (microwave, ultrasound, LIDAR, etc) are enhanced/magnified and presented to the screen).
  • the lower display device 104 may operate to provide a parking spot screen (e.g., providing data on parking locations, using data downloaded to the vehicle upon entering or nearing a parking garage, a vehicle airflow representation with different HVAC and vent position settings, a 3D representation of sound, with different settings, including if desired the monitoring of ambient sound, a 3D representation of rear and front viewing situations by various optical subsystems (mirrors, cameras, IR or other night vision, etc) and radar subsystems if used.
  • a parking spot screen e.g., providing data on parking locations, using data downloaded to the vehicle upon entering or nearing a parking garage, a vehicle airflow representation with different HVAC and vent position settings, a 3D representation of sound, with different settings, including if desired the monitoring of ambient sound, a 3D representation of rear and front viewing situations by various optical subsystems (mirrors, cameras, IR or other night vision, etc) and radar subsystems if used.
  • various optical subsystems mirrors, cameras, IR or other night vision
  • the lower display device 104 may occupy a larger portion of the width of the dash panel 100 as permitted by dash design constraints, may be located in front of the vehicle user, and may also be located in the center of the dash panel 100 . In another aspect, the lower display device 104 may occupy the space in front of the driver, in the instrument cluster area to provide while in operation traditional vehicle functional aspects, such as speed, RPM, engine temperature, etc., or change to others of the screens as may be permitted or limited based upon the vehicle operational mode.
  • the upper display device 106 operates to project a display to an upper display panel 108 .
  • the upper display device 106 provides a line-of-sight or heads-up display device for a vehicle operator to minimize distractions from the travel path by being able to view information while looking forward, and instead of angled down looking at lower instructions and/or display information.
  • the upper display device 106 may include, for example, a projector unit, a combiner, and a video generator.
  • the projector unit operates to produce an image where the light has a focal point perceived to be at infinity.
  • the combiner is provided by the upper display panel 108 , which operates to redirect the image projected by the upper display device 106 with the field of view to be viewed at substantially the same time, and may be provided as a coating to a windshield 110 to receive the light projected onto it from the projector of the upper display device 106 while allowing other light wavelengths to pass.
  • the upper display device 106 may also be provided as a separate surface having similar light wavelength characteristics.
  • the video generator of the upper display device 106 operates to interface with the systems and/or data to be displayed by the upper display device 106 , and generates imagery and symbology to be displayed by the upper display device 106 .
  • the upper display panel 108 may have an operational mode where at least a portion (or zone) of the upper display panel 108 provides mirrored and/or visual feedback to the vehicle user corresponding to the operations and/or function of the lower display device 104 , such as via a feedback application operable by the upper display panel 108 . A remaining portion of the upper display panel 108 may operate to provide navigational display functionality to the vehicle user.
  • the lower display device 104 may be switched between display states by an external button, an integral button, a tactile sensor display of the lower display device 104 , motion sensing input, etc.
  • the HMI control unit 210 operates to allow a user to switch among the screens, as described in detail with respect to FIGS. 2-10 , and operate to trigger and/or switch individual functions or applications based upon human command inputs.
  • the lower display device 104 may also provide other bases for human input, such as hand motions, head motions, finger motions, speech and/or voice, etc.
  • FIG. 2 is an illustration of a steering assembly 102 with an HMI control unit 210 .
  • the HMI control unit 210 includes a menu hotkey 212 , a plurality of hotkeys 214 , and a configurable application input module 216 having a configurable touch screen embodiment for user input.
  • the HMI control unit 210 is depicted in ghost lines to reflect that a control unit may be concealed within the steering assembly 102 , with surface components being accessible by a vehicle user. As may be appreciated, the HMI control unit 210 may be located apart from the surface input areas. For example, the HMI control unit 210 may be contained within a steering column, integrated with other electronic control units for the vehicle, etc.
  • the menu hotkey 212 , hotkeys 214 , and configurable application input module 216 are placed adjacent a control surface of the steering assembly 102 for access by the vehicle user, such as by a depression by the user's thumb, or a location suitable to receive a sliding motion on the configurable application input module 216 .
  • Each of the hotkeys 214 may operate to provide application selection input data, which correspondingly affects the display surface of the configurable application input module 216 .
  • the configurable application input module 216 includes a touch screen that presents a configurable graphic user interface for interaction with the vehicle user.
  • the touch screen is an input device that may be layered on the top of an electronic visual display of the HMI control unit 210 .
  • the vehicle user may provide input or control data through simple or multi-touch gestures by touching the screen of the configurable application input module 216 with one or more fingers.
  • the vehicle user can use the touch screen to react to what is displayed via the lower and upper display devices 104 and 106 (see, e.g., FIG. 1 ).
  • the configurable application input module 216 may be provided by suitable touch screen technologies, for example, as a resistive touch screen, a surface acoustic wave touch screen, a capacitive touch screen, a surface capacitance touch screen, etc.
  • a touch screen of the configurable application input module 216 includes a touch region generated by the user's finger.
  • touch data can be generated that identifies a plurality of pixels, depending on the type of touch screen employed. Each pixel in the touch region may be characterized by touch magnitudes corresponding to the pressure of the touch. In another embodiment, touch data can correspond to a binary indication that a pixel is either touched or not touched. While shown as a circular periphery, touch region for the configurable application input module 216 can be of another shape, including an irregular shape and a noncontiguous region.
  • the graphic user interface of the configurable application input module 216 is configurable based on the function selected via the menu hotkey 212 and/or the hotkeys 214 .
  • the application selection input data causes the configurable application input module 216 to display control features for that application, such as “skip,” “fast forward,” “play,” “pause,” etc.
  • the application selection input data cause the configurable application input module 216 to display control features for telephony data, such as “call,” “hang up,” etc. Configuration of the configurable application input module 216 is discussed in detail with reference to FIGS. 3-10 .
  • FIGS. 3 and 4 illustrate the HMI control unit 210 having at least a first operational mode and a second operational mode with regard to the lower display device 104 and the upper display device 106 .
  • FIG. 3 illustrates a first operational mode, in which the HMI control unit 210 directs inputs via the configurable application input module 216 to a corresponding application of the lower display device 104 .
  • the upper display device 106 provides a line-of-sight feedback that does not require the vehicle user to divert their attention from the road ahead.
  • the lower display device 104 operates a master vehicular display device relative to the upper display device 106 .
  • a hotkey of the plurality of hotkeys 214 operates to select an application for control input, which operates to cause a zone of the lower display device 104 to be active for receiving input from the vehicle user.
  • the menu hotkey 212 operates to provide a menu of selectable items in the alternative to an assigned hotkey 214 .
  • the menu hotkey 212 may present selections that pertain to a basic application (music, navigation, telephony, camera video, etc.), or when a display zone is selected, provide selections that relate to the display zone.
  • the menu hotkey 212 when pressed, provides an overlay selection window with audio selections such as audio source (over-the-air radio, compact disc, USB data files, steamed-audio data sources, wireless or wired tethered stand-alone device, etc.).
  • audio source over-the-air radio, compact disc, USB data files, steamed-audio data sources, wireless or wired tethered stand-alone device, etc.
  • the lower display device 104 may have several applications defined in each of display zones 112 , 114 , 116 , and 118 .
  • a first display zone 112 includes a navigation application to provide route and vehicle guidance to the user.
  • a second display zone 114 includes a music application for the playback of music, or tunable over-the-air selections, or Internet streamed data, etc.
  • the music application is selected.
  • each of the display zones 112 , 114 , 116 , and 118 may be static in size, or may be dynamically-increased in size relative to the others when activated/selected so that a user may have easier viewing of the respective graphic user interface controls.
  • the HMI control unit 210 permits a vehicle user to provide data input to the application of the second display zone 114 .
  • the configurable application input module 216 receives user input by depressing or touching the contextual control depicted by the module's display surface.
  • a first operational mode provides input on a first set of control inputs, which may correspond with the controls displayed by the second display zone 114 .
  • the HMI control unit 210 converts the input to a code, which is provided as a control message 224 to the lower display device 104 . Messaging and network architectures are discussed in detail with reference to FIGS. 5-10 .
  • feedback may be indicated to the vehicle user by a brief color image transition to a graphic user element 115 a .
  • a graphic user element may flash, change in color or hue, etc., to indicate the input was received.
  • Further feedback may be provided, such as a change in a media information label describing the content being played back to the user.
  • the display includes an enhancement zone 120 and an application mirror zone 122 .
  • the enhancement zone 120 provides heads-up enhancement, such as navigation indicators (direction, speed, turn indicators, etc.) that provide a form of augmented reality relating to the surrounding vehicle environment.
  • the application mirror zone 122 depicts a simplified status display of the active display zone of the lower display device 104 . In the example provided, the application mirror zone 122 depicts a corresponding graphic user element 115 b . Actions relating to the second display zone 114 are reflected in the mirror zone 122 via a mirrored feedback message 226 .
  • the mirrored feedback message 226 includes information relating to the display of the lower display device 104 , which is then visually provided to the vehicle user by the application mirror zone 122 .
  • the mirrored feedback message 226 includes information relating to the display of the lower display device 104 , which is then visually provided to the vehicle user by the application mirror zone 122 .
  • a vehicle user may view similar manipulations of a corresponding graphic user element 115 b of the application mirror zone 122 .
  • a timeout may occur with respect to the upper display device 106 to provide dynamic sizing of enhancement and application mirror zones 120 and 122 . That is, when a vehicle user is not actively inputting control commands to an active application of the lower display device 104 , the application mirror zone 122 may reduce to a further minimal information display, such as providing a graphic user display that reciting “media information.”
  • the enhancement zone 120 accordingly, may increase in size to provide information relating to the principal application activity.
  • FIG. 4 illustrates a second operational mode, in which the HMI control unit 210 directs inputs via the configurable application input module 216 to a corresponding application displayed in the upper display device 106 .
  • the upper display device 106 operates a master vehicular display device relative to the lower display device 104 .
  • the vehicle user may be engaging in an activity that requires additional time and focus away from the road, such as in the music application example, selecting a content source, conducting a content search, etc.
  • the vehicle user may transition to the second operational mode based on action input data from the configurable application input module 216 .
  • the action input data is distinguishable from general application command inputs presented by the touch screen of the configurable application input module 216 .
  • An example of such an action input data may be a continuous depression, or a scrolling motion, about the touch screen for a predetermined amount of time.
  • the HMI control unit 210 Upon indicating a transition in displays, the HMI control unit 210 generates a control message 302 indicating that the upper display device 106 operates as a master vehicular display device relative to the lower display device 104 .
  • the lower display device 104 serves to provide mirror feedback via a mirrored feedback message 304 .
  • the configurable application input module 216 now depicts another graphic user interface, which in this example is a “scrolling input function.”
  • the configurable application input module 216 permits a vehicle user to manipulate the active application mirror zone 122 (indicated by directional input 404 a ) to scroll through media source options of the active application mirror zone 122 , as indicated by corresponding scrolling action 404 b .
  • a graphic user element 402 a may be highlighted as a selected source, with the lower display reflecting a corresponding graphic user element 402 b being selected for feedback purposes. In effect, the lower display device 104 operates to mirror the active application mirror zone 122 .
  • the configurable application input module 216 When in the second operational mode, the configurable application input module 216 , the lower display device 104 , and the upper display device 106 , may operate to return to the first operational mode.
  • the transition between operational modes may occur based on an active command by the vehicle user (such as another continuous depression of the configurable application input module), or based on the expiration of an inactivity time period.
  • the first vehicular display device and the second vehicular display device return to the master/slave relation of the first operational mode.
  • FIG. 5 is a block diagram of a vehicle network 500 that includes the HMI control unit 210 , a central head unit 502 , an upper display control unit 510 , a lower display control unit 512 , and a gateway 504 .
  • the upper and lower display control units 510 and 512 relating the operation of the lower and upper display devices 104 and 106 and the HMI control unit 210 are displayed, with the understanding that other control units may be present and interacting within the vehicle network 500 .
  • a suitable configuration may operate at or about 17.8 kbps with 0 to 32 bytes of data with each message frame conveyed over the vehicle network 500 .
  • This aspect of the vehicle network 500 generally allows more (and greater featured) audio and video components to be integrated into a vehicle.
  • the vehicle network 500 includes the head unit 502 , which acts as a conduit to transmit messages.
  • the vehicle network 500 may be provided based on various network structures, such as, by way of example, a star-architecture, a daisy-chain architecture, a mesh architecture, a bus architecture, or other suitable network architecture.
  • the HMI control unit 210 , the upper display control unit 510 , and the lower display control unit 512 are coupled to the head unit 502 .
  • damage caused by line failure may be reduced by coupling the control units 210 , 510 , and 512 to the head unit 502 .
  • this head unit 502 may rebroadcast transmissions received from any control unit to all control units on the network, sometimes including the originating control unit. All peripheral nodes may communicate with all others by transmitting to, and receiving from, the head unit 502 .
  • the failure of a transmission line linking any control unit 210 , 510 , and 512 , to the head unit 502 will result in the isolation of that control unit from all others, but the remainder of the vehicle network 500 will be unaffected.
  • control units 210 , 510 , and 512 , head unit 502 , and gateway 504 may be coupled by wired/fiber couplings, which may include, for example, twisted pair cable, coaxial cable, optical fiber cable, and may also be coupled by wireless communications under wireless protocols, such as wireless area network, mesh network, Bluetooth, Near Field Communication, etc.
  • the HMI control unit 210 receives application selection input data 514 from hotkeys 214 .
  • the applicaion selection input data 514 includes, for example, an application selection for a display zone of the lower display control unit 512 , and mirrored application information for the upper display control unit 510 .
  • the HMI control unit 210 also receives menu input data 516 from the menu hotkey 212 , which based upon the functionality of the vehicle network 500 , operates to provide further input selections to a vehicle user via the lower display control unit 512 , and in a graphically-minimized context, via the a mirrored feedback through the upper display control unit 510 .
  • the configurable application input module 216 provides action input data 518 based on a user selection.
  • the action input data 518 operates to provide commands relative to an active, or selected, application.
  • the action input data 518 in a music application context may include commands such as “advance,” “play,” “pause,” etc.
  • Such action input data 518 is formatted by the HMI control unit 210 for transmission to the upper and lower display control units 510 and 512 .
  • Upper display control unit 510 is coupled for communication to the upper display device 106 for graphic presentation to a vehicle user.
  • Lower display control unit 512 is coupled for communication to the lower display device 104 for graphic presentation of an application to the vehicle user.
  • the graphic presentation of the upper display is abridged, or further concise, relative to that for the lower display device 104 in view of the available display space provided by the respective devices.
  • the action input data 518 may also include a transition command for an operational mode of the lower display control unit 512 and the lower display control unit 512 .
  • the first operational mode is that the configurable application input module 216 operates with the lower display device 104 in a first operational mode.
  • the mode may be considered a locked mode with regard to the inputs of the configurable application input module 216 , while the application mirror zone 122 of the upper display device 106 operates as a slave to the master of the lower display device 104 .
  • the application mirror zone 122 serves as a master display relative to the second display zone 114 .
  • the configurable application input module 216 is no longer “locked” to the lower display device 104 , and a vehicle user is able to provide input based on the graphic user interface of the upper display device 106 via the configurable application input module 216 .
  • the HMI control unit 210 provides mode configuration data 520 .
  • the mode configuration data 520 includes a graphic user interface related to the application selection input data 514 , or to the menu input data 516 . With respect to the operational mode, the mode configuration data 520 further includes a graphic user interface related to the first operational mode (see FIG. 3 ) and another graphic user interface related to the second operation mode (see FIG. 4 ).
  • the gateway 504 may be provided as a control unit.
  • the gateway 504 permits communication between vehicle networks with different communication data formats, such as a body electric area network (BEAN), a controller area network (CAN), which is based on an ISO standard bus for vehicles, etc.
  • BEAN body electric area network
  • CAN controller area network
  • the CAN manages the chassis electrical system control and responsible for critical-classified activities such as engine electrics and skid control
  • FIG. 6 is an illustration of a message frame for a message 600 that may be used in the transmission and/or receipt of messages such as the control message 302 and the mirror feedback message 304 (see, e.g., FIGS. 3 and 4 ).
  • the message frame provides a format for commands to devices in the vehicular network 500 .
  • the message frame may have a format including a header field with a start bit 602 , a master address field 604 , a slave address field 606 , a control field 608 , a data length field 610 , and application data field 612 , which corresponds with the data length field 610 .
  • other message formats may be implemented to provide communication within the vehicle network 500 , and may be based on packet-based network protocol specifications, such as IEEE 802.3, 802.11, 802.20, Bluetooth, etc., and variations thereof.
  • a second message is shown for a second control unit of the vehicle network 500 , which repeats the structure for another device via a master address field 614 and a slave address field 616 .
  • the input from the configurable application input module 216 , and the application selection input data 514 affect both upper and lower display control units 510 and 512 , and the graphic user interface displayed on the lower display device 104 and the upper display device 106 .
  • Messages may be generated addressed for the respective control unit of the vehicle network 500 , or the message may be broadcast, with the upper display control unit 510 and the lower display control unit 512 including sufficient intelligence to distinguish the present operational mode (that is, “locked” or “unlocked”), and treating the data accordingly.
  • Communication of the message 600 may be from one control unit to another, or may be broadcast (as indicated by a broadcast field) to all control units via the vehicle network 500 .
  • Each control unit on the vehicle network 500 bus has a unique identifier (address) that indicates its device type. Because the upper and lower display control units 510 and 512 may be classified with the same device type, a secondary identifier may be implemented to further distinguish the location of the display devices.
  • the master address field 604 indicates the HMI control unit 210 .
  • the slave address field 606 indicates the destination unit, which may be the upper display control unit 510 or the lower display control unit 512 .
  • the control field 608 may designate the operational mode for the lower and upper display devices 104 and 106 , such as “locked” in the first operational mode, or “unlocked” in the second operational mode.
  • FIG. 7 is a block diagram of a HMI control unit 210 .
  • the HMI control unit 210 includes a communication interface 702 , a processor 704 , and memory 706 , that are communicatively coupled via a data bus 708 .
  • the processor 704 in the HMI control unit 210 can be a conventional central processing unit or any other type of device, or multiple devices, capable of manipulating or processing information. As may be appreciated, the processor 704 may be a single processing device or a plurality of processing devices. Such a processing device may be a microprocessor, micro-controller, digital signal processor, microcomputer, central processing unit, field programmable gate array, programmable logic device, state machine, logic circuitry, analog circuitry, digital circuitry, and/or any device that manipulates signals (analog and/or digital) based on hard coding of the circuitry and/or operational instructions.
  • the memory and/or memory element 706 may be a single memory device, a plurality of memory devices, and/or embedded circuitry of the processor 704 .
  • Such a memory device may be a read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, and/or any device that stores digital information.
  • the processor 704 includes more than one processing device, the processing devices may be centrally located (e.g., directly coupled together via a wired and/or wireless bus structure) or may be distributed located (e.g., cloud computing via indirect coupling via a local area network and/or a wide area network).
  • the processor 704 implements one or more of its functions via a state machine, analog circuitry, digital circuitry, and/or logic circuitry
  • the memory and/or memory element storing the corresponding operational instructions may be embedded within, or external to, the circuitry comprising the state machine, analog circuitry, digital circuitry, and/or logic circuitry.
  • the memory element stores, and the processor 704 executes, hard coded and/or operational instructions corresponding to at least some of the steps and/or functions illustrated in FIGS. 1-10 to perform human-machine interface features and methods described herein.
  • the communication interface 702 generally governs and manages the user input data including application selection input data 514 , menu input data 516 , and action input data 518 , and also manages controller unit output data, which includes the mode configuration data 520 and the message 600 generation and transmission to the vehicle network 500 (see FIG. 5 ).
  • controller unit output data which includes the mode configuration data 520 and the message 600 generation and transmission to the vehicle network 500 (see FIG. 5 ).
  • the structure of the HMI control unit 210 may also be used as an acceptable architecture of the upper display control unit 510 and the lower display control unit 512 .
  • the upper and lower display control units 510 and 512 may each include a communication interface, a processor, and memory that may be communicatively coupled via a data bus.
  • other architectures may be implemented, with similar functional capabilities.
  • the processors for the upper and lower display control units 510 and 512 may be a conventional central processing unit or any other type of device, or multiple devices, capable of manipulating or processing information. As may be appreciated, the processor may be a single processing device or a plurality of processing devices. Such a processing device may be a microprocessor, micro-controller, digital signal processor, microcomputer, central processing unit, field programmable gate array, programmable logic device, state machine, logic circuitry, analog circuitry, digital circuitry, and/or any device that manipulates signals (analog and/or digital) based on hard coding of the circuitry and/or operational instructions.
  • the memory and/or memory element for the upper and lower display control units 510 and 512 may be a single memory device, a plurality of memory devices, and/or embedded circuitry of the processor related to the upper and lower display control units 510 and 512 .
  • Such a memory device may be a read-only memory, random access memory, volatile memory, non-volatile memory, static memory, dynamic memory, flash memory, cache memory, and/or any device that stores digital information.
  • the processing devices may be centrally located (e.g., directly coupled together via a wired and/or wireless bus structure) or may be distributed located (e.g., cloud computing via indirect coupling via a local area network and/or a wide area network).
  • the processor for each of the upper and lower display control units 510 and 512 may implement one or more of its functions via a state machine, analog circuitry, digital circuitry, and/or logic circuitry
  • the memory and/or memory element storing the corresponding operational instructions may be embedded within, or external to, the circuitry comprising the state machine, analog circuitry, digital circuitry, and/or logic circuitry.
  • the memory element stores, and the processor 704 executes, hard coded and/or operational instructions corresponding to at least some of the steps and/or functions illustrated in FIGS. 1-10 to perform human-machine interface features and methods described herein.
  • the communication interface for each of the upper and lower display control units 510 and 512 generally governs and manages the user input data received via the vehicle network 500 , and outbound data to/from the upper display device 106 and the lower display device 104 , respectively.
  • the communication interface for each of the upper and lower display control units 510 and 512 generally governs and manages the user input data received via the vehicle network 500 , and outbound data to/from the upper display device 106 and the lower display device 104 , respectively.
  • FIG. 8 presents a block diagram of a configurable application input module 216 .
  • the configurable application input module 216 is presented for use in conjunction with the HMI control unit 210 that hosts a touch screen 802 .
  • Touch screen 802 may include a display screen, such as a liquid crystal display (LCD), light emitting diode (LED), plasma display or other two dimensional or three dimensional display that displays graphics, text or video in either monochrome or color in response to display data 810 .
  • LCD liquid crystal display
  • LED light emitting diode
  • plasma display or other two dimensional or three dimensional display that displays graphics, text or video in either monochrome or color in response to display data 810 .
  • the configurable application input module 216 generates display data 810 for display on the touch screen 802 .
  • the display data 810 creates a visual representation of a graphic user interface when displayed on the touch screen 802 that can be used interactively by the vehicle user.
  • a graphic user interface (GUI) data generator 806 processes touch data 804 from the touch screen 802 to determine when the touch data 804 indicates user interaction with icons of the graphic user interface.
  • GUI graphic user interface
  • the GUI data generator 806 operates based on the touch data 804 to adjust the display data 810 to reflect the user interaction with the touch screen 802 .
  • the GUI data generator 806 generates action input data 518 based on the user interaction with the touch screen 802 .
  • the GUI data generator 806 receives mode configuration data 520 to generate the display data 810 , which in turn provides a graphic user interface displayed by the touch screen 802 .
  • the mode configuration data 520 indicates the application graphic user interface for which the configurable application input module 216 is being altered or presented for vehicle user input. The application as discussed herein, is based upon the vehicle user selection by application selection input data 514 and/or menu input data 516 (see FIG. 5 ).
  • the mode configuration data 520 includes data regarding the operational mode for the configurable application input module 216 . That is, the graphic user interface for the touch screen 802 has a version for a first operational mode (that is, a locked mode) and a second operational mode (that is, an unlocked mode) relating to the application selection by the vehicle user.
  • the configurable application input module 216 is in the first operational mode.
  • the configurable application input module 216 operates to manipulate the lower display device 104 , and the upper display device 106 mirrors the user inputs to the lower display device 104 (see FIGS. 3 and 4 ).
  • the configurable application input module 216 operates to manipulate the upper display device 106 , and the lower display device 104 mirrors the user inputs to the upper display device 106 .
  • the configurable application input module 216 operates to manipulate the upper display device 106 , and the lower display device 104 mirrors the user inputs to the upper display device 106 .
  • the configurable application input module 216 when in the second operational mode, then operates to manipulate the lower display device 104 , and the upper display device 106 mirrors the user inputs to the lower display device 104 .
  • the screens associated with each of the first and the second operational mode may be defined by the vehicle user via the configurable application input module 216 , may be initially defined by the HMI control unit 210 , may be defined by downloadable data that is accessible via the gateway 504 , etc.
  • the action input data 518 based on the user's manipulation of the touch screen 802 can be used in conjunction with another display generator of the upper display control unit 510 and the lower display control unit 512 (see FIG. 5 ) to generate display data for the lower display device 104 and the upper display device 106 .
  • the display of at least one other graphic user element 115 a and 115 b is adjusted based on the action input data 518 (see, e.g., FIG. 3 ).
  • touch data 804 can be used by the configurable application input module 216 to control, for example, the position, orientation and/or motion of an object that is displayed on the touch screen 802 , and also elements of the lower display device 104 and/or the upper display device 106 via the action input data 518 .
  • Such a graphical object can be a game object or character, a cursor, a highlighted area, tab, menu item, or other graphical object displayed by the touch screen 802 that may also be displayed by the lower display device 104 , and/or by the upper display device 106 as discussed herein.
  • FIG. 9 is an example process 900 for configuring a configurable application input module in a human machine interface (HMI) control unit for a plurality of vehicular display devices.
  • the operations described in connection the process 900 can be performed at the HMI control unit 210 (see, e.g., FIGS. 1-8 ).
  • the operations described in connection with the process 900 can be stored in the memory 706 , and executed by the processor 704 (see, e.g., FIG. 7 ).
  • the HMI control unit receives action input data from a configurable application input module 216 .
  • the HMI control unit places the configurable application module 216 is in a first operational mode.
  • a first vehicular display device operates as a master vehicular display device relative to a second vehicular display device.
  • the second vehicular display operates to provide mirrored feedback corresponding to the first vehicular display device. In this manner, the relational configuration of the displays expected or desired by the vehicle user remains consistent during vehicle operation.
  • the HMI control unit determines whether the action input data received from the configurable application input module 216 indicates a transition mode from the first operational mode to the second operational mode.
  • the action input data may indicate that a vehicle user provided a key or depression pattern (or motion pattern) to prompt the control to move from a locked operational mode to an unlocked operational mode.
  • the pattern may include a steady press by a finger for a duration of time, or circular pattern of a finger for a duration of time, or a pattern (such as a depression on a top-bottom-left-right circumference of the control).
  • the first and the second vehicular display devices may be defined, by way of example, as a lower display device and an upper display device, respectively.
  • a vehicle user may manipulate a display for the lower display device with a mirrored feedback to the vehicle user by a display of the upper display device.
  • the vehicle user may manipulate a display for the upper display device with a mirrored feedback to the vehicle user by a display of the lower display device.
  • the operational modes may be reversed for the display units based upon the designation of the display devices.
  • the first and the second vehicular display devices may be defined as an upper display device and a lower display device, respectively.
  • a vehicle user may manipulate a display for the upper display device with a mirrored feedback to the vehicle user by a display of the lower display device.
  • the vehicle user may manipulate a display for the lower display device with a mirrored feedback to the vehicle user by a display of the upper display device.
  • Such vehicle display designations may be placed as a default configuration upon manufacture and/or installation of the devices with a vehicle, may be altered by the vehicle user, etc.
  • the HMI control unit causes the second vehicular display device of the plurality of vehicular display devices to operate as the master vehicular display device and that the first vehicular display device to operate to provide the mirrored feedback relating to a subsequent action input data affecting a corresponding graphic user element of the second vehicular display device.
  • a control message is generated based on the action input data, the control message including a first identifier for the first vehicular display device and a second identifier for a second vehicular display device.
  • the control message operates to indicate that the second vehicular display device of the plurality of vehicular display devices operates as the master vehicular display device and that the first vehicular display device operates to provide the mirrored feedback relating to a subsequent action input data affecting a corresponding graphic user element of the second vehicular display device.
  • the vehicle display devices may receive similar input subsequent action input data from a vehicle user, the focus by the vehicle will be to a selected display, and the selection activity will take place with regard to the selected display.
  • the selection or manipulation of the graphic user element 402 a takes place relative to the active application mirror zone 122 of the upper display device 106 . In this manner, the attention of the vehicle user remains on the road and/or path ahead.
  • the lower display device 104 serves to provide mirror feedback via the mirrored feedback message 304 , which may be generated and transmitted via the upper display device 106 to the lower display device 104 , or may be generated by the HMI control unit 210 and transmitted to each of the lower and upper display devices 104 and 106 via the vehicle network 500 (see FIG. 5 ).
  • the HMI control unit 210 operates to transmit the control message to place the first and the second vehicular display devices in the second operational mode.
  • the HMI control unit 210 may revert to the first operational mode with respect to the first and the second vehicular display devices.
  • the HMI control unit 210 determines whether there has been inactivity for a period of time, resulting in an inactivity timeout. When an inactivity time period expires, then at operation 912 , the first vehicular display device and the second vehicular display device are returned to the first operational mode.
  • an inactivity control message including a first identifier for the first vehicular display device and a second identifier for a second vehicular display device and is transmitted at operation 912 to the vehicular display devices.
  • the control message includes data and/or instructions causing the first vehicular display device to again operate as the master vehicular display device and that the second vehicular display device again operates to provide the mirrored feedback relating to a subsequent action input data affecting a corresponding graphic user element of the first vehicular display device.
  • FIG. 10 is an example process 1000 in a configurable application input module of a human machine interface (HMI) device may display application controls (such as for music, navigation, telephony, vehicle status, etc.) to a vehicle user. Also, with regard to the graphic user interface, the configurable application input module may further be changed to different designs and display formats for its associated display screen, such as a first display for a first operational mode for a selected application and a second display for a second operational mode for the selected application.
  • HMI human machine interface
  • the operations described in connection with the process 1000 can be performed at the configurable application input module 216 (see, e.g., FIGS. 1-8 ).
  • the operations described in connection with the process 1000 can also be embodied as a non-transitory computer readable storage medium including program instructions executable by one or more processors that, when executed, cause the one or more processors to perform the operations described herein.
  • the operations described in connection with the process 1000 can be stored and executed in a memory device of the GUI data generator 806 (see, e.g., FIG. 8 ).
  • the HMI device configures a configurable application input module 216 (see FIGS. 1-8 ) with a first graphic user interface associated with application selection input data in a first operational mode.
  • the configurable application input module 216 receives mode configuration data, which may include data for an operational transition for operation with respect to vehicular display devices.
  • the configurable application input module 216 is configured with a second graphic user interface at operation 1008 , the second graphic user interface being associated with the application selection input data in a second operational mode.
  • the display screen of the configurable application input module 216 may revert to the first graphic user interface for the first operational mode with respect to the first and the second vehicular display devices.
  • the HMI device determines whether there has been inactivity for a period of time, resulting in an inactivity timeout. When an inactivity time period expires, then at operation 1012 , the configurable application input module 216 returns to the first operational mode with the first graphic user interface associated with the application selection input data.
  • the term “substantially” or “approximately,” as may be used herein, provides an industry-accepted tolerance to its corresponding term and/or relativity between items. Such an industry-accepted tolerance ranges from less than one percent to twenty percent and corresponds to, but is not limited to, component values, integrated circuit process variations, temperature variations, rise and fall times, and/or thermal noise. Such relativity between items range from a difference of a few percent to magnitude differences.
  • Coupled includes direct coupling and indirect coupling via another component, element, circuit, or module where, for indirect coupling, the intervening component, element, circuit, or module does not modify the information of a signal but may adjust its current level, voltage level, and/or power level.
  • inferred coupling (that is, where one element is coupled to another element by inference) includes direct and indirect coupling between two elements in the same manner as “coupled.”
  • the term “compares favorably,” as may be used herein, indicates that a comparison between two or more elements, items, signals, et cetera, provides a desired relationship. For example, when the desired relationship is that a first signal has a greater magnitude than a second signal, a favorable comparison may be achieved when the magnitude of the first signal is greater than that of the second signal, or when the magnitude of the second signal is less than that of the first signal.
  • a module includes a functional block that is implemented in hardware, software, and/or firmware that performs one or more functions such as the processing of an input signal to produce an output signal.
  • a module may contain submodules that themselves are modules.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Transportation (AREA)
  • Chemical & Material Sciences (AREA)
  • Combustion & Propulsion (AREA)
  • Mechanical Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Human Computer Interaction (AREA)
  • Computer Hardware Design (AREA)
  • Automation & Control Theory (AREA)
  • User Interface Of Digital Computer (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)
US15/272,699 2016-09-22 2016-09-22 Human machine interface (HMI) control unit for multiple vehicle display devices Active 2037-07-18 US10353658B2 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US15/272,699 US10353658B2 (en) 2016-09-22 2016-09-22 Human machine interface (HMI) control unit for multiple vehicle display devices
JP2017179809A JP6791822B2 (ja) 2016-09-22 2017-09-20 複数のビークルディスプレイデバイスのためのヒューマンマシンインターフェイス（ｈｍｉ）制御ユニット
EP17192204.0A EP3299208B1 (en) 2016-09-22 2017-09-20 Human machine interface (hmi) control unit for multiple vehicle display devices

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US15/272,699 US10353658B2 (en) 2016-09-22 2016-09-22 Human machine interface (HMI) control unit for multiple vehicle display devices

Publications (2)

Publication Number Publication Date
US20180081614A1 US20180081614A1 (en) 2018-03-22
US10353658B2 true US10353658B2 (en) 2019-07-16

Family

ID=59930208

Family Applications (1)

Application Number Title Priority Date Filing Date
US15/272,699 Active 2037-07-18 US10353658B2 (en) 2016-09-22 2016-09-22 Human machine interface (HMI) control unit for multiple vehicle display devices

Country Status (3)

Country Link
US (1) US10353658B2 (enExample)
EP (1) EP3299208B1 (enExample)
JP (1) JP6791822B2 (enExample)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10671205B2 (en) * 2016-09-30 2020-06-02 Toyota Jidosha Kabushiki Kaisha Operating apparatus for vehicle

Families Citing this family (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CA3216574A1 (en) 2013-02-26 2014-09-04 Polaris Industries Inc. Recreational vehicle interactive telemetry, mapping, and trip planning system
US9324195B2 (en) 2013-02-26 2016-04-26 Polaris Industries Inc. Recreational vehicle interactive, telemetry, mapping, and trip planning system
MX391895B (es) * 2016-02-10 2025-03-21 Polaris Inc Sistemas de administración para grupo de vehículos recreativos.
KR102279333B1 (ko) * 2017-03-07 2021-07-20 주식회사 만도 차량의 디스플레이 시스템 및 이의 구동 방법
US10198949B2 (en) * 2017-04-28 2019-02-05 Mastercard International Incorporated Method and system for parking verification via blockchain
JP6547155B2 (ja) * 2017-06-02 2019-07-24 本田技研工業株式会社 車両制御システム、車両制御方法、およびプログラム
WO2019087376A1 (ja) * 2017-11-06 2019-05-09 三菱電機株式会社 操作端末、プログラム及び空調システム
JP2020024560A (ja) * 2018-08-07 2020-02-13 本田技研工業株式会社 表示装置、表示制御方法、およびプログラム
JP1671702S (ja) * 2018-12-24 2020-11-02 乗用自動車
US20200257442A1 (en) * 2019-02-12 2020-08-13 Volvo Car Corporation Display and input mirroring on heads-up display
US10759441B1 (en) * 2019-05-06 2020-09-01 Cambridge Mobile Telematics Inc. Determining, scoring, and reporting mobile phone distraction of a driver
US10672249B1 (en) 2019-05-06 2020-06-02 Cambridge Mobile Telematics Inc. Determining, scoring, and reporting mobile phone distraction of a driver
JP2021075157A (ja) 2019-11-08 2021-05-20 トヨタ自動車株式会社 車両用入力装置
USD963666S1 (en) 2020-03-25 2022-09-13 Denso International America, Inc. Display screen or portion thereof with graphical user interface for a remote control vehicle
FR3111308B1 (fr) * 2020-06-15 2025-01-10 Psa Automobiles Sa Procédé et dispositif de réglage de fonctions de confort et/ou de sécurité d’un véhicule
US11513754B2 (en) * 2020-09-08 2022-11-29 Atieva, Inc. Presenting content on separate display devices in vehicle instrument panel
JP7534181B2 (ja) * 2020-10-13 2024-08-14 株式会社Subaru 車両の表示装置
CN112416277B (zh) * 2020-11-04 2024-09-03 上海郡码智能科技有限公司 车载系统的多屏显示方法、装置、车载系统和存储介质
CN114379573B (zh) * 2022-01-10 2024-05-24 海信集团控股股份有限公司 车辆及其控制方法
DE102022110485B4 (de) * 2022-04-29 2024-02-01 Peter Bongers Verfahren und Steuersystem zum Aufrufen von Funktionen und zur Betätigung von Einstellungen eines Fahrzeugs
KR20230162249A (ko) * 2022-05-20 2023-11-28 현대자동차주식회사 택시 서비스 제공 시스템 및 방법
CN117270801A (zh) * 2022-06-13 2023-12-22 华为技术有限公司 一种汽车座舱屏幕的操作方法及相关设备

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6240347B1 (en) 1998-10-13 2001-05-29 Ford Global Technologies, Inc. Vehicle accessory control with integrated voice and manual activation
US20060155431A1 (en) 2004-03-04 2006-07-13 Berg Frederick J Vehicle information system with remote communicator
US20100220250A1 (en) 2006-12-20 2010-09-02 Johnson Controls Technology Company Remote display reproduction system and method
US20110130921A1 (en) 2009-11-27 2011-06-02 Fujitsu Ten Limited In-vehicle display system
WO2012156029A1 (de) 2011-05-17 2012-11-22 Volkswagen Aktiengesellschaft Verfahren und system zum bereitstellen einer nutzerschnittstelle in einem fahrzeug
US20130218412A1 (en) * 2011-11-16 2013-08-22 Flextronics Ap, Llc Occupant sharing of displayed content in vehicles
WO2014096975A2 (en) 2012-12-20 2014-06-26 Johnson Controls Gmbh Vehicle display system and method
US20150066360A1 (en) * 2013-09-04 2015-03-05 Honda Motor Co., Ltd. Dashboard display navigation
GB2517961A (en) 2013-09-06 2015-03-11 Jaguar Land Rover Ltd A vehicle
US20160054967A1 (en) 2014-08-21 2016-02-25 Continental Automotive Systems, Inc. Cluster-radio information interchange system for enhancing human-machine interface experience

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6240347B1 (en) 1998-10-13 2001-05-29 Ford Global Technologies, Inc. Vehicle accessory control with integrated voice and manual activation
US20060155431A1 (en) 2004-03-04 2006-07-13 Berg Frederick J Vehicle information system with remote communicator
US20100220250A1 (en) 2006-12-20 2010-09-02 Johnson Controls Technology Company Remote display reproduction system and method
US20110130921A1 (en) 2009-11-27 2011-06-02 Fujitsu Ten Limited In-vehicle display system
WO2012156029A1 (de) 2011-05-17 2012-11-22 Volkswagen Aktiengesellschaft Verfahren und system zum bereitstellen einer nutzerschnittstelle in einem fahrzeug
US20130218412A1 (en) * 2011-11-16 2013-08-22 Flextronics Ap, Llc Occupant sharing of displayed content in vehicles
WO2014096975A2 (en) 2012-12-20 2014-06-26 Johnson Controls Gmbh Vehicle display system and method
US20150066360A1 (en) * 2013-09-04 2015-03-05 Honda Motor Co., Ltd. Dashboard display navigation
GB2517961A (en) 2013-09-06 2015-03-11 Jaguar Land Rover Ltd A vehicle
US20160054967A1 (en) 2014-08-21 2016-02-25 Continental Automotive Systems, Inc. Cluster-radio information interchange system for enhancing human-machine interface experience

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Extended European Search Report for Application No. 17192204.0 dated Feb. 27, 2018 in 9 pages.

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10671205B2 (en) * 2016-09-30 2020-06-02 Toyota Jidosha Kabushiki Kaisha Operating apparatus for vehicle

Also Published As

Publication number Publication date
EP3299208B1 (en) 2019-10-30
JP6791822B2 (ja) 2020-11-25
US20180081614A1 (en) 2018-03-22
JP2018077832A (ja) 2018-05-17
EP3299208A1 (en) 2018-03-28

Similar Documents

Publication Publication Date Title
US10353658B2 (en) Human machine interface (HMI) control unit for multiple vehicle display devices
KR20250156076A (ko) 위젯을 이용한 차량용 인포테인먼트 장치와 그의 동작 방법
US10466800B2 (en) Vehicle information processing device
US10528150B2 (en) In-vehicle device
US9649938B2 (en) Method for synchronizing display devices in a motor vehicle
US10591723B2 (en) In-vehicle projection display system with dynamic display area
US12263734B2 (en) Vehicle input device, vehicle input method, and non-transitory storage medium stored with vehicle input program
US20200057546A1 (en) User interface and methods for inputting and outputting information in a vehicle
US20220055482A1 (en) Systems And Methods For Horizon Digital Virtual Steering Wheel Controller
CN103909864A (zh) 车辆显示装置以及包括车辆显示装置的车辆
US10289217B2 (en) Multi-function control knobs for multimedia systems of vehicles
KR102708300B1 (ko) 차량 및 차량의 제어방법
WO2016084360A1 (ja) 車両用表示制御装置
KR101928637B1 (ko) 차량에서의 조작 방법 및 조작 시스템
KR102879111B1 (ko) 위젯을 이용한 차량용 인포테인먼트 장치와 그의 동작 방법
EP2685361B1 (en) Vehicle graphical user interface arrangement
JP7512976B2 (ja) 車両用表示制御装置、車両用表示装置、車両用表示制御方法およびプログラム
WO2018167815A1 (ja) 表示制御装置及び表示制御方法
KR20220062165A (ko) 사용자 인터페이스 장치, 그를 가지는 차량 및 그 제어 방법
US20250214441A1 (en) Cockpit system and method of controlling same
JP2020181279A (ja) 車両用メニュー表示制御装置、車載機器操作システム、及びｇｕｉプログラム
CN120942353A (zh) 用于头显式驾驶的方法、设备、介质和程序产品
US20220404923A1 (en) Transportation Apparatus and Device and Method for the Operation of an Operating-Force-Sensitive Input Device of a Transportation Apparatus
CN121062470A (zh) 座舱系统的显示处理方法、装置及可移动设备

Legal Events

Date Code Title Description
AS Assignment

Owner name: TOYOTA MOTOR SALES, U.S.A., INC., CALIFORNIA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:TSAI, DAVID;BASRA, STEVE;MCLAUGHLIN, JOHN;AND OTHERS;SIGNING DATES FROM 20160806 TO 20160920;REEL/FRAME:039920/0853

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: AWAITING TC RESP., ISSUE FEE NOT PAID

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT RECEIVED

STPP Information on status: patent application and granting procedure in general

Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED

STCF Information on status: patent grant

Free format text: PATENTED CASE

CC Certificate of correction
MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 4