WO2014070872A3 - System and method for multimodal interaction with reduced distraction in operating vehicles - Google Patents

System and method for multimodal interaction with reduced distraction in operating vehicles Download PDF

Info

Publication number
WO2014070872A3
WO2014070872A3 PCT/US2013/067477 US2013067477W WO2014070872A3 WO 2014070872 A3 WO2014070872 A3 WO 2014070872A3 US 2013067477 W US2013067477 W US 2013067477W WO 2014070872 A3 WO2014070872 A3 WO 2014070872A3
Authority
WO
WIPO (PCT)
Prior art keywords
service request
operating vehicles
multimodal interaction
input
reduced distraction
Prior art date
Application number
PCT/US2013/067477
Other languages
French (fr)
Other versions
WO2014070872A2 (en
Inventor
Fuliang Weng
Zhe Feng
Zhongnan Shen
Kui Xu
Original Assignee
Robert Bosch Gmbh
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Robert Bosch Gmbh filed Critical Robert Bosch Gmbh
Publication of WO2014070872A2 publication Critical patent/WO2014070872A2/en
Publication of WO2014070872A3 publication Critical patent/WO2014070872A3/en

Links

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/26—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 specially adapted for navigation in a road network
    • G01C21/34—Route searching; Route guidance
    • G01C21/36—Input/output arrangements for on-board computers
    • G01C21/3605—Destination input or retrieval
    • G01C21/362—Destination input or retrieval received from an external device or application, e.g. PDA, mobile phone or calendar application
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223—Execution procedure of a spoken command
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/226—Procedures used during a speech recognition process, e.g. man-machine dialogue using non-speech characteristics

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Theoretical Computer Science (AREA)
  • Computational Linguistics (AREA)
  • Automation & Control Theory (AREA)
  • Acoustics & Sound (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • User Interface Of Digital Computer (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

A method of interaction with an in-vehicle information system includes receiving first and second inputs from an operator with first and second input devices, respectively. The method further includes identifying a service request corresponding to the first input, and a parameter of the service request with a value that is included in the second input with controller in the in-vehicle information system. The controller executes stored program instructions to perform the identified service request with reference to the identified parameter.
PCT/US2013/067477 2012-10-30 2013-10-30 System and method for multimodal interaction with reduced distraction in operating vehicles WO2014070872A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201261720180P 2012-10-30 2012-10-30
US61/720,180 2012-10-30

Publications (2)

Publication Number Publication Date
WO2014070872A2 WO2014070872A2 (en) 2014-05-08
WO2014070872A3 true WO2014070872A3 (en) 2014-06-26

Family

ID=49627040

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2013/067477 WO2014070872A2 (en) 2012-10-30 2013-10-30 System and method for multimodal interaction with reduced distraction in operating vehicles

Country Status (1)

Country Link
WO (1) WO2014070872A2 (en)

Families Citing this family (89)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8677377B2 (en) 2005-09-08 2014-03-18 Apple Inc. Method and apparatus for building an intelligent automated assistant
US9318108B2 (en) 2010-01-18 2016-04-19 Apple Inc. Intelligent automated assistant
US8977255B2 (en) 2007-04-03 2015-03-10 Apple Inc. Method and system for operating a multi-function portable electronic device using voice-activation
US8676904B2 (en) 2008-10-02 2014-03-18 Apple Inc. Electronic devices with voice command and contextual data processing capabilities
US10706373B2 (en) 2011-06-03 2020-07-07 Apple Inc. Performing actions associated with task items that represent tasks to perform
US10276170B2 (en) 2010-01-18 2019-04-30 Apple Inc. Intelligent automated assistant
US10417037B2 (en) 2012-05-15 2019-09-17 Apple Inc. Systems and methods for integrating third party services with a digital assistant
KR102516577B1 (en) 2013-02-07 2023-04-03 애플 인크. Voice trigger for a digital assistant
US10652394B2 (en) 2013-03-14 2020-05-12 Apple Inc. System and method for processing voicemail
US10748529B1 (en) 2013-03-15 2020-08-18 Apple Inc. Voice activated device for use with a voice-based digital assistant
KR101922663B1 (en) 2013-06-09 2018-11-28 애플 인크. Device, method, and graphical user interface for enabling conversation persistence across two or more instances of a digital assistant
US10176167B2 (en) 2013-06-09 2019-01-08 Apple Inc. System and method for inferring user intent from speech inputs
WO2015020942A1 (en) 2013-08-06 2015-02-12 Apple Inc. Auto-activating smart responses based on activities from remote devices
WO2015184186A1 (en) 2014-05-30 2015-12-03 Apple Inc. Multi-command single utterance input method
US10170123B2 (en) 2014-05-30 2019-01-01 Apple Inc. Intelligent assistant for home automation
US9430463B2 (en) 2014-05-30 2016-08-30 Apple Inc. Exemplar-based natural language processing
US9715875B2 (en) 2014-05-30 2017-07-25 Apple Inc. Reducing the need for manual start/end-pointing and trigger phrases
US9338493B2 (en) 2014-06-30 2016-05-10 Apple Inc. Intelligent automated assistant for TV user interactions
WO2016002406A1 (en) * 2014-07-04 2016-01-07 クラリオン株式会社 In-vehicle interactive system and in-vehicle information appliance
US9668121B2 (en) 2014-09-30 2017-05-30 Apple Inc. Social reminders
US10127911B2 (en) 2014-09-30 2018-11-13 Apple Inc. Speaker identification and unsupervised speaker adaptation techniques
US10152299B2 (en) 2015-03-06 2018-12-11 Apple Inc. Reducing response latency of intelligent automated assistants
US9721566B2 (en) 2015-03-08 2017-08-01 Apple Inc. Competing devices responding to voice triggers
US9886953B2 (en) 2015-03-08 2018-02-06 Apple Inc. Virtual assistant activation
DE102015005853A1 (en) * 2015-05-06 2016-11-10 Airbus Ds Electronics And Border Security Gmbh Electronic system and method for planning and executing a vehicle mission
US10460227B2 (en) 2015-05-15 2019-10-29 Apple Inc. Virtual assistant in a communication session
US10083688B2 (en) 2015-05-27 2018-09-25 Apple Inc. Device voice control for selecting a displayed affordance
US10200824B2 (en) 2015-05-27 2019-02-05 Apple Inc. Systems and methods for proactively identifying and surfacing relevant content on a touch-sensitive device
US9578173B2 (en) 2015-06-05 2017-02-21 Apple Inc. Virtual assistant aided communication with 3rd party service in a communication session
US20160378747A1 (en) 2015-06-29 2016-12-29 Apple Inc. Virtual assistant for media playback
US10671428B2 (en) 2015-09-08 2020-06-02 Apple Inc. Distributed personal assistant
US10331312B2 (en) 2015-09-08 2019-06-25 Apple Inc. Intelligent automated assistant in a media environment
US10740384B2 (en) 2015-09-08 2020-08-11 Apple Inc. Intelligent automated assistant for media search and playback
US10747498B2 (en) 2015-09-08 2020-08-18 Apple Inc. Zero latency digital assistant
US10691473B2 (en) 2015-11-06 2020-06-23 Apple Inc. Intelligent automated assistant in a messaging environment
US10956666B2 (en) 2015-11-09 2021-03-23 Apple Inc. Unconventional virtual assistant interactions
US10223066B2 (en) 2015-12-23 2019-03-05 Apple Inc. Proactive assistance based on dialog communication between devices
US11227589B2 (en) 2016-06-06 2022-01-18 Apple Inc. Intelligent list reading
US10586535B2 (en) 2016-06-10 2020-03-10 Apple Inc. Intelligent digital assistant in a multi-tasking environment
DK201670540A1 (en) 2016-06-11 2018-01-08 Apple Inc Application integration with a digital assistant
DK179415B1 (en) 2016-06-11 2018-06-14 Apple Inc Intelligent device arbitration and control
DE102016221564A1 (en) * 2016-10-13 2018-04-19 Bayerische Motoren Werke Aktiengesellschaft Multimodal dialogue in a motor vehicle
US10497346B2 (en) 2017-01-04 2019-12-03 2236008 Ontario Inc. Three-dimensional simulation system
US11204787B2 (en) 2017-01-09 2021-12-21 Apple Inc. Application integration with a digital assistant
DK201770383A1 (en) 2017-05-09 2018-12-14 Apple Inc. User interface for correcting recognition errors
DK180048B1 (en) 2017-05-11 2020-02-04 Apple Inc. MAINTAINING THE DATA PROTECTION OF PERSONAL INFORMATION
US10726832B2 (en) 2017-05-11 2020-07-28 Apple Inc. Maintaining privacy of personal information
DK179496B1 (en) 2017-05-12 2019-01-15 Apple Inc. USER-SPECIFIC Acoustic Models
DK179745B1 (en) 2017-05-12 2019-05-01 Apple Inc. SYNCHRONIZATION AND TASK DELEGATION OF A DIGITAL ASSISTANT
DK201770427A1 (en) 2017-05-12 2018-12-20 Apple Inc. Low-latency intelligent automated assistant
DK201770411A1 (en) 2017-05-15 2018-12-20 Apple Inc. Multi-modal interfaces
CN110651324B (en) * 2017-05-15 2023-05-16 苹果公司 Multi-modal interface
US20180336275A1 (en) 2017-05-16 2018-11-22 Apple Inc. Intelligent automated assistant for media exploration
US20180336892A1 (en) 2017-05-16 2018-11-22 Apple Inc. Detecting a trigger of a digital assistant
DK179560B1 (en) 2017-05-16 2019-02-18 Apple Inc. Far-field extension for digital assistant services
US10818288B2 (en) 2018-03-26 2020-10-27 Apple Inc. Natural assistant interaction
US11145294B2 (en) 2018-05-07 2021-10-12 Apple Inc. Intelligent automated assistant for delivering content from user experiences
US10928918B2 (en) 2018-05-07 2021-02-23 Apple Inc. Raise to speak
DK180639B1 (en) 2018-06-01 2021-11-04 Apple Inc DISABILITY OF ATTENTION-ATTENTIVE VIRTUAL ASSISTANT
DK179822B1 (en) 2018-06-01 2019-07-12 Apple Inc. Voice interaction at a primary device to access call functionality of a companion device
US10892996B2 (en) 2018-06-01 2021-01-12 Apple Inc. Variable latency device coordination
US11027822B2 (en) * 2018-06-21 2021-06-08 Rockwell Collins, Inc. Control system for touchless operation of mechanical input devices
US11010561B2 (en) 2018-09-27 2021-05-18 Apple Inc. Sentiment prediction from textual data
US11170166B2 (en) 2018-09-28 2021-11-09 Apple Inc. Neural typographical error modeling via generative adversarial networks
US10839159B2 (en) 2018-09-28 2020-11-17 Apple Inc. Named entity normalization in a spoken dialog system
US11462215B2 (en) 2018-09-28 2022-10-04 Apple Inc. Multi-modal inputs for voice commands
US11475898B2 (en) 2018-10-26 2022-10-18 Apple Inc. Low-latency multi-speaker speech recognition
US11638059B2 (en) 2019-01-04 2023-04-25 Apple Inc. Content playback on multiple devices
US20200218488A1 (en) * 2019-01-07 2020-07-09 Nuance Communications, Inc. Multimodal input processing for vehicle computer
US11348573B2 (en) 2019-03-18 2022-05-31 Apple Inc. Multimodality in digital assistant systems
US11423908B2 (en) 2019-05-06 2022-08-23 Apple Inc. Interpreting spoken requests
US11475884B2 (en) 2019-05-06 2022-10-18 Apple Inc. Reducing digital assistant latency when a language is incorrectly determined
DK201970509A1 (en) 2019-05-06 2021-01-15 Apple Inc Spoken notifications
US11307752B2 (en) 2019-05-06 2022-04-19 Apple Inc. User configurable task triggers
US11140099B2 (en) 2019-05-21 2021-10-05 Apple Inc. Providing message response suggestions
US11496600B2 (en) 2019-05-31 2022-11-08 Apple Inc. Remote execution of machine-learned models
DK201970510A1 (en) 2019-05-31 2021-02-11 Apple Inc Voice identification in digital assistant systems
DK180129B1 (en) 2019-05-31 2020-06-02 Apple Inc. User activity shortcut suggestions
US11289073B2 (en) 2019-05-31 2022-03-29 Apple Inc. Device text to speech
US11360641B2 (en) 2019-06-01 2022-06-14 Apple Inc. Increasing the relevance of new available information
US11227599B2 (en) 2019-06-01 2022-01-18 Apple Inc. Methods and user interfaces for voice-based control of electronic devices
US11348581B2 (en) 2019-07-12 2022-05-31 Qualcomm Incorporated Multi-modal user interface
WO2021056255A1 (en) 2019-09-25 2021-04-01 Apple Inc. Text detection using global geometry estimators
CN111008532B (en) * 2019-12-12 2023-09-12 广州小鹏汽车科技有限公司 Voice interaction method, vehicle and computer readable storage medium
US11183193B1 (en) 2020-05-11 2021-11-23 Apple Inc. Digital assistant hardware abstraction
US11061543B1 (en) 2020-05-11 2021-07-13 Apple Inc. Providing relevant data items based on context
US11755276B2 (en) 2020-05-12 2023-09-12 Apple Inc. Reducing description length based on confidence
US11490204B2 (en) 2020-07-20 2022-11-01 Apple Inc. Multi-device audio adjustment coordination
US11438683B2 (en) 2020-07-21 2022-09-06 Apple Inc. User identification using headphones

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002063599A1 (en) * 2001-02-05 2002-08-15 International Business Machines Corporation System and method for multi-modal focus detection, referential ambiguity resolution and mood classification using multi-modal input
US20050288934A1 (en) * 2004-06-29 2005-12-29 Canon Kabushiki Kaisha Multimodal input method
US20100241431A1 (en) * 2009-03-18 2010-09-23 Robert Bosch Gmbh System and Method for Multi-Modal Input Synchronization and Disambiguation

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7716056B2 (en) 2004-09-27 2010-05-11 Robert Bosch Corporation Method and system for interactive conversational dialogue for cognitively overloaded device users

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2002063599A1 (en) * 2001-02-05 2002-08-15 International Business Machines Corporation System and method for multi-modal focus detection, referential ambiguity resolution and mood classification using multi-modal input
US20050288934A1 (en) * 2004-06-29 2005-12-29 Canon Kabushiki Kaisha Multimodal input method
US20100241431A1 (en) * 2009-03-18 2010-09-23 Robert Bosch Gmbh System and Method for Multi-Modal Input Synchronization and Disambiguation

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
BASTIAN PFLEGING ET AL: "Multimodal interaction in the car", PROCEEDINGS OF THE 4TH INTERNATIONAL CONFERENCE ON AUTOMOTIVE USER INTERFACES AND INTERACTIVE VEHICULAR APPLICATIONS, AUTOMOTIVEUI '12, 19 October 2012 (2012-10-19), New York, New York, USA, pages 155, XP055114585, ISBN: 978-1-45-031751-1, DOI: 10.1145/2390256.2390282 *
MICHAEL FELD ET AL: "The Automotive Ontology: Managing Knowledge Inside the Vehicle and Sharing it Between Cars", PROCEEDINGS OF THE 3RD INTERNATIONAL CONFERENCE ON AUTOMOTIVE USER INTERFACES AND INTERACTIVE VEHICULAR APPLICATIONS, AUTOMOTIVEUI '11, 2 December 2011 (2011-12-02), NY, USA, pages 79 - 86, XP055114619, Retrieved from the Internet <URL:http://dl.acm.org/ft_gateway.cfm?id=2381429&ftid=1295256&dwn=1&CFID=314795502&CFTOKEN=10673073> [retrieved on 20140417], DOI: 10.1145/2381416.2381429 *

Also Published As

Publication number Publication date
WO2014070872A2 (en) 2014-05-08

Similar Documents

Publication Publication Date Title
WO2014070872A3 (en) System and method for multimodal interaction with reduced distraction in operating vehicles
EP3407534A4 (en) In-car computer system, vehicle, key generation device, management method, key generation method, and computer program
MX2020013467A (en) System, method, and app for managing waste services.
EP3346542A4 (en) Battery management device, battery system, and hybrid vehicle control system
CA2975617C (en) Programmatically determining location information in connection with a transport service
EP3083297A4 (en) Air conditioning system, method for controlling the same and hybrid vehicle
EP3506553A4 (en) Vehicle information collection system, vehicle-mounted computer, vehicle information collection device, vehicle information collection method, and computer program
EP2639752A3 (en) Method and apparatus for providing late return detection of a shared vehicle
EP3023716A4 (en) Method for controlling vehicle air-conditioning system, and vehicle air-conditioning system
GB2525338A8 (en) Range extender control
EP3057076A4 (en) Parked vehicle detection device, vehicle management system, and control method
WO2011162554A3 (en) Component for network system and method for controlling same
GB2514291A (en) Method and system for prioritizing a plurality of variable speed devices
HK1187311A1 (en) Vehicle, vehiclelntelligent key device, remote control system and method for driving vehicle
WO2012135737A3 (en) Rules execution platform system and method
IN2015DN01573A (en)
EP2947856A4 (en) Telematic s controller, vehicle, and telematics control method
EP2738478A3 (en) Intelligent comfort management using natural language processing to interface with a comfort system controller
EP3770108A4 (en) Remote operation system for industrial vehicles, industrial vehicle, remote operation device, remote operation program for industrial vehicles, and remote operation method for industrial vehicles
EP3334085A4 (en) Management device, management system, key generation device, key generation system, key management system, vehicle, management method, key generation method, and computer program
HK1187314A1 (en) Vehicle, vehiclelntelligent key device, remote control system and method for driving vehicle
EP3544224A4 (en) Communication system, vehicle, server device, communication method, and computer program
EP3276876A4 (en) Management device, vehicle, management method, and computer program
WO2015036416A3 (en) Device and method for controlling mobility
WO2013092187A3 (en) Method and device for adapting a voltage limit in an electrical system

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13795032

Country of ref document: EP

Kind code of ref document: A2

122 Ep: pct application non-entry in european phase

Ref document number: 13795032

Country of ref document: EP

Kind code of ref document: A2