CA3158979C - Headless task completion within digital personal assistants - Google Patents

Headless task completion within digital personal assistants Download PDF

Info

Publication number
CA3158979C
CA3158979C CA3158979A CA3158979A CA3158979C CA 3158979 C CA3158979 C CA 3158979C CA 3158979 A CA3158979 A CA 3158979A CA 3158979 A CA3158979 A CA 3158979A CA 3158979 C CA3158979 C CA 3158979C
Authority
CA
Canada
Prior art keywords
application
user
voice
task
personal assistant
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CA3158979A
Other languages
English (en)
French (fr)
Other versions
CA3158979A1 (en
Inventor
Vishwac Sena KANNAN
Aleksandar Uzelac
Daniel J. Hwang
Robert L. Chambers
Thomas Soemo
Adina Magdalena TRUFINESCU
Khuram SHAHID
Ali Emami
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Microsoft Technology Licensing LLC
Original Assignee
Microsoft Technology Licensing LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Microsoft Technology Licensing LLC filed Critical Microsoft Technology Licensing LLC
Publication of CA3158979A1 publication Critical patent/CA3158979A1/en
Application granted granted Critical
Publication of CA3158979C publication Critical patent/CA3158979C/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44—Arrangements for executing specific programs
    • G06F9/451—Execution arrangements for user interfaces
    • G06F9/453—Help systems
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G06F9/4806—Task transfer initiation or dispatching
    • G06F9/4843—Task transfer initiation or dispatching by program, e.g. task dispatcher, supervisor, operating system
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/162—Interface to dedicated audio devices, e.g. audio drivers, interface to CODECs
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00—Arrangements for program control, e.g. control units
    • G06F9/06—Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46—Multiprogramming arrangements
    • G06F9/48—Program initiating; Program switching, e.g. by interrupt
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/08—Speech classification or search
    • G10L15/18—Speech classification or search using natural language modelling
    • G10L15/1822—Parsing for meaning understanding
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223—Execution procedure of a spoken command

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • Computational Linguistics (AREA)
  • Acoustics & Sound (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • User Interface Of Digital Computer (AREA)
  • Circuit For Audible Band Transducer (AREA)
  • Collating Specific Patterns (AREA)
  • Machine Translation (AREA)
  • Document Processing Apparatus (AREA)
CA3158979A 2015-01-09 2015-12-29 Headless task completion within digital personal assistants Active CA3158979C (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US14/593,584 US9959129B2 (en) 2015-01-09 2015-01-09 Headless task completion within digital personal assistants
US14/593,584 2015-01-09
CA2970725A CA2970725C (en) 2015-01-09 2015-12-29 Headless task completion within digital personal assistants

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
CA2970725A Division CA2970725C (en) 2015-01-09 2015-12-29 Headless task completion within digital personal assistants

Publications (2)

Publication Number Publication Date
CA3158979A1 CA3158979A1 (en) 2016-07-14
CA3158979C true CA3158979C (en) 2024-06-04

Family

ID=55305053

Family Applications (2)

Application Number Title Priority Date Filing Date
CA3158979A Active CA3158979C (en) 2015-01-09 2015-12-29 Headless task completion within digital personal assistants
CA2970725A Active CA2970725C (en) 2015-01-09 2015-12-29 Headless task completion within digital personal assistants

Family Applications After (1)

Application Number Title Priority Date Filing Date
CA2970725A Active CA2970725C (en) 2015-01-09 2015-12-29 Headless task completion within digital personal assistants

Country Status (18)

Country Link
US (1) US9959129B2 (enExample)
EP (2) EP3584787B1 (enExample)
JP (1) JP6666920B2 (enExample)
KR (1) KR102490776B1 (enExample)
CN (1) CN107111516B (enExample)
AU (1) AU2015375326B2 (enExample)
BR (1) BR112017013524B1 (enExample)
CA (2) CA3158979C (enExample)
CL (1) CL2017001755A1 (enExample)
CO (1) CO2017007028A2 (enExample)
IL (1) IL252458B (enExample)
MX (1) MX2017008926A (enExample)
MY (1) MY186956A (enExample)
NZ (2) NZ732357A (enExample)
PH (1) PH12017550012A1 (enExample)
RU (1) RU2710984C2 (enExample)
SG (1) SG11201705282XA (enExample)
WO (1) WO2016111881A1 (enExample)

Families Citing this family (206)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8677377B2 (en) 2005-09-08 2014-03-18 Apple Inc. Method and apparatus for building an intelligent automated assistant
US9318108B2 (en) 2010-01-18 2016-04-19 Apple Inc. Intelligent automated assistant
US8977255B2 (en) 2007-04-03 2015-03-10 Apple Inc. Method and system for operating a multi-function portable electronic device using voice-activation
US8676904B2 (en) 2008-10-02 2014-03-18 Apple Inc. Electronic devices with voice command and contextual data processing capabilities
US20120309363A1 (en) 2011-06-03 2012-12-06 Apple Inc. Triggering notifications associated with tasks items that represent tasks to perform
US10276170B2 (en) 2010-01-18 2019-04-30 Apple Inc. Intelligent automated assistant
US8682667B2 (en) 2010-02-25 2014-03-25 Apple Inc. User profiling for selecting user specific voice input processing information
US11087424B1 (en) 2011-06-24 2021-08-10 Google Llc Image recognition-based content item selection
US10972530B2 (en) * 2016-12-30 2021-04-06 Google Llc Audio-based data structure generation
US8688514B1 (en) 2011-06-24 2014-04-01 Google Inc. Ad selection using image data
US10630751B2 (en) 2016-12-30 2020-04-21 Google Llc Sequence dependent data message consolidation in a voice activated computer network environment
US10956485B2 (en) 2011-08-31 2021-03-23 Google Llc Retargeting in a search environment
US11093692B2 (en) 2011-11-14 2021-08-17 Google Llc Extracting audiovisual features from digital components
US10586127B1 (en) 2011-11-14 2020-03-10 Google Llc Extracting audiovisual features from content elements on online documents
US10417037B2 (en) 2012-05-15 2019-09-17 Apple Inc. Systems and methods for integrating third party services with a digital assistant
US20140136508A1 (en) 2012-11-09 2014-05-15 Palo Alto Research Center Incorporated Computer-Implemented System And Method For Providing Website Navigation Recommendations
KR20250004158A (ko) 2013-02-07 2025-01-07 애플 인크. 디지털 어시스턴트를 위한 음성 트리거
US10652394B2 (en) 2013-03-14 2020-05-12 Apple Inc. System and method for processing voicemail
US10748529B1 (en) 2013-03-15 2020-08-18 Apple Inc. Voice activated device for use with a voice-based digital assistant
US11030239B2 (en) 2013-05-31 2021-06-08 Google Llc Audio based entity-action pair based selection
KR101922663B1 (ko) 2013-06-09 2018-11-28 애플 인크. 디지털 어시스턴트의 둘 이상의 인스턴스들에 걸친 대화 지속성을 가능하게 하기 위한 디바이스, 방법 및 그래픽 사용자 인터페이스
US10176167B2 (en) 2013-06-09 2019-01-08 Apple Inc. System and method for inferring user intent from speech inputs
WO2015020942A1 (en) 2013-08-06 2015-02-12 Apple Inc. Auto-activating smart responses based on activities from remote devices
US9703757B2 (en) 2013-09-30 2017-07-11 Google Inc. Automatically determining a size for a content item for a web page
US10614153B2 (en) 2013-09-30 2020-04-07 Google Llc Resource size-based content item selection
US10431209B2 (en) 2016-12-30 2019-10-01 Google Llc Feedback controller for data transmissions
US10296160B2 (en) 2013-12-06 2019-05-21 Apple Inc. Method for extracting salient dialog usage from live data
US9966065B2 (en) 2014-05-30 2018-05-08 Apple Inc. Multi-command single utterance input method
US9430463B2 (en) 2014-05-30 2016-08-30 Apple Inc. Exemplar-based natural language processing
US10170123B2 (en) 2014-05-30 2019-01-01 Apple Inc. Intelligent assistant for home automation
US9633004B2 (en) 2014-05-30 2017-04-25 Apple Inc. Better resolution when referencing to concepts
US9715875B2 (en) 2014-05-30 2017-07-25 Apple Inc. Reducing the need for manual start/end-pointing and trigger phrases
US9338493B2 (en) 2014-06-30 2016-05-10 Apple Inc. Intelligent automated assistant for TV user interactions
US10127911B2 (en) 2014-09-30 2018-11-13 Apple Inc. Speaker identification and unsupervised speaker adaptation techniques
US9668121B2 (en) 2014-09-30 2017-05-30 Apple Inc. Social reminders
US9898170B2 (en) 2014-12-10 2018-02-20 International Business Machines Corporation Establishing user specified interaction modes in a question answering dialogue
US11127055B1 (en) * 2015-02-27 2021-09-21 State Farm Mutual Automobile Insurance Company System and method for navigating an automated process using audio and tactile feedback
US10152299B2 (en) 2015-03-06 2018-12-11 Apple Inc. Reducing response latency of intelligent automated assistants
US9886953B2 (en) 2015-03-08 2018-02-06 Apple Inc. Virtual assistant activation
US9721566B2 (en) 2015-03-08 2017-08-01 Apple Inc. Competing devices responding to voice triggers
US10460227B2 (en) 2015-05-15 2019-10-29 Apple Inc. Virtual assistant in a communication session
US10083688B2 (en) 2015-05-27 2018-09-25 Apple Inc. Device voice control for selecting a displayed affordance
US10200824B2 (en) 2015-05-27 2019-02-05 Apple Inc. Systems and methods for proactively identifying and surfacing relevant content on a touch-sensitive device
US9578173B2 (en) 2015-06-05 2017-02-21 Apple Inc. Virtual assistant aided communication with 3rd party service in a communication session
US20160378747A1 (en) 2015-06-29 2016-12-29 Apple Inc. Virtual assistant for media playback
US10650067B2 (en) 2015-08-13 2020-05-12 Samsung Electronics Co., Ltd. Cloud-enabled architecture for on-demand native application crawling
US10331312B2 (en) 2015-09-08 2019-06-25 Apple Inc. Intelligent automated assistant in a media environment
US10747498B2 (en) 2015-09-08 2020-08-18 Apple Inc. Zero latency digital assistant
US10671428B2 (en) 2015-09-08 2020-06-02 Apple Inc. Distributed personal assistant
US10740384B2 (en) 2015-09-08 2020-08-11 Apple Inc. Intelligent automated assistant for media search and playback
US11587559B2 (en) 2015-09-30 2023-02-21 Apple Inc. Intelligent device identification
US10691473B2 (en) 2015-11-06 2020-06-23 Apple Inc. Intelligent automated assistant in a messaging environment
US10956666B2 (en) 2015-11-09 2021-03-23 Apple Inc. Unconventional virtual assistant interactions
KR20170072652A (ko) * 2015-12-17 2017-06-27 삼성전자주식회사 디스플레이 장치 및 디스플레이 방법
US10079021B1 (en) * 2015-12-18 2018-09-18 Amazon Technologies, Inc. Low latency audio interface
US10223066B2 (en) 2015-12-23 2019-03-05 Apple Inc. Proactive assistance based on dialog communication between devices
US10120954B2 (en) 2015-12-31 2018-11-06 Samsung Electronics Co., Ltd. Transformation and presentation of on-demand native application crawling results
US10229677B2 (en) * 2016-04-19 2019-03-12 International Business Machines Corporation Smart launching mobile applications with preferred user interface (UI) languages
US10783178B2 (en) 2016-05-17 2020-09-22 Google Llc Generating a personal database entry for a user based on natural language user interface input of the user and generating output based on the entry in response to further natural language user interface input of the user
US11227589B2 (en) 2016-06-06 2022-01-18 Apple Inc. Intelligent list reading
US12223282B2 (en) 2016-06-09 2025-02-11 Apple Inc. Intelligent automated assistant in a home environment
US10586535B2 (en) 2016-06-10 2020-03-10 Apple Inc. Intelligent digital assistant in a multi-tasking environment
DK179415B1 (en) 2016-06-11 2018-06-14 Apple Inc Intelligent device arbitration and control
DK201670540A1 (en) 2016-06-11 2018-01-08 Apple Inc Application integration with a digital assistant
US12197817B2 (en) 2016-06-11 2025-01-14 Apple Inc. Intelligent device arbitration and control
US11093834B2 (en) * 2016-07-06 2021-08-17 Palo Alto Research Center Incorporated Computer-implemented system and method for predicting activity outcome based on user attention
US10474753B2 (en) 2016-09-07 2019-11-12 Apple Inc. Language identification using recurrent neural networks
US10395652B2 (en) * 2016-09-20 2019-08-27 Allstate Insurance Company Personal information assistant computing system
US11204787B2 (en) 2017-01-09 2021-12-21 Apple Inc. Application integration with a digital assistant
US11010601B2 (en) 2017-02-14 2021-05-18 Microsoft Technology Licensing, Llc Intelligent assistant device communicating non-verbal cues
US11100384B2 (en) 2017-02-14 2021-08-24 Microsoft Technology Licensing, Llc Intelligent device user interactions
US10467509B2 (en) 2017-02-14 2019-11-05 Microsoft Technology Licensing, Llc Computationally-efficient human-identifying smart assistant computer
US10574825B2 (en) * 2017-02-15 2020-02-25 Microsoft Technology Licensing, Llc Assisted-communication with intelligent personal assistant
US11323665B2 (en) * 2017-03-31 2022-05-03 Ecolink Intelligent Technology, Inc. Method and apparatus for interaction with an intelligent personal assistant
KR102339819B1 (ko) * 2017-04-05 2021-12-15 삼성전자주식회사 프레임워크를 이용한 자연어 표현 생성 방법 및 장치
US10726056B2 (en) * 2017-04-10 2020-07-28 Sap Se Speech-based database access
US11170768B2 (en) * 2017-04-17 2021-11-09 Samsung Electronics Co., Ltd Device for performing task corresponding to user utterance
KR102414122B1 (ko) * 2017-04-19 2022-06-29 삼성전자주식회사 사용자 발화를 처리하는 전자 장치 및 그 동작 방법
US20180316634A1 (en) * 2017-04-26 2018-11-01 Microsoft Technology Licensing, Llc Extending application functionality via conversational interfaces
KR102298947B1 (ko) 2017-04-28 2021-09-08 삼성전자주식회사 음성 데이터 처리 방법 및 이를 지원하는 전자 장치
KR102380717B1 (ko) * 2017-04-30 2022-03-31 삼성전자주식회사 사용자 발화를 처리하는 전자 장치 및 이 전자 장치의 제어 방법
DK201770383A1 (en) 2017-05-09 2018-12-14 Apple Inc. USER INTERFACE FOR CORRECTING RECOGNITION ERRORS
DK180048B1 (en) 2017-05-11 2020-02-04 Apple Inc. MAINTAINING THE DATA PROTECTION OF PERSONAL INFORMATION
US10395654B2 (en) 2017-05-11 2019-08-27 Apple Inc. Text normalization based on a data-driven learning network
US10726832B2 (en) 2017-05-11 2020-07-28 Apple Inc. Maintaining privacy of personal information
DK201770429A1 (en) 2017-05-12 2018-12-14 Apple Inc. LOW-LATENCY INTELLIGENT AUTOMATED ASSISTANT
US11301477B2 (en) 2017-05-12 2022-04-12 Apple Inc. Feedback analysis of a digital assistant
DK179496B1 (en) 2017-05-12 2019-01-15 Apple Inc. USER-SPECIFIC Acoustic Models
DK179745B1 (en) 2017-05-12 2019-05-01 Apple Inc. SYNCHRONIZATION AND TASK DELEGATION OF A DIGITAL ASSISTANT
US11436417B2 (en) 2017-05-15 2022-09-06 Google Llc Providing access to user-controlled resources by automated assistants
DK201770411A1 (en) 2017-05-15 2018-12-20 Apple Inc. Multi-modal interfaces
US10127227B1 (en) 2017-05-15 2018-11-13 Google Llc Providing access to user-controlled resources by automated assistants
US10311144B2 (en) 2017-05-16 2019-06-04 Apple Inc. Emoji word sense disambiguation
US20180336892A1 (en) 2017-05-16 2018-11-22 Apple Inc. Detecting a trigger of a digital assistant
US11048995B2 (en) 2017-05-16 2021-06-29 Google Llc Delayed responses by computational assistant
DK179549B1 (en) 2017-05-16 2019-02-12 Apple Inc. FAR-FIELD EXTENSION FOR DIGITAL ASSISTANT SERVICES
US10303715B2 (en) 2017-05-16 2019-05-28 Apple Inc. Intelligent automated assistant for media exploration
US10679192B2 (en) * 2017-05-25 2020-06-09 Microsoft Technology Licensing, Llc Assigning tasks and monitoring task performance based on context extracted from a shared contextual graph
US10769138B2 (en) 2017-06-13 2020-09-08 International Business Machines Corporation Processing context-based inquiries for knowledge retrieval
KR102365649B1 (ko) 2017-06-28 2022-02-22 삼성전자주식회사 화면 제어 방법 및 이를 지원하는 전자 장치
US10449440B2 (en) * 2017-06-30 2019-10-22 Electronic Arts Inc. Interactive voice-controlled companion application for a video game
US10366691B2 (en) * 2017-07-11 2019-07-30 Samsung Electronics Co., Ltd. System and method for voice command context
US11004444B2 (en) * 2017-09-08 2021-05-11 Amazon Technologies, Inc. Systems and methods for enhancing user experience by communicating transient errors
US10621317B1 (en) 2017-09-14 2020-04-14 Electronic Arts Inc. Audio-based device authentication system
US10423873B2 (en) * 2017-12-01 2019-09-24 International Business Machines Corporation Information flow analysis for conversational agents
US11113026B2 (en) * 2017-12-08 2021-09-07 Toonimo, Inc. System and method for voice-directed website walk-through
US10629192B1 (en) 2018-01-09 2020-04-21 Electronic Arts Inc. Intelligent personalized speech recognition
US20190213528A1 (en) * 2018-01-10 2019-07-11 Microsoft Technology Licensing, Llc Digital assistant task management
US10643632B2 (en) * 2018-01-12 2020-05-05 Wells Fargo Bank, N.A. Automated voice assistant personality selector
CN117348988A (zh) * 2018-01-12 2024-01-05 谷歌有限责任公司 用于暂停和恢复虚拟助理的系统、方法以及装置
US20210064640A1 (en) * 2018-01-17 2021-03-04 Sony Corporation Information processing apparatus and information processing method
US10592604B2 (en) 2018-03-12 2020-03-17 Apple Inc. Inverse text normalization for automatic speech recognition
KR102617265B1 (ko) 2018-03-13 2023-12-26 삼성전자주식회사 사용자 음성 입력을 처리하는 장치
CN108521500A (zh) * 2018-03-13 2018-09-11 努比亚技术有限公司 一种语音场景控制方法、设备及计算机可读存储介质
US10818288B2 (en) 2018-03-26 2020-10-27 Apple Inc. Natural assistant interaction
CN110321201A (zh) * 2018-03-29 2019-10-11 努比亚技术有限公司 一种后台程序处理方法、终端及计算机可读存储介质
BR102019007123A2 (pt) * 2018-04-16 2019-10-29 Panasonic Avionics Corp assistentes digitais e métodos associados para um veículo de transporte
US11307880B2 (en) 2018-04-20 2022-04-19 Meta Platforms, Inc. Assisting users with personalized and contextual communication content
US11676220B2 (en) 2018-04-20 2023-06-13 Meta Platforms, Inc. Processing multimodal user input for assistant systems
US11715042B1 (en) 2018-04-20 2023-08-01 Meta Platforms Technologies, Llc Interpretability of deep reinforcement learning models in assistant systems
US11886473B2 (en) 2018-04-20 2024-01-30 Meta Platforms, Inc. Intent identification for agent matching by assistant systems
US10963273B2 (en) 2018-04-20 2021-03-30 Facebook, Inc. Generating personalized content summaries for users
US10824432B2 (en) * 2018-04-30 2020-11-03 Servicenow, Inc. Systems and methods for providing multiple console sessions that enable line-by-line execution of scripts on a server application
EP3610375B1 (en) 2018-05-07 2024-04-10 Google LLC Platform selection for performing requested actions in audio-based computing environments
US11087761B2 (en) 2018-05-07 2021-08-10 Google Llc Platform selection for performing requested actions in audio-based computing environments
US10928918B2 (en) 2018-05-07 2021-02-23 Apple Inc. Raise to speak
CN110710158B (zh) * 2018-05-07 2022-08-09 谷歌有限责任公司 验证与数字助理应用交接的代理的操作状态
US11145294B2 (en) 2018-05-07 2021-10-12 Apple Inc. Intelligent automated assistant for delivering content from user experiences
KR102717736B1 (ko) * 2018-05-07 2024-10-16 구글 엘엘씨 연결 디바이스를 제어하기 위한 복합 그래픽 어시스턴트 인터페이스를 제공하는 방법, 시스템, 및 장치
US11487501B2 (en) 2018-05-16 2022-11-01 Snap Inc. Device control using audio data
KR20190136832A (ko) 2018-05-31 2019-12-10 휴렛-팩커드 디벨롭먼트 컴퍼니, 엘.피. 음성 명령을 프린팅 서비스를 지원하는 텍스트 코드 블록들로 변환
DK179822B1 (da) 2018-06-01 2019-07-12 Apple Inc. Voice interaction at a primary device to access call functionality of a companion device
US10892996B2 (en) 2018-06-01 2021-01-12 Apple Inc. Variable latency device coordination
DK201870355A1 (en) 2018-06-01 2019-12-16 Apple Inc. VIRTUAL ASSISTANT OPERATION IN MULTI-DEVICE ENVIRONMENTS
DK180639B1 (en) 2018-06-01 2021-11-04 Apple Inc DISABILITY OF ATTENTION-ATTENTIVE VIRTUAL ASSISTANT
DK201870357A1 (en) * 2018-06-03 2019-12-20 Apple Inc. ACCELERATED TASK PERFORMANCE
US10496705B1 (en) 2018-06-03 2019-12-03 Apple Inc. Accelerated task performance
AU2019100576C4 (en) * 2018-06-03 2020-01-30 Apple Inc. Accelerated task performance
CN110858481B (zh) * 2018-08-07 2024-04-30 三星电子株式会社 用于处理用户语音话语的系统和用于操作该系统的方法
WO2020032927A1 (en) 2018-08-07 2020-02-13 Google Llc Assembling and evaluating automated assistant responses for privacy concerns
KR102607666B1 (ko) * 2018-08-08 2023-11-29 삼성전자 주식회사 전자 장치에서 사용자 의도 확인을 위한 피드백 제공 방법 및 장치
US11010561B2 (en) 2018-09-27 2021-05-18 Apple Inc. Sentiment prediction from textual data
US11462215B2 (en) 2018-09-28 2022-10-04 Apple Inc. Multi-modal inputs for voice commands
US11170166B2 (en) 2018-09-28 2021-11-09 Apple Inc. Neural typographical error modeling via generative adversarial networks
US10839159B2 (en) 2018-09-28 2020-11-17 Apple Inc. Named entity normalization in a spoken dialog system
US11205422B2 (en) 2018-10-02 2021-12-21 International Business Machines Corporation Methods and systems for managing chatbots with data access
US11475898B2 (en) 2018-10-26 2022-10-18 Apple Inc. Low-latency multi-speaker speech recognition
KR20240146091A (ko) 2018-11-21 2024-10-07 구글 엘엘씨 서로 다른 데이터 소스들에 대한 질의의 응답들의 통합
KR102701868B1 (ko) 2018-12-03 2024-09-03 삼성전자주식회사 전자 장치 및 전자 장치의 제어 방법
US10878805B2 (en) 2018-12-06 2020-12-29 Microsoft Technology Licensing, Llc Expediting interaction with a digital assistant by predicting user responses
JP7286772B2 (ja) 2018-12-28 2023-06-05 グーグル エルエルシー 選択されたサジェスチョンによる自動アシスタントへのボイス入力の補足
US11638059B2 (en) 2019-01-04 2023-04-25 Apple Inc. Content playback on multiple devices
US11380214B2 (en) * 2019-02-19 2022-07-05 International Business Machines Corporation Memory retention enhancement for electronic text
EP3895161B1 (en) 2019-02-20 2023-07-26 Google LLC Utilizing pre-event and post-event input streams to engage an automated assistant
JP7280066B2 (ja) * 2019-03-07 2023-05-23 本田技研工業株式会社 エージェント装置、エージェント装置の制御方法、およびプログラム
US11348573B2 (en) 2019-03-18 2022-05-31 Apple Inc. Multimodality in digital assistant systems
WO2020226619A1 (en) * 2019-05-06 2020-11-12 Google Llc Rendering visual components on applications in response to voice commands
DK201970509A1 (en) 2019-05-06 2021-01-15 Apple Inc Spoken notifications
US11423908B2 (en) 2019-05-06 2022-08-23 Apple Inc. Interpreting spoken requests
US11475884B2 (en) 2019-05-06 2022-10-18 Apple Inc. Reducing digital assistant latency when a language is incorrectly determined
US11307752B2 (en) 2019-05-06 2022-04-19 Apple Inc. User configurable task triggers
US11140099B2 (en) 2019-05-21 2021-10-05 Apple Inc. Providing message response suggestions
CN113841172A (zh) * 2019-05-28 2021-12-24 瑞典爱立信有限公司 用于处置消息的网络节点以及在其中执行的方法
DK180129B1 (en) 2019-05-31 2020-06-02 Apple Inc. User activity shortcut suggestions
US11496600B2 (en) 2019-05-31 2022-11-08 Apple Inc. Remote execution of machine-learned models
DK201970510A1 (en) 2019-05-31 2021-02-11 Apple Inc Voice identification in digital assistant systems
US11289073B2 (en) 2019-05-31 2022-03-29 Apple Inc. Device text to speech
US11360641B2 (en) 2019-06-01 2022-06-14 Apple Inc. Increasing the relevance of new available information
US11468890B2 (en) 2019-06-01 2022-10-11 Apple Inc. Methods and user interfaces for voice-based control of electronic devices
CN118885147A (zh) * 2019-06-05 2024-11-01 谷歌有限责任公司 用于基于数字助理的应用的动作验证
US20200388280A1 (en) 2019-06-05 2020-12-10 Google Llc Action validation for digital assistant-based applications
US10926173B2 (en) 2019-06-10 2021-02-23 Electronic Arts Inc. Custom voice control of video game character
US10901689B1 (en) * 2019-07-11 2021-01-26 International Business Machines Corporation Dynamic augmented reality interface creation
EP4002087A4 (en) * 2019-07-19 2023-04-12 LG Electronics Inc. DISPLAY DEVICE AND ARTIFICIAL INTELLIGENCE SERVER CAPABLE OF CONTROLLING A HOME APPLIANCE VIA A USER'S VOICE
CN118535073A (zh) 2019-07-19 2024-08-23 谷歌有限责任公司 用于复杂应用gui的自动化助理控制的压缩口头话语
US11488406B2 (en) 2019-09-25 2022-11-01 Apple Inc. Text detection using global geometry estimators
US11567788B1 (en) 2019-10-18 2023-01-31 Meta Platforms, Inc. Generating proactive reminders for assistant systems
US11861674B1 (en) 2019-10-18 2024-01-02 Meta Platforms Technologies, Llc Method, one or more computer-readable non-transitory storage media, and a system for generating comprehensive information for products of interest by assistant systems
US11574634B2 (en) * 2019-11-27 2023-02-07 Google Llc Interfacing with applications via dynamically updating natural language processing
WO2021107951A1 (en) * 2019-11-27 2021-06-03 Google Llc Interfacing with applications via dynamically updating natural language processing
CN111261156A (zh) * 2019-12-30 2020-06-09 北京梧桐车联科技有限责任公司 语音采集方法、装置及计算机可读存储介质
US11907298B2 (en) * 2020-02-04 2024-02-20 Intuition Robotics, Ltd. System and method thereof for automatically updating a decision-making model of an electronic social agent by actively collecting at least a user response
US11782986B2 (en) 2020-03-27 2023-10-10 Trushant Mehta Interactive query based network communication through a media device
US12301635B2 (en) 2020-05-11 2025-05-13 Apple Inc. Digital assistant hardware abstraction
US11061543B1 (en) 2020-05-11 2021-07-13 Apple Inc. Providing relevant data items based on context
US11043220B1 (en) 2020-05-11 2021-06-22 Apple Inc. Digital assistant hardware abstraction
US11755276B2 (en) 2020-05-12 2023-09-12 Apple Inc. Reducing description length based on confidence
CN114072875B (zh) * 2020-05-29 2025-10-21 华为技术有限公司 一种语音信号处理方法及其相关设备
US11556707B2 (en) * 2020-06-18 2023-01-17 Google Llc Fulfillment of actionable requests ahead of a user selecting a particular autocomplete suggestion for completing a current user input
US11490204B2 (en) 2020-07-20 2022-11-01 Apple Inc. Multi-device audio adjustment coordination
US11438683B2 (en) 2020-07-21 2022-09-06 Apple Inc. User identification using headphones
US11587568B2 (en) * 2020-12-08 2023-02-21 Google Llc Streaming action fulfillment based on partial hypotheses
CN112698872A (zh) * 2020-12-21 2021-04-23 北京百度网讯科技有限公司 语音数据处理的方法、装置、设备及存储介质
US11947783B2 (en) * 2021-01-25 2024-04-02 Google Llc Undoing application operation(s) via user interaction(s) with an automated assistant
US11978449B2 (en) 2021-03-02 2024-05-07 Samsung Electronics Co., Ltd. Electronic device for processing user utterance and operation method therefor
US20220293128A1 (en) * 2021-03-10 2022-09-15 Comcast Cable Communications, Llc Systems and methods for improved speech and command detection
US20220366338A1 (en) * 2021-05-13 2022-11-17 At&T Intellectual Property I, L.P. Contextual presentation of multiple steps in performing a task
US12327553B2 (en) * 2021-05-17 2025-06-10 Google Llc Enabling natural conversations for an automated assistant
US12327557B2 (en) * 2021-05-18 2025-06-10 Google Llc Indexing application actions for voice-based execution
US12260858B2 (en) * 2021-07-21 2025-03-25 Google Llc Transferring dialog data from an initially invoked automated assistant to a subsequently invoked automated assistant
US12021806B1 (en) 2021-09-21 2024-06-25 Apple Inc. Intelligent message delivery
US12198690B1 (en) * 2021-11-23 2025-01-14 Amazon Technologies, Inc. Voice-based content attribution for speech processing applications
US20230229865A1 (en) * 2022-01-14 2023-07-20 Target Brands, Inc. Extensible digital assistant interface using natural language processing to respond to user intent
US20230297484A1 (en) * 2022-03-17 2023-09-21 International Business Machines Corporation Predicting target applications
US12394412B2 (en) 2022-05-10 2025-08-19 Apple Inc. Voice-activated shortcut registration
CN119937845A (zh) * 2023-10-31 2025-05-06 北京字跳网络技术有限公司 用于处理任务的方法、装置、设备和存储介质

Family Cites Families (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3725566B2 (ja) * 1992-12-28 2005-12-14 株式会社東芝 音声認識インターフェース
US6791529B2 (en) * 2001-12-13 2004-09-14 Koninklijke Philips Electronics N.V. UI with graphics-assisted voice control system
US7003464B2 (en) * 2003-01-09 2006-02-21 Motorola, Inc. Dialog recognition and control in a voice browser
DE102005016853A1 (de) * 2005-04-12 2006-10-19 Siemens Ag Verfahren zur Kontrolle von sprachgesteuerten Applikationen und zugehöriges Kontrollsystem
US8219134B2 (en) * 2006-12-13 2012-07-10 Quickplay Media Inc. Seamlessly switching among unicast, multicast, and broadcast mobile media content
US8595642B1 (en) 2007-10-04 2013-11-26 Great Northern Research, LLC Multiple shell multi faceted graphical user interface
US20130176328A1 (en) * 2009-12-25 2013-07-11 Nokia Corporation Method and apparatus for persistent display of navigation content
US20120209413A1 (en) 2011-02-14 2012-08-16 Microsoft Corporation Background Audio on Mobile Devices
US9063629B2 (en) * 2011-10-31 2015-06-23 Nokia Technologies Oy Responding to a received message in a locked user interaction mode
CN102510426A (zh) 2011-11-29 2012-06-20 安徽科大讯飞信息科技股份有限公司 个人助理应用访问方法及系统
US9052197B2 (en) 2012-06-05 2015-06-09 Apple Inc. Providing navigation instructions while device is in locked mode
CN103593230B (zh) 2012-08-13 2017-05-10 百度在线网络技术（北京）有限公司 移动终端的后台任务控制方法及移动终端
US10276157B2 (en) * 2012-10-01 2019-04-30 Nuance Communications, Inc. Systems and methods for providing a voice agent user interface
US9384732B2 (en) 2013-03-14 2016-07-05 Microsoft Technology Licensing, Llc Voice command definitions used in launching application with a command
US8767948B1 (en) * 2013-03-15 2014-07-01 Genesys Telecommunications Laboratories, Inc. Back office services of an intelligent automated agent for a contact center
US9658740B2 (en) * 2013-03-15 2017-05-23 Apple Inc. Device, method, and graphical user interface for managing concurrently open software applications
KR101505127B1 (ko) * 2013-03-15 2015-03-26 주식회사 팬택 음성 명령에 의한 오브젝트 실행 장치 및 방법
CN103257787B (zh) 2013-05-16 2016-07-13 小米科技有限责任公司 一种语音助手应用的开启方法和装置
WO2014197335A1 (en) 2013-06-08 2014-12-11 Apple Inc. Interpreting and acting upon commands that involve sharing information with remote devices
US9749209B2 (en) * 2013-11-01 2017-08-29 The Nielsen Company (Us), Llc Methods and apparatus to credit background applications
US9741343B1 (en) * 2013-12-19 2017-08-22 Amazon Technologies, Inc. Voice interaction application selection

Also Published As

Publication number Publication date
KR102490776B1 (ko) 2023-01-19
MY186956A (en) 2021-08-26
AU2015375326B2 (en) 2020-08-27
KR20170103801A (ko) 2017-09-13
CN107111516B (zh) 2021-07-30
CA2970725C (en) 2023-05-23
BR112017013524B1 (pt) 2023-03-07
BR112017013524A2 (pt) 2018-01-02
US20160203002A1 (en) 2016-07-14
JP6666920B2 (ja) 2020-03-18
CO2017007028A2 (es) 2017-09-29
NZ732357A (en) 2022-10-28
CL2017001755A1 (es) 2018-02-16
IL252458B (en) 2019-11-28
EP3243199B1 (en) 2019-06-19
AU2015375326A1 (en) 2017-06-15
RU2710984C2 (ru) 2020-01-14
EP3584787A1 (en) 2019-12-25
RU2017124103A (ru) 2019-01-09
PH12017550012A1 (en) 2017-10-18
CA3158979A1 (en) 2016-07-14
MX2017008926A (es) 2017-10-11
CN107111516A (zh) 2017-08-29
EP3243199A1 (en) 2017-11-15
SG11201705282XA (en) 2017-07-28
EP3584787B1 (en) 2022-02-23
WO2016111881A1 (en) 2016-07-14
CA2970725A1 (en) 2016-07-14
US9959129B2 (en) 2018-05-01
NZ770836A (en) 2022-10-28
RU2017124103A3 (enExample) 2019-07-17
JP2018511095A (ja) 2018-04-19
IL252458A0 (en) 2017-07-31

Similar Documents

Publication Publication Date Title
CA3158979C (en) Headless task completion within digital personal assistants
US12293763B2 (en) Application integration with a digital assistant
US11500672B2 (en) Distributed personal assistant
CN107112015B (zh) 发现第三方启用语音的资源的能力
JP2020144932A (ja) 提案される音声ベースのアクションクエリの提供
HK1240350A1 (en) Headless task completion within digital personal assistants
HK1240350B (zh) 数字个人助理内的无头任务完成

Legal Events

Date Code Title Description
EEER Examination request

Effective date: 20220516

EEER Examination request

Effective date: 20220516

EEER Examination request

Effective date: 20220516

EEER Examination request

Effective date: 20220516

EEER Examination request

Effective date: 20220516

EEER Examination request

Effective date: 20220516

EEER Examination request

Effective date: 20220516

EEER Examination request

Effective date: 20220516

EEER Examination request

Effective date: 20220516

EEER Examination request

Effective date: 20220516

EEER Examination request

Effective date: 20220516

EEER Examination request

Effective date: 20220516