CN113330409A - 人机交互方法、装置和系统 - Google Patents

人机交互方法、装置和系统 Download PDF

Info

Publication number
CN113330409A
CN113330409A CN201980079293.8A CN201980079293A CN113330409A CN 113330409 A CN113330409 A CN 113330409A CN 201980079293 A CN201980079293 A CN 201980079293A CN 113330409 A CN113330409 A CN 113330409A
Authority
CN
China
Prior art keywords
touch
operated
touch operation
voice instruction
position information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201980079293.8A
Other languages
English (en)
Inventor
赵光耀
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Huawei Technologies Co Ltd
Original Assignee
Huawei Technologies Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Huawei Technologies Co Ltd filed Critical Huawei Technologies Co Ltd
Publication of CN113330409A publication Critical patent/CN113330409A/zh
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
    • G06F3/0482—Interaction with lists of selectable items, e.g. menus
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04842—Selection of displayed objects or displayed text elements
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
    • G06F3/04847—Interaction techniques to control parameter settings, e.g. interaction with sliders or dials
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16—Sound input; Sound output
    • G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/038—Indexing scheme relating to G06F3/038
    • G06F2203/0381—Multimodal input, i.e. interface arrangements enabling the user to issue commands by simultaneous use of input devices of different nature, e.g. voice plus gesture on digitizer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/048—Indexing scheme relating to G06F3/048
    • G06F2203/04808—Several contacts: gestures triggering a specific function, e.g. scrolling, zooming, right-click, when the user establishes several contacts with the surface simultaneously; e.g. using several fingers or a combination of fingers and pen
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/22—Procedures used during a speech recognition process, e.g. man-machine dialogue
    • G10L2015/223—Execution procedure of a spoken command
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Acoustics & Sound (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

一种人机交互方法、装置和系统，涉及人机交互技术领域，有助于实现多模态的人机交互，提高用户体验。该人机交互方法应用于人机交互系统(10)，该人机交互系统(10)包括触摸屏(13)。该人机交互方法包括：接收具有重叠的触摸时长的第一触摸操作和第二触摸操作；在接收语音指令，该语音指令是在第一触摸操作和第二触摸操作的重叠的触摸时长内接收到的语音指令；响应第一触摸操作和第二触摸操作，确定该语音指令的待操作对象在触摸屏中的位置信息；基于待操作对象在触摸屏中的位置信息，执行该语音指令。

Description

PCT国内申请，说明书已公开。

Claims (37)

  1. PCT国内申请，权利要求书已公开。
CN201980079293.8A 2019-12-30 2019-12-30 人机交互方法、装置和系统 Pending CN113330409A (zh)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/130195 WO2021134358A1 (zh) 2019-12-30 2019-12-30 人机交互方法、装置和系统

Publications (1)

Publication Number Publication Date
CN113330409A true CN113330409A (zh) 2021-08-31

Family

ID=76686232

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201980079293.8A Pending CN113330409A (zh) 2019-12-30 2019-12-30 人机交互方法、装置和系统

Country Status (5)

Country Link
US (1) US20220335943A1 (zh)
EP (1) EP3992768A4 (zh)
JP (1) JP7413513B2 (zh)
CN (1) CN113330409A (zh)
WO (1) WO2021134358A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US12001782B2 (en) * 2021-09-24 2024-06-04 Google Llc Personalized autonomous spreadsheets
US12001783B2 (en) * 2021-09-24 2024-06-04 Google Llc Autonomous spreadsheet creation

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103970460A (zh) * 2013-01-30 2014-08-06 三星电子（中国）研发中心 基于触摸屏的操作方法和使用该方法的终端设备
CN105045481A (zh) * 2015-06-26 2015-11-11 深圳市金立通信设备有限公司 一种操作方法及终端
CN105677223A (zh) * 2016-01-07 2016-06-15 广东欧珀移动通信有限公司 一种按压触控方法和装置
CN109542323A (zh) * 2018-11-23 2019-03-29 网易（杭州）网络有限公司 交互控制方法及装置、存储介质、电子设备

Family Cites Families (14)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3203061B2 (ja) * 1992-09-07 2001-08-27 シャープ株式会社 音声電子黒板及び音声認識機能を備える表示装置
US5726688A (en) * 1995-09-29 1998-03-10 Ncr Corporation Predictive, adaptive computer interface
JPH11184669A (ja) * 1997-12-24 1999-07-09 Sharp Corp 情報処理装置及び情報処理方法並びに情報処理装置制御プログラムを記憶した媒体
JP4515653B2 (ja) 2001-03-15 2010-08-04 株式会社リコー 情報入力装置、情報入力装置の制御方法、プログラムおよびプログラムを記録したコンピュータ読み取り可能な記録媒体
US9223471B2 (en) * 2010-12-28 2015-12-29 Microsoft Technology Licensing, Llc Touch screen control
WO2013170383A1 (en) * 2012-05-16 2013-11-21 Xtreme Interactions Inc. System, device and method for processing interlaced multimodal user input
JP5921703B2 (ja) 2012-10-16 2016-05-24 三菱電機株式会社 情報表示装置および情報表示装置における操作制御方法
JP2014157520A (ja) 2013-02-18 2014-08-28 Fujitsu Ten Ltd 表示装置、及び、表示方法
US20160103655A1 (en) 2014-10-08 2016-04-14 Microsoft Corporation Co-Verbal Interactions With Speech Reference Point
CN105988707A (zh) * 2015-06-30 2016-10-05 乐视致新电子科技（天津）有限公司 一种控制电子设备的方法及装置
CN107436698A (zh) * 2016-05-27 2017-12-05 深圳市科潮达科技有限公司 触控系统及方法
US10558341B2 (en) * 2017-02-20 2020-02-11 Microsoft Technology Licensing, Llc Unified system for bimanual interactions on flexible representations of content
US20180318705A1 (en) * 2017-05-04 2018-11-08 Roblox Corporation Touch-Screen Based Game Controller
CN109388318B (zh) * 2018-09-30 2022-05-31 联想(北京)有限公司 一种控制方法及电子设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103970460A (zh) * 2013-01-30 2014-08-06 三星电子（中国）研发中心 基于触摸屏的操作方法和使用该方法的终端设备
CN105045481A (zh) * 2015-06-26 2015-11-11 深圳市金立通信设备有限公司 一种操作方法及终端
CN105677223A (zh) * 2016-01-07 2016-06-15 广东欧珀移动通信有限公司 一种按压触控方法和装置
CN109542323A (zh) * 2018-11-23 2019-03-29 网易（杭州）网络有限公司 交互控制方法及装置、存储介质、电子设备

Also Published As

Publication number Publication date
US20220335943A1 (en) 2022-10-20
JP2022547667A (ja) 2022-11-15
WO2021134358A1 (zh) 2021-07-08
EP3992768A4 (en) 2022-08-24
EP3992768A1 (en) 2022-05-04
JP7413513B2 (ja) 2024-01-15

Similar Documents

Publication Publication Date Title
TWI522894B (zh) 用於電子元件中的方法、電腦程式產品以及非暫時性電腦可讀記錄媒體
EP3206110B1 (en) Method of providing handwriting style correction function and electronic device adapted thereto
WO2020073980A1 (zh) 寄宿应用的处理方法、设备及计算机可读存储介质
EP2905679A1 (en) Electronic device and method of controlling electronic device
WO2019184490A1 (zh) 用于显示寄宿应用的图标的方法、设备和存储介质
CN105320417B (zh) 页面切换方法及客户端
US20220335943A1 (en) Human-computer interaction method, apparatus, and system
EP2869253A1 (en) Method for operating message application and electronic device implementing the same
WO2019000287A1 (zh) 一种图标显示方法及装置
WO2019205706A1 (zh) 应用组件的处理方法、设备及计算机可读存储介质
WO2020200263A1 (zh) 信息流中图片的处理方法、设备及计算机可读存储介质
CN107924286B (zh) 电子设备及电子设备的输入方法
WO2017067164A1 (zh) 多指并拢或打开手势的识别方法、装置及终端设备
WO2020135010A1 (zh) 寄宿应用的处理方法、设备及计算机可读存储介质
US9606665B2 (en) Object moving method and electronic device implementing the same
CN105580024A (zh) 一种截屏方法及装置
WO2019105391A1 (zh) 无线局域网络的连接方法、设备及计算机可读存储介质
WO2020221299A1 (zh) 即时通信方法、设备及计算机可读存储介质
US20170068374A1 (en) Changing an interaction layer on a graphical user interface
WO2020015704A1 (zh) 触摸控制方法及装置、存储介质、终端设备
CN109542272A (zh) 触摸屏扫描方法和装置、电子设备及计算机可读存储介质
CN112218134A (zh) 一种输入方法以及相关设备
WO2017067165A1 (zh) 多指滑动手势的识别方法、装置及终端设备
JP2015095219A (ja) 情報処理装置、情報処理方法および記憶媒体
CN107368245A (zh) 模式启用方法及装置

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination