CN102779000B - User interaction system and method - Google Patents

User interaction system and method Download PDF

Info

Publication number
CN102779000B
CN102779000B CN201210134998.5A CN201210134998A CN102779000B CN 102779000 B CN102779000 B CN 102779000B CN 201210134998 A CN201210134998 A CN 201210134998A CN 102779000 B CN102779000 B CN 102779000B
Authority
CN
China
Prior art keywords
user
display
unit
information
operation
Prior art date
Application number
CN201210134998.5A
Other languages
Chinese (zh)
Other versions
CN102779000A (en
Inventor
刘广松
Original Assignee
苏州触达信息技术有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 苏州触达信息技术有限公司 filed Critical 苏州触达信息技术有限公司
Priority to CN201210134998.5A priority Critical patent/CN102779000B/en
Publication of CN102779000A publication Critical patent/CN102779000A/en
Application granted granted Critical
Publication of CN102779000B publication Critical patent/CN102779000B/en

Links

Abstract

本发明实施方式公开了一种用户交互系统和方法。 Embodiments of the present invention discloses a system and method for user interaction. 信号运算处理单元,用于向视网膜显示单元提供显示信号;视网膜显示单元,用于将显示信号投影到用户视网膜,使用户在视觉上感觉出现虚拟界面,显示信号被显示在虚拟界面上;触摸屏单元,用于捕获用户在其上的触摸操作动作,并将对应于该触摸操作动作的触摸操作信息发送到信号运算处理单元;信号运算处理单元,进一步用于确定对应于该触摸操作信息的交互操作命令,并向视网膜显示单元实时提供对应于执行该交互操作命令后的显示信号。 A signal processing unit for providing a display signal to the display unit retina; retinal display unit for displaying a signal projected to the retina of the user, make the user feel visually appear on a virtual interface, the display signals are displayed on a virtual screen; touch screen unit for capturing a user operation on a touch operation, and the operation corresponding to the touch operation of the touch operation information to the signal processing unit; a signal processing unit, further configured to determine information corresponding to the touch operation of the interworking command, and retinal display unit which performs display in real time signals after the interaction commands. 本发明实施方式能够增强用户体验,将虚拟信息与现实实景融合在一起,而且提供触摸交互控制,给人提供增强现实的感官体验,可以衍生出大量有意义的应用。 Embodiments of the present invention can enhance the user experience, to blend virtual reality real information, and provide interactive touch control, giving the augmented reality sensory experience, it can be derived from a large number of interesting applications.

Description

一种用户交互系统和方法 A user interactive system and method

技术领域 FIELD

[0001] 本发明涉及电子应用(applicat1n)技术领域,特别地,涉及一种用户交互系统和方法。 [0001] The present invention relates to the field of electronic applications (applicat1n), in particular, to a system and method for user interaction.

背景技术 Background technique

[0002] 1959年美国学者B.Shackel首次提出了人机交互工程学的概念。 [0002] 1959, American scholar B.Shackel first proposed the concept of man-machine interaction engineering. 20世纪90年代后期以来,随着高速处理芯片,多媒体技术和互联网技术的迅速发展和普及,人机交互的研宄重点放在了智能化交互、多模态(多通道)_多媒体交互、虚拟交互以及人机协同交互等方面,也就是放在以人为在中心的人机交互技术方面。 Since the late 1990s, along with high-speed processing chip, multimedia technology and Internet technology rapid development and popularization of study based on focus on human-computer interaction intelligent interactive, multi-modal (multichannel) _ multimedia interactive, virtual aspects of human interaction and collaborative interaction and so on, that is, to put people at the center of human-computer interaction aspects.

[0003] 随着社会的进步与信息爆炸时代的来临,人们越来越多依靠各式各样的消费电子设备(如移动终端、个人数字助理(PDA)等)获取各种信息。 [0003] With the advent of the era of information explosion and the progress of society, more and more people rely on a wide range of consumer electronic devices (such as mobile terminals, personal digital assistants (PDA), etc.) to obtain a variety of information. 比如:打电话与别人沟通,浏览网页获取新闻和查看电子邮件等。 For example: a call to communicate with others, browse the Web for news, and view e-mail. 目前广泛应用的人机交互包括依靠传统的键盘鼠标等硬件设备,以及近几年逐渐流行起来的触摸屏等。 Human-computer interaction, including the widely used rely on traditional keyboard and mouse and other hardware devices, and in recent years increasingly popular touch-screens.

[0004] 人们对于现存的人机交互方式并不满足,人们期望新一代的人机交互能像人与人交互一样自然、准确和快捷。 [0004] people for the existing human-computer interaction is not satisfied, it is expected the new generation of human-computer interaction to be like natural human interaction, accurate and fast. 于是在20世纪90年代人机交互的研宄进到了多模态阶段,称为人机自然交互(Human-Computer Nature Interact1n,HCNI 或Human-Machine NatureInteract1n,HMNI)。 So in the 1990s to study based interactive multimodal into the stage, known as the natural human-computer interaction (Human-Computer Nature Interact1n, HCNI or Human-Machine NatureInteract1n, HMNI).

[0005]目前,如何将人机自然交互技术应用到各种应用中,依然是一个巨大的挑战。 [0005] At present, how to use natural human-computer interaction technology to a variety of applications, it is still a huge challenge.

发明内容 SUMMARY

[0006] 有鉴于此,本发明实施方式提出一种用户交互系统,以增加用户体验。 [0006] In view of this, embodiments of the present invention provides a user interactive system, in order to increase the user experience.

[0007] 本发明实施方式还提出一种用户交互方法,以增强用户体验。 [0007] The present invention also provides a embodiment of the method for user interaction, in order to enhance the user experience.

[0008] 本发明技术方案如下: [0008] The technical solution of the present invention is as follows:

[0009] 一种用户交互系统,该系统包括信号运算处理单元、视网膜显示单元和触摸屏单元,其中: [0009] A user interaction system, the system includes a signal processing unit, the display unit and the touch screen retina unit, wherein:

[0010] 信号运算处理单元,用于向视网膜显示单元提供显示信号; [0010] The signal processing unit, means for displaying a display signal to the retina;

[0011] 视网膜显示单元,用于将信号运算处理单元所提供的显示信号投影到用户视网膜,使用户在视觉上感觉出现虚拟界面,且所述显示信号被显示在所述虚拟界面上; [0011] retinal display unit, for projection display signal processing unit is supplied to the retina of the user, make the user feel visually appear in the virtual interface, and the display signals are displayed on the virtual interface;

[0012] 触摸屏单元,用于捕获用户在其上的触摸操作动作,并将对应于该触摸操作动作的触摸操作信息发送到信号运算处理单元; [0012] The touch screen unit, for capturing a user action on the touch operation thereon, and transmits the touch operation corresponds to the operation information of the touch operation to the signal processing unit;

[0013] 所述信号运算处理单元,进一步用于确定对应于该触摸操作信息的交互操作命令,并向视网膜显示单元实时提供对应于执行该交互操作命令后的显示信号。 [0013] The signal processing unit is further for determining a touch operation information corresponding to the interactive operation command, and retinal display unit which performs display in real time signals after the interaction commands.

[0014] 所述视网膜显示单元为用户头戴的眼镜式显示器或直接视网膜投影装置。 [0014] The retinal display unit for the user wearing the glasses-type display apparatus or a direct retinal projection.

[0015] 所述信号运算处理单元为移动终端、计算机或基于云计算的信息服务平台。 The [0015] signal processing unit of the mobile terminal, or a computer-based information service cloud computing platform.

[0016] 所述信号运算处理单元和视网膜显示单元在物理上集成为整体。 The [0016] signal processing unit and a display unit physically integrated retina integral.

[0017] 信号运算处理单元,用于向视网膜显示单元提供三维立体界面显示信号; [0017] The signal processing unit for displaying the three-dimensional interface unit provides a retinal display signal;

[0018] 视网膜显示单元,用于根据所述三维立体界面显示信号向用户显示三维立体界面。 [0018] retinal display unit according to the three-dimensional three-dimensional display interface display signals to a user interface.

[0019] 该系统进一步包括佩戴于用户头部的视角感知单元; [0019] The system further comprises a head of the user wearing the viewing angle sensing unit;

[0020] 视角感知单元,用于感测用户头部运动信息,并将所述用户头部运动信息发送到信息运算处理单元; [0020] Angle sensing means for sensing motion information of a user's head, and the head movement of the user information to the information processing unit;

[0021] 信息运算处理单元,进一步用于根据所述用户头部运动信息确定用户实时视角,并实时向视网膜显示单元提供基于该用户实时视角下的三维立体界面显示信号。 [0021] The information processing unit is further for determining from the perspective of the user head-motion real-time user information, and provides real-time three-dimensional interface based on the perspective of the user in real-time display signal to the display unit retina.

[0022] 该系统进一步包括动作捕获单元, [0022] The motion capture system further comprises means,

[0023] 所述动作捕获单元,用于捕获用户浏览该虚拟界面做出的肢体空间移动信息,并将所述肢体空间移动信息发送到信息运算处理单元; [0023] The motion capture means for capturing the user interface of the virtual space browsing limb made movement information, and transmits the information to the physical space of the mobile information arithmetic processing unit;

[0024] 信息运算处理单元,进一步用于确定对应于该用户肢体空间移动信息的交互操作命令,并实时向视网膜显示单元提供对应于执行该交互操作命令后的显示信号。 [0024] The information processing unit is further for determining a physical space corresponding to the mobile user interactive operation command information, and the display unit in real time to provide a retinal display signal corresponding to the execution of the command interaction.

[0025] 信号运算处理单元,用于向视网膜显示单元提供二维虚拟界面显示信号; [0025] The signal processing unit, an interface for providing a two-dimensional virtual retinal display signal to the display unit;

[0026] 视网膜显示单元,用于根据所述二维虚拟界面显示信号向用户显示二维虚拟界面。 [0026] retinal display unit according to the two-dimensional virtual display two-dimensional virtual display signal interface to a user interface.

[0027] 信号运算处理单元,进一步用于在虚拟界面上显示空间虚拟指针元素,并且所述空间虚拟指针元素在虚拟界面上的运动轨迹与用户在触摸屏单元上的触摸操作的轨迹保持一致。 [0027] The signal processing unit is further configured to display the virtual space on the virtual screen pointer element and the virtual space on the virtual trajectory user interface touch operation on the touch panel unit pointer element consistent trajectory.

[0028] 一种用户交互方法,该方法包括: [0028] A method of user interaction, the method comprising:

[0029] 提供显示信号; [0029] providing a display signal;

[0030] 将所提供的显示信号投影到用户视网膜,使用户在视觉上感觉出现虚拟界面,且所述显示信号被显示在所述虚拟界面上; [0030] The display signal is projected onto the retina of the user provided, make the user feel visually appear in the virtual interface, and the display signals are displayed on the virtual interface;

[0031] 捕获用户在触摸屏上的触摸操作,确定对应于该用户触摸操作的交互操作命令; [0031] The user captures a touch operation on the touch screen, to determine the touch operation corresponding to the user's interactive operation command;

[0032] 实时提供对应于执行该交互操作命令后的显示信号。 [0032] corresponding to the real time display signal after performing the interactive operation command.

[0033] 提供显示信号为:提供三维立体界面显示信号。 [0033] providing a display signal: providing a three-dimensional screen display signal.

[0034] 该方法进一步包括: [0034] The method further comprising:

[0035] 感测用户头部运动信息; [0035] The motion information sensed a user's head;

[0036] 根据所述用户头部运动信息确定用户实时视角,并实时提供基于该用户实时视角下的三维立体界面显示信号。 [0036] determining from the perspective of the user head-motion real-time user information, and provides real-time three-dimensional interface based on the perspective of the user in real-time display signal.

[0037] 该方法进一步包括: [0037] The method further comprising:

[0038] 捕获用户浏览该虚拟界面做出的肢体空间移动信息; [0038] capture the user's browser physical space of the virtual interface information to make a move;

[0039] 确定对应于该用户肢体空间移动信息的交互操作命令; [0039] corresponding to the user determined movement of the limb space of the interactive operation command information;

[0040] 实时提供对应于执行该交互操作命令后的显示信号。 [0040] corresponding to the real time display signal after performing the interactive operation command.

[0041] 所述捕获用户浏览该虚拟界面做出的肢体空间移动信息为:捕获用户浏览该虚拟界面而做出的精确定位操作和/或非精确定位操作。 [0041] The user captures the virtual space browsing interface limb movement information is made: the precise positioning operation to capture the user browses the virtual interface made and / or accurate positioning.

[0042] 所述用户在触摸屏上的触摸操作为单点触摸操作或多点触摸操作。 [0042] The user touch operation on the touch screen single-touch operation or a touch operation.

[0043] 所述单点触摸操作包括:点击、滑动或长按。 [0043] The one-touch operation comprising: a click, or press slide.

[0044] 所述多点触摸操作包括:旋转或缩放。 [0044] The multi-touch operation comprising: rotation or scaling.

[0045] 从上述技术方案中可以看出,在本发明实施方式中,信号运算处理单元,用于向视网膜显示单元提供显示信号;视网膜显示单元,用于将显示信号投影到用户视网膜,使用户在视觉上感觉出现虚拟界面,显示信号被显示在虚拟界面上;触摸屏单元,用于捕获用户在其上的触摸操作动作,并将对应于该触摸操作动作的触摸操作信息发送到信号运算处理单元;信号运算处理单元,进一步用于确定对应于该触摸操作信息的交互操作命令,并向视网膜显示单元实时提供对应于执行该交互操作命令后的显示信号。 [0045] As can be seen from the above technical solutions, in embodiments of the present invention, the signal processing unit, for providing a display signal to the display unit retina; retinal display unit for displaying a signal projected to the retina of the user, so that user appears on the visual sense of virtual screen, the display signal is displayed on the virtual interface; touch operation information of the touch screen unit, for capturing a user action on the touch operation thereon, and the operation corresponding to the touch operation to the signal processing unit ; a signal processing unit is further for determining a touch operation information corresponding to the interactive operation command, and retinal display unit which performs display in real time signals after the interaction commands. 由此可见,应用本发明实施方式以后,通过虚拟的界面与触摸屏之间的交互,实现了一种用户与硬件设备之间交互与获取信息的方式,极大地增强了用户体验。 Thus, after the application of embodiments of the present invention, through the interaction between the interface and touch-screen virtual, realized way between a user and an interaction with the hardware devices to access information, greatly enhancing the user experience.

[0046] 而且,本发明实施的交互方式非常自然,符合人性的基本肢体动作(比如手势)交互模式,而且降低了用户对操作设备的学习成本,符合人体自然地交互操控与便携信息处理硬件设备的分体设计,使人能够更集中精力于其所关注的信息而不是硬件设备本身。 [0046] Further, interact embodiment of the present invention is very natural, in line with the basic body movements human (such as gesture) the interactive mode, and reduces the user to learn the cost of operation of the device, in line with the body's natural interact manipulate the portable information processing hardware the split design, people can focus more attention on their information rather than hardware device itself.

[0047] 而且,本发明实施方式还提出了一种自然交互技术的三维立体虚拟信息自然交互界面,该交互界面包含众多三维立体可进行自然交互的元素。 [0047] Further, embodiments of the present invention also provides a natural three-dimensional information on the natural interaction in virtual interface, the interface comprising a number of three-dimensional elements may be made of natural interaction. 通过本发明实施方式所提出的解决方案,用户可以自然地用手控制上述三维立体虚拟信息自然交互界面中对应于用户手的虚拟指针,对三维立体虚拟信息自然交互界面进行自然交互。 Solutions proposed embodiment of the present invention, the user can manually controlling the natural three-dimensional virtual natural interaction interface information corresponding to the user's hand virtual pointer, information on the virtual three-dimensional natural interaction interface natural interaction.

[0048] 另外,本发明实施方式的独特显示方式使其受环境影响较小,给人提供高品质的感官体验,并能够保护信息的私密性。 [0048] Further, a unique display embodiment of the present invention it is less environmental impact, giving sensory experience to provide high quality, and can protect the privacy of information. 本发明实施方式通过直接视网膜扫描投影显示方式可以将虚拟信息与现实实景融合在一起,给人提供增强现实的感官体验,从而基于此可以衍生出大量的有意义的应用,进一步极大地提高用户体验。 Embodiments of the invention by direct retinal scan projection display virtual reality with real information can be fused together, giving sensory experience to provide augmented reality, so that this may be derived based on a large number of interesting applications, greatly improving the user experience is further .

[0049] 不仅于此,本发明实施方式可以应用与任何人机交互信息设备,其通用性将给人们带来极大便利。 [0049] Not only this, embodiments of the present invention can be applied to any human-computer interaction and information equipment, its versatility will bring great convenience.

附图说明 BRIEF DESCRIPTION

[0050] 图1为根据本发明实施方式的用户交互系统结构示意图; [0050] FIG. 1 is a schematic diagram of a user interaction system configuration of the embodiment of the present invention;

[0051] 图2为根据本发明实施方式的用户交互方法流程示意图; [0051] FIG. 2 is a schematic process flow according to a user interaction an embodiment of the present invention;

[0052] 图3为根据本发明实施方式的手势触控交互示意图; [0052] FIG. 3 is a diagram of a touch interaction according to an embodiment of the present invention, the gesture;

[0053] 图4为根据本发明实施方式的虚拟界面显示示意图; [0053] FIG. 4 is a virtual interface according to an embodiment of the present invention shows a schematic view;

[0054] 图5为根据本发明实施方式的虚拟界面显示示意图; [0054] FIG. 5 is a virtual interface according to an embodiment of the present invention shows a schematic view;

[0055] 图6为根据本发明实施方式的三维虚拟界面显示示意图。 [0055] FIG. 6 is a schematic three-dimensional virtual interface according to an embodiment of the present invention.

具体实施方式 Detailed ways

[0056] 为使本发明实施方式的目的、技术方案和优点表达得更加清楚明白,下面结合附图及具体实施例对本发明实施方式再作进一步详细的说明。 [0056] For purposes of embodiments of the present invention, technical solutions and advantages clearer to obtain expression, in conjunction with the accompanying drawings and specific embodiments further embodiment of the present invention will be further described in detail.

[0057] 在本发明实施方式中,针对现有技术下各种电子设备(诸如便携式电子设备)采用键盘等作为交互手段影响用户体验的缺陷,利用直接视网膜显示方式,使用户感觉前方一定距离出现虚拟的屏幕界面,关键信息可在屏幕界面中高亮显示,并通过识别用户针对该虚拟界面做出的触摸屏操作和/或空间肢体动作来实现交互。 [0057] In an embodiment of the present invention for various electronic devices (such as a portable electronic device) using the keyboard or the like user experience as an interactive means of a defect, by direct retinal display state of the art, to make users feel appears at a distance in front of virtual screen interface, key information can be highlighted in the display interface, and enable interaction for touch screen operation and / or body movements of the virtual space made by identifying the user interface.

[0058] 本发明实施方式使用视网膜直接扫描投影的方式产生虚拟屏幕界面,避免了采用物理显示屏导致的各种问题,而且还不会影响背景视场,其产生的虚拟屏幕界面可作为对现实实景的增强,能够广泛应用于增强现实(Augment Reality)技术。 [0058] Embodiments of the invention using a direct retinal scan projection screen interface generates a virtual manner, avoiding the problems caused by physical screen, but will not affect the background field of view, which generates a virtual reality interface may screen real enhancement can be widely used in augmented reality (Augment reality) technology.

[0059] 而且,本发明实施方式同时针对前述虚拟界面提出一种基于触摸屏识别的人性化交互方案,此交互方案能够无缝融合前述虚拟界面与触摸屏动作操控信息。 [0059] Further, embodiments of the present invention also provides a touch screen based user-friendly interactive program identification, this solution can interact with the virtual interface and seamless integration of the touchscreen operation control information for the virtual interface. 同样可以通过对一些基本的典型的触摸屏操作识别进行优化处理,形成一个稳定的交互开发平台,供开发者开发各式各样应用。 The same may be performed by some of the basic operation of a typical touch-screen identification optimization processing, to form a stable interactive development platform for application developers to develop a wide range.

[0060] 另外,本发明实施方式同时针对前述虚拟界面提出一种基于对人肢体动作(优选为人的手势)识别的人性化交互方案,此交互方案能够无缝融合前述虚拟界面与人体的肢体动作操控信息。 [0060] Further, embodiments of the present invention also proposed for the virtual interface based on user-friendly interactive program on human body movement (preferably a human gesture) recognizable, this scheme can interact seamlessly into the virtual interface and physical action of the human body manipulation of information. 类似地,通过对一些基本的典型的操作识别进行优化处理,形成一个稳定的交互开发平台,供开发者开发各式各样应用。 Similarly, by some typical basic operation recognition optimization processing, to form a stable interactive development platform for application developers to develop a wide range.

[0061] 图1为根据本发明实施方式的用户交互系统结构示意图。 [0061] FIG. 1 is a schematic system configuration of the user interaction according to an embodiment of the present invention.

[0062] 如图1所示,该系统包括:信号运算处理单元101、视网膜显示单元102和触摸屏单元103,其中: [0062] As shown, the system 1 comprises: a signal processing unit 101, display unit 102 and the retinal touch panel unit 103, wherein:

[0063] 信号运算处理单元101,用于向视网膜显示单元102提供显示信号; [0063] The signal processing unit 101, a display unit 102 into retinal display signal;

[0064] 视网膜显示单元102,用于将信号运算处理单元101所提供的显示信号投影到用户视网膜,使用户在视觉上感觉出现虚拟界面,且所述显示信号被显示在所述虚拟界面上; [0064] retinal display unit 102 for displaying the signal provided by the processing unit 101 is projected onto retina of the user, make the user feel visually appear in the virtual interface, and the display signals are displayed on the virtual interface;

[0065] 触摸屏单元103,用于捕获用户在其上的触摸操作动作,并将对应于该触摸操作动作的触摸操作信息发送到信号运算处理单元101; [0065] The touch panel unit 103, for capturing a user action on the touch operation thereon, and transmits an operation corresponding to the touch operation of touching the operation information to the signal processing unit 101;

[0066] 所述信号运算处理单元101,进一步用于确定对应于该触摸操作信息的交互操作命令,并向视网膜显示单元102实时提供对应于执行该交互操作命令后的显示信号。 The [0066] signal processing unit 101, is further for determining a touch operation information corresponding to the interactive operation command, the display unit 102 and the retina in real time corresponding to the display signal after performing the interactive operation command. 视网膜显示单元102进一步在虚拟界面中显示该对应于执行交互操作命令后的显示信号。 Retinal display unit 102 further displays the display signal corresponding to the operation command after the execution of the interactive virtual interface.

[0067] 其中,信号运算处理单元101可以是能提供显示信号以及计算能力的任意装置。 [0067] wherein the signal processing unit 101 may be any device providing a display signal and computing capabilities. 比如移动终端、计算机,甚至是基于云计算的信息服务平台等。 Such as a mobile terminal, a computer, or even a cloud-based information service platform.

[0068] 信号运算处理单元101可以通过其内置操作系统处理相应的交互处理命令完成某种运算(例如手机拨号,浏览网页等),并通过有线或无线方式实时更新相应显示信号,并输出显示信号给视网膜显示单元102。 [0068] The signal processing unit 101 may be processed through its built-in operating system interaction process corresponding command to complete a certain operation (e.g. phone dialer, Web browsing, etc.), a wired or wireless manner and immediately update the corresponding display signal, and outputs the display signal retina 102 to the display unit.

[0069] 优选地,信号运算处理单元101与视网膜显示单元102之间的通信方式可以有多种具体实施形式,包括但是不局限于:无线宽带传输、蓝牙传输、红外传输、移动通信传输或者有线传输等等。 [0069] Preferably, the signal processing unit 101 and the communication between the display unit 102 may have various retinal specific forms, including but not limited to: wireless broadband transmission, Bluetooth, infrared transmission, mobile transmission or wired communication transmission and so on.

[0070] 视网膜显示单元102通过上述通讯方式从信号运算处理单元101接收显示信号,对显示信号进行调制解码后,将显示图像直接投影到用户视网膜上,使用户感觉到前方出现一个虚拟的界面(优选为增广的屏幕)。 [0070] Retinal Display unit 102 receives the display signal from the signal processing unit 101 via the communication mode, the display signal is modulated after decoding, the display image projected directly onto the retina of the user, making the user feel appears in front of a virtual interface ( preferably augmented screen). 用户感兴趣的信息将通过这个虚拟界面得以展现。 Information of interest to the user will be revealed through the virtual interface.

[0071] 视网膜显示单元102可以通过多种方式产生这个虚拟界面。 [0071] Retinal Display unit 102 may generate the virtual interface through a variety of ways. 比如,视网膜显示单元102具体可以是用户头戴式眼镜式显示器。 For example, a retinal display unit 102 may be a user specific glasses head mounted display. 该用户头戴式眼镜式显示器有两个超微显示屏对应于人的左右眼,通过精密光学透镜放大超微显示屏上的图像,进而呈现于观看者眼中虚拟增广的屏幕图像。 The user has two head mounted goggle type display screen corresponding to the ultra left and right eyes, an enlarged image on the display by the ultra precision optical lens, thereby rendering the eyes of a viewer screen image of the virtual augmented.

[0072] 此外,视网膜显示单元102还可以通过视网膜直接投影的方式产生虚拟界面。 [0072] Further, the retinal display unit 102 may also generate a virtual interface by way of retinal direct projection. 比如,视网膜显示单元102具体可以为直接视网膜投影装置。 For example, a retinal display unit 102 may specifically be a direct retinal projection apparatus. 在这种方式中,利用人的视觉暂留原理,视网膜显示单元102 (即直接视网膜投影装置)中的显示芯片接收来自信号运算处理单元101的显示信号,进而对视网膜显示单元102中的微型激光发生器产生的红绿蓝激光进行调制,让低功率激光快速地按指定顺序在水平和垂直两个方向上循环扫描,撞击视网膜的一小块区域使其产生光感,使人们感觉到图像的存在,此种显示方式可以不影响背景视场,虚拟屏幕叠加在真实视场上,进而可以提供增强现实的感官体验。 In this manner, the use of human persistence of vision principle, retinal display chip unit 102 (i.e., the direct retinal projection means) receives the display signal from the signal processing unit 101, the unit 102 further displays miniature laser retinal red, green and blue modulated laser generator, so that low-power laser circulation quickly scanned in the order named on the horizontal and vertical directions, a small area of ​​the retina strikes to produce light-sensitive, so that people feel the image exist, such a display may not affect the background of the field of view, the virtual screen superimposed on the real field of view, in turn, can provide augmented reality sensory experience.

[0073] 在一个实施方式中,当虚拟界面出现以后,用户浏览虚拟界面,并且通过触摸屏单元103来触发交互过程。 [0073] In one embodiment, the virtual interface appears When the user browses the virtual interface, and to trigger the interaction via a touch screen unit 103.

[0074] 触摸屏单元103可以是带有触摸板或触摸屏的智能设备(例如带触摸屏的智能手机,平板电脑等移动终端)。 [0074] 103 may be a touch screen unit with a touch pad or a touch screen smart device (e.g., smart phone with a touch screen, tablet, etc. mobile terminals). 触摸屏单元103通过内置其中的软件实时记录用户在触摸屏上的触摸与滑动操作,并将这些触摸与滑动操作信息化,而且实时将触摸与滑动操作信息(简称触摸操作信息)通过特定的通信数据接口以有线或无线的方式传递给信号运算处理单元101。 The touch panel unit 103 is built-in software in real-time recording the user touches the slide operation on the touch screen, and the touch and slide operation information, and real-time touch and slide operation information (the touch operation information) through a specific communication data interface a wired or wireless manner is transmitted to the signal processing unit 101. 信号运算处理单元101将对其进行分析,并处理用户操作所代表的交互请求。 The signal processing unit 101 to analyze and process the user interaction request operation represented.

[0075] 在一个实施方式中,触摸屏单元103具体可以实施为五个基本种类:矢量压力传感技术触摸屏、电阻技术触摸屏、电容技术触摸屏、红外线技术触摸屏、表面声波技术触摸屏。 [0075] In one embodiment, the touch screen unit 103 may be implemented as specifically five basic categories: vector pressure sensor touch screen, resistive touch screen technology, capacitive technology touch screen, infrared touch screen technology, a surface acoustic wave touch screen. 按照触摸屏的工作原理和传输信息的介质,可以把触摸屏分为四种,它们分别为电阻式、电容感应式、红外线式以及表面声波式。 The medium of the working principle of the touch screen and the transmission of information, the touch screen can be divided into four, which were resistive, capacitive, infrared, and surface acoustic wave type.

[0076] 下面对本发明实施方式的应用场景进行示范性说明。 [0076] Next, the scenario of the embodiment of the present invention will be exemplary described.

[0077] 比如:用户带着一个眼镜式的显示器(即视网膜显示单元102),该眼镜式显示器与信号运算处理单元101集成在一起为一个整体(或者通过无线方式与信号运算处理单元101相连接)。 [0077] For example: Users with a glasses-type display (i.e. retinal display unit 102), the glasses-type display and integrated signal processing unit 101 together as a whole (or wirelessly connected to the signal processing unit 101 ). 由信号运算处理单元101向眼镜式显示器实时提供显示信号。 It provides a display signal 101 to a glasses-type display in real time by the signal processing unit. 通过此眼镜式显示器,用户在其视野前方可以看到一个虚拟的交互界面。 This goggle type display, the user can see a virtual interface in front of its field of view.

[0078] 示范性地:图4为根据本发明实施方式的虚拟界面显示示意图;图5为根据本发明实施方式的虚拟界面显示示意图。 [0078] exemplary manner: a virtual interface according to FIG. 4 embodiment of the present invention shows a schematic embodiment; FIG. 5 is a schematic diagram of the embodiment of a virtual interface of the present invention.

[0079] 用户将其带有触摸屏的智能设备(即触摸屏单元103)与信号运算处理单元101通过蓝牙或wifi等通信方式建立无线连接。 [0079] the user to the smart device (i.e., the touch panel unit 103) with a touch screen and the signal processing unit 101 to establish a wireless connection like Bluetooth or wifi communication. 通过在触摸屏上运行基于本发明实施方式开发的应用程序建立数据通信。 Establishing a data communication by running application on the touch screen based on the embodiments of the present invention developed.

[0080] 用户在其带有触摸屏的智能设备上打开该应用程序。 [0080] The user opens the application in which the smart device with a touch screen. 假设用户在智能设备的触摸屏上进行从左向右的滑动操作,应用程序将记录用户手指在触摸屏上的移动信息并根据特定的数据协议接口将该信息传输给信号运算处理单元101。 Suppose the user slide operation from left to right on the touch screen of the smart device, the application will record the user's finger movement information on the touch screen and to the signal processing unit 101 transmits the information in accordance with the interface protocol specific data.

[0081] 信号运算处理单元101根据特定的数据接口协议解析出用户的交互行为是从左向右的一段指定距离的滑动操作,并将该交互行为对应转化为用户所看到的交互界面下的有效的交互命令,例如用户的交互行为在当前用户所看到的交互界面下的交互命令为控制用户所看到的交互界面中一个指针从左向右滑动特定距离,这时信号运算处理单元101对该交互命令做出响应,实时更新交互界面显示信号给视网膜显示单元102,视网膜显示单元102将实时更新的交互界面实时呈现给用户。 [0081] The signal processing unit 101 parses the data interface protocol according to a particular user's interactions sliding operation is a specified distance from left to right, and the interactive activity corresponding to conversion to the interface the user sees the effective interactive commands, for example, a sliding pointer certain distance to the right from the left in the interactive command of the user interactions seen in the current user interface to control the user interface as seen in this case the signal processing unit 101 to respond to the interactive command signal is updated in real time display interface 102 to the display unit retina, retinal display unit 102 is updated in real time the real-time presentation to the user interface.

[0082] 在另一个实施方式中,用户可以在智能设备上的触摸板(或触摸屏)上进行多点触摸操作。 Multi-point touch operation on [0082] In another embodiment, a user may touch panel (or touch screen) on the smart device. 比如:用户用两个手指在触摸板(或触摸屏)上做放大操作(两个手指在触摸板上滑动且两个手指在触摸板上的触点距离增加),此交互行为被智能设备上的针对于本发明实施方式所开发的应用程序所记录并以特定的数据接口通过无线方式传输给信号运算处理单元101。 For example: Users do enlargement operation with two fingers on a touch panel (or touch screen) (slid two fingers on the touch pad and two contact fingers on the touch pad distance increases), on this interactive activity devices intelligently the recorded for application to the embodiments of the present invention is developed and transmitted wirelessly to the signal processing unit 101 in a particular data interface.

[0083] 信号运算处理单元101中预先保存有各种触摸操作动作与具体交互操作命令的对应关系。 [0083] The signal processing unit 101 stores in advance various touch operations interactive operation correspondence to a specific operation command. 当信号运算处理单元101收到触摸屏单元发送来的放大操作信息,解析出与该放大操作信息相对应的交互操作命令为:对当前交互显示界面中的一张图片进行放大操作。 When the signal processing unit 101 receives the touch screen unit zoom operation information transmitted, the information to resolve the amplifying interactive operation corresponding to the operation command: the current interactive display interface to zoom an image. 然后,信号运算处理单元101对图片进行放大操作,并且将执行完后的显示信号发送到视网膜显示单元102,然后视网膜显示单元102在虚拟界面上及时更新该显示信号。 Then, the signal processing unit 101 amplifies the picture operation, and performs a display signal transmitted to the retina after the display unit 102, the display unit 102 and the retinal update the display signal on the virtual screen.

[0084] 优选地,在触摸屏或触摸板上可识别的交互操作可以包括单点触摸操作和多点触摸操作。 [0084] Preferably, a touch pad or the touch screen can recognize the interaction may include a single touch operation and the multi-touch operation.

[0085] 单点触摸操作具体可以包括:点击、滑动和长按,等等。 [0085] The one-touch operation may specifically include: clicking, sliding, and long press, and the like. 多点触摸操作具体可以包括旋转(两个手指在触摸屏或触摸板上进行滑动,且手指在触摸屏或触摸板上的触点间距离基本不变,但触点间连线与触摸板边界夹角发生变化)以及缩放(两个手指在触摸屏或触摸板上滑动,且手指在触摸屏或触摸板上的触点间距离增大或缩小),等等。 DETAILED multi-touch operation may include rotating (sliding two fingers on a touch screen or a touch pad, and a substantially constant distance between the finger contacts on the touch screen or a touch pad, but the connection between the contacts with the touch panel border angle changes) and scaling (two fingers on a touch screen or touchpad slide, and the distance between the contacts is increased or reduced finger on the touch screen or a touch pad), and the like.

[0086] 在一个实施方式中,信号运算处理单元101,进一步用于在虚拟界面上显示空间虚拟指针元素,并且所述空间虚拟指针元素在虚拟界面上的运动轨迹与用户在触摸屏单元103上的触摸操作的轨迹保持一致。 [0086] In one embodiment, the signal processing unit 101, further for displaying the virtual space on the virtual interface element pointer, and the pointer element trajectory virtual space on the virtual user interface on the touch screen unit 103 keeping track of touch operations.

[0087] 优选地,信号运算处理单元101向视网膜显示单元102提供三维立体界面显示信号;视网膜显示单元102,用于根据所述三维立体界面显示信号向用户显示三维立体界面。 [0087] Preferably, the signal processing unit 101 to the display unit 102 provides a three-dimensional retina interface display signal; retinal display unit 102, according to the three-dimensional three-dimensional display interface display signals to a user interface. 这样,用户可以浏览三维立体界面,获得更加生动的交互感受。 In this way, the user can view three-dimensional interface, obtain a more vivid and interactive experience. 示范性地,图6为根据本发明实施方式的三维虚拟界面显示示意图。 Exemplarily, FIG. 6 is a schematic three-dimensional virtual interface according to an embodiment of the present invention.

[0088] 为了支持显示三维立体信号,视网膜显示单元102具体可以实施为头戴式设备,比如优选为眼镜式三维立体显示设备。 [0088] In order to support the display of three-dimensional signal, retinal display unit 102 may be implemented as specifically headsets, preferably such glasses type three-dimensional display apparatus. 该眼镜式三维立体显示设备通过控制对应于左右眼的微显示器所显示画面的微小差异使用户双眼看到的画面形成视差,大脑会解读双眼的视差并藉以判断物体远近与产生立体视觉。 The glasses type three-dimensional display device displays corresponding to the left and right eyes by controlling a microdisplay that the slight difference in picture screen of the user's eyes to see the formation of parallax, the brain interprets binocular parallax and thereby determines the object distance and generating the stereoscopic vision.

[0089] 当应用于三维显示的场景中时,该系统进一步包括佩戴于用户头部的视角感知单元(图中没有示出)。 [0089] When applied to a three-dimensional display of a scene, the system further comprising a head of the user wearing the viewing angle sensing unit (not shown). 视角感知单元,用于感测用户头部运动信息,并将所述用户头部运动信息发送到信息运算处理单元101。 Angle sensing means for sensing motion information of a user's head, the head of the user and transmits the movement information to the information processing unit 101. 此时,信息运算处理单元101,进一步用于根据所述用户头部运动信息确定用户实时视角,并实时向视网膜显示单元102提供基于该用户实时视角下的三维立体界面显示信号。 At this time, the information processing unit 101 is further used to determine in real time the perspective of a user according to the user's head motion information, and the display unit 102 in real time to the retinal-based real-time three-dimensional interface, the user of the viewing angle display signal.

[0090] 具体地,视角感知单元可以包括陀螺仪、加速计、电子罗盘等微电子传感器及数据传输模块。 [0090] Specifically, the viewing angle sensing unit may include a gyroscope, an accelerometer, other microelectronic sensors and electronic compass data transmission module. 视角感知单元可以固定佩戴于用户头部,用于感测用户头部运动信息,并将相应的数据信息通过数据传输模块传输给信息运算处理单元101,由信息运算处理单元101进一步分析得到用户实时的视角方向及变化信息。 Angle sensing means worn on the user's head may be fixed, for sensing motion information of a user's head, and the corresponding data information is transmitted to the information processing unit through a data transfer module 101, the information processing unit 101 is further analyzed to give the user real-time the viewing direction and change information. 视角感知单元可以与视网膜显示单元102在物理上集成为一体。 Angle sensing means may be associated with retinal display unit 102 physically integrated into one.

[0091] 以上虽然详细说明了基于三维虚拟界面的交互过程,本领域技术人员可以意识到,本发明实施方式并不局限于三维虚拟界面,而是可以适应于适用于任意的虚拟界面。 [0091] While the above detailed description of the interaction-based three-dimensional virtual interface, those skilled in the art may be appreciated, embodiments of the present invention is not limited to the three-dimensional virtual interface, but may be adapted to be applied to any virtual interface.

[0092] 在一个优选实施方式中,当二维或三维虚拟界面出现以后,用户还可以通过各种肢体动作(比如,优选为通过手势)来触发交互过程。 [0092] In a preferred embodiment, the two-dimensional or three-dimensional virtual When the screen appears, the user can also trigger the interaction (such as through gestures, preferably) through a variety of body movements.

[0093] 对应的,还系统进一步包括动作捕获单元104。 [0093] corresponding to the motion capture system further comprises further means 104.

[0094] 所述动作捕获单元104,用于捕获用户浏览该虚拟界面做出的肢体空间移动信息,并将所述肢体空间移动信息发送到信息运算处理单元101; [0094] The motion capture unit 104 for capturing the user interface of the virtual space browsing limb made movement information, and transmits the information to the physical space of the mobile information arithmetic processing unit 101;

[0095] 信息运算处理单元101,进一步用于确定对应于该用户肢体空间移动信息的交互操作命令,并实时向视网膜显示单元102提供对应于执行该交互操作命令后的显示信号. [0095] The information processing unit 101, is further for determining a physical space corresponding to the mobile user interactive operation command information, and the display unit 102 in real time to provide the retina corresponding to the display signal after performing the interactive operation command.

[0096] 具体地,动作捕获单元104通过实时拍摄视场中景象捕获用户肢体动作,并将获取的包含景深信息图像数据实时传给信息运算处理单元101。 [0096] Specifically, the operation capturing unit 104 captures the user body movements in real-time picture imaging field, and the acquired depth information comprising real-time image data to the information processing unit 101. 然后,信息运算处理单元101可以通过一系列软件算法,分析得到用户肢体动作(优选为手势)轨迹,进而分析得到用户交互命令意图。 Then, the information processing unit 101 through a series of software algorithms, to give the user body movement analysis (preferably a gesture) trajectory, and further analysis is intended to give the user interaction commands. 信号运算处理单元101进一步用于向视网膜显示单元102实时提供对应于执行交互用户肢体动作操作命令后的显示信号。 The signal processing unit 101 is further configured to provide real-time retinal display unit 102 performs display signal corresponding to the operation command interactive user body movements.

[0097] 图3为根据本发明实施方式的手势触控交互示意图。 [0097] FIG. 3 is a diagram of a touch interaction according to an embodiment of the present invention gesture. 比如,假如用户的手从右向左划过动作捕获单元104的视场区域,动作捕获单元104时记录并发送图像数据给信号运算处理单元101。 For example, if the user's hand from right to left across the motion capture field area unit 104, and transmits the image data to the recording signal processing unit 101 when the capture unit 104 operation. 信号运算处理单元101通过一系列软件算法从图像数据中分析得出用户手势轨迹为从右向左划动,再通过软件算法确定为某种交互命令(例如:返回上一页)。 Analysis of the signal processing unit 101 through a series of software algorithm derived from the image data a user gesture track swipe from right to left, and then determined by the software algorithm of some interactive commands (e.g.: Back).

[0098] 在实际交互过程中,动作捕获单元104可以捕获一系列的交互动作。 [0098] In the actual interaction process, motion capture unit 104 may capture a series of interactive actions. 比如:可以捕获“开始交互/确定/选择/点击”,“移动(上、下、左、右、前、后)”,“放大”,“缩小”,“旋转”,“退出/结束交互”等的手势动作。 For example: You can capture "start interactive / OK / Select / click", "move (up, down, left, right, front, back)", "Zoom", "Zoom", "rotate", "Exit / end interaction" and other gestures. 信号运算处理单元101在预先保存交互动作与交互命令之间的对应关系。 The signal processing unit 101 in a correspondence relationship stored in advance between the interaction with the interactive command. 信号运算处理单元101得到交互动作后,首先确定出与交互动作线对应的交互命令,并执行该交互命令,而且控制视网膜显示单元102输出相应的交互后显示状态。 The signal processing unit 101 to obtain the interaction, and the interaction is determined that the first line corresponding to the interactive command, and executes the interactive command and control state retinal display unit 102 outputs a corresponding interaction.

[0099] 优选地,信号运算处理单元101具备自学习能力以及一定的用户自定义扩展操作功能,用户可以按照自身的手势习惯训练提高系统的手势识别能力并可以根据用户自身喜好自定义各种操作的手势以及操作方式。 [0099] Preferably, the signal processing unit 101 includes a self-learning ability and a certain user-defined extended operation function, the user can gesture recognition system to improve the diet in accordance with their training gestures and customize various operations according to the user's own preferences gestures and mode of operation. 用户交互识别软件中预设了很多参量,例如人的肤色信息,手臂的长度信息等等,初始情况下这些参量初始值基于统计平均以尽量满足大多数用户,通过软件算法中实现系统的自学习能力,也就是随着用户不断使用,软件能够根据用户自身特点修正其中一些参量使识别交互更倾向于针对特定用户特点,进而提高系统的手势识别能力。 User interaction recognition software in a number of preset parameters, such as people of color information, the arm length information, etc., the initial values ​​of these parameters Initially based on statistical average in order to try to meet most users to achieve self-learning system through software algorithms capability, that is, as users continue to use the software in which the user can correct some of its own characteristics to identify the parameters for a particular user to interact more inclined features thereby increasing the gesture recognition system.

[0100] 此外,用户识别交互软件还应提供用户自定义操作接口,比如用户喜爱的特定手势轨迹代表某种用户自定义的操作命令,从而实现系统的个性化可定制特点。 [0100] In addition, the software also provides user interaction to identify user-defined user interface, such as user's favorite track specific gesture operation on behalf of a certain user-defined command in order to achieve individual characteristics of the system can be customized.

[0101] 更具体地,用户对虚拟界面的交互操作分为两类:一类是识别非精确定位操作,比如“翻页”,“前进”,“后退”等命令。 [0101] More particularly, user interaction, the virtual interface is divided into two types: one is to identify the precise positioning of non-operation, such as "page", "forward", "back" commands. 另一类是实现精确定位操作,比如点击虚拟界面中的按钮或选择一个特定区域等操作。 Another is to achieve accurate positioning operation, such as clicking a button to select a virtual interface or a particular region of operation and the like.

[0102] 对于非精确定位操作的识别,只需要记录分析手的移动轨迹信息即可。 [0102] For precise positioning of non-recognition operation, only the movement trajectory information to record analysis hand. 比如,非精确定位操作可以包括:例如手从右向左划动、手从左向右划动、手从上到下划动以及、手从下到上划动或,以及两手分开、聚拢等。 For example, non-precise positioning operations may include: a hand, for example, from right to left swipe, hand swipe from left to right, top to bottom hand paddling and hand paddling or from bottom to top, and hands separately, like bunching .

[0103] 为了实现精确操作的识别,需要实时跟踪手的运动轨迹,并将该轨迹实时映射到虚拟界面上,与虚拟操作界面上的坐标值--对应,从而实现对界面的精确操作。 [0103] In order to achieve accurate recognition operation, it requires real-time tracking of the hand trajectory, and the real-time trajectory is mapped onto the virtual screen, the coordinate values ​​on the virtual interface - corresponding to, in order to achieve accurate operation of the interface.

[0104] 实际上,触摸屏单元103和动作捕获单元104分别构成了该系统的交互命令输入设备。 [0104] In fact, the operation of the touch screen unit 103 and the capture unit 104 are configured the interactive commands input device of the system. 优选地,可以为触摸屏单元102和动作捕获单元104设置命令优先级,从而避免来自不同命令输入设备之间的命令冲突。 Preferably, a touch panel unit 102 and a motion capture unit 104 sets the priority order so as to avoid a conflict between the command input devices from different commands. 比如,可以设置触摸屏单元102具有较高的优先级,而动作捕获单元104具有较低优先级。 For example, the touch screen unit 102 may be provided with a higher priority, and motion capture unit 104 has a lower priority.

[0105] 这样,当触摸屏单元102和动作捕获单元104同时向信号运算处理单元101提供交互命令的时候,优先执行触摸屏单元102的交互命令,从而避免命令冲突。 [0105] Thus, when the touch panel unit 102 and a motion capture unit 104 while providing interactive commands to the signal processing unit 101, execution priority touch screen unit 102 of the interactive commands, commands to avoid conflicts.

[0106] 基于上述分析,本发明实施方式还提出了一种用户交互方法。 [0106] Based on the above, embodiments of the present invention further provides a method for user interaction.

[0107] 图2为根据本发明实施方式的用户交互方法流程示意图。 [0107] FIG. 2 is a schematic process flow according to a user interaction an embodiment of the present invention.

[0108] 如图2所示,该方法包括: [0108] 2, the method comprising:

[0109] 步骤201:提供显示信号。 [0109] Step 201: display signal.

[0110] 步骤202:将所提供的显示信号投影到用户视网膜,使用户在视觉上感觉出现虚拟界面,且所述显示信号被显示在所述虚拟界面上。 [0110] Step 202: The projection display signal supplied to the retina of the user, make the user feel visually appear in the virtual interface, and the display signals are displayed on the virtual interface.

[0111] 步骤203:捕获用户在触摸屏上的触摸操作,确定对应于该用户触摸操作的交互操作命令。 [0111] Step 203: Capture a user touch operation on the touch screen, to determine the touch operation corresponding to the user's interactive operation command.

[0112] 步骤204:实时提供对应于执行该交互操作命令后的显示信号。 [0112] Step 204: Real-time signal provided corresponding to the display after execution of the command interaction.

[0113] 在上述流程中,优选在步骤201中提供三维立体界面显示信号,从而后续流程中可以显示该三维立体界面。 [0113] In the above process, it is preferable to provide three-dimensional display screen signal in step 201, so that subsequent processes may display the three-dimensional interface. 该方法进一步包括:感测用户头部运动信息;根据所述用户头部运动信息确定用户实时视角,并实时提供基于该用户实时视角下的三维立体界面显示信号。 The method further comprising: sensing motion information of a user's head; determining in real time the perspective of the user information according to the user's head movement, and interfaces in real time based on the three-dimensional perspective of the user in real-time display signal.

[0114] 在该方法中,进一步包括:捕获用户浏览该虚拟界面做出的肢体空间移动信息;确定对应于该用户肢体空间移动信息的交互操作命令;实时提供对应于执行该交互操作命令后的显示信号。 [0114] In this method, further comprising: capturing a physical space of the user browses the virtual movement information to make the interface; determining physical space corresponding to the mobile user interaction command information; real time execution of the interactive operation corresponding to the command display signal.

[0115] 而且,捕获用户浏览该虚拟界面做出的肢体空间移动信息可以包括:捕获用户浏览该虚拟界面而做出的精确定位操作和/或非精确定位操作。 [0115] Further, a user browses to capture body movement information of the virtual space may make the interface comprising: capturing a user to browse the virtual interface made precise positioning operation and / or an exact positioning. 用户对虚拟界面的交互操作分为两类:一类是识别非精确定位操作,比如“翻页”,“前进”,“后退”等命令。 User interaction virtual interface is divided into two types: one is to identify the precise positioning of non-operation, such as "page", "forward", "back" commands. 另一类是实现精确定位操作,比如点击虚拟界面中的按钮或选择一个特定区域等操作。 Another is to achieve accurate positioning operation, such as clicking a button to select a virtual interface or a particular region of operation and the like.

[0116] 对于非精确定位操作的识别,只需要记录分析手的移动轨迹信息即可。 [0116] For precise positioning of non-recognition operation, only the movement trajectory information to record analysis hand. 比如,非精确定位操作可以包括:例如手从右向左划动、手从左向右划动、手从上到下划动以及、手从下到上划动或,以及两手分开、聚拢等。 For example, non-precise positioning operations may include: a hand, for example, from right to left swipe, hand swipe from left to right, top to bottom hand paddling and hand paddling or from bottom to top, and hands separately, like bunching .

[0117] 为了实现精确操作的识别,需要实时跟踪手的运动轨迹,并将该轨迹实时映射到虚拟界面上,与虚拟操作界面上的坐标值--对应,从而实现对界面的精确操作。 [0117] In order to achieve accurate recognition operation, it requires real-time tracking of the hand trajectory, and the real-time trajectory is mapped onto the virtual screen, the coordinate values ​​on the virtual interface - corresponding to, in order to achieve accurate operation of the interface.

[0118] 优选地,在触摸屏或触摸板上可识别的交互操作可以包括单点触摸操作和多点触摸操作。 [0118] Preferably, a touch pad or the touch screen can recognize the interaction may include a single touch operation and the multi-touch operation.

[0119] 单点触摸操作具体可以包括:点击、滑动和长按,等等。 [0119] a single-touch operation may specifically include: clicking, sliding, and long press, and the like. 多点触摸操作具体可以包括旋转(两个手指在触摸屏或触摸板上进行滑动,且手指在触摸屏或触摸板上的触点间距离基本不变,但触点间连线与触摸板边界夹角发生变化)以及缩放(两个手指在触摸屏或触摸板上滑动,且手指在触摸屏或触摸板上的触点间距离增大或缩小),等等。 DETAILED multi-touch operation may include rotating (sliding two fingers on a touch screen or a touch pad, and a substantially constant distance between the finger contacts on the touch screen or a touch pad, but the connection between the contacts with the touch panel border angle changes) and scaling (two fingers on a touch screen or touchpad slide, and the distance between the contacts is increased or reduced finger on the touch screen or a touch pad), and the like.

[0120] 综上所述,在本发明实施方式中,信号运算处理单元,用于向视网膜显示单元提供显示信号;视网膜显示单元,用于将显示信号投影到用户视网膜,使用户在视觉上感觉出现虚拟界面,显示信号被显示在虚拟界面上;触摸屏单元,用于捕获用户在其上的触摸操作动作,并将对应于该触摸操作动作的触摸操作信息发送到信号运算处理单元;信号运算处理单元,进一步用于确定对应于该触摸操作信息的交互操作命令,并向视网膜显示单元实时提供对应于执行该交互操作命令后的显示信号。 [0120] As described above, in the embodiment of the present invention, the signal processing unit, for providing a display signal to the display unit retina; retinal display unit for displaying a signal projected to the retina of the user, making the user feel visually virtual screen appears, the display signal is displayed on the virtual interface; touch screen unit, for capturing a user action on the touch operation thereon, and transmits the touch operation corresponds to the operation information of the touch operation to the signal processing unit; a signal processing operation means is further for determining a touch operation information corresponding to the interactive operation command, and retinal display unit which performs display in real time signals after the interaction commands. 由此可见,应用本发明实施方式以后,通过虚拟界面和触摸屏之间的交互,实现了一种用户与硬件设备之间交互与获取信息的方式,极大地增强了用户体验。 Thus, after the application of embodiments of the present invention, through the interaction between the virtual interface and touch screen to achieve a way between a user and an interaction with the hardware devices to access information, greatly enhancing the user experience.

[0121] 而且,本发明实施的交互方式非常自然,符合人性的基本肢体动作(比如手势)交互模式,而且降低了用户对操作设备的学习成本。 [0121] Further, embodiments of the present invention interact very nature, consistent with the basic operation of the human body (such as gesture) the interactive mode, the user to learn and reduces the cost of operation of the device. 本发明实施方式符合人体自然地交互操控与便携信息处理硬件设备的分体设计,使人能够更集中精力于其所关注的信息而不是硬件设备本身。 Split design embodiment of the present invention is consistent with human nature interact with the manipulation of portable information processing hardware, people can focus more attention on their information rather than hardware device itself.

[0122] 而且,本发明实施方式提出了一种自然交互技术的三维立体虚拟信息自然交互界面,该交互界面包含众多三维立体可进行自然交互的元素。 [0122] Further, embodiments of the present invention provides a natural three-dimensional information on the natural interaction in virtual interface, the interface comprising a number of three-dimensional elements may be made of natural interaction. 通过本发明实施方式所提出的解决方案,用户可以自然地用手控制上述三维立体虚拟信息自然交互界面中对应于用户手的虚拟指针,对三维立体虚拟信息自然交互界面进行自然交互。 Solutions proposed embodiment of the present invention, the user can manually controlling the natural three-dimensional virtual natural interaction interface information corresponding to the user's hand virtual pointer, information on the virtual three-dimensional natural interaction interface natural interaction.

[0123] 另外,本发明实施方式的独特显示方式使其受环境影响较小,给人提供高品质的感官体验,并能够保护信息的私密性。 [0123] Further, a unique display embodiment of the present invention it is less environmental impact, giving sensory experience to provide high quality, and can protect the privacy of information. 本发明实施方式通过直接视网膜扫描投影显示方式可以将虚拟信息与现实实景融合在一起,给人提供增强现实的感官体验,从而基于此可以衍生出大量的有意义的应用,进一步极大地提高用户体验。 Embodiments of the invention by direct retinal scan projection display virtual reality with real information can be fused together, giving sensory experience to provide augmented reality, so that this may be derived based on a large number of interesting applications, greatly improving the user experience is further .

[0124] 不仅于此,本发明实施方式可以应用与任何人机交互信息设备,其通用性将给人们带来极大便利。 [0124] Not only this, embodiments of the present invention can be applied to any human-computer interaction and information equipment, its versatility will bring great convenience.

[0125] 以上所述,仅为本发明实施方式的较佳实施例而已,并非用于限定本发明实施方式的保护范围。 [0125] The above are merely preferred embodiments of the present invention embodiment example only, and are not intended to limit the embodiment of the present invention. 凡在本发明实施方式的精神和原则之内,所作的任何修改、等同替换、改进等,均应包含在本发明实施方式的保护范围之内。 Any modification within the spirit and principle of the embodiment of the present invention, made, equivalent substitutions, improvements, etc., should be included in the protection scope of embodiments of the present invention.

Claims (10)

1.一种用户交互系统,其特征在于,该系统包括信号运算处理单元、视网膜显示单元和触摸屏单元,其中: 信号运算处理单元,用于向视网膜显示单元提供显示信号; 视网膜显示单元,用于将信号运算处理单元所提供的显示信号投影到用户视网膜,使用户在视觉上感觉出现虚拟界面,且所述显示信号被显示在所述虚拟界面上; 触摸屏单元,用于捕获用户在其上的触摸操作动作,并将对应于该触摸操作动作的触摸操作信息发送到信号运算处理单元; 所述信号运算处理单元,进一步用于确定对应于该触摸操作信息的交互操作命令,并向视网膜显示单元实时提供对应于执行该交互操作命令后的显示信号;该系统进一步包括动作捕获单元; 所述动作捕获单元,用于捕获用户浏览该虚拟界面做出的肢体空间移动信息,并将所述肢体空间移动信息发送到信息 A user interaction system, characterized in that the system includes a signal processing unit, the display unit and the touch screen retinal cell, wherein: a signal processing unit for providing a display signal to the display unit retina; retinal display unit for the display signal processing unit is projected onto the retina of the user provided, make the user feel visually appear in the virtual interface, and the display signals are displayed on the virtual interface; touch screen unit, for capturing a user thereon operation of a touch operation and a touch operation corresponding to the touch operation of the operation information to the signal processing unit; said signal processing unit is further for determining a touch operation information corresponding to the interactive operation command, and the display unit retina performing display corresponding to the real time signals after the interaction commands; the system further comprises motion capture unit; the motion capture means for capturing the user interface of the virtual space browsing limb made movement information, and the physical space movement information to the information 算处理单元; 信息运算处理单元,进一步用于确定对应于该用户肢体空间移动信息的交互操作命令,并实时向视网膜显示单元提供对应于执行该交互操作命令后的显示信号;其中为触摸屏单元和动作捕获单元设置命令优先级,从而避免来自不同命令输入设备之间的命令冲突; 信号运算处理单元,进一步用于在虚拟界面上显示空间虚拟指针元素,并且所述空间虚拟指针元素在虚拟界面上的运动轨迹与用户在触摸屏单元上的触摸操作的轨迹保持一致; 信号运算处理单元,用于向视网膜显示单元提供三维立体界面显示信号; 视网膜显示单元,用于根据所述三维立体界面显示信号向用户显示三维立体界面; 该系统进一步包括佩戴于用户头部的视角感知单元; 视角感知单元,用于感测用户头部运动信息,并将所述用户头部运动信息发送到信息运算处理单 Calculation processing unit; information arithmetic processing unit further configured to determine corresponding to the user limbs spatial movement information interaction commands and real-time display signal corresponding to perform the interactive operation command to the retina of the display unit; wherein is a touch screen unit, and capturing unit operation setting command priority order to avoid conflicts between the different commands from an input device; and a signal processing unit is further for displaying the virtual space on the virtual interface element pointer, and the pointer on the virtual space in the virtual interface element trajectory trajectory of the user's touch operation on the touch screen unit of consistency; a signal processing unit for displaying the three-dimensional interface unit provides a retinal display signal; retinal display unit according to the display signal to the three-dimensional interface displaying three-dimensional user interface; the system further comprising a user's head in wearing perspective sensing unit; Angle sensing means for sensing the user's head motion information, and transmits the information to the user's head motion information arithmetic processing unit ; 信息运算处理单元,进一步用于根据所述用户头部运动信息确定用户实时视角,并实时向视网膜显示单元提供基于该用户实时视角下的三维立体界面显示信号。 ; Information arithmetic processing unit is further configured according to the user head-motion real-time information for determining the perspective of the user, and provide real-time three-dimensional interface based on the perspective of the user in real-time display signal to the display unit retina.
2.根据权利要求1所述的用户交互系统,其特征在于,所述视网膜显示单元为用户头戴的眼镜式显示器或直接视网膜投影装置。 2. The user interaction system according to claim 1, wherein the display unit is a retina of a user wearing a glasses-type display apparatus or a direct retinal projection.
3.根据权利要求1所述的用户交互系统,其特征在于,所述信号运算处理单元为移动终端、计算机或基于云计算的信息服务平台。 The user interaction system according to claim 1, characterized in that the signal processing unit of the mobile terminal, or a computer-based information service cloud computing platform.
4.根据权利要求1或3中任一项所述的用户交互系统,其特征在于,所述信号运算处理单元和视网膜显示单元在物理上集成为整体。 1 according to claim 3 or to any one of the user interaction system, characterized in that the signal processing unit and a display unit physically integrated retina integral.
5.根据权利要求1或3中任一项所述的用户交互系统,其特征在于, 信号运算处理单元,用于向视网膜显示单元提供二维虚拟界面显示信号; 视网膜显示单元,用于根据所述二维虚拟界面显示信号向用户显示二维虚拟界面。 The user interaction system as claimed in any one or claims 1-3, characterized in that the signal processing unit, a signal for providing a virtual interface to a two-dimensional display unit to display the retina; retinal display unit according to the said two-dimensional virtual display two-dimensional virtual display signal interface to a user interface.
6.一种用户交互方法,其特征在于,该方法包括: 提供显示信号; 将所提供的显示信号投影到用户视网膜,使用户在视觉上感觉出现虚拟界面,且所述显示信号被显示在所述虚拟界面上; 捕获用户在触摸屏上的触摸操作,确定对应于该用户触摸操作的交互操作命令; 实时提供对应于执行该交互操作命令后的显示信号; 该方法进一步包括: 捕获用户浏览该虚拟界面做出的肢体空间移动信息; 确定对应于该用户肢体空间移动信息的交互操作命令; 实时提供对应于执行该交互操作命令后的显示信号;其中为触摸操作和动作捕获操作设置命令优先级,从而避免来自不同命令输入设备之间的命令冲突; 在虚拟界面上显示空间虚拟指针元素,并且所述空间虚拟指针元素在虚拟界面上的运动轨迹与用户在触摸屏单元上的触摸操作的轨迹保持一致; 所述提供显 A method of user interaction, which is characterized in that, the method comprising: providing a display signal; The projection display signal supplied to the retina of the user, make the user feel visually appear in the virtual interface, and the display signals are displayed on the the above virtual interface; capturing a user touch operation on the touch screen, is determined corresponding to the user touch operation interaction commands; real-time display signal corresponding to perform the interactive operation command; and the method further comprising: capturing a user to browse the virtual interface to make spatial limb movement information; determining physical space corresponding to the mobile user interaction command information; real time signal after performing display corresponding to the interactive operation command; wherein the capture operation setting command touch operation and the action priority, thereby avoiding conflicting command from the command input between different devices; virtual display space on the virtual interface element pointer, and the pointer element trajectory virtual space with a virtual user interface on a touch operation on the touch panel unit keeping track ; providing a significant 信号为:提供三维立体界面显示信号;该方法进一步包括: 感测用户头部运动信息; 根据所述用户头部运动信息确定用户实时视角,并实时提供基于该用户实时视角下的三维立体界面显示信号。 Signal: providing a three-dimensional screen display signal; the method further comprising: sensing motion information of a user's head; determining in real time the perspective of the user information according to the user's head movement, and interfaces in real time based on the three-dimensional perspective of the user in real-time display signal.
7.根据权利要求6所述的用户交互方法,其特征在于,所述捕获用户浏览该虚拟界面做出的肢体空间移动信息为:捕获用户浏览该虚拟界面而做出的精确定位操作和/或非精确定位操作。 7. A method of user interaction according to claim 6, wherein the capture limb space of the mobile user browses the virtual interface information is made: the precise positioning operation to capture the user browses the virtual interface made and / or Inexact positioning operation.
8.根据权利要求6-7中任一项所述的用户交互方法,其特征在于,所述用户在触摸屏上的触摸操作为单点触摸操作或多点触摸操作。 8. A method of user interaction 6-7 according to any one of the preceding claims, wherein the user touch operation on the touch screen is a single-touch operation or a touch operation.
9.根据权利要求8所述的用户交互方法,其特征在于,所述单点触摸操作包括:点击、滑动或长按。 9. A method of user interaction according to claim 8, wherein said single-touch operation includes: clicking, sliding or press.
10.根据权利要求8所述的用户交互方法,其特征在于,所述多点触摸操作包括:旋转或缩放。 10. A method of user interaction according to claim 8, wherein the multi-touch operation comprising: rotation or scaling.
CN201210134998.5A 2012-05-03 2012-05-03 User interaction system and method CN102779000B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201210134998.5A CN102779000B (en) 2012-05-03 2012-05-03 User interaction system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201210134998.5A CN102779000B (en) 2012-05-03 2012-05-03 User interaction system and method

Publications (2)

Publication Number Publication Date
CN102779000A CN102779000A (en) 2012-11-14
CN102779000B true CN102779000B (en) 2015-05-20

Family

ID=47123925

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201210134998.5A CN102779000B (en) 2012-05-03 2012-05-03 User interaction system and method

Country Status (1)

Country Link
CN (1) CN102779000B (en)

Families Citing this family (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103823548A (en) * 2012-11-19 2014-05-28 联想(北京)有限公司 Electronic equipment, wearing-type equipment, control system and control method
US20140168261A1 (en) * 2012-12-13 2014-06-19 Jeffrey N. Margolis Direct interaction system mixed reality environments
CN103914128B (en) * 2012-12-31 2017-12-29 联想(北京)有限公司 The head-mounted electronic device and input method
CN104063039A (en) * 2013-03-18 2014-09-24 朱慧灵 Human-computer interaction method of wearable computer intelligent terminal
CN103246351B (en) * 2013-05-23 2016-08-24 刘广松 A user interactive system and method
CN104238730A (en) * 2013-06-21 2014-12-24 上海复旦上科多媒体有限公司 Smart grid visualization platform and demonstration control method
CN103970272B (en) * 2014-04-10 2018-04-27 北京智谷睿拓技术服务有限公司 Interaction method, apparatus and a user equipment
CN104298350A (en) * 2014-09-28 2015-01-21 联想(北京)有限公司 Information processing method and wearable electronic device
CN104391575A (en) * 2014-11-21 2015-03-04 深圳市哲理网络科技有限公司 Head mounted display device
CN106155284B (en) * 2015-04-02 2019-03-08 联想(北京)有限公司 Electronic equipment and information processing method
CN106527696A (en) * 2016-10-31 2017-03-22 宇龙计算机通信科技(深圳)有限公司 Method for implementing virtual operation and wearable device
CN106569725A (en) * 2016-11-09 2017-04-19 北京小米移动软件有限公司 A method and device for providing input for smart glasses, and touch device

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1115152A (en) * 1993-12-03 1996-01-17 德克萨斯仪器股份有限公司 Visual information system
CN1770063A (en) * 2004-10-01 2006-05-10 通用电气公司 Method and apparatus for surgical operating room information display gaze detection and user prioritization for control
CN1785608A (en) * 2005-11-10 2006-06-14 上海大学 Control platform of multifinger mechanical skillful closed ring real time action
CN101258436A (en) * 2005-09-08 2008-09-03 瑞士电信流动电话公司 Communication device, system and method
CN101662720A (en) * 2008-08-26 2010-03-03 索尼株式会社 Sound processing apparatus, sound image localized position adjustment method and video processing apparatus

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102906623A (en) * 2010-02-28 2013-01-30 奥斯特豪特集团有限公司 Local advertising content on an interactive head-mounted eyepiece

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN1115152A (en) * 1993-12-03 1996-01-17 德克萨斯仪器股份有限公司 Visual information system
CN1770063A (en) * 2004-10-01 2006-05-10 通用电气公司 Method and apparatus for surgical operating room information display gaze detection and user prioritization for control
CN101258436A (en) * 2005-09-08 2008-09-03 瑞士电信流动电话公司 Communication device, system and method
CN1785608A (en) * 2005-11-10 2006-06-14 上海大学 Control platform of multifinger mechanical skillful closed ring real time action
CN101662720A (en) * 2008-08-26 2010-03-03 索尼株式会社 Sound processing apparatus, sound image localized position adjustment method and video processing apparatus

Also Published As

Publication number Publication date
CN102779000A (en) 2012-11-14

Similar Documents

Publication Publication Date Title
CN103097996B (en) Motion control apparatus and method touchscreen
US8659548B2 (en) Enhanced camera-based input
US9535516B2 (en) System for projecting content to a display surface having user-controlled size, shape and location/direction and apparatus and methods useful in conjunction therewith
US8823642B2 (en) Methods and systems for controlling devices using gestures and related 3D sensor
US9910498B2 (en) System and method for close-range movement tracking
CN103261997B (en) A method and apparatus for controlling the display of information input by the user
US9063563B1 (en) Gesture actions for interface elements
US9733789B2 (en) Interfacing with a device via virtual 3D objects
CN103347437B (en) Gaze detection 3d mapping environment
EP2214088A2 (en) Information processing
US20130335573A1 (en) Input method designed for augmented reality goggles
US20160259423A1 (en) Touch fee interface for augmented reality systems
AU2013239179B2 (en) Enhanced virtual touchpad and touchscreen
RU2638004C2 (en) Device for information processing, method for managing display and program
US20100128112A1 (en) Immersive display system for interacting with three-dimensional content
WO2014147686A1 (en) Head-mounted device for user interactions in an amplified reality environment
US20110107216A1 (en) Gesture-based user interface
US10203764B2 (en) Systems and methods for triggering actions based on touch-free gesture detection
US9372544B2 (en) Gesture recognition techniques
US20130154913A1 (en) Systems and methods for a gaze and gesture interface
US9696795B2 (en) Systems and methods of creating a realistic grab experience in virtual reality/augmented reality environments
US9165381B2 (en) Augmented books in a mixed reality environment
US8432362B2 (en) Keyboards and methods thereof
US9016857B2 (en) Multi-touch interactions on eyewear
CN103729055B (en) Multi display device, a stylus, a display control method and a multi-multi-display system

Legal Events

Date Code Title Description
C06 Publication
C10 Request of examination as to substance
COR Bibliographic change or correction in the description

Free format text: CORRECT: ADDRESS; FROM: 100085 HAIDIAN, BEIJING TO: 215021 SUZHOU, JIANGSU PROVINCE

ASS Succession or assignment of patent right

Owner name: SUZHOU CHUDA INFORMATION TECHNOLOGY CO., LTD.

Free format text: FORMER OWNER: TOUCHAIR (BEIJING) TECHNOLOGY CO., LTD.

Effective date: 20140211

C14 Granted