WO2018082162A1 - 用于虚拟现实设备的功能触发方法、装置及虚拟现实设备 - Google Patents

用于虚拟现实设备的功能触发方法、装置及虚拟现实设备 Download PDF

Info

Publication number
WO2018082162A1
WO2018082162A1 PCT/CN2016/111118 CN2016111118W WO2018082162A1 WO 2018082162 A1 WO2018082162 A1 WO 2018082162A1 CN 2016111118 W CN2016111118 W CN 2016111118W WO 2018082162 A1 WO2018082162 A1 WO 2018082162A1
Authority
WO
WIPO (PCT)
Prior art keywords
action
left eye
right eye
function
virtual reality
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2016/111118
Other languages
English (en)
French (fr)
Inventor
马杨杨
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Goertek Techology Co Ltd
Original Assignee
Goertek Techology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Goertek Techology Co Ltd filed Critical Goertek Techology Co Ltd
Publication of WO2018082162A1 publication Critical patent/WO2018082162A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013—Eye tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/012—Walk-in-place systems for allowing a user to walk in a virtual environment while constraining him to a given position in the physical environment

Definitions

  • the present invention relates to the field of virtual reality device technologies, and in particular, to a function triggering method, apparatus, and virtual reality device for a virtual reality device.
  • the general mobile phone product video recording is achieved by clicking the photo or video button on the touch screen to achieve the corresponding function, or there are special shortcut keys to achieve the photo.
  • virtual reality devices generally do not have a touch screen. If you want to take a photo or video, one method is to add an additional touch pad or touch screen, and the other is to add a button specifically for taking pictures.
  • a function triggering method for a virtual reality device includes at least two infrared cameras, and the function triggering method includes:
  • a function triggering apparatus for a virtual reality device comprising at least two infrared cameras, the function triggering device comprising:
  • control module configured to control the first infrared camera to collect the left eye image, and control the second infrared camera to collect the right eye image;
  • An identification module configured to identify an action of the left eye according to the left eye image, according to the right eye The image identifies the action of the right eye;
  • a determining module configured to determine whether the action of the left eye and/or the action of the right eye is a specified action corresponding to the current application completed within a set time
  • a triggering module wherein the action of the left eye and/or the action of the right eye is a specified action corresponding to a current application completed within a set time, triggering the current application execution to correspond to the specified action The function.
  • a virtual reality device comprising the function triggering device according to the second aspect of the invention.
  • a virtual reality device comprising at least two infrared cameras, a processor and a memory, the two infrared cameras respectively for collecting a left eye image and a right eye image, wherein the memory is used for And storing instructions for controlling the processor to perform the function triggering method according to the first aspect of the present invention.
  • a computer readable storage medium storing program code for performing the function triggering method according to the first aspect of the invention.
  • FIG. 1 is a flow chart of an embodiment of a function triggering method for a virtual reality device according to the present invention
  • FIG. 2 is a block schematic diagram of an implementation structure of a function triggering apparatus for a virtual reality device according to the present invention
  • FIG. 3 is a block schematic diagram of an implementation structure of a virtual reality device in accordance with the present invention.
  • the hardware circuit occupies space, thereby increasing the hardware cost of processing the virtual reality device, and providing a virtual reality device.
  • the function trigger method In order to solve the problem that the hardware device triggers the corresponding function of the virtual reality device by increasing the hardware circuit, the hardware circuit occupies space, thereby increasing the hardware cost of processing the virtual reality device, and providing a virtual reality device.
  • the function trigger method In order to solve the problem that the hardware device triggers the corresponding function of the virtual reality device by increasing the hardware circuit, the hardware circuit occupies space, thereby increasing the hardware cost of processing the virtual reality device, and providing a virtual reality device.
  • the virtual reality device is, for example, a head-mounted virtual reality device
  • the occlusion of the eye is equivalent to a nighttime state during the wearing of the user, so it is necessary to use a camera with strong nighttime image capturing capability to track the movement and position of the eye, and
  • the infrared camera has strong night image capturing capability. Therefore, the virtual reality device includes at least two infrared cameras, and the infrared camera has low power supply, and the infrared rays used are respectively two kinds: near infrared rays or short-wave infrared rays, and the wavelength is 0.76 to 1.5 micrometers.
  • the human body is deep, about 5 to 10 mm; far infrared or long-wave infrared, with a wavelength of 1.5 to 400 ⁇ m, is mostly absorbed by the superficial skin, and the penetration depth is less than 2 mm.
  • FIG. 1 is a flow chart of an embodiment of a function triggering method for a virtual reality device in accordance with the present invention.
  • the function triggering method comprises the following steps:
  • Step S110 controlling the first infrared camera to collect the left eye image, and controlling the second infrared camera to collect the right eye image.
  • the first infrared camera is controlled to record the left eye
  • the second infrared camera is controlled to record the right eye
  • the first infrared camera is controlled to take a picture on the left eye
  • the second infrared camera is controlled to take a picture on the right eye.
  • Shooting of the first infrared camera and the second infrared camera The frequencies can be, for example, the same, all taken every 100 ms. Since both infrared cameras are used to track the user's eyes, the pixel resolution requirements of the two infrared cameras are relatively low, and the resolution can be, for example, 320*240.
  • step S120 the motion of the left eye is recognized based on the left eye image, and the motion of the right eye is recognized based on the right eye image.
  • the left eye image and the right eye image are all square. Since the infrared camera is only sensitive to black and white or shade color, the output data is also two-dimensional, and the human The eye structure is generally dark in the middle, and the surrounding is light-colored. Therefore, the infrared camera is used to track the eye. When the user looks at the front, if the two-dimensional data output by the infrared camera is mapped to the corresponding square image, respectively.
  • the dark infrared camera data in the middle surrounded by infrared camera data representing the light color
  • the dark data on the image will also move in the same direction or in the opposite direction;
  • the corresponding two-dimensional data of the infrared camera maps the image of a single color, thereby judging the state of the left and right eyes of the user and the left eye. And the position of the right eye, and then according to the position and state of the left and right eyes of the user at different times, the motion of the left eye of the user and the action of the right eye can be judged.
  • step S120 may specifically include:
  • the state of the left eye is detected as blinking, and the state of the right eye is blinking; at time t1, the state of the left eye is detected as closed eyes, and the state of the right eye is closed eyes; at time t2, detection is performed.
  • the state to the left eye is blinking, and the state of the right eye is blinking; at time t3, the state of the left eye is closed and the state of the right eye is closed; at time t4, the state of the left eye is detected as
  • the blinking or right eye state is blinking, it can be considered that the user's left eye and right eye motions are blinking.
  • the position of the left eye map on the left eye image is the left side of the square area, and the position of the right eye maps the position on the right eye image as the left side of the square area;
  • the left eye is detected.
  • the position of the map on the left eye image is above the square area, and the position map of the right eye
  • the position on the image of the right eye is above the square area;
  • the position of the left eye map on the left eye image is detected as the right side of the square area, and the position of the right eye is mapped to the square area.
  • the position of the left eye map on the left eye image is below the square area
  • the position of the right eye maps the position on the right eye image below the square area
  • the left eye is detected
  • the position mapped on the left eye image is the left side of the square area
  • the position of the right eye maps the position on the right eye image to the left side of the square area, and it can be considered that the actions of the left eye and the right eye of the user are blinking eyes.
  • the interval between the times t0, t1, t2, t3, and t4 is short, and the interval between the times t0 and t4 may be less than the set time.
  • step S130 it is determined whether the action of the left eye and/or the action of the right eye is the designated action corresponding to the current application completed within the set time. If yes, step S140 is performed, and if no, step S110 is continued.
  • the specified action includes, for example, at least one of blinking, closed eyes, and blinking, or any combination.
  • the specified action corresponding to triggering a certain function may be blinking the left eye and closing the eye of the right eye, or the left eye and the right eye may simultaneously turn the eye, or may be the left eye closed eye.
  • blinking is a normal physiological need of the human eye in a normal state
  • the setting is specified.
  • the blinking time of the blink of the action is 500ms, so that the interference of the normal blinking motion can be eliminated.
  • the function triggering method further includes: presetting a specified action corresponding to different functions for different applications.
  • the application may be, for example, a video playing application, a shooting application, or other APP installed in a virtual reality device.
  • a specified action of blinking in the video playing application for the corresponding playing or pause function may be preset, and the shooting action is a designated action corresponding to the photographing function.
  • Step S140 triggering the current application to perform a function corresponding to the specified action.
  • the virtual reality device further includes a color camera
  • the current application is a shooting application, for example, the specified action corresponding to the camera function in the shooting application is left.
  • the shooting application is turned on, if both the left eye motion and the right eye motion are detected to be closed for 2 seconds, for example, the shooting application is triggered to perform the photographing function.
  • the color camera is controlled to capture the current preview screen.
  • the method further includes:
  • the alarm prompts the user.
  • the specific alarm prompting the user may include, for example, any one of vibration, sound generation, and music playing, or any combination. In this way, the user can be reminded that the function corresponding to the specified action is completed, thereby further improving the user experience.
  • the action of the left eye and/or the action of the right eye is tracked by the infrared camera, and the current application is triggered when the action of the left eye and/or the action of the right eye is the specified action corresponding to the current application completed within the set time.
  • Performing functions corresponding to specified actions can reduce hardware costs and improve user experience.
  • the present invention also provides a function triggering device for a virtual reality device, the virtual reality device comprising at least two infrared cameras.
  • FIG. 2 is a block schematic diagram of an implementation structure of a function triggering apparatus for a virtual reality device according to the present invention.
  • the function triggering device 200 includes a control module 210, an identification module 220, a determining module 230, and a triggering module 240.
  • the control module 210 is configured to control the first infrared camera to collect the left eye image, and the second infrared camera to collect the right eye image.
  • the identification module 220 is configured to recognize the motion of the left eye according to the left eye image, and recognize the motion of the right eye according to the right eye image.
  • the determining module 230 is configured to determine whether the action of the left eye and/or the action of the right eye is a specified action corresponding to the current application that is completed within the set time.
  • the triggering module 240 When the triggering module 240 is used for the action of the left eye and/or the action of the right eye is a specified action corresponding to the current application that is completed within the set time, the current application is triggered to perform a function corresponding to the specified action.
  • the foregoing identification module is specifically configured to: identify a left eye The position of the dark data in the image is used to determine the state and position of the left eye, and then the motion of the left eye is determined according to the state and position of the left eye at different times; the position of the dark data in the image of the right eye is recognized to determine the state of the right eye and Position, and then determine the action of the right eye according to the state and position of the right eye at different times.
  • the specified action includes at least one of blinking, closed eyes and blinking eyes or any combination.
  • the function triggering device further includes an alarm module, configured to issue an alarm to prompt the user after the triggering module 240 triggers the current application to perform the function corresponding to the specified motion.
  • the function triggering device further includes a setting module for presetting the specified actions corresponding to different functions for different applications.
  • the present invention also provides a virtual reality device.
  • the virtual reality device includes the aforementioned function triggering device 200 for a virtual reality device.
  • FIG. 3 is a block schematic diagram of an implementation structure of the virtual reality device in accordance with another aspect of the present invention.
  • the virtual reality device 300 includes at least two infrared cameras, a memory 301 and a processor 302.
  • the two infrared cameras are respectively used for acquiring a left eye image and a right eye image.
  • the memory 301 is configured to store an instruction.
  • the instructions are for controlling the processor 302 to operate to perform the above-described function triggering method for the virtual reality device.
  • the processor can be, for example, a central processing unit CPU, a microprocessor MCU, or the like.
  • the memory includes, for example, a ROM (Read Only Memory), a RAM (Random Access Memory), a nonvolatile memory such as a hard disk, and the like.
  • the virtual reality device 300 further includes an interface device 303, an input device 304, a display device 305, a communication device 306, a speaker 307, a microphone 308, and the like.
  • an interface device 303 an input device 304
  • a display device 305 a display device 305
  • a communication device 306 a speaker 307
  • a microphone 308 a microphone
  • the virtual reality device of the present invention may relate only to some of the devices, such as the processor 301, the memory 302, the display device 305, and the like.
  • the communication device 306 can be wired or wirelessly communicated, for example.
  • the interface device 303 includes, for example, a headphone jack, a USB interface, etc., such that the USB interface of the interface device 303 establishes a connection with other electronic devices such as a tracking light ball, a handle, etc. via a USB data line.
  • the input device 304 described above may include, for example, a touch screen, a button, and the like.
  • the display device 305 described above is, for example, a liquid crystal display, a touch display, or the like.
  • the virtual reality device of the present invention may be, for example, a head mounted virtual reality product.
  • the invention can be an apparatus, method and/or computer program product.
  • the computer program product can comprise a computer readable storage medium having computer readable program instructions embodied thereon for causing a processor to implement various aspects of the present invention.
  • the computer readable storage medium can be a tangible device that can hold and store the instructions used by the instruction execution device.
  • the computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • Non-exhaustive list of computer readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM) Or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, for example, with instructions stored thereon A raised structure in the hole card or groove, and any suitable combination of the above.
  • a computer readable storage medium as used herein is not to be interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (eg, a light pulse through a fiber optic cable), or through a wire The electrical signal transmitted.
  • the computer readable program instructions described herein can be downloaded from a computer readable storage medium to various computing/processing devices, or via a network, such as the Internet, a local area network, a wide area network, and/or Or the wireless network is downloaded to an external computer or an external storage device.
  • the network may include copper transmission cables, fiber optic transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and/or edge servers.
  • a network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in each computing/processing device .
  • Computer program instructions for performing the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine related instructions, microcode, firmware instructions, state setting data, or in one or more programming languages.
  • the computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer, partly on the remote computer, or entirely on the remote computer or server. carried out.
  • the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or wide area network (WAN), or can be connected to an external computer (eg, using an Internet service provider to access the Internet) connection).
  • the customized electronic circuit such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), can be customized by utilizing state information of computer readable program instructions.
  • Computer readable program instructions are executed to implement various aspects of the present invention.
  • the computer readable program instructions can be provided to a general purpose computer, a special purpose computer, or a processor of other programmable data processing apparatus to produce a machine such that when executed by a processor of a computer or other programmable data processing apparatus Means for implementing the functions/acts specified in one or more of the blocks of the flowcharts and/or block diagrams.
  • the computer readable program instructions can also be stored in a computer readable storage medium that causes the computer, programmable data processing device, and/or other device to operate in a particular manner, thereby storing instructions
  • the computer readable medium then includes an article of manufacture that includes instructions for implementing various aspects of the functions/acts recited in one or more of the flowcharts.
  • the computer readable program instructions can also be loaded onto a computer, other programmable data processing device, or other device to perform a series of operational steps on a computer, other programmable data processing device or other device to produce a computer-implemented process.
  • instructions executed on a computer, other programmable data processing apparatus, or other device implement the functions/acts recited in one or more of the flowcharts and/or block diagrams.
  • each block in the flowchart or block diagram can represent a module, a program segment, or a portion of an instruction that includes one or more components for implementing the specified logical functions.
  • Executable instructions can also occur in a different order than those illustrated in the drawings. For example, two consecutive blocks may be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending upon the functionality involved.
  • each block of the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts can be implemented in a dedicated hardware-based system that performs the specified function or function. Or it can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, implementation by software, and implementation by a combination of software and hardware are equivalent.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)

Abstract

一种用于虚拟现实设备的功能触发方法、装置及虚拟现实设备，包括：控制第一红外摄像头采集左眼图像，控制第二红外摄像头采集右眼图像（S110）；根据左眼图像识别出左眼的动作，根据右眼图像识别出右眼的动作（S120）；判断左眼的动作和/或右眼的动作是否为在设定时间内完成的对应当前应用的指定动作（S130）；如是，则触发当前应用执行与指定动作对应的功能（S140）。能够降低硬件成本，提升用户体验。

Description

用于虚拟现实设备的功能触发方法、装置及虚拟现实设备 技术领域
本发明涉及虚拟现实设备技术领域，更具体地，涉及一种用于虚拟现实设备的功能触发方法、装置及虚拟现实设备。
背景技术
一般的手机产品上拍照录像是通过点击触摸屏上的拍照或者录像按键来实现相应的功能，或者有专门的快捷键来实现拍照。相比之下，虚拟现实设备一般都没有触摸屏，如果要进行拍照或者录像，一个方法是增加额外的触摸板或者触摸屏，另外就是增加个按键专门用来进行拍照。
发明内容
根据本发明的第一方面，提供了一种用于虚拟现实设备的功能触发方法，所述虚拟现实设备包括至少两个红外摄像头，所述功能触发方法包括：
控制第一红外摄像头采集左眼图像，控制第二红外摄像头采集右眼图像；
根据所述左眼图像识别出左眼的动作，根据所述右眼图像识别出右眼的动作；
判断所述左眼的动作和/或所述右眼的动作是否为在设定时间内完成的对应当前应用的指定动作，如是，则触发所述当前应用执行与所述指定动作对应的功能。
根据本发明的第二方面，提供了一种用于虚拟现实设备的功能触发装置，所述虚拟现实设备包括至少两个红外摄像头，所述功能触发装置包括：
控制模块，用于控制第一红外摄像头采集左眼图像，控制第二红外摄像头采集右眼图像；
识别模块，用于根据所述左眼图像识别出左眼的动作，根据所述右眼 图像识别出右眼的动作；
判断模块，用于判断所述左眼的动作和/或所述右眼的动作是否为在设定时间内完成的对应当前应用的指定动作；以及，
触发模块，用于所述左眼的动作和/或所述右眼的动作为在设定时间内完成的对应当前应用的指定动作的情况下，触发所述当前应用执行与所述指定动作对应的功能。
根据本发明的第三方面，提供了一种虚拟现实设备，包括根据本发明第二方面所述的功能触发装置。
根据本发明的第四方面，提供了一种虚拟现实设备，包括至少两个红外摄像头、处理器和存储器，所述两个红外摄像头分别用于采集左眼图像和右眼图像，所述存储器用于存储指令，所述指令用于控制所述处理器执行根据本发明第一方面所述的功能触发方法。
根据本发明的第五方面，提供了一种计算机可读存储介质，其存储有用于执行根据本发明的第一方面所述功能触发方法的程序代码。
通过以下参照附图对本发明的示例性实施例的详细描述，本发明的其它特征及其优点将会变得清楚。
附图说明
被结合在说明书中并构成说明书的一部分的附图示出了本发明的实施例，并且连同其说明一起用于解释本发明的原理。
图1为根据本发明一种用于虚拟现实设备的功能触发方法的一种实施方式的流程图；
图2为根据本发明一种用于虚拟现实设备的功能触发装置的一种实施结构的方框原理图；
图3为根据本发明一种虚拟现实设备的一种实施结构的方框原理图。
具体实施方式
现在将参照附图来详细描述本发明的各种示例性实施例。应注意到：除非另外具体说明，否则在这些实施例中阐述的部件和步骤的相对布置、 数字表达式和数值不限制本发明的范围。
以下对至少一个示例性实施例的描述实际上仅仅是说明性的，决不作为对本发明及其应用或使用的任何限制。
对于相关领域普通技术人员已知的技术、方法和设备可能不作详细讨论，但在适当情况下，所述技术、方法和设备应当被视为说明书的一部分。
在这里示出和讨论的所有例子中，任何具体值应被解释为仅仅是示例性的，而不是作为限制。因此，示例性实施例的其它例子可以具有不同的值。
应注意到：相似的标号和字母在下面的附图中表示类似项，因此，一旦某一项在一个附图中被定义，则在随后的附图中不需要对其进行进一步讨论。
为了解决现有技术中存在的通过增加硬件电路触发虚拟现实设备对应功能的方式，增加了硬件电路占用空间，进而增加了加工虚拟现实设备的硬件成本的问题，提供了一种用于虚拟现实设备的功能触发方法。
其中，由于虚拟现实设备例如是头戴式虚拟现实设备在用户佩戴过程中，眼部被遮挡相当于是夜间状态，因此需使用夜间图像捕捉能力较强的摄像头来跟踪眼的动作及位置，又由于红外摄像头的夜间图像捕捉能力强，因此，该虚拟现实设备包括至少两个红外摄像头，红外摄像头供电低，使用的红外线分别为两种：近红外线或称短波红外线，波长0.76～1.5微米，穿入人体组织较深,约5～10毫米；远红外线或称长波红外线，波长1.5～400微米，多被表层皮肤吸收，穿透组织深度小于2毫米。
图1为根据本发明一种用于虚拟现实设备的功能触发方法的一种实施方式的流程图。
根据图1所示，该功能触发方法包括以下步骤：
步骤S110，控制第一红外摄像头采集左眼图像，控制第二红外摄像头采集右眼图像。
具体为，控制第一红外摄像头对左眼进行录像，控制第二红外摄像头对右眼进行录像；也可以是控制第一红外摄像头对左眼进行拍照，控制第二红外摄像头对右眼进行拍照，第一红外摄像头和第二红外摄像头的拍摄 频率例如可以相同，均为每100ms拍摄一次。由于这两个红外摄像头均用于跟踪用户眼，因此这两个红外摄像头的像素分辨率要求都比较低，其分辨率例如可以是320*240。
步骤S120，根据左眼图像识别出左眼的动作，根据右眼图像识别出右眼的动作。
在红外摄像头的分辨率为320*240的情况下，左眼图像和右眼图像对应的就都是方形，由于红外摄像头只对黑白或者是深浅色彩敏感，输出的数据也是二维的，而人的眼结构一般都是中间深色，周围是浅色的，因此，用红外摄像头来跟踪眼、当用户目光看向前方时，如果把红外摄像头输出的二维数据分别映射到对应的方形的图像上，此时应该是代表深色的红外摄像头数据在中间，周围是代表浅色的红外摄像头数据，眼转动时，图像上的深色数据也会随着同方向或者逆方向的移动；当用户闭上眼睛时，此时眼被遮挡，不会再有深浅之分，对应的红外摄像头的二维数据映射的是单一颜色的图像，由此可以判断用户左眼和右眼的状态及左眼和右眼位置，再根据用户左眼和右眼在不同时刻的位置及状态就能够判断用户左眼的动作和右眼的动作。
因此，步骤S120具体可以包括：
识别出左眼图像中深色数据的位置以判断左眼的状态及位置，再根据不同时刻左眼的状态及位置确定左眼的动作；识别出右眼图像中深色数据的位置以判断右眼的状态及位置，再根据不同时刻右眼的状态及位置确定右眼的动作。
例如在t0时刻，检测到左眼的状态为睁眼、右眼的状态为睁眼；在t1时刻，检测到左眼的状态为闭眼、右眼的状态为闭眼；在t2时刻，检测到左眼的状态为睁眼、右眼的状态为睁眼；在t3时刻，检测到左眼的状态为闭眼、右眼的状态为闭眼；在t4时刻，检测到左眼的状态为睁眼、右眼的状态为睁眼，则可以认为用户的左眼和右眼的动作均为眨眼。
例如在t0时刻，检测到左眼映射在左眼图像上的位置为方形区域的左侧，右眼的位置映射右眼图像上的位置为方形区域的左侧；在t1时刻，检测到左眼映射在左眼图像上的位置为方形区域的上方，右眼的位置映射 右眼图像上的位置为方形区域的上方；在t2时刻，检测到左眼映射在左眼图像上的位置为方形区域的右侧，右眼的位置映射右眼图像上的位置为方形区域的右侧；在t3时刻，检测到左眼映射在左眼图像上的位置为方形区域的下方，右眼的位置映射右眼图像上的位置为方形区域的下方；在t4时刻，检测到左眼映射在左眼图像上的位置为方形区域的左侧，右眼的位置映射右眼图像上的位置为方形区域的左侧，则可以认为用户的左眼和右眼的动作均为转眼。
再例如，在t0、t1、t2、t3、t4时刻检测到左眼的状态均为闭眼，则可以认为用户的左眼的动作为闭眼。
其中，上述t0、t1、t2、t3、t4时刻之间的间隔时间很短，且t0至t4时刻之间的间隔时间可以小于设定时间。
步骤S130，判断左眼的动作和/或右眼的动作是否为在设定时间内完成的对应当前应用的指定动作，如是，则执行步骤S140，如否，则继续执行步骤S110。
其中，指定动作例如至少包括眨眼、闭眼和转眼中的任意一种或者是任意组合。例如，在当前应用中触发某一个功能对应的指定动作可以为左眼眨眼且右眼闭眼，也可以是左眼和右眼同时转眼，还可以是左眼闭眼。
进一步地，由于眨眼是人眼在正常状态下的正常生理需要，为了避免用户正常眨眼对人眼动作检测结果的干扰，例如可以在正常眨眼时的闭眼时间为100ms的情况下，设置作为指定动作的眨眼的闭眼时间为500ms，这样，就能够排除正常眨眼动作的干扰。
在此基础上，该功能触发方法还包括：针对不同应用，预先设定对应不同功能的指定动作。具体的，该应用例如可以是视频播放应用、拍摄应用或者是虚拟现实设备内安装的其他APP等。例如，可以预先设定眨眼在视频播放应用中为对应播放或者是暂停功能的指定动作，在拍摄应用中为对应拍照功能的指定动作。
步骤S140，触发当前应用执行与指定动作对应的功能。
在本发明的一个具体实施例中，该虚拟现实设备还包括彩色摄像头，当前应用为拍摄应用，例如在拍摄应用中的拍照功能对应的指定动作为左 眼和右眼均闭眼、且打开拍摄应用的情况下，检测到左眼动作和右眼动作均在设定时间例如是2s内持续为闭眼时，则将触发该拍摄应用执行拍照功能，具体为控制彩色摄像头拍摄当前预览画面。
进一步地，为了使得用户能够清楚地知道与指定动作对应的功能执行完毕，在本发明的一个具体实施例中，执行完步骤S140之后还包括：
报警提示用户。
其中，具体的报警提示用户的方式例如可以包括震动、发出声音和播放音乐中的任意一种或者是任意组合。这样，能够提醒用户与指定动作对应的功能执行完毕，进一步提升用户体验。
这样，通过红外摄像头跟踪左眼的动作和/或右眼的动作，在左眼的动作和/或右眼的动作为设定时间内完成的对应当前应用的指定动作的情况下，触发当前应用执行与指定动作对应的功能，能够降低硬件成本，提升用户体验。
与上述功能触发方法对应的，本发明还提供了一种用于虚拟现实设备的功能触发装置，该虚拟现实设备包括至少两个红外摄像头。
图2为根据本发明一种用于虚拟现实设备的功能触发装置的一种实施结构的方框原理图。
根据图2所示，该功能触发装置200包括控制模块210、识别模块220、判断模块230和触发模块240。
上述控制模块210用于控制第一红外摄像头采集左眼图像，控制第二红外摄像头采集右眼图像。
上述识别模块220用于根据左眼图像识别出左眼的动作，根据右眼图像识别出右眼的动作。
上述判断模块230用于判断左眼的动作和/或右眼的动作是否为在设定时间内完成的对应当前应用的指定动作。
上述触发模块240用于左眼的动作和/或右眼的动作为在设定时间内完成的对应当前应用的指定动作的情况下，触发当前应用执行与指定动作对应的功能。
在本发明的一个具体实施例中，上述识别模块具体用于：识别出左眼 图像中深色数据的位置以判断左眼的状态及位置，再根据不同时刻左眼的状态及位置确定左眼的动作；识别出右眼图像中深色数据的位置以判断右眼的状态及位置，再根据不同时刻右眼的状态及位置确定右眼的动作。
其中，指定动作至少包括眨眼、闭眼和转眼中的任意一种或者任意组合。
进一步地，该功能触发装置还包括报警模块，用于在触发模块240触发当前应用执行与指定动对应的功能之后发出警报提示用户。
在此基础上，该功能触发装置还包括设定模块，用于针对不同应用，预先设定对应不同功能的指定动作。
本发明还提供了一种虚拟现实设备，在一方面，该虚拟现实设备包括前述的用于虚拟现实设备的功能触发装置200。
图3为根据本发明另一方面的该虚拟现实设备的实施结构的方框原理图。
根据图3所示，该虚拟现实设备300包括至少两个红外摄像头、存储器301和处理器302，两个红外摄像头分别用于采集左眼图像和右眼图像；该存储器301用于存储指令，该指令用于控制处理器302进行操作以执行上述用于虚拟现实设备的功能触发方法。
该处理器例如可以是中央处理器CPU、微处理器MCU等。该存储器例如包括ROM(只读存储器)、RAM(随机存取存储器)、诸如硬盘的非易失性存储器等。
除此之外，根据图3所示，该虚拟现实设备300还包括接口装置303、输入装置304、显示装置305、通信装置306、扬声器307、麦克风308等等。尽管在图3中示出了多个装置，但是，本发明虚拟现实设备可以仅涉及其中的部分装置，例如，处理器301、存储器302、显示装置305等。
上述通信装置306例如能够进行有有线或无线通信。
上述接口装置303例如包括耳机插孔、USB接口等，以使接口装置303的USB接口通过USB数据线与其他电子设备例如是追踪光球、手柄等建立连接。
上述输入装置304例如可以包括触摸屏、按键等。
上述显示装置305例如是液晶显示屏、触摸显示屏等。
本发明虚拟现实设备例如可以是头戴式虚拟现实产品。
上述各实施例主要重点描述与其他实施例的不同之处，但本领域技术人员应当清楚的是，上述各实施例可以根据需要单独使用或者相互结合使用。
本说明书中的各个实施例均采用递进的方式描述，各个实施例之间相同相似的部分相互参见即可，每个实施例重点说明的都是与其他实施例的不同之处，但本领域技术人员应当清楚的是，上述各实施例可以根据需要单独使用或者相互结合使用。另外，对于装置实施例而言，由于其是与方法实施例相对应，所以描述得比较简单，相关之处参见方法实施例的对应部分的说明即可。以上所描述的系统实施例仅仅是示意性的，其中作为分离部件说明的模块可以是或者也可以不是物理上分开的。
本发明可以是装置、方法和/或计算机程序产品。计算机程序产品可以包括计算机可读存储介质，其上载有用于使处理器实现本发明的各个方面的计算机可读程序指令。
计算机可读存储介质可以是可以保持和存储由指令执行设备使用的指令的有形设备。计算机可读存储介质例如可以是――但不限于――电存储设备、磁存储设备、光存储设备、电磁存储设备、半导体存储设备或者上述的任意合适的组合。计算机可读存储介质的更具体的例子(非穷举的列表)包括：便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、静态随机存取存储器(SRAM)、便携式压缩盘只读存储器(CD-ROM)、数字多功能盘(DVD)、记忆棒、软盘、机械编码设备、例如其上存储有指令的打孔卡或凹槽内凸起结构、以及上述的任意合适的组合。这里所使用的计算机可读存储介质不被解释为瞬时信号本身，诸如无线电波或者其他自由传播的电磁波、通过波导或其他传输媒介传播的电磁波(例如，通过光纤电缆的光脉冲)、或者通过电线传输的电信号。
这里所描述的计算机可读程序指令可以从计算机可读存储介质下载到各个计算/处理设备，或者通过网络、例如因特网、局域网、广域网和/ 或无线网下载到外部计算机或外部存储设备。网络可以包括铜传输电缆、光纤传输、无线传输、路由器、防火墙、交换机、网关计算机和/或边缘服务器。每个计算/处理设备中的网络适配卡或者网络接口从网络接收计算机可读程序指令，并转发该计算机可读程序指令，以供存储在各个计算/处理设备中的计算机可读存储介质中。
用于执行本发明操作的计算机程序指令可以是汇编指令、指令集架构(ISA)指令、机器指令、机器相关指令、微代码、固件指令、状态设置数据、或者以一种或多种编程语言的任意组合编写的源代码或目标代码，所述编程语言包括面向对象的编程语言—诸如Smalltalk、C++等，以及常规的过程式编程语言—诸如“C”语言或类似的编程语言。计算机可读程序指令可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络—包括局域网(LAN)或广域网(WAN)—连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。在一些实施例中，通过利用计算机可读程序指令的状态信息来个性化定制电子电路，例如可编程逻辑电路、现场可编程门阵列(FPGA)或可编程逻辑阵列(PLA)，该电子电路可以执行计算机可读程序指令，从而实现本发明的各个方面。
这里参照根据本发明实施例的方法、装置(系统)和计算机程序产品的流程图和/或框图描述了本发明的各个方面。应当理解，流程图和/或框图的每个方框以及流程图和/或框图中各方框的组合，都可以由计算机可读程序指令实现。
这些计算机可读程序指令可以提供给通用计算机、专用计算机或其它可编程数据处理装置的处理器，从而生产出一种机器，使得这些指令在通过计算机或其它可编程数据处理装置的处理器执行时，产生了实现流程图和/或框图中的一个或多个方框中规定的功能/动作的装置。也可以把这些计算机可读程序指令存储在计算机可读存储介质中，这些指令使得计算机、可编程数据处理装置和/或其他设备以特定方式工作，从而，存储有指令的 计算机可读介质则包括一个制造品，其包括实现流程图和/或框图中的一个或多个方框中规定的功能/动作的各个方面的指令。
也可以把计算机可读程序指令加载到计算机、其它可编程数据处理装置、或其它设备上，使得在计算机、其它可编程数据处理装置或其它设备上执行一系列操作步骤，以产生计算机实现的过程，从而使得在计算机、其它可编程数据处理装置、或其它设备上执行的指令实现流程图和/或框图中的一个或多个方框中规定的功能/动作。
附图中的流程图和框图显示了根据本发明的多个实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或指令的一部分，所述模块、程序段或指令的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。对于本领域技术人员来说公知的是，通过硬件方式实现、通过软件方式实现以及通过软件和硬件结合的方式实现都是等价的。
以上已经描述了本发明的各实施例，上述说明是示例性的，并非穷尽性的，并且也不限于所披露的各实施例。在不偏离所说明的各实施例的范围和精神的情况下，对于本技术领域的普通技术人员来说许多修改和变更都是显而易见的。本文中所用术语的选择，旨在最好地解释各实施例的原理、实际应用或对市场中的技术的技术改进，或者使本技术领域的其它普通技术人员能理解本文披露的各实施例。本发明的范围由所附权利要求来限定。

Claims (11)

  1. 一种用于虚拟现实设备的功能触发方法，所述虚拟现实设备包括至少两个红外摄像头，其特征在于，所述功能触发方法包括：
    控制第一红外摄像头采集左眼图像，控制第二红外摄像头采集右眼图像；
    根据所述左眼图像识别出左眼的动作，根据所述右眼图像识别出右眼的动作；
    判断所述左眼的动作和/或所述右眼的动作是否为在设定时间内完成的对应当前应用的指定动作，如是，则触发所述当前应用执行与所述指定动作对应的功能。
  2. 根据权利要求1所述的功能触发方法，其特征在于，所述触发所述当前应用执行与所述指定动对应的功能之后还包括：
    发出警报提示用户。
  3. 根据权利要求1或2所述的功能触发方法，其特征在于，所述根据所述左眼图像识别出左眼的动作，根据所述右眼图像识别出右眼的动作具体为：
    识别出所述左眼图像中深色数据的位置以判断所述左眼的状态及位置，再根据不同时刻所述左眼的状态及位置确定所述左眼的动作；识别出所述右眼图像中深色数据的位置以判断所述右眼的状态及位置，再根据不同时刻所述右眼的状态及位置确定所述右眼的动作。
  4. 根据权利要求1-3中任一项所述的功能触发方法，其特征在于，所述指定动作至少包括眨眼、闭眼和转眼中的任意一种或者任意组合。
  5. 一种用于虚拟现实设备的功能触发装置，所述虚拟现实设备包括至少两个红外摄像头，其特征在于，所述功能触发装置包括：
    控制模块，用于控制第一红外摄像头采集左眼图像，控制第二红外摄像头采集右眼图像；
    识别模块，用于根据所述左眼图像识别出左眼的动作，根据所述右眼图像识别出右眼的动作；
    判断模块，用于判断所述左眼的动作和/或所述右眼的动作是否为在 设定时间内完成的对应当前应用的指定动作；以及，
    触发模块，用于所述左眼的动作和/或所述右眼的动作为在设定时间内完成的对应当前应用的指定动作的情况下，触发所述当前应用执行与所述指定动作对应的功能。
  6. 根据权利要求5所述的功能触发装置，其特征在于，所述功能触发装置还包括：
    报警模块，用于在所述触发模块触发所述当前应用执行与所述指定动对应的功能之后发出警报提示用户。
  7. 根据权利要求5或6所述的功能触发装置，其特征在于，所述识别模块具体用于：
    识别出所述左眼图像中深色数据的位置以判断所述左眼的状态及位置，再根据不同时刻所述左眼的状态及位置确定所述左眼的动作；识别出所述右眼图像中深色数据的位置以判断所述右眼的状态及位置，再根据不同时刻所述右眼的状态及位置确定所述右眼的动作。
  8. 根据权利要求5-7中任一项所述的功能触发装置，其特征在于，所述指定动作至少包括眨眼、闭眼和转眼中的任意一种或者任意组合。
  9. 一种虚拟现实设备，其特征在于，包括根据权利要求5-8中任一项所述的功能触发装置。
  10. 一种虚拟现实设备，其特征在于，包括至少两个红外摄像头、处理器和存储器，所述两个红外摄像头分别用于采集左眼图像和右眼图像，所述存储器用于存储指令，所述指令用于控制所述处理器执行所述权利要求1-4任一项所述的功能触发方法。
  11. 一种计算机可读存储介质，其特征在于，存储有用于执行根据权利要求1-4中任一项所述功能触发方法的程序代码。
PCT/CN2016/111118 2016-11-02 2016-12-20 用于虚拟现实设备的功能触发方法、装置及虚拟现实设备 Ceased WO2018082162A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610946056.5A CN106598214A (zh) 2016-11-02 2016-11-02 用于虚拟现实设备的功能触发方法、装置及虚拟现实设备
CN201610946056.5 2016-11-02

Publications (1)

Publication Number Publication Date
WO2018082162A1 true WO2018082162A1 (zh) 2018-05-11

Family

ID=58589644

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/111118 Ceased WO2018082162A1 (zh) 2016-11-02 2016-12-20 用于虚拟现实设备的功能触发方法、装置及虚拟现实设备

Country Status (2)

Country Link
CN (1) CN106598214A (zh)
WO (1) WO2018082162A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112817441A (zh) * 2021-01-14 2021-05-18 深圳卡多希科技有限公司 一种按键与人眼识别结合的方法和装置
CN114979614A (zh) * 2022-05-16 2022-08-30 北京芯海视界三维科技有限公司 显示模式确定方法及显示模式确定装置

Families Citing this family (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107292261B (zh) * 2017-06-16 2021-07-13 深圳天珑无线科技有限公司 一种拍照方法及其移动终端
CN107704849A (zh) * 2017-10-28 2018-02-16 上海爱优威软件开发有限公司 双重验证功能的人脸识别方法及系统
CN107982913B (zh) * 2017-12-11 2023-10-24 歌尔科技有限公司 一种游戏手柄和获取游戏手柄控制方向的方法
CN108345844B (zh) * 2018-01-26 2020-11-20 上海歌尔泰克机器人有限公司 控制无人机拍摄的方法及装置、虚拟现实设备及系统
CN112613389A (zh) * 2020-12-18 2021-04-06 上海影创信息科技有限公司 眼部手势控制方法和系统及其vr眼镜

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102981616A (zh) * 2012-11-06 2013-03-20 中兴通讯股份有限公司 增强现实中对象的识别方法及系统和计算机
US9116545B1 (en) * 2012-03-21 2015-08-25 Hayes Solos Raffle Input detection
CN105373225A (zh) * 2015-10-26 2016-03-02 深圳中发明科技开发有限公司 用眼进行控制的方法以及装置
CN105929926A (zh) * 2016-04-08 2016-09-07 惠州Tcl移动通信有限公司 Vr眼镜及其省电方法
CN106055102A (zh) * 2016-05-30 2016-10-26 北京奇艺世纪科技有限公司 一种虚拟现实设备控制方法及装置
CN106062665A (zh) * 2013-09-11 2016-10-26 深圳市汇顶科技股份有限公司 基于用户的眼睛运动和位置的光学感测和跟踪的用户界面

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9116545B1 (en) * 2012-03-21 2015-08-25 Hayes Solos Raffle Input detection
CN102981616A (zh) * 2012-11-06 2013-03-20 中兴通讯股份有限公司 增强现实中对象的识别方法及系统和计算机
CN106062665A (zh) * 2013-09-11 2016-10-26 深圳市汇顶科技股份有限公司 基于用户的眼睛运动和位置的光学感测和跟踪的用户界面
CN105373225A (zh) * 2015-10-26 2016-03-02 深圳中发明科技开发有限公司 用眼进行控制的方法以及装置
CN105929926A (zh) * 2016-04-08 2016-09-07 惠州Tcl移动通信有限公司 Vr眼镜及其省电方法
CN106055102A (zh) * 2016-05-30 2016-10-26 北京奇艺世纪科技有限公司 一种虚拟现实设备控制方法及装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112817441A (zh) * 2021-01-14 2021-05-18 深圳卡多希科技有限公司 一种按键与人眼识别结合的方法和装置
CN114979614A (zh) * 2022-05-16 2022-08-30 北京芯海视界三维科技有限公司 显示模式确定方法及显示模式确定装置

Also Published As

Publication number Publication date
CN106598214A (zh) 2017-04-26

Similar Documents

Publication Publication Date Title
WO2018082162A1 (zh) 用于虚拟现实设备的功能触发方法、装置及虚拟现实设备
US11482238B2 (en) Audio-visual sound enhancement
RU2684189C2 (ru) Адаптивное распознавание событий
KR102039427B1 (ko) 스마트 글라스
EP3641280A1 (en) Unlocking of a mobile terminal by face-recognition of a slidable camera
US20220236801A1 (en) Method, computer program and head-mounted device for triggering an action, method and computer program for a computing device and computing device
BR112016026613B1 (pt) Sistema e método para fornecer retroalimentação háptica para auxiliar na captura de imagens
CN106464793B (zh) 摄像装置和摄像辅助方法
CN111630477A (zh) 提供增强现实服务的设备及其操作方法
JP6402718B2 (ja) 情報処理装置、制御方法およびプログラム
JP2017531221A (ja) 仮想現実環境の中に没入したときのつまずき対抗
US10044464B1 (en) Blocking the capture of photo and video content
JP6750697B2 (ja) 情報処理装置、情報処理方法及びプログラム
WO2017124899A1 (zh) 一种信息处理方法及装置、电子设备
US11055533B1 (en) Translating sound events to speech and AR content
EP3087727B1 (en) An emotion based self-portrait mechanism
JP6600359B2 (ja) モニタリング
KR20230063667A (ko) 전자 장치 및 그 동작 방법
RU2679719C2 (ru) Способ и устройство для управления рабочим состоянием
CN114783432A (zh) 智能眼镜的播放控制方法及智能眼镜、存储介质
US9992407B2 (en) Image context based camera configuration
JP2015111371A (ja) 情報処理装置、情報処理方法およびプログラム
CN111464781A (zh) 图像显示方法、图像显示装置、存储介质与电子设备
US20170155892A1 (en) Wearable stereoscopic camera system for 3d virtual reality imaging and networked area learning
CN114638949A (zh) 虚拟对象显示方法及装置、电子设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16920454

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16920454

Country of ref document: EP

Kind code of ref document: A1