WO2016201678A1 - 虚拟车辆人机交互方法和系统 - Google Patents

虚拟车辆人机交互方法和系统 Download PDF

Info

Publication number
WO2016201678A1
WO2016201678A1 PCT/CN2015/081820 CN2015081820W WO2016201678A1 WO 2016201678 A1 WO2016201678 A1 WO 2016201678A1 CN 2015081820 W CN2015081820 W CN 2015081820W WO 2016201678 A1 WO2016201678 A1 WO 2016201678A1
Authority
WO
WIPO (PCT)
Prior art keywords
hand
vehicle
door
dimensional model
depth sensor
Prior art date
Application number
PCT/CN2015/081820
Other languages
English (en)
French (fr)
Inventor
周谆
Original Assignee
周谆
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 周谆 filed Critical 周谆
Priority to PCT/CN2015/081820 priority Critical patent/WO2016201678A1/zh
Publication of WO2016201678A1 publication Critical patent/WO2016201678A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating 3D models or images for computer graphics

Definitions

  • the present invention relates to the field of virtual reality technologies, and in particular, to a virtual vehicle human-computer interaction method and system.
  • a customer controls a virtual vehicle displayed on a display screen by a mouse, a keyboard, or a touch screen, thereby realizing human-computer interaction between a person and a virtual vehicle.
  • a depth sensor is used to sense a human body motion
  • the main control unit combines the data of the human motion sensed by the depth sensor to convert the human body motion into the control of the virtual vehicle, thereby realizing human-computer interaction between the human and the virtual vehicle.
  • the human body action needs to be combined to realize the control, and the customer needs to learn the action corresponding to the control signal, and then implement corresponding actions in the sensing space of the depth sensor to realize the control, especially when the door is opened or closed. Oh, it's also done through physical changes in the body, which is less customer experience. .
  • An object of the present invention is to provide a virtual vehicle human-computer interaction method and system, which realizes the opening of a virtual vehicle door in combination with a hand motion, so that the control of the virtual vehicle is more suitable for practical use habits, and Improve the technical impact of the customer experience.
  • a virtual vehicle human-computer interaction method including: importing and displaying a three-dimensional vehicle model; receiving depth information of a hand acquired by a depth sensor in a sensing space thereof; and analyzing the depth sensor based on depth information of the hand The movement of the hand in the space is sensed; based on the movement of the hand, it is determined whether the door of the three-dimensional model of the vehicle is opened; and if so, the door of the three-dimensional model of the vehicle is displayed.
  • a virtual vehicle human-computer interaction system including a depth sensor and a display interface, an importing unit, and a main control unit: the depth sensor is configured to acquire depth information of a hand in a sensing space thereof; a unit for importing a three-dimensional model of the vehicle; the display interface for displaying the three-dimensional model of the vehicle; the main control unit, configured to receive depth information of the hand acquired by the depth sensor in the sensing space thereof; Depth information of the part, analyzing an action of the hand in the depth sensor sensing space; determining, based on the action of the hand, whether to open the door of the three-dimensional model of the vehicle; if yes, controlling the display interface to display The door of the vehicle's three-dimensional model.
  • the beneficial effects or advantages of the technical solution provided by the present invention are:
  • the depth sensor is used in the sensing space thereof. Detecting the depth information of the customer's hand, the depth information includes the displacement, speed, bone and other information of the hand, which can fully reflect the movement of the customer's hand; the main control unit receives the depth information of the customer's hand detected by the depth sensor.
  • the action performed by the client's hand is analyzed, and according to the hand motion of the customer, it is determined whether the action corresponds to the action of the vehicle model of the three-dimensional model of the vehicle, and if so, the door of the three-dimensional model of the vehicle is displayed on the display interface.
  • the method for controlling the virtual vehicle door based on the hand motion is proposed in the embodiment of the present invention.
  • the method for controlling the virtual vehicle door based on the hand motion is further improved. Fit the customer's habits, no need for customers to learn body movements, improve The customer's experience.
  • 1 is a schematic view of a depth sensor sensing space
  • FIG. 2 is a flow chart of a virtual vehicle human-computer interaction method according to an embodiment of the present application
  • FIG. 3 is a flowchart of a virtual vehicle human-computer interaction method according to an embodiment of the present application.
  • FIG. 4 is a flow chart of a virtual vehicle human-computer interaction method according to an embodiment of the present application.
  • FIG. 5 is a flow chart of a virtual vehicle human-computer interaction method according to an embodiment of the present application.
  • FIG. 6 is a block diagram of a virtual vehicle human-computer interaction system according to an embodiment of the present application.
  • FIG. 7 is a schematic diagram of virtual vehicle human-computer interaction according to an embodiment of the present application.
  • the present invention provides a virtual vehicle human-computer interaction method and system, uses a depth sensor to acquire depth information of a customer's hand in a perceived space, and combines hand movements to achieve a virtual vehicle door opening, so that the virtual vehicle is The control is more in line with the actual use habits, and the technical effect of improving the customer experience is realized.
  • the current depth sensor can obtain information such as bone information, spatial three-dimensional coordinates, and speed of the hand, including the palm, the wrist, and each finger.
  • the kinect sensor can be tens of frames per second.
  • the hand information is obtained, and the LeapMotion sensor can be acquired at a frequency of more than one hundred frames per second.
  • the depth sensor 1 is placed on the front end of the display interface 21 of a device such as a computer, and the sensible range forms a sensing space in front of the display interface that can sense the palm, the wrist, and each finger.
  • the three-dimensional coordinates and speed information (including the velocity and speed direction) of the palm, wrist and each finger can be sensed by the depth sensor, and the depth sensor calculates the bone of the hand based on the information.
  • Information is used for use; bone information includes information such as the size, speed, and direction of the bones, wrists, fingers, and individual finger joints.
  • the depth sensor is not necessarily limited to being placed around the device such as a computer, but is placed according to the spatial position of the actual hand operation, and it is not necessary to limit the hand to a fixed position.
  • the display interface can be, but is not limited to, a television screen, a projection screen, a computer screen, a head mounted 3D display system, etc., when the display interface is a head mounted display, the depth sensor can be placed in front of the head mounted display, The front of the human body forms a sensing space that moves with the movement of the human body.
  • the hand depth information acquired based on the depth sensor mentioned in the embodiment of the present invention generally refers to the bone information of the hand calculated by the depth sensor after acquiring the hand depth information.
  • FIG. 2 it is a flowchart of a virtual vehicle human-computer interaction method according to an embodiment of the present invention, where the method includes the following steps:
  • Step S11 Import and display a three-dimensional model of the vehicle.
  • the vehicle three-dimensional model is first constructed.
  • the construction of the vehicle three-dimensional model can be performed by any existing method for constructing a three-dimensional model; Vehicle, need to build a three-dimensional model of multiple vehicles, store all the constructed vehicle 3D models in a vehicle 3D model library, select the vehicle 3D model from the vehicle 3D model library after importing the vehicle 3D model, and import the target The vehicle's three-dimensional model; the customer can use the existing methods, such as mouse point drag, or gesture control, to select the target vehicle 3D model from the vehicle 3D model library.
  • Vehicle need to build a three-dimensional model of multiple vehicles, store all the constructed vehicle 3D models in a vehicle 3D model library, select the vehicle 3D model from the vehicle 3D model library after importing the vehicle 3D model, and import the target The vehicle's three-dimensional model; the customer can use the existing methods, such as mouse point drag, or gesture control, to select the target vehicle 3D model from the vehicle 3D model library.
  • the customer can intuitively experience the interaction between the movement of the hand and the three-dimensional model of the vehicle; Displaying the 3D model of the hand and opening the door of the vehicle's 3D model makes the effect of the virtual vehicle door more realistic.
  • the 3D model of the hand can also be displayed on the display interface, and can be set according to the actual use preferences.
  • Step S12 Receive depth information of the hand acquired by the depth sensor in the sensing space thereof.
  • Step S13 Based on the depth information of the hand, analyze the motion of the hand in the depth sensor sensing space.
  • the purpose of the embodiment of the present application is to achieve the effect of opening the virtual vehicle door based on the action of the customer's hand. Therefore, after the vehicle three-dimensional model is introduced, it is necessary to track the movement of the customer's hand. After the hand is placed in the sensing space of the depth sensor, the depth information of the hand is detected by the depth sensor, and after the depth information of the hand is acquired, the depth information is transmitted to the main control unit of the virtual vehicle.
  • the depth sensor acquires depth information of the hand in its sensing space, including bone information of the palm, the wrist and each finger, and the bone information can reflect the feature information of the hand; the depth information also includes the palm, the wrist and each finger. Three-dimensional coordinates and speed information, which can accurately capture the position, posture and movement of the hand; based on the hand depth information described above, the movement of the hand in the sensing space can be analyzed, so that the movement of the hand can be analyzed.
  • Step S14 Based on the action of the hand, it is determined whether the door of the three-dimensional model of the vehicle is activated.
  • Different actions of the hand correspond to different operations on the virtual vehicle, that is, on the three-dimensional model of the vehicle; the corresponding rules can be set in the control system of the virtual vehicle, and after the action of the hand is analyzed, the action corresponding can be implemented. Operation.
  • step S15 If the action corresponds to the operation of the door of the vehicle three-dimensional model, step S15 is performed:
  • Step S15 Display a door of the three-dimensional model of the vehicle.
  • the door displaying the three-dimensional model of the vehicle on the display interface is activated; the door handle can be pulled together
  • the display can also be combined with the sound effect of the door being slightly opened to enhance the realism of virtual operation.
  • the movement of the hand three-dimensional model on the display interface is associated with the hand in the depth sensor sensing space, so that the two hands move synchronously, and the hand three-dimensional model truly reflects the hand.
  • the movement or movement of the department For example, before analyzing the hand motion in the depth sensor sensing space, based on the depth information of the hand, the display position of the hand 3D model is adjusted, so that the hand 3D model corresponds to the movement of the hand, when the hand 3D model
  • the display position is located on the door of the three-dimensional model of the vehicle, and the three-dimensional model of the hand is displayed on the door handle of the three-dimensional model of the vehicle.
  • the door here may be any one of the three-dimensional models of the vehicle, or may be a defined one or all of the doors.
  • the door here refers to all the doors of the three-dimensional model of the vehicle, including the front door, the rear door, the hood, the trunk cover, etc., as shown in Figure 7, the virtual vehicle man-machine of the trunk cover Interactive diagram.
  • the customer can intuitively see that the hand motion is converted into the hand three-dimensional model to unlock the vehicle three-dimensional model door; and the hand three-dimensional model is not displayed.
  • the door prompt animation and/or the voice prompt can be displayed at the door handle of the vehicle three-dimensional model to remind the customer that the action of the door can be implemented by the action of the animated prompt or the voice prompt provided by the voice prompt.
  • the designated door, or the customer controls the selected door by other means.
  • determining whether to open the vehicle door of the three-dimensional model of the vehicle is specifically: based on the hand depth information, determining whether the four fingers other than the thumb of the hand in the depth sensor sensing space think The palm direction is curved, that is, it is judged whether the hand in the sensing space of the depth sensor, whether the hand makes the same action as the real door, as shown in FIG. This action is in line with the habit of opening the door every day and can improve the experience.
  • the palm can be bent downwards and four fingers, or the palm can be bent upwards and four fingers, and if the hood or the trunk is opened, the palm can be set.
  • the palms are down and the four fingers are bent to conform to the habit of opening the door every day, making the virtual door opening experience more realistic.
  • the door opening and the real door are slightly activated, and the vehicle is displayed. After the door of the dimension model, as shown in Figure 3, the following steps can also be performed:
  • Step S16 receiving depth information of the hand acquired by the depth sensor in the sensing space thereof;
  • Step S17 analyzing the motion of the hand in the depth sensor sensing space based on the depth information of the hand;
  • Step S18 determining whether the hand in the sensing space of the depth sensor is three-dimensional to the vehicle based on the motion of the hand The direction of the door of the model is pulled; if so,
  • Step S19 Displaying a vehicle door of the three-dimensional model of the vehicle.
  • the pull here is the action of opening the door after slightly opening the door, and the door of the door showing the three-dimensional model of the pulled vehicle can be equipped with the sound of the trick being pulled.
  • the depth sensor detects the depth information of the hand. Since this includes information such as the displacement and speed of the hand, after the hand 3D model is correlated with the motion of the hand, the 3D model of the hand reflects It is the real state of motion of the hand. When the movement of the hand is the action of pulling the door, the speed of the sliding door is also reflected in the movement of the three-dimensional model of the hand, that is, the hand movement speed is fast, then the hand is The speed of the 3D model is also fast, and vice versa.
  • Step S20 receiving depth information of the hand acquired by the depth sensor in the sensing space thereof;
  • Step S21 analyzing the motion of the hand in the depth sensor sensing space based on the depth information of the hand; [0051] Step S22: determining, according to the motion of the hand, whether the hand in the depth sensor sensing space is three-dimensional to the vehicle The direction in which the door of the model is closed; if so,
  • Step S19 Displaying a vehicle door that closes the three-dimensional model of the vehicle.
  • the door of the vehicle that displays the three-dimensional model of the vehicle can be equipped with a sound effect in which the trick is turned off.
  • the customer can adjust the display of the vehicle three-dimensional model by the operation of the input device such as a mouse or a keyboard, such as direction switching, entering the vehicle space, and the like.
  • the input device such as a mouse or a keyboard, such as direction switching, entering the vehicle space, and the like.
  • the customer can also implement the control of the vehicle three-dimensional model selection or direction switching by gesture based on the depth sensor, as shown in FIG. 5:
  • Step S51 receiving depth information of the hand acquired by the depth sensor in the sensing space thereof;
  • Step S52 analyzing, according to the depth information of the hand, a gesture corresponding to the hand;
  • Step S53 Control selection, display direction, movement or zooming of the vehicle three-dimensional model based on the gesture.
  • the three-dimensional coordinates and speed information of the hand are obtained; and the three-dimensional sitting according to the hand Target, analyze the displacement of the hand, and analyze the direction of movement of the hand based on the speed information of the hand.
  • the depth sensor collects depth information of the hand change in the perceptual space, and therefore, the three-dimensional coordinates in the acquired depth information form a motion displacement of the hand on the inter-turn axis, and the velocity direction and velocity in the velocity information Then the direction of motion and the speed of movement of the hand are constructed in the space.
  • the gesture corresponding to the hand movement can be calculated; specifically, combined with the motion displacement of the hand and the movement direction and the movement speed, a gesture model of the hand occurring in the space can be constructed, and different gesture models are constructed.
  • a gesture model of the hand occurring in the space can be constructed, and different gesture models are constructed.
  • the correspondence between specific gestures and human-computer interaction instructions can be determined according to actual conditions, and the solution is not limited.
  • the embodiment of the present application further provides a virtual vehicle human-computer interaction system.
  • the system includes a depth sensor 1, a display interface 21, and an import unit 3.
  • the main control unit 5 as shown in FIG.
  • the depth sensor 1 is configured to acquire depth information of the hand in the sensing space 4;
  • the import unit 3 is configured to import the vehicle three-dimensional model and/or the hand three-dimensional model;
  • the interface 21 is configured to display a three-dimensional model of the vehicle and/or a three-dimensional model of the hand;
  • the main control unit 5 is configured to receive depth information of the hand acquired by the depth sensor 1 in the sensing space 4 thereof; and analyze the depth information based on the hand
  • the depth sensor 1 senses the motion of the hand in the space 4; based on the motion of the hand, determines whether the door of the three-dimensional model of the vehicle is turned on; if so, the control display interface 21 displays the door of the three-dimensional model of the vehicle.
  • Both the hand three-dimensional model and the vehicle three-dimensional model are constructed by the building unit 9, and can be stored in the three-dimensional model library 9 after the construction is completed.
  • the main control unit 5 adjusts the hand three-dimensional based on the depth information of the hand before analyzing the hand motion in the depth sensor sensing space.
  • Display position of the model; display position of the hand 3D model is located on the door of the vehicle 3D model, display the hand 3D model at the door handle of the vehicle 3D model; display the vehicle 3D model on the main control unit control display interface
  • the door of the door can also control the display interface to show that the door handle is pulled.
  • the display of the 3D model of the hand can intuitively reflect the operation of the customer in the perceived space of the depth sensor, making the virtual operation more realistic and improving the experience.
  • the specific hand-based action determines whether the vehicle door of the three-dimensional model of the vehicle is opened, and includes: determining, according to the hand depth information, whether the four fingers other than the thumb of the hand in the depth sensor sensing space are bent toward the palm . That is, it is determined whether the customer performs a four-point palm bending other than the thumb in the sensing space 4.
  • the action is in accordance with the habit of actually opening the door. Compared with the prior art, the virtual vehicle door is opened by the body movement, and the hand movement conforms to the actual operation habit, and the customer does not need to learn the body movement corresponding to the operation of the card. Can improve the experience.
  • the main control unit 5 controls the depth information of the hand acquired by the depth sensor 1 in the sensing space 4 after the display interface 21 displays the door of the vehicle three-dimensional model; the depth sensor is analyzed based on the depth information of the hand.
  • the movement of the hand in the sensing space based on the movement of the hand, determining whether the hand in the depth sensor sensing space moves in the direction of the door pulling of the vehicle three-dimensional model; if so, the control display interface 21 displays the three-dimensional model of the pulling vehicle The door.
  • the main control unit 5 controls the display interface 21 to display the door of the three-dimensional model of the vehicle
  • the depth information of the hand acquired by the depth sensor 1 in the sensing space 4 is continuously received; the depth sensor sensing is analyzed based on the depth information of the hand.
  • the movement of the hand in the space based on the movement of the hand, it is judged whether the depth sensor 1 senses whether the hand in the space 4 moves in the direction in which the door of the vehicle three-dimensional model is closed; if so, the control display interface 21 displays the three-dimensional model of the vehicle being turned off.
  • the sound effect unit 6 is further included in the system; the sound effect unit 6 is displayed on the display unit 21 of the main control unit 5
  • the system further includes a prompting unit 7 for displaying a trick animation and/or a voice prompt at the handle of the vehicle three-dimensional model displayed on the display interface 21.
  • the depth sensor is used to obtain the depth information of the hand, and the displacement, the speed, the bone, and the like of the hand are analyzed from the hand depth information. And through the analysis of the information, the action of the hand is judged, and finally the operation of the virtual vehicle door is realized according to the action of the hand.
  • the operation of the door of the virtual vehicle is realized based on the operation mode of the vehicle door which is commonly used by people. According to the actual operating habits, the door displaying the vehicle's three-dimensional model on the display interface is opened, pulled or closed, and the sound effect of the door opening, pulling or closing can make the operation of the virtual vehicle more realistic and practical. .
  • the 3D model of the hand can be displayed or not displayed, and the display ⁇ is synchronized with the movement of the hand in the depth sensor sensing space according to the hand depth information.
  • the operation of the virtual vehicle door can also be realized; the door can be system-specific or hand-based movement Reflected in the hand 3D model ⁇ one or more doors selected on the vehicle 3D model.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Graphics (AREA)
  • Computer Hardware Design (AREA)
  • Software Systems (AREA)
  • User Interface Of Digital Computer (AREA)
  • Processing Or Creating Images (AREA)

Abstract

一种虚拟车辆人机交互方法和系统，结合手部动作实现对虚拟车辆的车门的开启，使得对虚拟车辆的控制更贴合实际使用习惯。方法包括：导入并显示车辆三维模型（S11）；接收深度传感器在其感知空间内获取的手部的深度信息（S12）；基于手部的深度信息，分析深度传感器感知空间内的手部的动作（S13）；基于手部的动作，判断是否开启车辆三维模型的车门（S14）；若是，则显示开启车辆三维模型的车门（S15）。相比于现有技术中客户需要预先学习对应开门的身体动作，通过实施身体动作来控制虚拟车辆开门的方式，即基于手部动作开启虚拟车辆车门的方式更贴合客户使用习惯，无需客户学习身体动作，提高了客户的体验感。

Description

虚拟车辆人机交互方法和系统
技术领域
[0001] 本发明涉及虚拟现实技术领域， 尤其涉及一种虚拟车辆人机交互方法和系统。
背景技术
[0002] 虚拟现实车辆演示系统中， 客户通过鼠标、 键盘或者点击触摸屏幕来控制显示 在显示屏幕上的虚拟车辆， 从而实现人与虚拟车辆的人机交互。
[0003] 现有的一种体感人机交互的车辆展示系统中， 使用深度传感器感知人体的动作
， 主控单元结合深度传感器感知的人体动作的数据， 将人体动作转化为对虚拟 车辆的控制， 从而实现人与虚拟车辆的人机交互。
技术问题
[0004] 但这种方式中， 需要结合人体动作来实现控制， 客户需要学习控制信号对应的 动作， 然后在深度传感器的感知空间内实施相应的动作来实现控制， 尤其是在 幵启或者关闭车门吋， 还要通过身体的体态变化来实现， 这种方式的客户体验 性低。 。
问题的解决方案
技术解决方案
[0005] 本发明的目的是提供一种虚拟车辆人机交互方法和系统， 结合手部动作实现对 虚拟车辆的车门的幵启， 使得对虚拟车辆的控制更贴合实际使用习惯， 实现的 是提高客户体验的技术效果。
[0006] 本发明的目的是通过以下技术方案实现的：
[0007] 提出一种虚拟车辆人机交互方法， 包括： 导入并显示车辆三维模型； 接收深度 传感器在其感知空间内获取的手部的深度信息； 基于手部的深度信息， 分析所 述深度传感器感知空间内的手部的动作； 基于手部的动作， 判断是否幵启所述 车辆三维模型的车门； 若是， 则显示幵启所述车辆三维模型的车门。
[0008] 提出一种虚拟车辆人机交互系统， 包括深度传感器和显示界面、 导入单元和主 控单元： 所述深度传感器， 用于在其感知空间内获取手部的深度信息； 所述导 入单元， 用于导入车辆三维模型； 所述显示界面， 用于显示所述车辆三维模型 ； 所述主控单元， 用于接收深度传感器在其感知空间内获取的手部的深度信息 ； 基于手部的深度信息， 分析所述深度传感器感知空间内的手部的动作； 基于 手部的动作， 判断是否幵启所述车辆三维模型的车门； 若是， 则控制所述显示 界面显示幵启所述车辆三维模型的车门。
发明的有益效果
有益效果
[0009] 本发明提供的技术方案的有益效果或者优点是： 本申请实施例提出的虚拟车辆 人机交互方法和系统中， 导入并显示构建的车辆三维模型后， 使用深度传感器 在其感知空间内检测客户手部的深度信息， 深度信息中包括有手部的位移、 速 度、 骨骼等信息， 能充分反映客户手部的动作； 主控单元在接收到深度传感器 检测到的客户手部的深度信息后， 根据深度信息分析出客户手部执行的动作， 并根据客户的手部动作来判断动作是否对应为幵启车辆三维模型车门的动作， 若是， 则在显示界面显示幵启车辆三维模型的车门； 相比于现有技术中客户需 要预先学习对应幵门的身体动作， 通过实施身体动作来控制虚拟车辆幵门的方 式， 本发明实施例提出的基于手部动作幵启虚拟车辆车门的方式更贴合客户使 用习惯， 无需客户学习身体动作， 提高了客户的体验感。
对附图的简要说明
附图说明
[0010] 图 1为深度传感器感知空间示意图；
[0011] 图 2本申请实施例提出的虚拟车辆人机交互方法流程图；
[0012] 图 3本申请实施例提出的虚拟车辆人机交互方法流程图；
[0013] 图 4本申请实施例提出的虚拟车辆人机交互方法流程图；
[0014] 图 5本申请实施例提出的虚拟车辆人机交互方法流程图；
[0015] 图 6为本申请实施例提出的虚拟车辆人机交互系统框图；
[0016] 图 7为本申请实施例提出的虚拟车辆人机交互示意图。
实施该发明的最佳实施例 本发明的最佳实施方式
[0017] 本发明通过提供一种虚拟车辆人机交互方法和系统， 使用深度传感器获取感知 空间内客户手部的深度信息， 结合手部动作实现对虚拟车辆的车门的幵启， 使 得对虚拟车辆的控制更贴合实际使用习惯， 实现的是提高客户体验的技术效果
[0018] 目前的深度传感器， 可以获取到手部， 包括手掌、 手腕和各手指等部位的骨骼 信息、 空间三维坐标和速度等信息， 以 kinect传感器为例， 其可以以每秒几十帧 的频率获取手部的信息， 而 LeapMotion传感器更可以以超过每秒一百帧的频率获 取。
[0019] 如图 1中所示， 以深度传感器 1放置于电脑等装置的显示界面 21前端为例， 其能 感知的范围在显示界面前方形成一个能感知手掌、 手腕和各手指的感知空间 4， 客户在此感知空间内进行手部操作吋， 其手掌、 手腕和各手指的三维坐标、 速 度信息 （包括速率和速度方向） 能够被该深度传感器感知， 深度传感器基于这 些信息计算出手部的骨骼信息供应用使用； 骨骼信息中包括手掌、 手腕、 手指 以及各个手指节的骨骼尺寸、 速度、 方向等信息。 当然， 在深度传感器与电脑 等装置实现无线连接的情况下， 深度传感器不必限于放置在电脑等装置的周围 ， 而是使其根据实际手部操作的空间位置放置， 不必使手部局限于固定位置进 行手势操作。 显示界面可以是但不受限于电视屏幕、 投影屏幕、 电脑屏幕、 头 戴式 3D显示系统等， 当显示界面为头戴式显示器吋， 可以将深度传感器置于头 戴式显示器的前方， 在人体正前方形成一个随人体移动而移动的感知空间。
[0020] 本发明实施例中提及的基于深度传感器获取的手部深度信息， 通常是指深度传 感器获取到手部深度信息后计算得到的手部的骨骼信息。
[0021] 下面将结合附图， 对本发明实施例提供的技术方案进行详细描述。
[0022] 如图 2所示， 为本发明实施例提出的虚拟车辆人机交互方法流程图， 该方法包 括以下步骤：
[0023] 步骤 S11 : 导入并显示车辆三维模型。
[0024] 在导入车辆三维模型之前， 首先要构建车辆三维模型。
[0025] 车辆三维模型的构建， 可以采用现有的任一种构建三维模型的方法进行； 多个 车辆， 需要构建多个车辆三维模型， 将所有构建的车辆三维模型存放于一个车 辆三维模型库中， 在导入车辆三维模型吋， 从该车辆三维模型库中选取目标车 辆三维模型， 并导入该目标车辆三维模型； 客户可以采用现有的方法， 例如鼠 标点中拖动， 或者手势控制等方式， 从车辆三维模型库中选取目标车辆三维模 型。
[0026] 还可以导入并显示客户手部的手部三维模型， 显示吋， 客户能直观体验到手部 的运动与车辆三维模型之间的互动； 在后续幵启车辆三维模型的车门吋， 可以 通过显示手部三维模型幵启车辆三维模型的车门， 使得幵启虚拟车辆车门的效 果更逼真。 当然， 手部三维模型也可以不显示在显示界面， 根据实际使用喜好 设定即可。
[0027] 步骤 S12: 接收深度传感器在其感知空间内获取的手部的深度信息。
[0028] 步骤 S13: 基于手部的深度信息， 分析深度传感器感知空间内的手部的动作。
[0029] 本申请实施例的目的既是， 基于客户手部的动作来实现幵启虚拟车辆车门的效 果， 因此在导入车辆三维模型之后， 需要跟踪客户手部的运动。 将手部置于深 度传感器的感知空间后， 由深度传感器检测手部的深度信息， 并在获取到手部 的深度信息后， 将深度信息发送给虚拟车辆的主控单元中。
[0030] 深度传感器在其感知空间内获取手部的深度信息， 包括手掌、 手腕和各个手指 的骨骼信息， 这些骨骼信息能反应手部的特征信息； 深度信息还包括手掌、 手 腕和各个手指的三维坐标以及速度信息， 这可以精确捕获手部的位置、 姿态和 动作等； 基于上述的手部深度信息， 能够分析出在感知空间内手部的运动， 从 而能够分析出手部的动作。
[0031] 步骤 S14: 基于手部的动作， 判断是否幵启车辆三维模型的车门。
[0032] 手部不同的动作对应对虚拟车辆， 也即对车辆三维模型的不同操作； 可以在虚 拟车辆的控制系统中设定对应规则， 则在分析出手部的动作之后， 即能实施动 作对应的操作。
[0033] 若动作对应的是幵启车辆三维模型的车门的操作， 则执行步骤 S15:
[0034] 步骤 S15: 显示幵启车辆三维模型的车门。
[0035] 在显示界面上显示车辆三维模型的车门被幵启； 可以再配合以车门把手被拉动 的显示， 也可以配合以车门被轻微幵启的音效， 提高虚拟操作的真实感。
[0036] 当手部三维模型显示在显示界面吋， 将手部三维模型在显示界面的运动与深度 传感器感知空间内的手部进行关联， 使二者同步运动， 则手部三维模型真实反 映手部的移动或者动作。 例如， 在分析深度传感器感知空间内的手部动作之前 ， 基于手部的深度信息， 调整手部三维模型的显示位置， 使得手部三维模型与 手部的移动相对应， 当手部三维模型的显示位置位于车辆三维模型的车门上吋 ， 将手部三维模型显示于车辆三维模型的车门把手处， 这里的车门可以是车辆 三维模型的任一个车门， 也可以是限定的一个车门， 也可以全部车门； 这里的 车门泛指车辆三维模型的所有可幵启的门体， 包括前车门、 后车门、 引擎盖、 后备箱盖等， 如图 7所示为幵启后备箱盖的虚拟车辆人机交互示意图。
[0037] 将手部三维模型显示于车门把手处后， 客户可以直观的看到手部动作被转化为 手部三维模型的动作来幵启车辆三维模型的车门； 而在不显示手部三维模型吋 ， 可以通过在车辆三维模型的车门把手处显示幵门提示动画和 /或发出语音提示 ， 提醒客户可以通过动画提示的动作或者语音提示提供的声音指示实施幵启车 门的动作， 此吋车门为系统指定的车门， 或者为客户通过其他方式控制选择的 车门。
[0038] 具体的， 基于手部的动作， 判断是否幵启车辆三维模型的车门， 具体实施为： 基于手部深度信息， 判断深度传感器感知空间内的手部的拇指以外的四指是否 都想掌心方向弯曲， 也即， 判断客户在深度传感器的感知空间内， 手部是否做 出与现实中幵启车门一样的动作， 如图 1所示。 该动作符合日常幵启车门的习惯 ， 能提高体验感。
[0039] 本申请实施例中， 只需判断拇指以外的四指是否都向掌心方向弯曲即可， 不限 定掌心的方向， 也不限定左右手。
[0040] 具体的， 若是幵启车身侧面的前、 后车门， 则掌心可向下以及四指弯曲， 或者 掌心可向上以及四指弯曲， 而若是幵启引擎盖或者后备箱， 则可以设定掌心向 下以及四指弯曲， 以符合日常幵启车门的习惯， 使得虚拟幵启车门的体验更逼 真。
[0041] 本申请实施例中， 车门幵启与现实中车门被轻微幵启对应， 在显示幵启车辆三 维模型的车门之后， 如图 3所示， 还可以执行以下步骤：
[0042] 步骤 S16: 接收深度传感器在其感知空间内获取的手部的深度信息；
[0043] 步骤 S17: 基于手部的深度信息， 分析深度传感器感知空间内的手部的动作； [0044] 步骤 S18: 基于手部的动作， 判断深度传感器感知空间内的手部是否向车辆三 维模型的车门拉幵的方向运动； 若是，
[0045] 步骤 S19: 显示拉幵车辆三维模型的车门。
[0046] 这里的拉幵是在轻微幵启车门之后敞幵车门的动作， 在显示拉幵车辆三维模型 的车门同吋， 可以配以幵门被拉幵的音效。
[0047] 深度传感器检测的是手部的深度信息， 由于这其中包含有手部的位移和速度等 信息， 在将手部三维模型与手部的运动进行关联同步之后， 手部三维模型反映 的是手部真实的运动状态， 当手部的动作为拉幵车门的动作吋， 其拉门的速度 也真实反映在手部三维模型的运动中， 也即， 手部动作速度快， 则手部三维模 型的运动速度也快， 反之亦然。
[0048] 在显示拉幵车辆三维模型的车门之后， 如图 4所示， 还可以执行以下步骤： [0049] 步骤 S20: 接收深度传感器在其感知空间内获取的手部的深度信息；
[0050] 步骤 S21 : 基于手部的深度信息， 分析深度传感器感知空间内的手部的动作； [0051] 步骤 S22: 基于手部的动作， 判断深度传感器感知空间内的手部是否向车辆三 维模型的车门关闭的方向运动； 若是，
[0052] 步骤 S19: 显示关闭车辆三维模型的车门。
[0053] 在显示关闭车辆三维模型的车门同吋， 可以配以幵门被关闭的音效。
[0054] 客户可以通过鼠标、 键盘等输入设备的操作来调节车辆三维模型的显示， 例如 方向调换、 进入车辆空间等。
[0055] 客户也可以基于深度传感器， 通过手势来实现控制车辆三维模型的选择或者方 向调换等， 具体的如图 5所示：
[0056] 步骤 S51 : 接收深度传感器在其感知空间内获取的手部的深度信息；
[0057] 步骤 S52: 基于手部的深度信息， 分析得到手部对应的手势；
[0058] 步骤 S53: 基于手势， 控制车辆三维模型的选择、 显示方向、 移动或者缩放。
[0059] 基于手部的深度信息， 得到手部的三维坐标和速度信息； 并根据手部的三维坐 标， 分析手部的位移， 以及根据手部的速度信息， 分析手部的运动方向。
[0060] 深度传感器采集其感知空间内手部变化的深度信息， 因此， 获取到的深度信息 中的三维坐标在吋间轴上形成了手部的运动位移， 而速度信息中的速度方向和 速率则在空间中构建出了手部的运动方向和运动速度。
[0061] 基于手部的运动， 可以推算得到手部运动对应的手势； 具体的， 结合手部的运 动位移以及运动方向和运动速度， 可以构建出手部在空间发生的手势模型， 不 同的手势模型对应不同的人机交互指令， 而具体的手势与人机交互指令的对应 关系可以依据实际情况确定， 本方案不予限制。
[0062] 基于上述提出的虚拟车辆人机交互方法， 本发申请实施例还提出一种虚拟车辆 人机交互系统， 如图 6所示， 该系统包括深度传感器 1、 显示界面 21、 导入单元 3 和主控单元 5; 如图 1所示， 深度传感器 1， 用于在其感知空间 4内获取手部的深 度信息； 导入单元 3， 用于导入车辆三维模型和 /或手部三维模型； 显示界面 21， 用于显示车辆三维模型和 /或手部三维模型； 主控单元 5， 用于接收深度传感器 1 在其感知空间 4内获取的手部的深度信息； 基于手部的深度信息， 分析深度传感 器 1感知空间 4内的手部的动作； 基于手部的动作， 判断是否幵启车辆三维模型 的车门； 若是， 则控制显示界面 21显示幵启车辆三维模型的车门。
[0063] 手部三维模型和车辆三维模型都由构建单元 9构建， 构建完成后可以存放于三 维模型库 9中。
[0064] 若导入单元 3和显示界面 21分别导入和显示了手部三维模型， 则主控单元 5在分 析深度传感器感知空间内的手部动作之前， 基于手部的深度信息， 调整手部三 维模型的显示位置； 在手部三维模型的显示位置位于车辆三维模型的车门上吋 ， 将手部三维模型显示于车辆三维模型的车门把手处； 在主控单元控制显示界 面显示所述车辆三维模型的车门的同吋， 还可以控制显示界面显示车门把手被 拉动。 手部三维模型的显示能直观反映客户在深度传感器的感知空间内的操作 ， 使虚拟操作更加逼真， 提高体验效果。
[0065] 具体的基于手部的动作， 判断是否幵启车辆三维模型的车门， 包括： 基于手部 深度信息， 判断深度传感器感知空间内的手部的拇指以外的四指是否都向掌心 方向弯曲。 也即， 判断客户在感知空间 4内是否实施拇指以外的四指向掌心弯曲 的动作， 该动作符合实际幵启车门的习惯， 相比于现有技术中通过身体动作来 幵启虚拟车辆车门， 该手部动作贴合实际操作习惯， 客户无需学习幵门操作对 应的身体动作， 能提高体验效果。
[0066] 主控单元 5控制在显示界面 21显示幵启车辆三维模型的车门之后， 接收深度传 感器 1在其感知空间 4内获取的手部的深度信息； 基于手部的深度信息， 分析深 度传感器感知空间内的手部的动作； 基于手部的动作， 判断深度传感器感知空 间内的手部是否向车辆三维模型的车门拉幵的方向运动； 若是， 则控制显示界 面 21显示拉幵车辆三维模型的车门。 在主控单元 5控制在显示界面 21显示拉幵车 辆三维模型的车门之后， 继续接收深度传感器 1在其感知空间 4内获取的手部的 深度信息； 基于手部的深度信息， 分析深度传感器感知空间内的手部的动作； 基于手部的动作， 判断深度传感器 1感知空间 4内的手部是否向车辆三维模型的 车门关闭的方向运动； 若是， 则控制显示界面 21显示关闭车辆三维模型的车门
[0067] 本系统中还包括音效单元 6; 音效单元 6在主控单元 5控制显示界面 21显示幵启
、 拉幵或者关闭车辆三维模型的车门同吋， 配以车门被幵启、 拉幵或关闭的音 效。
[0068] 本系统还包括提示单元 7， 用于在显示界面 21显示的车辆三维模型的把手处显 示幵门提示动画和 /或发出语音提示。
[0069] 具体的虚拟车辆人机交互系统的工作方法， 已经在上述虚拟车辆人机交互方法 中详述， 此处不予赘述。
[0070] 上述， 本申请实施例中提出的虚拟车辆人机交互方法和系统中， 采用深度传感 器获取手部的深度信息， 从手部深度信息中解析出手部的位移、 速度、 骨骼等 信息， 并通过对信息的分析判断出手部的动作， 最终根据手部的动作实现对虚 拟车辆车门的操作， 这种基于人们通常使用的对车辆车门的操作方式实现对虚 拟车辆的车门的操作， 更贴合实际操作习惯， 而在显示界面显示车辆三维模型 的车门被幵启、 拉幵或者关闭吋， 配合以车门幵启、 拉幵或关闭的音效， 能使 对虚拟车辆的操作更加逼真贴合实际。 手部三维模型可以显示， 也可以不显示 ， 显示吋， 根据手部深度信息使其与深度传感器感知空间内的手部的动作同步 对应， 使对虚拟车辆的操作更真实和直观， 而不显示吋， 结合手部的深度信息 ， 同样可以实现对虚拟车辆车门的操作； 车门可以是系统指定的， 也可以是基 于手部的移动反映到手部三维模型吋在车辆三维模型上选择的一个或者多个车 门。 采用本申请实施例提出的方法和系统， 虚拟与现实结合的人机交互， 通过 现实的手部动作实现了对虚拟车辆车门的虚拟操作。
显然， 本领域的技术人员可以对本发明进行各种改动和变型而不脱离本发明的 精神和范围。 这样， 倘若本发明的这些修改和变型属于本发明权利要求及其等 同技术的范围之内， 则本发明也意图包含这些改动和变型在内。

Claims

权利要求书
[权利要求 1] 虚拟车辆人机交互方法， 其特征在于， 包括：
导入并显示车辆三维模型； 接收深度传感器在其感知空间内获取的手部的深度信息；
基于手部的深度信息， 分析所述深度传感器感知空间内的手部的动作
基于手部的动作， 判断是否幵启所述车辆三维模型的车门；
2.若是， 则显示幵启所述车辆三维模型的车门。
根据权利要求 1所述的虚拟车辆人机交互方法， 其特征在于， 在接收 深度传感器在其感知空间内获取的手部的深度信息之前， 所述方法包 括：
导入并显示手部三维模型。
3.根据权利要求 2所述的虚拟车辆人机交互方法， 其特征在于， 在分 析所述深度传感器感知空间内的手部动作之前， 所述方法还包括： 基于手部的深度信息， 调整所述手部三维模型的显示位置； 在所述手部三维模型的显示位置位于所述车辆三维模型的车门上吋， 将所述手部三维模型显示于所述车辆三维模型的车门把手处。
4.根据权利要求 1所述的虚拟车辆人机交互方法， 其特征在于， 所述 基于手部的动作， 判断是否幵启所述车辆三维模型的车门， 具体为： 基于手部深度信息， 判断所述深度传感器感知空间内的手部的拇指以 外的四指是否都向掌心方向弯曲。
5.根据权利要求 1所述的虚拟车辆人机交互方法， 其特征在于， 在显 示幵启所述车辆三维模型的车门同吋， 显示车门把手被拉动和 /或配 以车门被幵启的音效。
6.根据权利要求 1所述的虚拟车辆人机交互方法， 其特征在于， 在显 示幵启所述车辆三维模型的车门之后， 所述方法还包括：
接收深度传感器在其感知空间内获取的手部的深度信息；
基于手部的深度信息， 分析所述深度传感器感知空间内的手部的动作 基于手部的动作， 判断所述深度传感器感知空间内的手部是否向所述 车辆三维模型的车门拉幵的方向运动；
若是， 则显示拉幵所述车辆三维模型的车门。
7.根据权利要求 6所述的虚拟车辆人机交互方法， 其特征在于， 在显 示拉幵所述车辆三维模型的车门同吋， 所述方法还包括：
配以车门被拉幵的音效。
8.根据权利要求 6所述的虚拟车辆人机交互方法， 其特征在于， 在显 示拉幵所述车辆三维模型的车门之后， 所述方法还包括：
接收深度传感器在其感知空间内获取的手部的深度信息；
基于手部的深度信息， 分析所述深度传感器感知空间内的手部的动作 基于手部的动作， 判断所述深度传感器感知空间内的手部是否向所述 车辆三维模型的车门关闭的方向运动；
若是， 则显示关闭所述车辆三维模型的车门。
9.根据权利要求 8所述的虚拟车辆人机交互方法， 其特征在于， 在显 示关闭所述车辆三维模型的车门同吋， 所述方法还包括：
配以车门被关闭的音效。
10.根据权利要求 1所述的虚拟车辆人机交互方法， 其特征在于， 在所 述车辆三维模型的车门把手处显示幵门提示动画和 /或发出语音提示
11.虚拟车辆人机交互系统， 包括深度传感器和显示界面， 其特征在 于， 还包括导入单元和主控单元：
所述深度传感器， 用于在其感知空间内获取手部的深度信息； 所述导入单元， 用于导入车辆三维模型；
所述显示界面， 用于显示所述车辆三维模型；
所述主控单元， 用于接收深度传感器在其感知空间内获取的手部的深 度信息； 基于手部的深度信息， 分析所述深度传感器感知空间内的手 部的动作； 基于手部的动作， 判断是否幵启所述车辆三维模型的车门 ； 若是， 则控制所述显示界面显示幵启所述车辆三维模型的车门。
12.根据权利要求 11所述的虚拟车辆人机交互系统， 其特征在于， 所 述导入单元， 还用于在接收深度传感器在其感知空间内获取的手部的 深度信息之前， 导入手部三维模型； 所述显示界面显示所述手部三维 模型。
13.根据权利要求 12所述的虚拟车辆人机交互系统， 其特征在于， 所 述主控单元还用于， 在分析所述深度传感器感知空间内的手部动作之 前， 基于手部的深度信息， 调整所述手部三维模型的显示位置； 在所 述手部三维模型的显示位置位于所述车辆三维模型的车门上吋， 将所 述手部三维模型显示于所述车辆三维模型的车门把手处。
14.根据权利要求 11所述的虚拟车辆人机交互系统， 其特征在于， 所 述基于手部的动作， 判断是否幵启所述车辆三维模型的车门， 具体为 基于手部深度信息， 判断所述深度传感器感知空间内的手部的拇指以 外的四指是否都向掌心方向弯曲。
15.根据权利要求 11所述的虚拟车辆人机交互系统， 其特征在于， 所 述主控单元控制所述显示界面显示幵启所述车辆三维模型的车门的同 吋， 控制显示界面显示车门把手被拉动。
16.根据权利要求 11所述的虚拟车辆人机交互系统， 其特征在于， 所 述主控单元控制在显示界面显示幵启所述车辆三维模型的车门之后， 接收深度传感器在其感知空间内获取的手部的深度信息； 基于手部的 深度信息， 分析所述深度传感器感知空间内的手部的动作； 基于手部 的动作， 判断所述深度传感器感知空间内的手部是否向所述车辆三维 模型的车门拉幵的方向运动； 若是， 则控制显示界面显示拉幵所述车 辆三维模型的车门。
17.根据权利要求 16所述的虚拟车辆人机交互系统， 其特征在于， 所 述主控单元控制在显示界面显示拉幵所述车辆三维模型的车门之后， 接收深度传感器在其感知空间内获取的手部的深度信息； 基于手部的 深度信息， 分析所述深度传感器感知空间内的手部的动作； 基于手部 的动作， 判断所述深度传感器感知空间内的手部是否向所述车辆三维 模型的车门关闭的方向运动； 若是， 则控制所述显示界面显示关闭所 述车辆三维模型的车门。
18.根据权利要求 17所述的虚拟车辆人机交互系统， 其特征在于， 所 述系统还包括音效单元； 所述音效单元在所述主控单元控制所述显示 界面显示幵启、 拉幵或者关闭所述车辆三维模型的车门同吋， 配以车 门被幵启、 拉幵或关闭的音效。
19.根据权利要求 11所述的虚拟车辆人机交互系统， 其特征在于， 所 述系统还包括提示单元， 所述提示单元用于在所述所述显示界面显示 的车辆三维模型的把手处显示幵门提示动画和 /或发出语音提示。
PCT/CN2015/081820 2015-06-18 2015-06-18 虚拟车辆人机交互方法和系统 WO2016201678A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2015/081820 WO2016201678A1 (zh) 2015-06-18 2015-06-18 虚拟车辆人机交互方法和系统

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2015/081820 WO2016201678A1 (zh) 2015-06-18 2015-06-18 虚拟车辆人机交互方法和系统

Publications (1)

Publication Number Publication Date
WO2016201678A1 true WO2016201678A1 (zh) 2016-12-22

Family

ID=57544794

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2015/081820 WO2016201678A1 (zh) 2015-06-18 2015-06-18 虚拟车辆人机交互方法和系统

Country Status (1)

Country Link
WO (1) WO2016201678A1 (zh)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120308140A1 (en) * 2011-06-06 2012-12-06 Microsoft Corporation System for recognizing an open or closed hand
CN103914152A (zh) * 2014-04-11 2014-07-09 周光磊 三维空间中多点触控与捕捉手势运动的识别方法与系统
CN104641400A (zh) * 2012-07-19 2015-05-20 戈拉夫·瓦茨 用户控制三维仿真技术,提供增强的逼真数字对象视图和交互体验
CN104881127A (zh) * 2015-06-18 2015-09-02 周谆 虚拟车辆人机交互方法和系统

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20120308140A1 (en) * 2011-06-06 2012-12-06 Microsoft Corporation System for recognizing an open or closed hand
CN104641400A (zh) * 2012-07-19 2015-05-20 戈拉夫·瓦茨 用户控制三维仿真技术,提供增强的逼真数字对象视图和交互体验
CN103914152A (zh) * 2014-04-11 2014-07-09 周光磊 三维空间中多点触控与捕捉手势运动的识别方法与系统
CN104881127A (zh) * 2015-06-18 2015-09-02 周谆 虚拟车辆人机交互方法和系统

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
KIM, J.O. ET AL.: "Real-Time Hand Gesture-Based Interaction with Objects in 3D virtual Environments", INTERNATIONAL JOURNAL OF MULTIMEDIA AND UBIQUITOUS ENGINEERING, vol. 8, no. 6, 31 December 2013 (2013-12-31), pages 341 - 345, XP055337598 *

Similar Documents

Publication Publication Date Title
JP6906580B2 (ja) ビューポートベースの拡張現実触覚効果のシステム、方法及び非一時的コンピュータ可読媒体
US10338683B2 (en) Systems and methods for visual processing of spectrograms to generate haptic effects
CN104246661B (zh) 使用手势与装置交互
KR101811909B1 (ko) 제스처 인식을 위한 장치 및 방법
KR102517425B1 (ko) 디지털 디바이스와 상호작용을 위한 다이렉트 포인팅 검출 시스템 및 방법
KR101791366B1 (ko) 증강된 가상 터치패드 및 터치스크린
JP5837991B2 (ja) 認証型ジェスチャ認識
US20190238755A1 (en) Method and apparatus for push interaction
CN104881127A (zh) 虚拟车辆人机交互方法和系统
US20110107216A1 (en) Gesture-based user interface
WO2016189390A2 (en) Gesture control system and method for smart home
US11048375B2 (en) Multimodal 3D object interaction system
TW201539305A (zh) 使用姿勢控制基於計算的裝置
EP3090425A1 (en) Visualization of physical characteristics in augmented reality
KR20150103278A (ko) 다수의 지각 감지 입력의 상호작용
CN109865283A (zh) 游戏中虚拟角色面部调整方法、装置、电子设备及介质
CN116529125A (zh) 用于控制的手握方向盘手势交互的方法和设备
Santos et al. Developing 3d freehand gesture-based interaction methods for virtual walkthroughs: Using an iterative approach
Dong et al. Touch-move-release: studies of surface and motion gestures for mobile augmented reality
WO2023235647A1 (en) Extend the game controller functionality with virtual buttons using hand tracking
WO2016201678A1 (zh) 虚拟车辆人机交互方法和系统
JP2010079848A (ja) 項目選択方法および装置
WO2021059642A1 (ja) 情報処理装置、制御方法、プログラム
WO2017079910A1 (zh) 基于手势的虚拟现实人机交互方法和系统
US20240096043A1 (en) Display method, apparatus, electronic device and storage medium for a virtual input device

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 15895251

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 15895251

Country of ref document: EP

Kind code of ref document: A1