CN109395375A - A kind of 3d gaming method of interface interacted based on augmented reality and movement - Google Patents
A kind of 3d gaming method of interface interacted based on augmented reality and movement Download PDFInfo
- Publication number
- CN109395375A CN109395375A CN201811086686.5A CN201811086686A CN109395375A CN 109395375 A CN109395375 A CN 109395375A CN 201811086686 A CN201811086686 A CN 201811086686A CN 109395375 A CN109395375 A CN 109395375A
- Authority
- CN
- China
- Prior art keywords
- data
- hand
- augmented reality
- mobile
- coordinate system
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/20—Input arrangements for video game devices
- A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
- A63F13/212—Input arrangements for video game devices characterised by their sensors, purposes or types using sensors worn by the player, e.g. for measuring heart beat or leg activity
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/40—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
- A63F13/42—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
- A63F13/428—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Human Computer Interaction (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biophysics (AREA)
- Cardiology (AREA)
- General Health & Medical Sciences (AREA)
- Heart & Thoracic Surgery (AREA)
- User Interface Of Digital Computer (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention proposes a kind of based on augmented reality and the mobile 3d gaming method of interface interacted.This method is divided into four steps: data collection, and data calculate, and data transmission, hand is rebuild.Data collection section is completed the acquisition to hand position and bearing data by Leap Motion；In data calculating section, the PC connecting with Leap Motion carries out coordinate system conversion to the hand data that acquisition comes and reduces noise data using Interval Kalman filter device and section particle filter, calculates hand direction；In TCP data segment, PC sends the data handled well to by server the client on HoloLens；In scene phase of regeneration, hand is rebuild out by HoloLens using the data received in virtual scene.The present invention can capture hand motion not against any handheld device, improve man-machine interactive compatibility, enhance the sense of reality of game.
Description
Technical field
The present invention relates to a kind of human-computer interaction technologies, and in particular to a kind of side that human-computer interaction is carried out based on hand motion
Method.
Background technique
Human-computer interaction is always the historical pith of development of computer.From earliest machine language, mouse finally
Mark, keyboard, and now universal touch screen etc..Keep man-machine interactive process more intuitive naturally, existing for computer
The development and application of every field has tremendous influence.
In recent years, human-computer interaction develops a variety of different interaction channels, including speech recognition, bio-identification and body-sensing are handed over
Mutually.Speech recognition application identifies robot in intelligent sounds such as Cortana, Siri；Bio-identification mainly utilizes human body distinctive
The physiological characteristics such as fingerprint, iris complete the functions such as mobile phone unlock, daily payment;In terms of body feeling interaction, Kinect is known as drawing
The product in epoch, as a 3-D camera, it can capture the depth of human body and be moved in the case where not wearing any equipment
Make.
And in aspects of game play, main interactive mode is to complete by game paddle or mouse and keyboard to game object
Control.AR or VR game newly developed needs handheld device to control game object.Although some game using Kinect into
Row body feeling interaction, but this interaction can be completed without any handheld device, but Kinect can not be moved, it means that dynamically catch
It is fixed for catching range.In order to keep player more intuitive naturally, we are captured using Leap Motion when controlling game object
Player's hand motion, i.e. player can complete interactive process by naked hand, without any handheld device.On the other hand, it passes
The scope of activities of the game interaction of system is excessively limited to, we devise a kind of moveable wearable device, no matter being in player
Where game interaction can be carried out.
Summary of the invention
In order to realize that naked hand interaction and the moveable function of equipment, the present invention provide a kind of based on augmented reality and movement
Interactive 3d gaming interface method.
The purpose of the present invention is achieved through the following technical solutions.
A kind of 3d gaming method of interface interacted based on augmented reality and movement, is captured by Leap Motion
Hand motion simultaneously obtains coordinate data, carries out coordinate system conversion to data, reduces the white noise of data, calculated by computer
Hand finally is rebuild out by hand direction in virtual scene.
Further, method includes the following steps:
S1, hand original coordinates and bearing data are obtained；
S2, coordinate system conversion is carried out, calculates accurate hand position and direction；
S3, the client sent by server treated data in AR equipment；
S4, AR equipment reconstruct hand model in virtual scene.
Further, step S1 obtains the three-dimensional coordinate information and bearing data of hand using Leap Motion.
Further, the original right hand cartesian coordinate that step S2 is obtained Leap Motion using cartesian coordinate system
System is converted into world coordinate system, and is mapped in the Virtual Space that AR equipment is established；Use the area Interval Kalman filter Qi He
Between particle filter removal due to calculating mass data and the white noise that generates, make calculated hand position data and direction number
According to more accurate.
Further, the server at the end PC transmits the data calculated to the client in AR equipment in step S3
End.
Further, AR equipment used in step S4 is HoloLens, and HoloLens is by the data handled well virtual
It rebuilds out in scene.
The present invention in conjunction with AR glasses HoloLens, incites somebody to action gesture induction device Leap Motion using cartesian coordinate system
Hand coordinate system is converted into world coordinate system and projects in the virtual scene of AR glasses building, allows equipment with user
Movement and move, expand the range of AR scene of game.The present invention directly acquires the number of coordinates for surveying naked hand using Leap Motion
According to direction of rotation data, and reduced due to long-time data stacking using Interval Kalman filter device and section particle filter
The white noise of generation, accurately calculates hand position and direction, enhances the Stability and veracity of interactive system.
Compared with prior art, the present invention having the following advantages that and technical effect:
1, body feeling interaction is carried out without handheld device, can directly captures naked manual work, makes the interactive process of game more
Add intuitive naturally, giving user more true game experiencing.
2, equipment is removable.The disadvantage of excessive, the mobile inconvenience of traditional body-sensing equipment volume is overcome, the user's of increase
Scope of activities.
Detailed description of the invention
Fig. 1 is a kind of based on augmented reality and the mobile 3d gaming method of interface flow chart interacted.
Specific embodiment
Specific implementation of the invention is described further below in conjunction with attached drawing and example, but implementation and protection of the invention
It is without being limited thereto.If it is noted that having process (such as Interval Kalman filter and particle filter of not special detailed description below
Algorithm), it is that those skilled in the art can refer to prior art understanding or realize.
At this stage there are three types of main man-machine interaction modes: 1) mouse；2) touch screen；3) body-sensing.And in the development side of game
Face, the game of mouse interactive class and touch screen interactive class game are more, but both interactive modes are for based on AR technological development
It is not applicable for game.And game this respect is being carried out by body feeling interaction, and mainly rely on Kinect --- it is a
3-D cameras capture human action completes interaction.However, Kinect device is more huge, need to be fixedly mounted, therefore player
Scope of activities is restricted.The characteristics of AR technology is that virtual 3-D object is shown on the basis of realistic space, and this requires AR to swim
Play can obtain more living scenes, avoid the dull and dull of game, and the irremovable feature of Kinect leads to its nothing
The exploitation of method adaptation AR game.
It is provided in this embodiment a kind of removable based on augmented reality in order to solve the above-mentioned problems of the prior art
Game interaction mode.Using Leap Motion and HoloLens small in size, light-weight as interactive device, with this reality
The existing moveable feature of equipment.Leap Motion captures hand data, carries out coordinate conversion to data by the end PC and accurately estimates
The data handled well are reconstructed to come in Virtual Space by meter using HoloLens.
As shown in Figure 1.The entire method of this example includes following four step:
S1, the three-dimensional coordinate information and bearing data that hand is obtained using Leap Motion；
S2, world coordinates is converted into using the original right-handed Cartesian coordinate system that cartesian coordinate system obtains Leap Motion
System, and be mapped in the Virtual Space that HoloLens is established；It is gone using Interval Kalman filter device and section particle filter
Except the white noise generated due to calculating mass data, keep calculated hand position data and bearing data more accurate；
S3, the client sent by server treated data on HoloLens；
S4, HoloLens reconstruct hand model in virtual scene.
The 3d gaming method of interface interacted based on augmented reality and movement of the present embodiment.This method is divided into four
Step: data collection, data calculate, and data transmission, hand is rebuild.Data collection section is completed by Leap Motion to hand
The acquisition of position and direction data；In data calculating section, hand data that the PC connecting with Leap Motion carrys out acquisition into
Row coordinate system converts and uses Interval Kalman filter device and section particle filter to reduce noise data, calculates hand direction；
In TCP data segment, PC sends the data handled well to by server the client on HoloLens；In scene rebuilding
Hand is rebuild out by stage, HoloLens using the data received in virtual scene.The present invention is not against any
Handheld device can capture hand motion, improve man-machine interactive compatibility, enhance the sense of reality of game.
Claims (6)
1. a kind of based on augmented reality and the mobile 3d gaming method of interface interacted, which is characterized in that pass through Leap
Motion captures hand motion and obtains coordinate data, carries out coordinate system conversion to data, reduces the white noise of data, pass through meter
Calculation machine calculates hand direction, finally rebuilds out in virtual scene by hand.
2. according to claim 1 a kind of based on augmented reality and the mobile 3d gaming interface method interacted, feature
Be method includes the following steps:
S1, hand original coordinates and bearing data are obtained；
S2, coordinate system conversion is carried out, calculates accurate hand position and direction；
S3, the client sent by server treated data in AR equipment；
S4, AR equipment reconstruct hand model in virtual scene.
3. according to claim 1 a kind of based on augmented reality and the mobile 3d gaming interface method interacted, feature
It is that step S1 obtains the three-dimensional coordinate information and bearing data of hand using Leap Motion.
4. according to claim 1 a kind of based on augmented reality and the mobile 3d gaming interface method interacted, feature
It is that step S2 is converted into the world using the original right-handed Cartesian coordinate system that cartesian coordinate system obtains Leap Motion and sits
Mark system, and be mapped in the Virtual Space that AR equipment is established；It is gone using Interval Kalman filter device and section particle filter
Except the white noise generated due to calculating mass data, keep calculated hand position data and bearing data more accurate.
5. according to claim 1 a kind of based on augmented reality and the mobile 3d gaming interface method interacted, feature
It is that the server at the end PC in step S3 transmits the data calculated to the client in AR equipment.
6. according to claim 1 a kind of based on augmented reality and the mobile 3d gaming interface method interacted, feature
It is that AR equipment used in step S4 is HoloLens, HoloLens structure again in virtual scene by the data handled well
It builds out.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811086686.5A CN109395375A (en) | 2018-09-18 | 2018-09-18 | A kind of 3d gaming method of interface interacted based on augmented reality and movement |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811086686.5A CN109395375A (en) | 2018-09-18 | 2018-09-18 | A kind of 3d gaming method of interface interacted based on augmented reality and movement |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109395375A true CN109395375A (en) | 2019-03-01 |
Family
ID=65464968
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811086686.5A Pending CN109395375A (en) | 2018-09-18 | 2018-09-18 | A kind of 3d gaming method of interface interacted based on augmented reality and movement |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109395375A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110070777A (en) * | 2019-06-13 | 2019-07-30 | 大连民族大学 | A kind of Hezhe's fish-skin draws simulation training system and implementation method |
| CN110561424A (en) * | 2019-07-28 | 2019-12-13 | 华南理工大学 | online robot kinematic calibration method based on multi-sensor hybrid filter |
| CN110877335A (en) * | 2019-09-27 | 2020-03-13 | 华南理工大学 | Self-adaptive unmarked mechanical arm track tracking method based on hybrid filter |
| CN112000219A (en) * | 2020-03-30 | 2020-11-27 | 华南理工大学 | Movable gesture interaction device and method for augmented reality game |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101807114A (en) * | 2010-04-02 | 2010-08-18 | 浙江大学 | Natural interactive method based on three-dimensional gestures |
| US20120270653A1 (en) * | 2011-04-25 | 2012-10-25 | Imimtek, Inc. | Method and system to create three-dimensional mapping in a two-dimensional game |
| CN104007819A (en) * | 2014-05-06 | 2014-08-27 | 清华大学 | Gesture recognition method and device and Leap Motion system |
| CN106444861A (en) * | 2016-11-21 | 2017-02-22 | 清华大学深圳研究生院 | Space robot teleoperation system based on three-dimensional gestures |
| CN108334199A (en) * | 2018-02-12 | 2018-07-27 | 华南理工大学 | The multi-modal exchange method of movable type based on augmented reality and device |
-
2018
- 2018-09-18 CN CN201811086686.5A patent/CN109395375A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101807114A (en) * | 2010-04-02 | 2010-08-18 | 浙江大学 | Natural interactive method based on three-dimensional gestures |
| US20120270653A1 (en) * | 2011-04-25 | 2012-10-25 | Imimtek, Inc. | Method and system to create three-dimensional mapping in a two-dimensional game |
| CN104007819A (en) * | 2014-05-06 | 2014-08-27 | 清华大学 | Gesture recognition method and device and Leap Motion system |
| CN106444861A (en) * | 2016-11-21 | 2017-02-22 | 清华大学深圳研究生院 | Space robot teleoperation system based on three-dimensional gestures |
| CN108334199A (en) * | 2018-02-12 | 2018-07-27 | 华南理工大学 | The multi-modal exchange method of movable type based on augmented reality and device |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110070777A (en) * | 2019-06-13 | 2019-07-30 | 大连民族大学 | A kind of Hezhe's fish-skin draws simulation training system and implementation method |
| CN110070777B (en) * | 2019-06-13 | 2021-07-09 | 大连民族大学 | Huchizhui fish skin painting simulation training system and implementation method |
| CN110561424A (en) * | 2019-07-28 | 2019-12-13 | 华南理工大学 | online robot kinematic calibration method based on multi-sensor hybrid filter |
| CN110877335A (en) * | 2019-09-27 | 2020-03-13 | 华南理工大学 | Self-adaptive unmarked mechanical arm track tracking method based on hybrid filter |
| CN112000219A (en) * | 2020-03-30 | 2020-11-27 | 华南理工大学 | Movable gesture interaction device and method for augmented reality game |
| CN112000219B (en) * | 2020-03-30 | 2022-06-14 | 华南理工大学 | Movable gesture interaction method for augmented reality game |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109003301B (en) | Human body posture estimation method based on OpenPose and Kinect and rehabilitation training system | |
| CN109395375A (en) | A kind of 3d gaming method of interface interacted based on augmented reality and movement | |
| CN107632699B (en) | Natural human-machine interaction system based on the fusion of more perception datas | |
| Ren et al. | Depth camera based hand gesture recognition and its applications in human-computer-interaction | |
| WO2021169839A1 (en) | Action restoration method and device based on skeleton key points | |
| Wang et al. | Real-time hand-tracking with a color glove | |
| CN102638653B (en) | Automatic face tracing method on basis of Kinect | |
| Segen et al. | Human-computer interaction using gesture recognition and 3D hand tracking | |
| CN1304931C (en) | Head carried stereo vision hand gesture identifying device | |
| CN110728739B (en) | Virtual human control and interaction method based on video stream | |
| WO2023071964A1 (en) | Data processing method and apparatus, and electronic device and computer-readable storage medium | |
| CN108509026B (en) | Remote maintenance support system and method based on enhanced interaction mode | |
| CN109885163A (en) | A kind of more people's interactive cooperation method and systems of virtual reality | |
| JP2004094288A (en) | Instructed position detecting device and autonomous robot | |
| WO2008029467A1 (en) | Gesture input system, method and program | |
| CN104090660A (en) | Motion collecting and feedback method and system based on stereoscopic vision | |
| CN102831380A (en) | Body action identification method and system based on depth image induction | |
| Leibe et al. | Toward spontaneous interaction with the perceptive workbench | |
| KR20170086317A (en) | Apparatus and Method for Generating 3D Character Motion via Timing Transfer | |
| CN102306053B (en) | Virtual touch screen-based man-machine interaction method and device and electronic equipment | |
| CN104517100B (en) | Gesture pre-judging method and system | |
| CN102830798A (en) | Mark-free hand tracking method of single-arm robot based on Kinect | |
| CN115129164B (en) | Interaction control method and system based on virtual reality and virtual reality equipment | |
| CN111596767A (en) | Gesture capturing method and device based on virtual reality | |
| CN106598211A (en) | Gesture interaction system and recognition method for multi-camera based wearable helmet |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190301 |
|
| RJ01 | Rejection of invention patent application after publication |