CN105068649A - Binocular gesture recognition device and method based on virtual reality helmet - Google Patents
Binocular gesture recognition device and method based on virtual reality helmet Download PDFInfo
- Publication number
- CN105068649A CN105068649A CN201510489700.6A CN201510489700A CN105068649A CN 105068649 A CN105068649 A CN 105068649A CN 201510489700 A CN201510489700 A CN 201510489700A CN 105068649 A CN105068649 A CN 105068649A
- Authority
- CN
- China
- Prior art keywords
- gesture
- information
- user
- eye camera
- binocular
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
The invention discloses a binocular gesture recognition method based on a virtual reality helmet. The method comprises the following steps of: S1, separately collecting gesture information of a user through a left-eye camera and a right-eye camera, and transmitting the collected gesture information to a data processing module; S2, performing three-dimensional reconstruction on the gesture information by the data processing module to generate gesture three-dimensional information of the user, and transmitting the information to a display screen; and S3, acquiring the gesture three-dimensional information by the display screen and performing synchronous real-time display. The invention further provides a binocular gesture recognition device based on the virtual reality helmet. The left-eye camera and the right-eye camera are used for collecting operation gestures of the user, and the collected information is subjected to image and data processing to obtain the gesture three-dimensional information of the user; and the synchronous real-time display of the information is performed on the corresponding display screen, so as to realize consistent synchronization of the gesture operation of the user and display operation of a display; and therefore, direct and accurate operation experience is provided for the user. Compared with the prior art, the binocular gesture recognition device and method have the advantages of being good in visibility, high in accuracy and strong in experience feeling, etc.
Description
technical field:
The present invention relates to virtual implementing helmet, what be specifically related to is a kind of binocular gesture identifying device based on virtual implementing helmet and method.
background technology:
Virtual implementing helmet, is also helmet-mounted display, is virtual reality display the earliest, and it utilizes Helmet Mounted Display people's vision to external world, the sense of hearing to be closed, and guides user to produce a kind of sensation in virtual environment.Its displaying principle is the image that right and left eyes screen shows right and left eyes respectively, and human eye produces stereoscopic sensation after obtaining the discrepant information of this band in brain.Helmet Mounted Display is as the display device of virtual reality, and have small and exquisite and that closure is strong feature, in military training, virtual driving, has a wide range of applications in the project such as virtual city and game.
Utilize virtual implementing helmet when carrying out above-mentioned application at present, usually need to operate by telepilot or handle, the problem that its ubiquity operation intuitive is poor, accuracy is not high, the experience sense of user is not strong.
summary of the invention:
For this reason, the object of the present invention is to provide the binocular gesture identifying device based on virtual implementing helmet that a kind of intuitive is good, accuracy is high, experience sense is strong and method.
For achieving the above object, the present invention mainly adopts following technical scheme:
Based on a binocular gesture identifying device for virtual implementing helmet, comprise virtual implementing helmet, be provided with in described virtual implementing helmet:
One left eye camera and a right eye camera, be respectively used to the gesture information gathering user;
One data processing module, is connected with described left eye camera and the communication of right eye camera respectively, for obtaining the gesture information of left eye camera and right eye camera collection, and carries out three-dimensional reconstruction to described gesture information, generates the gesture three-dimensional information of user;
One display screen, is connected with described data processing module communication, for obtaining described gesture three-dimensional information, and carries out synchronous display in real time.
Preferably, described data processing module, also for carrying out analyzing and processing to described gesture information, extracting and obtaining the hand position information of user.
Preferably, described left eye camera is for gathering the left-side images information of user's gesture; Right eye camera is for gathering the gesture image right information of user.
Preferably, described gesture information comprises slip, clicks and grasping information.
In addition, present invention also offers a kind of binocular gesture identification method based on virtual implementing helmet, comprise step:
S1, by left eye camera and right eye camera, gather the gesture information of user respectively, and the gesture information collected is sent to data processing module;
S2, data processing module, by carrying out three-dimensional reconstruction to described gesture information, generate the gesture three-dimensional information of user, and send to display screen;
S3, display screen obtain gesture three-dimensional information, and carry out synchronous display in real time.
Preferably, step S1 specifically comprises:
By left eye camera with gathering images of gestures information on the left of user;
And by right eye camera collection user right images of gestures information.
Preferably, step S2 specifically comprises:
Data processing module, by carrying out analyzing and processing to the gesture information obtained, therefrom extracts and obtains the hand position information of user, then carrying out three-dimensional reconstruction to described gesture information, generating the gesture three-dimensional information of user.
The present invention uses left eye camera and the operating gesture of right eye camera to user to gather, the gesture three-dimensional information of user is obtained after figure and data processing are carried out to the information gathered, and correspondence carries out synchronous display in real time on a display screen, operate synchronous consistent with the display of the gesture operation with display that realize user, thus for user provide intuitively, operating experience accurately.The present invention, compared with conventional art, has that intuitive is good, accuracy is high and the advantage such as experience sense is strong.
accompanying drawing illustrates:
Fig. 1 is the schematic perspective view of virtual implementing helmet of the present invention;
Fig. 2 is that the present invention carries out the process flow diagram of gesture identification by virtual implementing helmet.
embodiment:
For setting forth thought of the present invention and object, below in conjunction with the drawings and specific embodiments, the present invention is described further.
The present embodiment provides the virtual implementing helmet that a kind of intuitive is good, accuracy is high, experience sense is strong, utilize this virtual implementing helmet can realize binocular gesture identification, it is by the gesture motion of two the camera collection users in left and right, three-dimensional reconstruction is carried out to it, generate the gesture three-dimensional information of user, thus simultaneous display corresponding to display screen operates.
Shown in Figure 1, Fig. 1 is the schematic perspective view of virtual implementing helmet of the present invention.Virtual implementing helmet of the present invention, this helmet includes left eye camera 1, right eye camera 2, helmet back casing 4 and helmet procapsid 3.
Left eye camera 1, right eye camera 2 are arranged in procapsid 3, and correspondence is arranged at the left and right sides of procapsid 3.
In helmet back casing 4, data processing module and display screen are also installed.
Wherein left eye camera 1 is connected with data processing module communication respectively with right eye camera 2, when user do gesture motion in the front of left eye camera 1 and right eye camera 2 time, right eye camera 2 correspondingly can gather a graphical information (right side gesture information); Left eye camera 1 also can collect a graphical information (left side gesture information) simultaneously.
Gesture motion described here can be various different actions, and it includes but not limited to the actions such as slip, click, grasping.
Data processing module is connected with described left eye camera and the communication of right eye camera, also be connected with display screen communication simultaneously, this data processing module is for obtaining the gesture information of left eye camera and right eye camera collection, and three-dimensional reconstruction is carried out to described gesture information, generate the gesture three-dimensional information of user, and the gesture three-dimensional information that will obtain, synchronous display is in real time on a display screen.
Present invention also offers a kind of binocular gesture identification method based on virtual implementing helmet, be further described below in conjunction with accompanying drawing 2 and specific embodiment.
As shown in Figure 2, Fig. 2 is that the present invention carries out the process flow diagram of gesture identification by virtual implementing helmet.The present invention is based on the binocular gesture identification method of virtual implementing helmet, specifically comprise step as follows:
S1, by left eye camera and right eye camera, gather the gesture information of user respectively, and the gesture information collected is sent to data processing module;
Wherein by left eye camera images of gestures information on the left of collection user; By right eye camera collection user right images of gestures information.
Here the left side images of gestures information collected and right side images of gestures information are an one-side messages of gesture motion, and this information can only demonstrate the effect of plane, and is difficult to accurately locate.
S2, data processing module, by carrying out three-dimensional reconstruction to described gesture information, generate the gesture three-dimensional information of user, and send to display screen;
Data processing module, by carrying out analyzing and processing to the gesture information obtained, therefrom extracts and obtains the hand position information of user, then carrying out three-dimensional reconstruction to described gesture information, generating the gesture three-dimensional information of user.
Because two, the left and right graphical information obtained is the information of plane, by data processing module, the two is combined, the position relationship data of both extractions, then undertaken calculating the three-dimensional information obtaining user by three-dimensional algorithm, namely carry out three-dimensional reconstruction.
S3, display screen obtain gesture three-dimensional information, and carry out synchronous display in real time.
Virtual implementing helmet is used to carry out playing games or experiencing virtual community for user, operated by the mode such as telepilot or handle, there is visual sense difference, experience bad phenomenon, invention has been improvement, its achieve virtual implementing helmet play games or experiencing virtual community process in, without the need to the operation by telepilot or handle, it only needs user can to realize in helmet display screen corresponding display by gesture operation, make the operation of user more directly perceived, experience sense is stronger.
More than that the binocular gesture identifying device based on virtual implementing helmet provided by the present invention and method are described in detail, apply specific case herein to set forth structural principle of the present invention and embodiment, above embodiment just understands method of the present invention and core concept thereof for helping; Meanwhile, for one of ordinary skill in the art, according to thought of the present invention, all will change in specific embodiments and applications, in sum, this description should not be construed as limitation of the present invention.
Claims (7)
1. based on a binocular gesture identifying device for virtual implementing helmet, comprise virtual implementing helmet, it is characterized in that, be provided with in described virtual implementing helmet:
One left eye camera and a right eye camera, be respectively used to the gesture information gathering user;
One data processing module, is connected with described left eye camera and the communication of right eye camera respectively, for obtaining the gesture information of left eye camera and right eye camera collection, and carries out three-dimensional reconstruction to described gesture information, generates the gesture three-dimensional information of user;
One display screen, is connected with described data processing module communication, for obtaining described gesture three-dimensional information, and carries out synchronous display in real time.
2. as claimed in claim 1 based on the binocular gesture identifying device of virtual implementing helmet, it is characterized in that, described data processing module, also for carrying out analyzing and processing to described gesture information, extracting and obtaining the hand position information of user.
3., as claimed in claim 2 based on the binocular gesture identifying device of virtual implementing helmet, it is characterized in that, described left eye camera is for gathering the left-side images information of user's gesture; Right eye camera is for gathering the gesture image right information of user.
4. as claimed in claim 3 based on the binocular gesture identifying device of virtual implementing helmet, it is characterized in that, described gesture information comprises slip, clicks and grasping information.
5., based on a binocular gesture identification method for virtual implementing helmet, it is characterized in that, comprise step:
S1, by left eye camera and right eye camera, gather the gesture information of user respectively, and the gesture information collected is sent to data processing module;
S2, data processing module, by carrying out three-dimensional reconstruction to described gesture information, generate the gesture three-dimensional information of user, and send to display screen;
S3, display screen obtain gesture three-dimensional information, and carry out synchronous display in real time.
6., as claimed in claim 5 based on the binocular gesture identifying device of virtual implementing helmet, it is characterized in that, step S1 specifically comprises:
By left eye camera with gathering images of gestures information on the left of user;
And by right eye camera collection user right images of gestures information.
7., as claimed in claim 5 based on the binocular gesture identifying device of virtual implementing helmet, it is characterized in that, step S2 specifically comprises:
Data processing module, by carrying out analyzing and processing to the gesture information obtained, therefrom extracts and obtains the hand position information of user, then carrying out three-dimensional reconstruction to described gesture information, generating the gesture three-dimensional information of user.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510489700.6A CN105068649A (en) | 2015-08-12 | 2015-08-12 | Binocular gesture recognition device and method based on virtual reality helmet |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510489700.6A CN105068649A (en) | 2015-08-12 | 2015-08-12 | Binocular gesture recognition device and method based on virtual reality helmet |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105068649A true CN105068649A (en) | 2015-11-18 |
Family
ID=54498035
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510489700.6A Pending CN105068649A (en) | 2015-08-12 | 2015-08-12 | Binocular gesture recognition device and method based on virtual reality helmet |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105068649A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105487673A (en) * | 2016-01-04 | 2016-04-13 | 京东方科技集团股份有限公司 | Man-machine interactive system, method and device |
| CN105808014A (en) * | 2016-05-19 | 2016-07-27 | 京东方科技集团股份有限公司 | Array substrate and manufacturing method thereof as well as touch display device |
| CN105867626A (en) * | 2016-04-12 | 2016-08-17 | 京东方科技集团股份有限公司 | Head-mounted virtual reality equipment, control method thereof and virtual reality system |
| CN105975158A (en) * | 2016-05-11 | 2016-09-28 | 乐视控股（北京）有限公司 | Virtual reality interaction method and device |
| CN106251735A (en) * | 2016-10-13 | 2016-12-21 | 西南大学 | A kind of virtual driving system |
| CN106371206A (en) * | 2016-08-31 | 2017-02-01 | 安徽协创物联网技术有限公司 | Wide visual angle virtual reality device |
| CN106371597A (en) * | 2016-08-31 | 2017-02-01 | 安徽协创物联网技术有限公司 | Virtuality and reality exchanging light VR (Virtual Reality) all-in-one machine |
| CN106598211A (en) * | 2016-09-29 | 2017-04-26 | 莫冰 | Gesture interaction system and recognition method for multi-camera based wearable helmet |
| CN106598235A (en) * | 2016-11-29 | 2017-04-26 | 歌尔科技有限公司 | Gesture recognition method and apparatus for virtual reality device, and virtual reality device |
| WO2017092432A1 (en) * | 2015-12-01 | 2017-06-08 | 乐视控股（北京）有限公司 | Method, device, and system for virtual reality interaction |
| CN107943293A (en) * | 2017-11-24 | 2018-04-20 | 联想(北京)有限公司 | A kind of information interacting method and information processor |
| CN109460150A (en) * | 2018-11-12 | 2019-03-12 | 北京特种机械研究所 | A kind of virtual reality human-computer interaction system and method |
| US11507190B2 (en) | 2016-07-26 | 2022-11-22 | Huawei Technologies Co., Ltd. | Gesture control method applied to VR device, and apparatus |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1413566A (en) * | 2002-08-30 | 2003-04-30 | 延自强 | Virtual reality acupuncture point location method and system |
| CN1604014A (en) * | 2003-09-30 | 2005-04-06 | 佳能株式会社 | Image display apparatus and method |
| US20050234333A1 (en) * | 2004-03-31 | 2005-10-20 | Canon Kabushiki Kaisha | Marker detection method and apparatus, and position and orientation estimation method |
| CN201965308U (en) * | 2010-12-29 | 2011-09-07 | 鼎创电子股份有限公司 | Head-wearing instant video recording and playing device capable of three-dimensionally imaging and receiving and transmitting signals |
| CN102350700A (en) * | 2011-09-19 | 2012-02-15 | 华南理工大学 | Method for controlling robot based on visual sense |
| CN103559734A (en) * | 2013-10-17 | 2014-02-05 | 中国科学院自动化研究所 | Binocular imaging three-dimensional visual system device and control method |
| TW201443699A (en) * | 2013-03-28 | 2014-11-16 | Sony Corp | Image processing device and method, and program |
-
2015
- 2015-08-12 CN CN201510489700.6A patent/CN105068649A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1413566A (en) * | 2002-08-30 | 2003-04-30 | 延自强 | Virtual reality acupuncture point location method and system |
| CN1604014A (en) * | 2003-09-30 | 2005-04-06 | 佳能株式会社 | Image display apparatus and method |
| US20050234333A1 (en) * | 2004-03-31 | 2005-10-20 | Canon Kabushiki Kaisha | Marker detection method and apparatus, and position and orientation estimation method |
| CN201965308U (en) * | 2010-12-29 | 2011-09-07 | 鼎创电子股份有限公司 | Head-wearing instant video recording and playing device capable of three-dimensionally imaging and receiving and transmitting signals |
| CN102350700A (en) * | 2011-09-19 | 2012-02-15 | 华南理工大学 | Method for controlling robot based on visual sense |
| TW201443699A (en) * | 2013-03-28 | 2014-11-16 | Sony Corp | Image processing device and method, and program |
| CN103559734A (en) * | 2013-10-17 | 2014-02-05 | 中国科学院自动化研究所 | Binocular imaging three-dimensional visual system device and control method |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2017092432A1 (en) * | 2015-12-01 | 2017-06-08 | 乐视控股（北京）有限公司 | Method, device, and system for virtual reality interaction |
| CN105487673A (en) * | 2016-01-04 | 2016-04-13 | 京东方科技集团股份有限公司 | Man-machine interactive system, method and device |
| US10585488B2 (en) | 2016-01-04 | 2020-03-10 | Boe Technology Group Co., Ltd. | System, method, and apparatus for man-machine interaction |
| CN105867626A (en) * | 2016-04-12 | 2016-08-17 | 京东方科技集团股份有限公司 | Head-mounted virtual reality equipment, control method thereof and virtual reality system |
| CN105975158A (en) * | 2016-05-11 | 2016-09-28 | 乐视控股（北京）有限公司 | Virtual reality interaction method and device |
| US10324571B2 (en) | 2016-05-19 | 2019-06-18 | Boe Technology Group Co., Ltd. | Array substrate, manufacturing method thereof and touch display device |
| CN105808014A (en) * | 2016-05-19 | 2016-07-27 | 京东方科技集团股份有限公司 | Array substrate and manufacturing method thereof as well as touch display device |
| CN105808014B (en) * | 2016-05-19 | 2019-07-02 | 京东方科技集团股份有限公司 | A kind of array substrate and preparation method thereof, touch control display apparatus |
| US11507190B2 (en) | 2016-07-26 | 2022-11-22 | Huawei Technologies Co., Ltd. | Gesture control method applied to VR device, and apparatus |
| CN106371206A (en) * | 2016-08-31 | 2017-02-01 | 安徽协创物联网技术有限公司 | Wide visual angle virtual reality device |
| CN106371597A (en) * | 2016-08-31 | 2017-02-01 | 安徽协创物联网技术有限公司 | Virtuality and reality exchanging light VR (Virtual Reality) all-in-one machine |
| CN106598211A (en) * | 2016-09-29 | 2017-04-26 | 莫冰 | Gesture interaction system and recognition method for multi-camera based wearable helmet |
| CN106251735A (en) * | 2016-10-13 | 2016-12-21 | 西南大学 | A kind of virtual driving system |
| CN106598235A (en) * | 2016-11-29 | 2017-04-26 | 歌尔科技有限公司 | Gesture recognition method and apparatus for virtual reality device, and virtual reality device |
| CN106598235B (en) * | 2016-11-29 | 2019-10-22 | 歌尔科技有限公司 | Gesture identification method, device and virtual reality device for virtual reality device |
| WO2018098862A1 (en) * | 2016-11-29 | 2018-06-07 | 歌尔科技有限公司 | Gesture recognition method and device for virtual reality apparatus, and virtual reality apparatus |
| CN107943293A (en) * | 2017-11-24 | 2018-04-20 | 联想(北京)有限公司 | A kind of information interacting method and information processor |
| CN109460150A (en) * | 2018-11-12 | 2019-03-12 | 北京特种机械研究所 | A kind of virtual reality human-computer interaction system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105068649A (en) | Binocular gesture recognition device and method based on virtual reality helmet | |
| CN107656613B (en) | Human-computer interaction system based on eye movement tracking and working method thereof | |
| CN102647606B (en) | Stereoscopic image processor, stereoscopic image interaction system and stereoscopic image display method | |
| CN107004296B (en) | Method and system for reconstructing an occlusion face of a virtual reality environment | |
| CN103873840B (en) | Display methods and display equipment | |
| CN103810353A (en) | Real scene mapping system and method in virtual reality | |
| CN105869215B (en) | A kind of virtual reality imaging system | |
| CN105959666A (en) | Method and device for sharing 3d image in virtual reality system | |
| CN203746012U (en) | Three-dimensional virtual scene human-computer interaction stereo display system | |
| JP2020513704A (en) | Video data processing method, apparatus and equipment | |
| CN111226187A (en) | System and method for interacting with a user through a mirror | |
| CN204990186U (en) | Desktop formula stereoscopic display interaction system | |
| CN105184622A (en) | Network shopping for consumer by utilization of virtual technology | |
| CN105721857A (en) | Helmet with double cameras | |
| CN103257703A (en) | Augmented reality device and method | |
| CN103810399A (en) | Sandplay evaluation system | |
| CN106127846A (en) | Virtual reality terminal and vision virtual method thereof and device | |
| CN109068122A (en) | Image display method, device, equipment and storage medium | |
| CN102880352A (en) | Non-contact interface operation method and non-contact interface operation system | |
| CN108388148A (en) | A kind of Chinese medicine production visualization virtual reality display system | |
| CN104510568A (en) | Method for realizing visualization of blind by virtual reality technology | |
| CN210804322U (en) | Virtual and real scene operation platform based on 5G and MR technology | |
| CN202058337U (en) | Stereoscopic display interactive helmet | |
| CN202713528U (en) | System for synchronous acquisition of 3D data | |
| CN114866757B (en) | Stereoscopic display system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20151118 |
|
| RJ01 | Rejection of invention patent application after publication |