CN106502424A - Based on the interactive augmented reality system of speech gestures and limb action - Google Patents
Based on the interactive augmented reality system of speech gestures and limb action Download PDFInfo
- Publication number
- CN106502424A CN106502424A CN201611070594.9A CN201611070594A CN106502424A CN 106502424 A CN106502424 A CN 106502424A CN 201611070594 A CN201611070594 A CN 201611070594A CN 106502424 A CN106502424 A CN 106502424A
- Authority
- CN
- China
- Prior art keywords
- augmented reality
- video
- limb action
- module
- interactive
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Processing Or Creating Images (AREA)
Abstract
The invention discloses a kind of augmented reality system interactive based on speech gestures and limb action, including video acquisition module：Collection video；Position changes acquisition module：The information of collection position change；Gesture and limb action identification module：The information that video flowing or position change is identified processing；Audio collection module：Collection audio frequency；Phonetic control command identification module：The audio frequency for collecting is parsed and is recognized, generated phonetic order；Augmented reality control module：The video and audio frequency changed after receiving the input of phonetic order, gesture instruction and limb action in augmented reality system carries out interactive processing；Augmented reality output module：The content of the video of augmented reality is shown, while playing the audio content in augmented reality.The scenes such as family's projection, projection robot, video display box are present invention can be widely used to, it is more convenient to provide the user, more natural and more intelligent service.
Description
Technical field
The present invention relates to a kind of augmented reality (AR) system, more particularly to a kind of interactive based on speech gestures and limb action
Augmented reality system.
Background technology
The method and system of the augmented reality (AR) interactive with people, the main gesture instruction input by people, come real at present
Now interactive with augmented reality (AR).Based on the shortcoming of the method and system of the interactive augmented reality (AR) of only gesture instruction it is,
Consumer's Experience is not convenient enough, not natural and not intelligent enough enough.Its reason is that gesture instruction is single, then has current machine vision to people
Gesture identification degree also than relatively low, cause the single function of strengthening system and the interactive user for only interacting by the gesture of people
Experience is than relatively low.Simultaneously gesture instruction exist obvious defect have following some：1st, people can complete a gesture instruction
Speed is slow.2nd, current computer vision level can only identify very limited amount of gesture instruction.3rd, people is for seldom
Gesture instruction is relatively difficult to Precise Representation out.These shortcomings are only determined by gesture and augmented reality (AR) interaction
The function of method and system is relatively simple, and it is not intelligent enough to cause its Consumer's Experience, not convenient enough and not natural enough.
Content of the invention
It is existing that the technical problem to be solved is to provide a kind of enhancing interactive based on speech gestures and limb action
Real system, can be realized the interactive operation of augmented reality, be can be widely applied to based on the input such as voice, gesture and limb action
The scenes such as family's projection, projection robot, video display box, it is more convenient to provide the user, more natural and more intelligent clothes
Business.
The technical scheme that the present invention is adopted for solving above-mentioned technical problem is to provide one kind based on speech gestures and limbs
The augmented reality system of interactive motion, including video acquisition module：Video is gathered by single high-definition camera, or is passed through
Multiple high-definition camera collections have the video of the depth of field；Position changes acquisition module：By sensor acquisition position
The information of change；Gesture and limb action identification module：Video flowing from video acquisition module is parsed, identification is sold
Gesture instruction and limb action, and human body position in the scene；Or the information to the position change from sensor
Processed, identified limb action；Audio collection module：Sound is gathered by the MIC arrays of single MIC or spatial distribution
Frequently；Phonetic control command identification module：The audio frequency for collecting is parsed and is recognized, generated phonetic order；Augmented reality control
Molding block：Change the video and sound in augmented reality system after receiving the input of phonetic order, gesture instruction and limb action
Frequency carries out interactive processing；Augmented reality output module：The content of the video of augmented reality is shown, while play strengthening now
Audio content in reality.
The above-mentioned augmented reality system interactive based on speech gestures and limb action, wherein, the position change
Acquisition module gathers the information of position change by gyroscope, acceleration, infrared ray or gravity sensor.
The above-mentioned augmented reality system interactive based on speech gestures and limb action, wherein, the augmented reality control
After module receives phonetic order, gesture instruction and limb action, by changing the state of the scene and dummy object for showing, real
When complete corresponding interactive processing.
The above-mentioned augmented reality system interactive based on speech gestures and limb action, wherein, the augmented reality output
Module shows video content using LCD screen or projector.
Present invention contrast prior art has following beneficial effect：The present invention provide based on speech gestures and limb action
Interactive augmented reality system, can be widely applied for the scenes such as family's projection, projection robot, video display box, provides the user
More convenient, more intelligent service.
Description of the drawings
Fig. 1 is the augmented reality system architecture schematic diagram interactive based on speech gestures and limb action of the invention.
Specific embodiment
The invention will be further described with reference to the accompanying drawings and examples.
Fig. 1 is the augmented reality system architecture schematic diagram interactive based on speech gestures and limb action of the invention.
Fig. 1 is referred to, the augmented reality system interactive based on speech gestures and limb action that the present invention is provided, including such as
Lower seven modules：
1. video acquisition module
High-definition camera gathers video, or the video with the collection of many high-definition cameras with the depth of field；
2. position changes acquisition module：Limbs position is gathered by gyroscope, acceleration, infrared ray or gravity sensor
Put the information of change.
3. gesture and limb action identification module：Video flowing from video acquisition module is parsed, gesture is recognized
Instruction and limb action, and its human body position in the scene；Or by parsing the limbs position of the sensor acquisition such as gyroscope
The information of change is put, limb action is recognized.
4. audio collection module
Audio frequency is gathered by many MIC arrays of single MIC or spatial distribution.
5. phonetic control command identification module
Instruction translates into manipulable order by sound identification module.
6. augmented reality (AR) control module
The operation life of augmented reality system is translated into after the input for receiving phonetic order, gesture instruction and limb action
Order, changes the video (scene and void including display in augmented reality (AR) system according to the operational order of augmented reality system
Intend the state of object) and audio frequency, complete the video (scene and void including display that augmented reality (AR) is controlled by the instruction of people
Intend the state of object) and audio frequency.
7. augmented reality (AR) output module
The content of the video of augmented reality (AR) (including the scene that shows and the state of dummy object) be shown to screen,
In the display devices such as projector.Audio content in augmented reality is played simultaneously；The augmented reality system of the present invention passes through first
Single high-definition camera or multiple photographic head collect image information, then by the gesture instruction in Machine Vision Recognition image.
Or the information of the position change that arrives by sensor collections such as gyroscope, acceleration, infrared ray, gravity, then parse receipts
The positional information for collecting, identification limbs instruction.Again by voice MIC or MIC array acquisition voice, the voice that collects is parsed,
Identification phonetic order.Then augmented reality (AR) control module is receiving the defeated of phonetic order, gesture instruction and limb action
The operational order of augmented reality system is translated into after entering, and augmented reality (AR) system is changed according to the operational order of augmented reality system
Video (state of scene and dummy object including display) and audio content in system, completes to strengthen by the instruction control of people
The video (state of scene and dummy object including display) of real (AR) and audio frequency；.Augmented reality (AR) output module handle
The content of the video (state of scene and dummy object including display) of augmented reality (AR) is shown to screen, projector etc. and shows
Show in equipment, while playing the audio content in augmented reality；So as to realize the augmented reality (AR) interactive with people method and
System so as to realize based on voice, the method and system of the gesture augmented reality (AR) interactive with limb action.
The augmented reality system interactive based on speech gestures and limb action that the present invention is provided, can be widely applied for family
The scenes such as projection, projection robot, video display box, it is more convenient to provide the user, more intelligent service.Concrete advantage is such as
Under：1st, the method for interaction is very abundant.2nd, such exchange method be people life in very general.3rd, such interaction is also people
Live in simple and practical method.Interactive with augmented reality (AR) by such exchange method, it is possible to achieve to go out function
The augmented reality system that enriches very much.Its interactive mode and the life style of people be closely so that such interacts with people
The Consumer's Experience of augmented reality (AR) method and system is more convenient, more natural and more intelligent.
Although the present invention is disclosed as above with preferred embodiment, so which is not limited to the present invention, any this area skill
Art personnel, without departing from the spirit and scope of the present invention, when a little modification and perfect can be made, therefore protection model of the invention
Enclose when by being defined that claims are defined.
Claims (4)
1. a kind of augmented reality system interactive based on speech gestures and limb action, it is characterised in that include：
Video acquisition module：Video is gathered by single high-definition camera, or is had by the collection of multiple high-definition cameras
The video of the depth of field；
Position changes acquisition module：The information changed by sensor acquisition position；
Gesture and limb action identification module：Video flowing from video acquisition module is parsed, gesture instruction is identified
And limb action, and human body position in the scene；Or at the information to the position change from sensor
Reason, identifies limb action；
Audio collection module：Audio frequency is gathered by the MIC arrays of single MIC or spatial distribution；
Phonetic control command identification module：The audio frequency for collecting is parsed and is recognized, generated phonetic order；
Augmented reality control module：Change augmented reality system after receiving the input of phonetic order, gesture instruction and limb action
Video and audio frequency in system carries out interactive processing；
Augmented reality output module：The content of the video of augmented reality is shown, while playing the audio frequency in augmented reality
Content.
2. as claimed in claim 1 based on the augmented reality system that speech gestures and limb action are interactive, it is characterised in that institute
State position change acquisition module and position change is gathered by gyroscope, acceleration, infrared ray or gravity sensor
Information.
3. as claimed in claim 1 based on the augmented reality system that speech gestures and limb action are interactive, it is characterised in that institute
State after augmented reality control module receives phonetic order, gesture instruction and limb action, by changing the scene and void that show
Intend the state of object, complete corresponding interactive processing in real time.
4. as claimed in claim 1 based on the augmented reality system that speech gestures and limb action are interactive, it is characterised in that institute
State augmented reality output module and video content is shown using LCD screen or projector.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611070594.9A CN106502424A (en) | 2016-11-29 | 2016-11-29 | Based on the interactive augmented reality system of speech gestures and limb action |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611070594.9A CN106502424A (en) | 2016-11-29 | 2016-11-29 | Based on the interactive augmented reality system of speech gestures and limb action |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106502424A true CN106502424A (en) | 2017-03-15 |
Family
ID=58328761
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611070594.9A Pending CN106502424A (en) | 2016-11-29 | 2016-11-29 | Based on the interactive augmented reality system of speech gestures and limb action |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106502424A (en) |
Cited By (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106919120A (en) * | 2017-05-05 | 2017-07-04 | 美载（厦门）网络科技有限公司 | One kind can alternative projection robot |
| CN107564521A (en) * | 2017-09-10 | 2018-01-09 | 绵阳西真科技有限公司 | A kind of guest-meeting robot voice interface method and system |
| CN108052927A (en) * | 2017-12-29 | 2018-05-18 | 北京奇虎科技有限公司 | Gesture processing method and processing device based on video data, computing device |
| CN108062533A (en) * | 2017-12-28 | 2018-05-22 | 北京达佳互联信息技术有限公司 | Analytic method, system and the mobile terminal of user's limb action |
| CN108733200A (en) * | 2017-04-18 | 2018-11-02 | 芦伟杰 | A kind of AR screens motion sensing control power distribution method |
| CN109032358A (en) * | 2018-08-27 | 2018-12-18 | 百度在线网络技术（北京）有限公司 | The control method and device of AR interaction dummy model based on gesture identification |
| CN109828660A (en) * | 2018-12-29 | 2019-05-31 | 深圳云天励飞技术有限公司 | A kind of method and device of the control application operating based on augmented reality |
| CN109976519A (en) * | 2019-03-14 | 2019-07-05 | 浙江工业大学 | A kind of interactive display unit and its interactive display method based on augmented reality |
| CN110244837A (en) * | 2019-04-26 | 2019-09-17 | 北京圣威特科技有限公司 | Augmented reality and the experience glasses and its imaging method being superimposed with virtual image |
| CN110262662A (en) * | 2019-06-20 | 2019-09-20 | 河北识缘信息科技发展有限公司 | A kind of intelligent human-machine interaction method |
| CN110941328A (en) * | 2018-09-25 | 2020-03-31 | 未来市股份有限公司 | Interactive display method and device based on gesture recognition |
| CN111045510A (en) * | 2018-10-15 | 2020-04-21 | 中国移动通信集团山东有限公司 | Man-machine interaction method and system based on augmented reality |
| CN111093036A (en) * | 2020-01-17 | 2020-05-01 | 福州大学 | VR panoramic video interactive playing method based on limb recognition |
| CN112462939A (en) * | 2020-11-24 | 2021-03-09 | 苏州华恒展览设计营造有限公司 | Interactive projection method and system |
| CN112904999A (en) * | 2020-12-30 | 2021-06-04 | 江苏奥格视特信息科技有限公司 | Augmented reality somatosensory interaction method and system based on laser radar |
| CN113160806A (en) * | 2020-01-07 | 2021-07-23 | 京东方科技集团股份有限公司 | Projection system and control method thereof |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104063039A (en) * | 2013-03-18 | 2014-09-24 | 朱慧灵 | Human-computer interaction method of wearable computer intelligent terminal |
| CN104615243A (en) * | 2015-01-15 | 2015-05-13 | 深圳市掌网立体时代视讯技术有限公司 | Head-wearable type multi-channel interaction system and multi-channel interaction method |
| CN205507230U (en) * | 2015-12-18 | 2016-08-24 | 济南中景电子科技有限公司 | Virtual reality glass |
-
2016
- 2016-11-29 CN CN201611070594.9A patent/CN106502424A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104063039A (en) * | 2013-03-18 | 2014-09-24 | 朱慧灵 | Human-computer interaction method of wearable computer intelligent terminal |
| CN104615243A (en) * | 2015-01-15 | 2015-05-13 | 深圳市掌网立体时代视讯技术有限公司 | Head-wearable type multi-channel interaction system and multi-channel interaction method |
| CN205507230U (en) * | 2015-12-18 | 2016-08-24 | 济南中景电子科技有限公司 | Virtual reality glass |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108733200A (en) * | 2017-04-18 | 2018-11-02 | 芦伟杰 | A kind of AR screens motion sensing control power distribution method |
| CN106919120A (en) * | 2017-05-05 | 2017-07-04 | 美载（厦门）网络科技有限公司 | One kind can alternative projection robot |
| CN107564521A (en) * | 2017-09-10 | 2018-01-09 | 绵阳西真科技有限公司 | A kind of guest-meeting robot voice interface method and system |
| CN108062533A (en) * | 2017-12-28 | 2018-05-22 | 北京达佳互联信息技术有限公司 | Analytic method, system and the mobile terminal of user's limb action |
| CN108052927B (en) * | 2017-12-29 | 2021-06-01 | 北京奇虎科技有限公司 | Gesture processing method and device based on video data and computing equipment |
| CN108052927A (en) * | 2017-12-29 | 2018-05-18 | 北京奇虎科技有限公司 | Gesture processing method and processing device based on video data, computing device |
| CN109032358A (en) * | 2018-08-27 | 2018-12-18 | 百度在线网络技术（北京）有限公司 | The control method and device of AR interaction dummy model based on gesture identification |
| CN110941328A (en) * | 2018-09-25 | 2020-03-31 | 未来市股份有限公司 | Interactive display method and device based on gesture recognition |
| CN111045510A (en) * | 2018-10-15 | 2020-04-21 | 中国移动通信集团山东有限公司 | Man-machine interaction method and system based on augmented reality |
| CN109828660A (en) * | 2018-12-29 | 2019-05-31 | 深圳云天励飞技术有限公司 | A kind of method and device of the control application operating based on augmented reality |
| CN109976519A (en) * | 2019-03-14 | 2019-07-05 | 浙江工业大学 | A kind of interactive display unit and its interactive display method based on augmented reality |
| CN109976519B (en) * | 2019-03-14 | 2022-05-03 | 浙江工业大学 | Interactive display device based on augmented reality and interactive display method thereof |
| CN110244837A (en) * | 2019-04-26 | 2019-09-17 | 北京圣威特科技有限公司 | Augmented reality and the experience glasses and its imaging method being superimposed with virtual image |
| CN110262662A (en) * | 2019-06-20 | 2019-09-20 | 河北识缘信息科技发展有限公司 | A kind of intelligent human-machine interaction method |
| CN113160806A (en) * | 2020-01-07 | 2021-07-23 | 京东方科技集团股份有限公司 | Projection system and control method thereof |
| CN111093036A (en) * | 2020-01-17 | 2020-05-01 | 福州大学 | VR panoramic video interactive playing method based on limb recognition |
| CN112462939A (en) * | 2020-11-24 | 2021-03-09 | 苏州华恒展览设计营造有限公司 | Interactive projection method and system |
| CN112904999A (en) * | 2020-12-30 | 2021-06-04 | 江苏奥格视特信息科技有限公司 | Augmented reality somatosensory interaction method and system based on laser radar |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106502424A (en) | Based on the interactive augmented reality system of speech gestures and limb action | |
| CN104219584B (en) | Panoramic video exchange method and system based on augmented reality | |
| CN106358036B (en) | A kind of method that virtual reality video is watched with default visual angle | |
| CN108986189B (en) | Method and system for capturing and live broadcasting of real-time multi-person actions based on three-dimensional animation | |
| CN104102412B (en) | A kind of hand-held reading device and method thereof based on augmented reality | |
| CN105340014B (en) | Touch optimization design for video editing | |
| CN205622745U (en) | Real -time synthesis system of virtual reality true man | |
| CN201570011U (en) | Terminal control device and terminal | |
| CN106730815A (en) | The body-sensing interactive approach and system of a kind of easy realization | |
| CN103533445B (en) | Flying theater playing system based on active interaction | |
| CN104396234B (en) | Transmission of video and reconstruct | |
| CN201178467Y (en) | Interactive floor/wall system | |
| CN206649496U (en) | A kind of object identification and projection interactive device | |
| CN103841299A (en) | Virtual studio system | |
| CN104606882A (en) | Motion sensing game interaction method and system | |
| CN207123961U (en) | Immersion multi-person synergy trainer for the three-dimensional arc curtain formula of Substation Training | |
| CN106210562A (en) | A kind of implementation method of live specially good effect | |
| CN109116987A (en) | A kind of holographic display system based on Kinect gesture control | |
| CN106571108A (en) | Advisement player having voice interaction function | |
| CN203607077U (en) | Virtual traveling machine | |
| CN107566863A (en) | A kind of exchange of information methods of exhibiting, device and equipment, set top box | |
| CN102572308A (en) | Interactive panoramic system | |
| CN105163081A (en) | Real object display method and system based on asynchronous rendering | |
| CN202548758U (en) | Interactive projection system | |
| CN110971924A (en) | Method, device, storage medium and system for beautifying in live broadcast process |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20170315 |