CN106502424A - Based on the interactive augmented reality system of speech gestures and limb action - Google Patents

Based on the interactive augmented reality system of speech gestures and limb action Download PDF

Info

Publication number
CN106502424A
CN106502424A CN201611070594.9A CN201611070594A CN106502424A CN 106502424 A CN106502424 A CN 106502424A CN 201611070594 A CN201611070594 A CN 201611070594A CN 106502424 A CN106502424 A CN 106502424A
Authority
CN
China
Prior art keywords
augmented reality
video
limb action
module
interactive
Prior art date
Application number
CN201611070594.9A
Other languages
Chinese (zh)
Inventor
覃永平
蒋武锋
李晓城
Original Assignee
上海小持智能科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海小持智能科技有限公司 filed Critical 上海小持智能科技有限公司
Priority to CN201611070594.9A priority Critical patent/CN106502424A/en
Publication of CN106502424A publication Critical patent/CN106502424A/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures

Abstract

The invention discloses a kind of augmented reality system interactive based on speech gestures and limb action, including video acquisition module:Collection video;Position changes acquisition module:The information of collection position change;Gesture and limb action identification module:The information that video flowing or position change is identified processing;Audio collection module:Collection audio frequency;Phonetic control command identification module:The audio frequency for collecting is parsed and is recognized, generated phonetic order;Augmented reality control module:The video and audio frequency changed after receiving the input of phonetic order, gesture instruction and limb action in augmented reality system carries out interactive processing;Augmented reality output module:The content of the video of augmented reality is shown, while playing the audio content in augmented reality.The scenes such as family's projection, projection robot, video display box are present invention can be widely used to, it is more convenient to provide the user, more natural and more intelligent service.

Description

Based on the interactive augmented reality system of speech gestures and limb action

Technical field

The present invention relates to a kind of augmented reality (AR) system, more particularly to a kind of interactive based on speech gestures and limb action Augmented reality system.

Background technology

The method and system of the augmented reality (AR) interactive with people, the main gesture instruction input by people, come real at present Now interactive with augmented reality (AR).Based on the shortcoming of the method and system of the interactive augmented reality (AR) of only gesture instruction it is, Consumer's Experience is not convenient enough, not natural and not intelligent enough enough.Its reason is that gesture instruction is single, then has current machine vision to people Gesture identification degree also than relatively low, cause the single function of strengthening system and the interactive user for only interacting by the gesture of people Experience is than relatively low.Simultaneously gesture instruction exist obvious defect have following some:1st, people can complete a gesture instruction Speed is slow.2nd, current computer vision level can only identify very limited amount of gesture instruction.3rd, people is for seldom Gesture instruction is relatively difficult to Precise Representation out.These shortcomings are only determined by gesture and augmented reality (AR) interaction The function of method and system is relatively simple, and it is not intelligent enough to cause its Consumer's Experience, not convenient enough and not natural enough.

Content of the invention

It is existing that the technical problem to be solved is to provide a kind of enhancing interactive based on speech gestures and limb action Real system, can be realized the interactive operation of augmented reality, be can be widely applied to based on the input such as voice, gesture and limb action The scenes such as family's projection, projection robot, video display box, it is more convenient to provide the user, more natural and more intelligent clothes Business.

The technical scheme that the present invention is adopted for solving above-mentioned technical problem is to provide one kind based on speech gestures and limbs The augmented reality system of interactive motion, including video acquisition module:Video is gathered by single high-definition camera, or is passed through Multiple high-definition camera collections have the video of the depth of field;Position changes acquisition module:By sensor acquisition position The information of change;Gesture and limb action identification module:Video flowing from video acquisition module is parsed, identification is sold Gesture instruction and limb action, and human body position in the scene;Or the information to the position change from sensor Processed, identified limb action;Audio collection module:Sound is gathered by the MIC arrays of single MIC or spatial distribution Frequently;Phonetic control command identification module:The audio frequency for collecting is parsed and is recognized, generated phonetic order;Augmented reality control Molding block:Change the video and sound in augmented reality system after receiving the input of phonetic order, gesture instruction and limb action Frequency carries out interactive processing;Augmented reality output module:The content of the video of augmented reality is shown, while play strengthening now Audio content in reality.

The above-mentioned augmented reality system interactive based on speech gestures and limb action, wherein, the position change Acquisition module gathers the information of position change by gyroscope, acceleration, infrared ray or gravity sensor.

The above-mentioned augmented reality system interactive based on speech gestures and limb action, wherein, the augmented reality control After module receives phonetic order, gesture instruction and limb action, by changing the state of the scene and dummy object for showing, real When complete corresponding interactive processing.

The above-mentioned augmented reality system interactive based on speech gestures and limb action, wherein, the augmented reality output Module shows video content using LCD screen or projector.

Present invention contrast prior art has following beneficial effect:The present invention provide based on speech gestures and limb action Interactive augmented reality system, can be widely applied for the scenes such as family's projection, projection robot, video display box, provides the user More convenient, more intelligent service.

Description of the drawings

Fig. 1 is the augmented reality system architecture schematic diagram interactive based on speech gestures and limb action of the invention.

Specific embodiment

The invention will be further described with reference to the accompanying drawings and examples.

Fig. 1 is the augmented reality system architecture schematic diagram interactive based on speech gestures and limb action of the invention.

Fig. 1 is referred to, the augmented reality system interactive based on speech gestures and limb action that the present invention is provided, including such as Lower seven modules:

1. video acquisition module

High-definition camera gathers video, or the video with the collection of many high-definition cameras with the depth of field;

2. position changes acquisition module:Limbs position is gathered by gyroscope, acceleration, infrared ray or gravity sensor Put the information of change.

3. gesture and limb action identification module:Video flowing from video acquisition module is parsed, gesture is recognized Instruction and limb action, and its human body position in the scene;Or by parsing the limbs position of the sensor acquisition such as gyroscope The information of change is put, limb action is recognized.

4. audio collection module

Audio frequency is gathered by many MIC arrays of single MIC or spatial distribution.

5. phonetic control command identification module

Instruction translates into manipulable order by sound identification module.

6. augmented reality (AR) control module

The operation life of augmented reality system is translated into after the input for receiving phonetic order, gesture instruction and limb action Order, changes the video (scene and void including display in augmented reality (AR) system according to the operational order of augmented reality system Intend the state of object) and audio frequency, complete the video (scene and void including display that augmented reality (AR) is controlled by the instruction of people Intend the state of object) and audio frequency.

7. augmented reality (AR) output module

The content of the video of augmented reality (AR) (including the scene that shows and the state of dummy object) be shown to screen, In the display devices such as projector.Audio content in augmented reality is played simultaneously;The augmented reality system of the present invention passes through first Single high-definition camera or multiple photographic head collect image information, then by the gesture instruction in Machine Vision Recognition image. Or the information of the position change that arrives by sensor collections such as gyroscope, acceleration, infrared ray, gravity, then parse receipts The positional information for collecting, identification limbs instruction.Again by voice MIC or MIC array acquisition voice, the voice that collects is parsed, Identification phonetic order.Then augmented reality (AR) control module is receiving the defeated of phonetic order, gesture instruction and limb action The operational order of augmented reality system is translated into after entering, and augmented reality (AR) system is changed according to the operational order of augmented reality system Video (state of scene and dummy object including display) and audio content in system, completes to strengthen by the instruction control of people The video (state of scene and dummy object including display) of real (AR) and audio frequency;.Augmented reality (AR) output module handle The content of the video (state of scene and dummy object including display) of augmented reality (AR) is shown to screen, projector etc. and shows Show in equipment, while playing the audio content in augmented reality;So as to realize the augmented reality (AR) interactive with people method and System so as to realize based on voice, the method and system of the gesture augmented reality (AR) interactive with limb action.

The augmented reality system interactive based on speech gestures and limb action that the present invention is provided, can be widely applied for family The scenes such as projection, projection robot, video display box, it is more convenient to provide the user, more intelligent service.Concrete advantage is such as Under:1st, the method for interaction is very abundant.2nd, such exchange method be people life in very general.3rd, such interaction is also people Live in simple and practical method.Interactive with augmented reality (AR) by such exchange method, it is possible to achieve to go out function The augmented reality system that enriches very much.Its interactive mode and the life style of people be closely so that such interacts with people The Consumer's Experience of augmented reality (AR) method and system is more convenient, more natural and more intelligent.

Although the present invention is disclosed as above with preferred embodiment, so which is not limited to the present invention, any this area skill Art personnel, without departing from the spirit and scope of the present invention, when a little modification and perfect can be made, therefore protection model of the invention Enclose when by being defined that claims are defined.

Claims (4)

1. a kind of augmented reality system interactive based on speech gestures and limb action, it is characterised in that include:
Video acquisition module:Video is gathered by single high-definition camera, or is had by the collection of multiple high-definition cameras The video of the depth of field;
Position changes acquisition module:The information changed by sensor acquisition position;
Gesture and limb action identification module:Video flowing from video acquisition module is parsed, gesture instruction is identified And limb action, and human body position in the scene;Or at the information to the position change from sensor Reason, identifies limb action;
Audio collection module:Audio frequency is gathered by the MIC arrays of single MIC or spatial distribution;
Phonetic control command identification module:The audio frequency for collecting is parsed and is recognized, generated phonetic order;
Augmented reality control module:Change augmented reality system after receiving the input of phonetic order, gesture instruction and limb action Video and audio frequency in system carries out interactive processing;
Augmented reality output module:The content of the video of augmented reality is shown, while playing the audio frequency in augmented reality Content.
2. as claimed in claim 1 based on the augmented reality system that speech gestures and limb action are interactive, it is characterised in that institute State position change acquisition module and position change is gathered by gyroscope, acceleration, infrared ray or gravity sensor Information.
3. as claimed in claim 1 based on the augmented reality system that speech gestures and limb action are interactive, it is characterised in that institute State after augmented reality control module receives phonetic order, gesture instruction and limb action, by changing the scene and void that show Intend the state of object, complete corresponding interactive processing in real time.
4. as claimed in claim 1 based on the augmented reality system that speech gestures and limb action are interactive, it is characterised in that institute State augmented reality output module and video content is shown using LCD screen or projector.
CN201611070594.9A 2016-11-29 2016-11-29 Based on the interactive augmented reality system of speech gestures and limb action CN106502424A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611070594.9A CN106502424A (en) 2016-11-29 2016-11-29 Based on the interactive augmented reality system of speech gestures and limb action

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611070594.9A CN106502424A (en) 2016-11-29 2016-11-29 Based on the interactive augmented reality system of speech gestures and limb action

Publications (1)

Publication Number Publication Date
CN106502424A true CN106502424A (en) 2017-03-15

Family

ID=58328761

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611070594.9A CN106502424A (en) 2016-11-29 2016-11-29 Based on the interactive augmented reality system of speech gestures and limb action

Country Status (1)

Country Link
CN (1) CN106502424A (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108062533A (en) * 2017-12-28 2018-05-22 北京达佳互联信息技术有限公司 Analytic method, system and the mobile terminal of user's limb action
CN109032358A (en) * 2018-08-27 2018-12-18 百度在线网络技术(北京)有限公司 The control method and device of AR interaction dummy model based on gesture identification

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104063039A (en) * 2013-03-18 2014-09-24 朱慧灵 Human-computer interaction method of wearable computer intelligent terminal
CN104615243A (en) * 2015-01-15 2015-05-13 深圳市掌网立体时代视讯技术有限公司 Head-wearable type multi-channel interaction system and multi-channel interaction method
CN205507230U (en) * 2015-12-18 2016-08-24 济南中景电子科技有限公司 Virtual reality glass

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104063039A (en) * 2013-03-18 2014-09-24 朱慧灵 Human-computer interaction method of wearable computer intelligent terminal
CN104615243A (en) * 2015-01-15 2015-05-13 深圳市掌网立体时代视讯技术有限公司 Head-wearable type multi-channel interaction system and multi-channel interaction method
CN205507230U (en) * 2015-12-18 2016-08-24 济南中景电子科技有限公司 Virtual reality glass

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108062533A (en) * 2017-12-28 2018-05-22 北京达佳互联信息技术有限公司 Analytic method, system and the mobile terminal of user's limb action
CN109032358A (en) * 2018-08-27 2018-12-18 百度在线网络技术(北京)有限公司 The control method and device of AR interaction dummy model based on gesture identification

Similar Documents

Publication Publication Date Title
JP2971381B2 (en) Multimedia program editing method and a multi-media program editing system
US20100159430A1 (en) Educational system and method using virtual reality
CN101729792B (en) Image processing apparatus, image processing method, and program
US20120056989A1 (en) Image recognition apparatus, operation determining method and program
JP2006191526A (en) Digital photographing device selecting optimum taking opportune moment
CN101465957B (en) System for implementing remote control interaction in virtual three-dimensional scene
US9384588B2 (en) Video playing method and system based on augmented reality technology and mobile terminal
US8073203B2 (en) Generating effects in a webcam application
US9842433B2 (en) Method, apparatus, and smart wearable device for fusing augmented reality and virtual reality
WO2000077572A1 (en) Method and apparatus for image projection, and apparatus for controlling image projection
CN103404169B (en) Focus on the method and system of microphone array
US20130222427A1 (en) System and method for implementing interactive augmented reality
CN107079141A (en) Image mosaic for 3 D video
EP2583155A2 (en) Natural user input for driving interactive stories
CN103136986B (en) Sign Language Recognition Method and system
KR20120138187A (en) System for constructiing mixed reality using print medium and method therefor
US20120281119A1 (en) Image data creation support device and image data creation support method
EP1877897B1 (en) Interactive multimedia applications device
CN101833232B (en) Visual support and match analysis system for ping-pong match and method for running same
CN101539804A (en) Real time human-machine interaction method and system based on augmented virtual reality and anomalous screen
WO2006023268A3 (en) Portable augmented reality device and method
JP2014127987A (en) Information processing apparatus and recording medium
KR100490726B1 (en) Apparatus and method for video based shooting game
JP5706340B2 (en) Method for controlling media by face detection and hot spot movement
CN102221887B (en) Interactive projection system and method

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20170315