CN105138111A - Single camera based somatosensory interaction method and system - Google Patents
Single camera based somatosensory interaction method and system Download PDFInfo
- Publication number
- CN105138111A CN105138111A CN201510402373.6A CN201510402373A CN105138111A CN 105138111 A CN105138111 A CN 105138111A CN 201510402373 A CN201510402373 A CN 201510402373A CN 105138111 A CN105138111 A CN 105138111A
- Authority
- CN
- China
- Prior art keywords
- model
- single camera
- human body
- profile
- contour outline
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Landscapes
- Processing Or Creating Images (AREA)
- Image Analysis (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
Embodiments of the invention disclose a single camera based somatosensory interaction method and system. The method comprises: based on a single camera, obtaining a human body contour model; based on the obtained human body contour model, obtaining a body skeleton model; based on the body skeleton model, identifying a somatosensory action instruction corresponding to the body skeleton model; and making a response to the somatosensory action instruction. According to the embodiments of the invention, the single camera based somatosensory interaction method and system can enable a user to perform non-contact operation completely, so that the generated effect is very obvious. A player without professional somatosensory hardware of Kinect and the like can experience low-performance somatosensory games; the operation of controlling a computer in a relatively long distance by using an action is more convenient (except character input); and the single camera based somatosensory interaction method and system can be used for action correction in certain situations.
Description
Technical field
The present invention relates to field of computer technology, especially relate to a kind of body sense exchange method based on single camera and system.
Background technology
Use multiple optical camera and infrared pick-up head to obtain image and carry out three-dimensional modeling, then carrying out action recognition, is the cardinal principle of current body sense technology.Its core technology is the image aberration utilizing fixing multiple cameras to obtain, and uses triangulation location, the three-dimensional model of accurate Calculation human body and position.
Use infrared camera seizure human action, has realized and the technology of commercialization has Microsoft Kinect, the body sense hardware technologies such as TrackIR, LeapMotion.
For Microsoft Kinect.Kinect is a kind of 3D body sense video camera, and have the functions such as instant motion capture, image identification, microphone input, speech recognition, it does not need to use any controller just can realize various operation.
There are three cameras, from left to right respectively: infrared transmitter, RGB camera lens, infrared C MOS video camera.Infrared transmitter and CMOS camera can obtain 3D degree of depth image, and the acquisition of RGB camera lens is chromatic image.In addition, also have Array Microphone, can sound be obtained.
The workflow of Kinect, first, by CMOS infrared sensor and LightCoding technology detecting 3D image, generating depth map picture.Depth image, represents with black and white spectroscopy mode: black represents infinite distance, pure white represent infinitely near.The corresponding object of gray zone between black and white is to the physical distance of sensor.Obtain after depth image, telling in the visual field may be the mobile object of human body, and generate skeletal graph, the object of any " greatly " font is all likely tracked as " player " by Kinect.Then use skeleton tracing system, image data is converted to action command.
Skeleton model technology, sets up the three-dimensional coordinate in each joint of human body by treating depth data, can determine the position of the various piece of human body.Under standing state, a skeleton model can record 20 articulation points (can obtain the spatial positional information of a node) altogether, and the pattern of being seated can follow the tracks of 10 articulation points.
The complete skeleton model of maximum two width of this system energy Proactive traceback, identifies the action of 2 people, additionally can follow the trail of 4 people, but not generate detailed skeleton data, only comprise positional information under passive homing pattern.Kinect identification, mainly through following three visual cues identification player identification: the clothing color of the face of player, the height of player and player.
Equipment cost is high, uses the equipment of infrared transmitter and infrared camera can not popularized at present.Current most equipment all can only provide a camera (mobile phone has major-minor two cameras, but in fact only has a camera to work towards same scene), cannot carry out three-dimensional modeling.
Summary of the invention
The object of invention proposes a kind of single camera to make can carry out body sense operation without the computer equipment of body sense hardware device or smart machine.
In order to solve the problem, the present invention proposes a kind of body sense exchange method based on single camera, comprising:
Human body contour outline model is obtained based on single camera;
Health skeleton model is obtained based on the human body contour outline model obtained;
Based on the body sense action command corresponding to health skeleton model described in the identification of health skeleton model;
Respond for described body sense action command.
Described human body contour outline model comprises trunk profile and limbs profile.
The described human body contour outline model based on obtaining obtains health skeleton model and comprises:
From human body contour outline model, isolate trunk profile and limbs profile, from trunk profile and limbs profile, build skeleton trunci model and limbs skeleton model respectively.
Also comprise before described method:
Set up the body sense action command corresponding one by one with health skeleton model, and instruction database is made in organizator emotion.
Also comprise before described acquisition human body contour outline model:
Initialization is carried out to the first human body contour outline model obtained, and camera is calibrated.
Accordingly, present invention also offers a kind of body sense interactive system based on single camera, described system comprises:
Profile acquisition module, for obtaining human body contour outline model based on single camera;
Skeleton model module, for obtaining health skeleton model based on the human body contour outline model obtained;
Identification module, for based on the body sense action command corresponding to health skeleton model described in the identification of health skeleton model;
Processing module, for responding for described body sense action command.
Described human body contour outline model comprises trunk profile and limbs profile.
Described skeleton model module also for isolating trunk profile and limbs profile from human body contour outline model, builds skeleton trunci model and limbs skeleton model respectively from trunk profile and limbs profile.
Described system also comprises body emotion and makes command library, and for setting up the body sense action command corresponding one by one with health skeleton model, and instruction database is made in organizator emotion.
It is characterized in that, described system also comprises: initialization of calibration module, for carrying out initialization to the first human body contour outline model obtained, and calibrates camera.
By implementing the embodiment of the present invention, because the present invention can make the complete touch free operation of user, so bring, the good results are evident.Do not have the player of the professional body sense hardware such as Kinect can experience the somatic sensation television game of low performance; Convenient compared with far distance controlled computing machine (except text event detection) is come with action; The action that can be used in some cases is corrected.
Accompanying drawing explanation
In order to be illustrated more clearly in the embodiment of the present invention or technical scheme of the prior art, be briefly described to the accompanying drawing used required in embodiment or description of the prior art below, apparently, accompanying drawing in the following describes is only some embodiments of the present invention, for those of ordinary skill in the art, under the prerequisite not paying creative work, other accompanying drawing can also be obtained according to these accompanying drawings.
Fig. 1 is the body sense exchange method process flow diagram based on single camera in the embodiment of the present invention;
Fig. 2 is the body sense interactive system structural representation based on single camera in the embodiment of the present invention.
Embodiment
Below in conjunction with the accompanying drawing in the embodiment of the present invention, be clearly and completely described the technical scheme in the embodiment of the present invention, obviously, described embodiment is only the present invention's part embodiment, instead of whole embodiments.Based on the embodiment in the present invention, those of ordinary skill in the art, not making the every other embodiment obtained under creative work prerequisite, belong to the scope of protection of the invention.
Fig. 1 shows the body sense exchange method process flow diagram based on single camera in the embodiment of the present invention, before enforcement flow process of the present invention, need to set up the body sense action command corresponding one by one with health skeleton model, and instruction database is made in organizator emotion, the easy like this corresponding relation found between human body contour outline model and body sense action command, thus ensure instructions parse and carry out response process etc., specifically comprise the steps:
S101, initialization is carried out to the first human body contour outline model obtained, and camera is calibrated;
Have deviation or error owing to obtaining human body contour outline for the first time, need to calibrate for camera, win the error rate of skeleton pattern, this calibration comprises the focusing to image, noise reduction etc.
S102, obtain human body contour outline model based on single camera;
Human body contour outline model comprises trunk profile and limbs profile, and human body whole body profile is made up of main trunk and limbs, by extracting the matching of these Information Assurance skeleton models and body sense action command.
S103, obtain health skeleton model based on the human body contour outline model obtained;
From human body contour outline model, isolate trunk profile and limbs profile, from trunk profile and limbs profile, build skeleton trunci model and limbs skeleton model respectively,
S104, based on the body sense action command corresponding to health skeleton model described in the identification of health skeleton model;
S105, to respond for described body sense action command.
Fig. 2 shows the body sense interactive system structural representation based on single camera in the embodiment of the present invention, and this system comprises:
Profile acquisition module, for obtaining human body contour outline model based on single camera;
Skeleton model module, for obtaining health skeleton model based on the human body contour outline model obtained;
Identification module, for based on the body sense action command corresponding to health skeleton model described in the identification of health skeleton model;
Processing module, for responding for described body sense action command.
Described human body contour outline model comprises trunk profile and limbs profile.
Described skeleton model module also for isolating trunk profile and limbs profile from human body contour outline model, builds skeleton trunci model and limbs skeleton model respectively from trunk profile and limbs profile.
Described system also comprises body emotion and makes command library, and for setting up the body sense action command corresponding one by one with health skeleton model, and instruction database is made in organizator emotion.
Described system also comprises: initialization of calibration module, for carrying out initialization to the first human body contour outline model obtained, and calibrates camera.
To sum up, because the present invention can make the complete touch free operation of user, so bring, the good results are evident.Do not have the player of the professional body sense hardware such as Kinect can experience the somatic sensation television game of low performance; Convenient compared with far distance controlled computing machine (except text event detection) is come with action; The action that can be used in some cases is corrected.
One of ordinary skill in the art will appreciate that all or part of step in the various methods of above-described embodiment is that the hardware that can carry out instruction relevant by program has come, this program can be stored in a computer-readable recording medium, storage medium can comprise: ROM (read-only memory) (ROM, ReadOnlyMemory), random access memory (RAM, RandomAccessMemory), disk or CD etc.
In addition, the body sense exchange method based on single camera provided the embodiment of the present invention above and system are described in detail, apply specific case herein to set forth principle of the present invention and embodiment, the explanation of above embodiment just understands method of the present invention and core concept thereof for helping; Meanwhile, for one of ordinary skill in the art, according to thought of the present invention, all will change in specific embodiments and applications, in sum, this description should not be construed as limitation of the present invention.
Claims (10)
1., based on a body sense exchange method for single camera, it is characterized in that, comprising:
Human body contour outline model is obtained based on single camera;
Health skeleton model is obtained based on the human body contour outline model obtained;
Based on the body sense action command corresponding to health skeleton model described in the identification of health skeleton model;
Respond for described body sense action command.
2., as claimed in claim 1 based on the body sense exchange method of single camera, it is characterized in that, described human body contour outline model comprises trunk profile and limbs profile.
3. as claimed in claim 2 based on the body sense exchange method of single camera, it is characterized in that, the described human body contour outline model based on obtaining obtains health skeleton model and comprises:
From human body contour outline model, isolate trunk profile and limbs profile, from trunk profile and limbs profile, build skeleton trunci model and limbs skeleton model respectively.
4., as claimed in claim 3 based on the body sense exchange method of single camera, it is characterized in that, also comprise before described method:
Set up the body sense action command corresponding one by one with health skeleton model, and instruction database is made in organizator emotion.
5. the body sense exchange method based on single camera as described in any one of Claims 1-4, is characterized in that, also comprises before described acquisition human body contour outline model:
Initialization is carried out to the first human body contour outline model obtained, and camera is calibrated.
6., based on a body sense interactive system for single camera, it is characterized in that, described system comprises:
Profile acquisition module, for obtaining human body contour outline model based on single camera;
Skeleton model module, for obtaining health skeleton model based on the human body contour outline model obtained;
Identification module, for based on the body sense action command corresponding to health skeleton model described in the identification of health skeleton model;
Processing module, for responding for described body sense action command.
7., as claimed in claim 6 based on the body sense interactive system of single camera, it is characterized in that, described human body contour outline model comprises trunk profile and limbs profile.
8. as claimed in claim 7 based on the body sense interactive system of single camera, it is characterized in that, described skeleton model module also for isolating trunk profile and limbs profile from human body contour outline model, builds skeleton trunci model and limbs skeleton model respectively from trunk profile and limbs profile.
9. as claimed in claim 8 based on the body sense interactive system of single camera, it is characterized in that, described system also comprises body emotion and makes command library, and for setting up the body sense action command corresponding one by one with health skeleton model, and instruction database is made in organizator emotion.
10. the body sense interactive system based on single camera as described in any one of claim 6 to 9, it is characterized in that, described system also comprises: initialization of calibration module, for carrying out initialization to the first human body contour outline model obtained, and calibrates camera.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510402373.6A CN105138111A (en) | 2015-07-09 | 2015-07-09 | Single camera based somatosensory interaction method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510402373.6A CN105138111A (en) | 2015-07-09 | 2015-07-09 | Single camera based somatosensory interaction method and system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN105138111A true CN105138111A (en) | 2015-12-09 |
Family
ID=54723482
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510402373.6A Pending CN105138111A (en) | 2015-07-09 | 2015-07-09 | Single camera based somatosensory interaction method and system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105138111A (en) |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105894533A (en) * | 2015-12-31 | 2016-08-24 | 乐视移动智能信息技术（北京）有限公司 | Method and system for realizing body motion-sensing control based on intelligent device and intelligent device |
| CN106582005A (en) * | 2016-11-14 | 2017-04-26 | 深圳市豆娱科技有限公司 | Data synchronous interaction method and device in virtual games |
| CN106709440A (en) * | 2016-12-16 | 2017-05-24 | 电子科技大学 | User identification method of bank automatic teller machine (ATM) |
| CN106730815A (en) * | 2016-12-09 | 2017-05-31 | 福建星网视易信息系统有限公司 | The body-sensing interactive approach and system of a kind of easy realization |
| CN108983956A (en) * | 2017-11-30 | 2018-12-11 | 成都通甲优博科技有限责任公司 | Body feeling interaction method and device |
| CN109558835A (en) * | 2018-11-28 | 2019-04-02 | 科大智能机器人技术有限公司 | A kind of control method and its system of the automatic tractor based on human bioequivalence |
| CN110427100A (en) * | 2019-07-03 | 2019-11-08 | 武汉子序科技股份有限公司 | A kind of movement posture capture system based on depth camera |
| CN110549975A (en) * | 2018-05-31 | 2019-12-10 | 南京华捷艾米软件科技有限公司 | 3D Sensor-based automobile radar system and control method thereof |
| CN111290577A (en) * | 2020-01-22 | 2020-06-16 | 北京明略软件系统有限公司 | Non-contact input method and device |
| CN113058261A (en) * | 2021-04-22 | 2021-07-02 | 杭州当贝网络科技有限公司 | Somatosensory action recognition method and system based on reality scene and game scene |
| CN113058258A (en) * | 2021-04-22 | 2021-07-02 | 杭州当贝网络科技有限公司 | Method, system and storage medium for recognizing expected somatosensory action based on player game |
| CN113058260A (en) * | 2021-04-22 | 2021-07-02 | 杭州当贝网络科技有限公司 | Method, system and storage medium for recognizing motion of body based on player portrait |
| WO2024088073A1 (en) * | 2022-10-28 | 2024-05-02 | 华为技术有限公司 | Motion sensing interaction method, electronic device, system, and readable storage medium |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011076057A1 (en) * | 2009-12-22 | 2011-06-30 | 聂清永 | Input apparatus based on body feeling and input method thereof |
| CN201893831U (en) * | 2010-11-11 | 2011-07-06 | 上海华勤通讯技术有限公司 | Mobile phone with X-ray function |
| CN103198519A (en) * | 2013-03-15 | 2013-07-10 | 苏州跨界软件科技有限公司 | Virtual character photographic system and virtual character photographic method |
| CN104598012A (en) * | 2013-10-30 | 2015-05-06 | 中国艺术科技研究所 | Interactive advertising equipment and working method thereof |
-
2015
- 2015-07-09 CN CN201510402373.6A patent/CN105138111A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2011076057A1 (en) * | 2009-12-22 | 2011-06-30 | 聂清永 | Input apparatus based on body feeling and input method thereof |
| CN201893831U (en) * | 2010-11-11 | 2011-07-06 | 上海华勤通讯技术有限公司 | Mobile phone with X-ray function |
| CN103198519A (en) * | 2013-03-15 | 2013-07-10 | 苏州跨界软件科技有限公司 | Virtual character photographic system and virtual character photographic method |
| CN104598012A (en) * | 2013-10-30 | 2015-05-06 | 中国艺术科技研究所 | Interactive advertising equipment and working method thereof |
Cited By (19)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105894533A (en) * | 2015-12-31 | 2016-08-24 | 乐视移动智能信息技术（北京）有限公司 | Method and system for realizing body motion-sensing control based on intelligent device and intelligent device |
| WO2017113674A1 (en) * | 2015-12-31 | 2017-07-06 | 乐视控股（北京）有限公司 | Method and system for realizing motion-sensing control based on intelligent device, and intelligent device |
| CN106582005A (en) * | 2016-11-14 | 2017-04-26 | 深圳市豆娱科技有限公司 | Data synchronous interaction method and device in virtual games |
| CN106730815A (en) * | 2016-12-09 | 2017-05-31 | 福建星网视易信息系统有限公司 | The body-sensing interactive approach and system of a kind of easy realization |
| CN106709440A (en) * | 2016-12-16 | 2017-05-24 | 电子科技大学 | User identification method of bank automatic teller machine (ATM) |
| CN108983956A (en) * | 2017-11-30 | 2018-12-11 | 成都通甲优博科技有限责任公司 | Body feeling interaction method and device |
| CN108983956B (en) * | 2017-11-30 | 2021-07-06 | 成都通甲优博科技有限责任公司 | Somatosensory interaction method and device |
| CN110549975A (en) * | 2018-05-31 | 2019-12-10 | 南京华捷艾米软件科技有限公司 | 3D Sensor-based automobile radar system and control method thereof |
| CN109558835A (en) * | 2018-11-28 | 2019-04-02 | 科大智能机器人技术有限公司 | A kind of control method and its system of the automatic tractor based on human bioequivalence |
| CN110427100A (en) * | 2019-07-03 | 2019-11-08 | 武汉子序科技股份有限公司 | A kind of movement posture capture system based on depth camera |
| CN111290577A (en) * | 2020-01-22 | 2020-06-16 | 北京明略软件系统有限公司 | Non-contact input method and device |
| CN111290577B (en) * | 2020-01-22 | 2024-03-22 | 北京明略软件系统有限公司 | Non-contact input method and device |
| CN113058261A (en) * | 2021-04-22 | 2021-07-02 | 杭州当贝网络科技有限公司 | Somatosensory action recognition method and system based on reality scene and game scene |
| CN113058258A (en) * | 2021-04-22 | 2021-07-02 | 杭州当贝网络科技有限公司 | Method, system and storage medium for recognizing expected somatosensory action based on player game |
| CN113058260A (en) * | 2021-04-22 | 2021-07-02 | 杭州当贝网络科技有限公司 | Method, system and storage medium for recognizing motion of body based on player portrait |
| CN113058260B (en) * | 2021-04-22 | 2024-02-02 | 杭州当贝网络科技有限公司 | Method, system and storage medium for identifying motion of body feeling based on player image |
| CN113058258B (en) * | 2021-04-22 | 2024-04-19 | 杭州当贝网络科技有限公司 | Method, system and storage medium for identifying motion based on expected motion of player game |
| CN113058261B (en) * | 2021-04-22 | 2024-04-19 | 杭州当贝网络科技有限公司 | Somatosensory motion recognition method and system based on reality scene and game scene |
| WO2024088073A1 (en) * | 2022-10-28 | 2024-05-02 | 华为技术有限公司 | Motion sensing interaction method, electronic device, system, and readable storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105138111A (en) | Single camera based somatosensory interaction method and system | |
| TWI751161B (en) | Terminal equipment, smart phone, authentication method and system based on face recognition | |
| CN102193626B (en) | Gesture recognition apparatus, and method for controlling gesture recognition apparatus | |
| Ma et al. | Kinect sensor-based long-distance hand gesture recognition and fingertip detection with depth information | |
| US20180268207A1 (en) | Method for automatic facial impression transformation, recording medium and device for performing the method | |
| Fossati et al. | Consumer depth cameras for computer vision: research topics and applications | |
| JP6456347B2 (en) | INSITU generation of plane-specific feature targets | |
| CN111402290A (en) | Action restoration method and device based on skeleton key points | |
| CN104899563A (en) | Two-dimensional face key feature point positioning method and system | |
| CN113706699A (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| CN110211222A (en) | A kind of AR immersion tourism guide method, device, storage medium and terminal device | |
| CN113689503B (en) | Target object posture detection method, device, equipment and storage medium | |
| CN109116981A (en) | A kind of mixed reality interactive system of passive touch feedback | |
| US20230154115A1 (en) | Method and apparatus for providing multi-user-involved augmented reality content for diorama application | |
| CN109859857A (en) | Mask method, device and the computer readable storage medium of identity information | |
| CN112669422A (en) | Simulated 3D digital human generation method and device, electronic equipment and storage medium | |
| KR20190119212A (en) | System for performing virtual fitting using artificial neural network, method thereof and computer recordable medium storing program to perform the method | |
| US11403768B2 (en) | Method and system for motion prediction | |
| Kowalski et al. | Holoface: Augmenting human-to-human interactions on hololens | |
| US10713833B2 (en) | Method and device for controlling 3D character using user's facial expressions and hand gestures | |
| Cao et al. | Leveraging convolutional pose machines for fast and accurate head pose estimation | |
| WO2024045454A1 (en) | Target identification method, storage medium and device | |
| WO2017113674A1 (en) | Method and system for realizing motion-sensing control based on intelligent device, and intelligent device | |
| WO2023142555A1 (en) | Data processing method and apparatus, computer device, storage medium, and computer program product | |
| JP2023089947A (en) | Feature tracking system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20151209 |