CN109656355A - A kind of exchange method and device of mobile phone and other display equipment - Google Patents
A kind of exchange method and device of mobile phone and other display equipment Download PDFInfo
- Publication number
- CN109656355A CN109656355A CN201811245646.0A CN201811245646A CN109656355A CN 109656355 A CN109656355 A CN 109656355A CN 201811245646 A CN201811245646 A CN 201811245646A CN 109656355 A CN109656355 A CN 109656355A
- Authority
- CN
- China
- Prior art keywords
- mobile phone
- human body
- artis
- display equipment
- depth
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04M—TELEPHONIC COMMUNICATION
- H04M1/00—Substation equipment, e.g. for use by subscribers
- H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
- H04M1/724—User interfaces specially adapted for cordless or mobile telephones
- H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
- H04M1/72409—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories
- H04M1/72415—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories for remote control of appliances
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
- G06F2203/01—Indexing scheme relating to G06F3/01
- G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Health & Medical Sciences (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Multimedia (AREA)
- Computer Networks & Wireless Communication (AREA)
- Signal Processing (AREA)
- Processing Or Creating Images (AREA)
Abstract
This disclosure relates to the exchange method and device of a kind of mobile phone and other display equipment, this method comprises the following steps: S100: the depth map of human body is obtained by the depth camera in mobile phone；S200: human skeleton is positioned and is tracked using the depth map of the human body；S300: the artis that the hand of human body is chosen from the human skeleton is tracked, and converts the control instruction of mobile phone for the track that the artis of hand moves to control mobile phone；S400: the control instruction is transmitted to the other display equipment communicated with by mobile phone, realizes control of the mobile phone to other display equipment.The method will show the function to be realized of human body attitude by other display equipment, and the human-computer interaction of mobile phone is combined with other display equipment, so that the effect of human-computer interaction is more intuitive and three-dimensional.
Description
Technical field
The disclosure belongs to image procossing, natural interaction and technical field of integrated circuits, in particular to a kind of mobile phone and other
Show the exchange method and device of equipment.
Background technique
The man-machine interaction mode of natural harmony is dreamboat of the mankind to Manipulation of the machine, causes a machine to understand people in nature
The order that state is transmitted.Depth information is obtained using image processing techniques to carry out the real-time identification of 3-D image and act to catch
It catches, enables a person to interact and be possibly realized with the natural ways such as expression, gesture, body-sensing movement and terminal.Depth perception technology is
The core technology of natural human-machine interaction has in fields such as machine vision, intelligent monitoring, three-dimensional reconstruction, body feeling interaction, 3D printings
Broad application prospect is gradually extended to other intelligent terminal, including smart television, smart phone, PC/ from game machine peripheral hardware
Tablet computer, intelligent appliance etc. bring control mode and completely new man-machine interaction experience as " science fiction " for user.
Carrying out information exchange to mobile phone using limb action has convenient, practical feature.And the active based on structure light
Depth perception makes human-computer interaction more stable, smooth.The human-computer interaction of mobile phone is combined with display other display equipment,
So that the effect of human-computer interaction is more intuitive and three-dimensional.
Summary of the invention
In consideration of it, including the following steps: present disclose provides the exchange method of a kind of mobile phone and other display equipment
S100: the depth map of human body is obtained by the depth camera in mobile phone；
S200: human skeleton is positioned and is tracked using the depth map of the human body；
S300: the artis that the hand of human body is chosen from the human skeleton is tracked, by the artis of hand
The track of movement is converted into the control instruction of mobile phone to control mobile phone；
S400: the control instruction is transmitted to the other display equipment communicated with by mobile phone, realizes mobile phone to other
Show the control of equipment.
The disclosure additionally provides the interactive device of a kind of mobile phone and other display equipment, comprising:
Three dimensional depth sension unit: for obtaining the depth map of human body by the depth camera in mobile phone；
Skeleton tracing unit: human skeleton is positioned and is tracked using the depth map of the human body；
Human body gesture identification unit: the artis that the hand of human body is chosen from the human skeleton is tracked, will
The track of the artis movement of hand is converted into the control instruction of mobile phone to control mobile phone；
Other display equipment display unit: the control instruction is transmitted to the other display equipment communicated with by mobile phone,
Realize control of the mobile phone to other display equipment.
Through the above technical solutions, being incuded by posture of the depth camera in mobile phone to human body, to the bone of human body
Bone is tracked, to carry out human-computer interaction, the human action for then sensing mobile phone is converted into the control instruction of mobile phone, so
After execute corresponding function, mobile phone is communicated by communication protocol with other display equipment, finally by liquid crystal display show people
The function to be realized of body posture.
Detailed description of the invention
Fig. 1 is a kind of process of the exchange method of mobile phone and other display equipment provided in an embodiment of the present disclosure
Schematic diagram；
Fig. 2 is that the interactive device structure of a kind of mobile phone and other display equipment provided in an embodiment of the present disclosure is shown
It is intended to.
Specific embodiment
The present invention is further described in detail with reference to the accompanying drawing.
In one embodiment, referring to Fig. 1, it discloses the exchange methods of a kind of mobile phone and other display equipment, including
Following steps:
S100: the depth map of human body is obtained by the depth camera in mobile phone；
S200: human skeleton is positioned and is tracked using the depth map of the human body；
S300: the artis that the hand of human body is chosen from the human skeleton is tracked, by the artis of hand
The track of movement is converted into the control instruction of mobile phone to control mobile phone；
S400: the control instruction is transmitted to the other display equipment communicated with by mobile phone, realizes mobile phone to other
Show the control of equipment.
In another embodiment, step S100 further comprises:
S101: mobile phone is opened, mobile phone is allowed to be in normal working condition；
S102: the infrared speckle encoding that people station includes in the depth camera in the position from mobile phone 30-200cm, mobile phone
The projector projects infrared speckle encoding pattern to human body, while the infrared receiver camera that the depth camera in mobile phone includes is opened
Begin to receive the infrared speckle encoding pattern after human body reflection；
S103: by the infrared speckle encoding pattern received give chip for cell phone carry out image pretreatment, image enhancement,
The depth map of human body is obtained after Block- matching depth calculation step.
In another embodiment, step S200 further comprises:
S201: using the depth map of the human body, according to the method for deep learning to the characteristic value of the artis of human body into
Row extracts；
S202: the characteristic value for the artis extracted is trained, according to the characteristic value obtained after training to human body
Bone carries out accurate positioning and tracking.
Wherein, what the deep learning method was taken is a kind of deep learning method based on convolutional neural networks, choosing
The number of plies of neural network is 5 layers, and 5 layers of convolution kernel number is respectively 4,12,36,108,324, and such neural network has
Conducive to the accuracy and speed for improving characteristics extraction.The training of characteristic value is also the method using deep learning, builds nerve net
Network model.The number of plies of the neural network of selection is 5 layers, and 5 layers of convolution kernel number is respectively 4,12,36,108,324, such
Neural network is conducive to improve the accuracy and speed of training.
In another embodiment, the artis of the human body in step S200 is about 28, mainly includes brain joint
Point, both shoulders artis, swivel of hand artis, foot joint artis.
In another embodiment, the track that the artis by hand in step S300 moves is converted into the control of mobile phone
Instruction specifically:
Convert the track that the artis of hand moves from top to bottom to the page-turning instruction from top to bottom of mobile phone；And/or
Convert the track that the artis of hand moves from top to bottom to the rolling sliding rail instruction of mobile phone；And/or
Convert the track that the artis of hand moves from left to right to the determine instruction of mobile phone；And/or
Convert the artis of hand to from the right track toward left movement the cancellation instruction of mobile phone.
In another embodiment, wherein the communication in step S400 is specially interchanger or I/0 interface.
In another embodiment, wherein the other display equipment is the video screen of liquid crystal or the ppt that teaching uses
Other display equipment, when the other display equipment is the video screen of liquid crystal, mobile phone is equivalent to a remote manipulator；
When the other display equipment is the ppt other display equipment that teaching uses, mobile phone is equivalent to a player.
In another embodiment, referring to fig. 2, it discloses the interactive device of a kind of mobile phone and other display equipment, packets
It includes:
Three dimensional depth sension unit: for obtaining the depth map of human body by the depth camera in mobile phone；
Skeleton tracing unit: human skeleton is positioned and is tracked using the depth map of the human body；
Human body gesture identification unit: the artis that the hand of human body is chosen from the human skeleton is tracked, will
The track of the artis movement of hand is converted into the control instruction of mobile phone to control mobile phone；
Other display equipment display unit: the control instruction is transmitted to the other display equipment communicated with by mobile phone,
Realize control of the mobile phone to other display equipment.
In another embodiment, three dimensional depth sension unit further comprises:
Mobile phone initialization unit: mobile phone is opened, mobile phone is allowed to be in normal working condition；
Infrared speckle encoding pattern acquiring unit: depth camera of the people station in the position from mobile phone 30-200cm, mobile phone
In include the infrared speckle encoding projector project infrared speckle encoding pattern to human body, while the depth camera packet in mobile phone
The infrared receiver camera contained starts to receive the infrared speckle encoding pattern after human body reflection；
The depth map acquiring unit of human body: it gives the infrared speckle encoding pattern received to chip for cell phone and carries out image
It pre-processes, image enhancement, obtains the depth map of human body after Block- matching depth calculation step.
In another embodiment, skeleton tracing unit further comprises:
Characteristics extraction unit: using the depth map of the human body, according to the method for deep learning to the artis of human body
Characteristic value extract；
Characteristic value training unit: the characteristic value for the artis extracted is trained, according to the feature obtained after training
Value carries out accurate positioning and tracking to human skeleton.
It should be noted that " other display equipment " in Figure of description with display screen as an example.It can manage
Solution, other display equipment can be the various appropriate equipment with display screen, for example, monitor, the smart phone with display screen/
Smart television/intelligence projection etc..
Although embodiment of the present invention is described in conjunction with attached drawing above, the invention is not limited to above-mentioned
Specific embodiments and applications field, above-mentioned specific embodiment are only schematical, directiveness, rather than restricted
's.Those skilled in the art are under the enlightenment of this specification and in the range for not departing from the claims in the present invention and being protected
In the case where, a variety of forms can also be made, these belong to the column of protection of the invention.
Claims (10)
1. the exchange method of a kind of mobile phone and other display equipment, includes the following steps:
S100: the depth map of human body is obtained by the depth camera in mobile phone；
S200: human skeleton is positioned and is tracked using the depth map of the human body；
S300: the artis that the hand of human body is chosen from the human skeleton is tracked, and the artis of hand is moved
Track be converted into the control instruction of mobile phone to control mobile phone；
S400: the control instruction is transmitted to the other display equipment communicated with by mobile phone, realizes mobile phone to other displays
The control of equipment.
2. step S100 further comprises according to the method described in claim 1, preferred:
S101: mobile phone is opened, mobile phone is allowed to be in normal working condition；
S102: the infrared speckle encoding projection that people station includes in the depth camera in the position from mobile phone 30-200cm, mobile phone
Device projects infrared speckle encoding pattern to human body, while the infrared receiver camera that the depth camera in mobile phone includes starts to connect
Infrared speckle encoding pattern after receiving human body reflection；
S103: the infrared speckle encoding pattern received is given to the pretreatment that chip for cell phone carries out image, image enhancement, block
With obtaining the depth map of human body after depth calculation step.
3. according to the method described in claim 1, step S200 further comprises:
S201: it using the depth map of the human body, is mentioned according to characteristic value of the method for deep learning to the artis of human body
It takes；
S202: the characteristic value for the artis extracted is trained, according to the characteristic value obtained after training to human skeleton
Carry out accurate positioning and tracking.
4. according to the method described in claim 1, the artis of the human body in step S200 is about 28, mainly including brain
Artis, both shoulders artis, swivel of hand artis, foot joint artis.
5. according to the method described in claim 1, the track that the artis by hand in step S300 moves is converted into mobile phone
Control instruction specifically:
Convert the track that the artis of hand moves from top to bottom to the page-turning instruction from top to bottom of mobile phone；And/or
Convert the track that the artis of hand moves from top to bottom to the rolling sliding rail instruction of mobile phone；And/or
Convert the track that the artis of hand moves from left to right to the determine instruction of mobile phone；And/or
Convert the artis of hand to from the right track toward left movement the cancellation instruction of mobile phone.
6. according to the method described in claim 1, wherein, the communication in step S400 is specially interchanger or I/O interface.
7. according to the method described in claim 1, wherein, the other display equipment is that the video screen of liquid crystal or teaching use
Ppt other display equipment, when the other display equipment is the video screen of liquid crystal, mobile phone is equivalent to one long-range distant
Control device；When the other display equipment is the ppt other display equipment that teaching uses, mobile phone is equivalent to a player.
8. the interactive device of a kind of mobile phone and other display equipment, comprising:
Three dimensional depth sension unit: for obtaining the depth map of human body by the depth camera in mobile phone；
Skeleton tracing unit: human skeleton is positioned and is tracked using the depth map of the human body；
Human body gesture identification unit: the artis that the hand of human body is chosen from the human skeleton is tracked, by hand
Artis movement track be converted into the control instruction of mobile phone to control mobile phone；
Other display equipment display unit: the control instruction is transmitted to the other display equipment communicated with by mobile phone, is realized
Control of the mobile phone to other display equipment.
9. device according to claim 8, three dimensional depth sension unit further comprises:
Mobile phone initialization unit: mobile phone is opened, mobile phone is allowed to be in normal working condition；
Infrared speckle encoding pattern acquiring unit: people wraps at station in the depth camera in the position from mobile phone 30-200cm, mobile phone
The infrared speckle encoding projector contained projects infrared speckle encoding pattern to human body, while the depth camera in mobile phone includes
Infrared receiver camera starts to receive the infrared speckle encoding pattern after human body reflection；
The depth map acquiring unit of human body: the infrared speckle encoding pattern received is given to the pre- place that chip for cell phone carries out image
It manages, image enhancement, obtains the depth map of human body after Block- matching depth calculation step.
10. device according to claim 8, skeleton tracing unit further comprises:
Characteristics extraction unit: using the depth map of the human body, according to the method for deep learning to the spy of the artis of human body
Value indicative extracts；
Characteristic value training unit: the characteristic value for the artis extracted is trained, according to the characteristic value pair obtained after training
Human skeleton carries out accurate positioning and tracking.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811245646.0A CN109656355A (en) | 2018-10-23 | 2018-10-23 | A kind of exchange method and device of mobile phone and other display equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811245646.0A CN109656355A (en) | 2018-10-23 | 2018-10-23 | A kind of exchange method and device of mobile phone and other display equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109656355A true CN109656355A (en) | 2019-04-19 |
Family
ID=66110438
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811245646.0A Pending CN109656355A (en) | 2018-10-23 | 2018-10-23 | A kind of exchange method and device of mobile phone and other display equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109656355A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021147989A1 (en) * | 2020-01-22 | 2021-07-29 | 京东方科技集团股份有限公司 | Human-machine interaction method and apparatus, and motion sensing game device |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105807926A (en) * | 2016-03-08 | 2016-07-27 | 中山大学 | Unmanned aerial vehicle man-machine interaction method based on three-dimensional continuous gesture recognition |
| CN106325509A (en) * | 2016-08-19 | 2017-01-11 | 北京暴风魔镜科技有限公司 | Three-dimensional gesture recognition method and system |
| CN107080940A (en) * | 2017-03-07 | 2017-08-22 | 中国农业大学 | Body feeling interaction conversion method and device based on depth camera Kinect |
| CN107253192A (en) * | 2017-05-24 | 2017-10-17 | 湖北众与和智能装备科技有限公司 | It is a kind of based on Kinect without demarcation human-computer interactive control system and method |
-
2018
- 2018-10-23 CN CN201811245646.0A patent/CN109656355A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105807926A (en) * | 2016-03-08 | 2016-07-27 | 中山大学 | Unmanned aerial vehicle man-machine interaction method based on three-dimensional continuous gesture recognition |
| CN106325509A (en) * | 2016-08-19 | 2017-01-11 | 北京暴风魔镜科技有限公司 | Three-dimensional gesture recognition method and system |
| CN107080940A (en) * | 2017-03-07 | 2017-08-22 | 中国农业大学 | Body feeling interaction conversion method and device based on depth camera Kinect |
| CN107253192A (en) * | 2017-05-24 | 2017-10-17 | 湖北众与和智能装备科技有限公司 | It is a kind of based on Kinect without demarcation human-computer interactive control system and method |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2021147989A1 (en) * | 2020-01-22 | 2021-07-29 | 京东方科技集团股份有限公司 | Human-machine interaction method and apparatus, and motion sensing game device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109003301B (en) | Human body posture estimation method based on OpenPose and Kinect and rehabilitation training system | |
| Yang et al. | Gesture interaction in virtual reality | |
| JP7061694B2 (en) | Image processing methods and equipment, imaging equipment, and storage media | |
| CN105389539B (en) | A kind of three-dimension gesture Attitude estimation method and system based on depth data | |
| CN102779000B (en) | User interaction system and method | |
| CN102681651B (en) | A kind of user interactive system and method | |
| CN207397530U (en) | For the immersion multi-person synergy training device of the virtual implementing helmet formula of Substation Training | |
| CN103544724A (en) | System and method for realizing fictional cartoon character on mobile intelligent terminal by augmented reality and card recognition technology | |
| CN104407701A (en) | Individual-oriented clustering virtual reality interactive system | |
| CN107948499A (en) | A kind of image capturing method and mobile terminal | |
| CN109218648A (en) | A kind of display control method and terminal device | |
| CN107241579A (en) | A kind of utilization AR realizes the method and apparatus of Telemedicine Consultation | |
| CN106648068A (en) | Method for recognizing three-dimensional dynamic gesture by two hands | |
| CN107943282A (en) | A kind of man-machine interactive system and method based on augmented reality and wearable device | |
| CN113221726A (en) | Hand posture estimation method and system based on visual and inertial information fusion | |
| CN109839827B (en) | Gesture recognition intelligent household control system based on full-space position information | |
| CN107961531B (en) | Virtual social contact system based on motion capture clothing and working method thereof | |
| CN109395375A (en) | A kind of 3d gaming method of interface interacted based on augmented reality and movement | |
| CN208239980U (en) | A kind of virtual interacting device based on data glove | |
| CN105892627A (en) | Virtual augmented reality method and apparatus, and eyeglass or helmet using same | |
| CN109656355A (en) | A kind of exchange method and device of mobile phone and other display equipment | |
| Chen et al. | Lisee: A headphone that provides all-day assistance for blind and low-vision users to reach surrounding objects | |
| CN111312363B (en) | Double-hand coordination enhancement system based on virtual reality | |
| CN113703583A (en) | Multi-mode cross fusion virtual image fusion system, method and device | |
| CN106599811A (en) | Facial expression tracking method of VR heat-mounted display |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20190419 |