CN112363629B - Novel non-contact man-machine interaction method and system - Google Patents
Novel non-contact man-machine interaction method and system Download PDFInfo
- Publication number
- CN112363629B CN112363629B CN202011395956.8A CN202011395956A CN112363629B CN 112363629 B CN112363629 B CN 112363629B CN 202011395956 A CN202011395956 A CN 202011395956A CN 112363629 B CN112363629 B CN 112363629B
- Authority
- CN
- China
- Prior art keywords
- point
- display screen
- computer interaction
- target
- depth camera
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
- 230000003993 interaction Effects 0.000 title claims abstract description 40
- 238000000034 method Methods 0.000 title claims abstract description 34
- 230000009471 action Effects 0.000 claims abstract description 8
- 238000001514 detection method Methods 0.000 claims description 17
- 238000013528 artificial neural network Methods 0.000 claims description 15
- 238000013135 deep learning Methods 0.000 claims description 9
- 230000007246 mechanism Effects 0.000 claims description 4
- 230000001960 triggered effect Effects 0.000 description 8
- 238000010586 diagram Methods 0.000 description 6
- 230000002452 interceptive effect Effects 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 210000004556 brain Anatomy 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 230000003190 augmentative effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- DMSMPAJRVJJAGA-UHFFFAOYSA-N benzo[d]isothiazol-3-one Chemical compound C1=CC=C2C(=O)NSC2=C1 DMSMPAJRVJJAGA-UHFFFAOYSA-N 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 208000013409 limited attention Diseases 0.000 description 1
- 238000005259 measurement Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012545 processing Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- General Health & Medical Sciences (AREA)
- Computing Systems (AREA)
- Computational Linguistics (AREA)
- Data Mining & Analysis (AREA)
- Evolutionary Computation (AREA)
- Biomedical Technology (AREA)
- Molecular Biology (AREA)
- Biophysics (AREA)
- Artificial Intelligence (AREA)
- Life Sciences & Earth Sciences (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention is suitable for the technical field of human-computer interaction, and provides a novel non-contact human-computer interaction method and a system, wherein the method comprises the following steps: s100, automatically detecting three vertexes A, B, C and a target point F of a display screen in a two-dimensional plane image of the depth camera; s200, combining the internal reference of the depth camera, and converting the plane pixel coordinates of the top point A, B, C and the target point F into three-dimensional coordinates under a depth camera coordinate system; s300, calculating a three-dimensional coordinate of a projection point F' of the target point F on the display screen in a depth camera coordinate system; s400, calculating the plane pixel coordinate of the projection point F' on the display screen; and S500, identifying the action of the target point F, and calling a related system mouse interface to trigger a mouse event. The invention solves the problem of controlling the screen content by using detectable target objects such as finger tips and the like in a mouse-free environment, does not need any calibration, and has the advantages of small calculated amount and simple hardware equipment.
Description
Technical Field
The invention belongs to the technical field of human-computer interaction, and particularly relates to a novel non-contact human-computer interaction method and system.
Background
At present, a non-contact human-computer interaction technology is widely applied to the fields of interactive games, interactive museums, interactive tourist halls, VR/AR (virtual reality/augmented reality) and the like, a user can perform interactive operation with an electronic product by using various non-contact gestures in front of display areas such as a computer display screen and an intelligent television, for example, the user can adjust the volume by using the gestures, and the human-computer interaction technology without mouse operation has great market value and economic value.
In patent application publication No. CN102841679A entitled non-contact human-computer interaction method and apparatus, a non-contact human-computer interaction method and apparatus are disclosed, which mainly use the camera calibration principle, in which the position information and direction information of the camera need to be acquired to calculate the calibration result, and a gravity sensing module and a sliding resistance measurement module need to be introduced, so that the hardware design is complicated.
With the emergence of depth cameras and the development of deep learning techniques in the field of computer vision, more and more human-computer interaction systems based on the depth cameras are provided. The depth camera is different from a general camera in that depth information of a photographed object, that is, three-dimensional position and size information can be obtained in addition to planar image information, so that the entire computing system can obtain three-dimensional stereoscopic data of an environment and an object. The depth camera has the capability of three-dimensional sensing and identification, and can complete the application of three-dimensional modeling and the like through further deepening treatment.
Disclosure of Invention
The embodiment of the invention aims to provide a novel non-contact human-computer interaction method, and aims to solve the problems of complex calculation process and complex hardware in the prior art of non-contact human-computer interaction by utilizing a camera calibration principle.
The embodiment of the invention is realized in such a way that a novel non-contact man-machine interaction method is provided, which comprises the following steps:
s100, automatically detecting three vertexes A, B, C and a target point F of a display screen by using a deep learning system in a two-dimensional plane image of a depth camera to obtain plane pixel coordinates of each point;
s200, combining the internal reference of the depth camera, and converting the plane pixel coordinates of the top point A, B, C and the target point F into three-dimensional coordinates under a depth camera coordinate system;
s300, calculating a three-dimensional coordinate of a projection point F' of the target point F on the display screen in a depth camera coordinate system;
s400, calculating the plane pixel coordinate of the projection point F' on the display screen;
and S500, identifying the action of the target point F, and calling a related system mouse interface to trigger a mouse event.
Further, the step S100 includes the following sub-steps:
s110, automatically detecting a vertex A, B, C of the display screen by using a target detection algorithm based on deep learning to obtain plane pixel coordinates of each point;
s120, establishing a two-stage target detection deep neural network structure, and automatically detecting a target point F, wherein the two-stage target detection deep neural network structure comprises the following steps:
s121, establishing a target object detection depth neural network for detecting a target object in the two-dimensional plane image, expanding the detected target object area and positioning the target object according to the expanded area;
and S122, establishing a target point detection depth neural network for detecting a target point of the target object and positioning the target point to obtain a plane pixel coordinate of the target point F.
Further, the step S120 further includes: and establishing a neural network with a dual-channel attention mechanism for improving the positioning precision of the detected target point.
Further, the step S300 includes:
from the vertex A, B, C, a normal vector n of the screen plane passing through the vertex a is calculated, and from the line segment FF ' parallel to the normal vector n and the line segment AF ' perpendicular to the normal vector n, the three-dimensional coordinates of the projected point F ' are calculated.
Further, the step S400 includes:
and calculating the ratio of the abscissa u to the ordinate v in the plane pixel coordinate of the projection point F ', and then combining the screen resolution of the display screen to obtain the plane pixel coordinate of the projection point F'.
Further, the step S400 includes:
respectively calculating the distance D from the projection point F' to the line segment ABFABAnd the distance D from the projection point F' to the line segment ACFACObtaining the ratio of the abscissa u to the ordinate v in the plane pixel coordinate of the projection point F', and then combining the screen resolution of the display screen according to the u ═ DFABV ═ screen horizontal pixels, (/ AC) ((D))FACAB) screen vertical pixels, resulting in planar pixel coordinates F '(u, v) of projection point F'.
Another objective of an embodiment of the present invention is to provide a new non-contact human-computer interaction system, which includes a computer, a depth camera, and a display screen, where the computer is connected to the depth camera and the display screen, respectively, and the system performs human-computer interaction by using the new non-contact human-computer interaction method.
It is a further object of an embodiment of the present invention to provide a computer-readable storage medium storing a program for electronic data exchange, the program being for executing the above-mentioned new contactless human-computer interaction method.
Compared with the prior art, the novel non-contact human-computer interaction method and system provided by the invention have the beneficial effects that: the problem of controlling the screen content by using detectable target objects such as finger tips and the like in a mouse-free environment is solved, the novel non-contact human-computer interaction method does not need any calibration, and the method has the advantages of small calculated amount and simple hardware equipment. The invention is based on deep learning, two-dimensional plane information and three-dimensional depth information are obtained through a depth camera, the projected coordinates of a target object on a display screen are obtained through calculation, a mouse event is triggered, and non-contact human-computer interaction is realized.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the technical descriptions of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to derive other drawings without creative efforts.
Fig. 1 is a flowchart of a new contactless human-computer interaction method according to an embodiment of the present invention.
Fig. 2 is a two-dimensional plane image captured by the depth camera in the embodiment of the present invention.
FIG. 3 is a schematic diagram of a two-stage deep neural network structure for target detection in an embodiment of the present invention.
FIG. 4 is a schematic diagram of the position relationship between the depth camera, the finger tip and the display screen in the embodiment of the invention.
FIG. 5 is a flow chart of a method for calculating three-dimensional coordinates of a proxel F' in a depth camera coordinate system in an embodiment of the present invention.
FIG. 6 is a diagram illustrating a relationship between a current position and an original position of a fingertip when a mouse double-click event is triggered according to an embodiment of the present invention.
FIG. 7 is a flowchart of triggering a mouse double-click event in an embodiment of the present invention.
Fig. 8 is a schematic structural diagram of a new contactless human-computer interaction system according to an embodiment of the present invention.
Detailed Description
In order to make the technical problems, technical solutions and advantageous effects to be solved by the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
Referring to fig. 1, fig. 1 is a flowchart of a new contactless human-computer interaction method provided by an embodiment of the present invention, where the method includes the following steps:
s100, automatically detecting three vertexes A, B, C and a target point F of a display screen by using a deep learning system in a two-dimensional plane image of a depth camera to obtain plane pixel coordinates A (ua, va), B (ub, vb), C (uc, vc) and F (u0, v0) of each point;
s200, combining the internal reference of the depth camera, converting the plane pixel coordinate of each point into a three-dimensional coordinate under a depth camera coordinate system to obtain a three-dimensional coordinate A (x) of each point1,y1,z1),B(x2,y2,z2),C(x3,y3,z3),F(x0,y0,z0)；
S300, calculating a three-dimensional coordinate F ' (x ', y ', z ') of a projection point F ' of the target point F on the display screen in a depth camera coordinate system;
s400, calculating plane pixel coordinates F '(u, v) of a projection point F' on a display screen, and comprising:
respectively calculating the distance D from the projection point F' to the line segment ABFABAnd the distance D from the projection point F' to the line segment ACFACObtaining the horizontal coordinate of the plane pixel of the projection point FPlotting the ratio of u to the ordinate v, and then combining the screen resolution of the display screen (screen horizontal pixels x screen vertical pixels), based on u ═ DFABV ═ screen horizontal pixels, (/ AC) ((D))FACAB) screen vertical pixels, obtaining the planar pixel coordinates F '(u, v) of the projection point F';
s500, recognizing the action of the target point F, displaying a mouse mark at the position of the projection point F', calling a related system mouse interface to trigger a mouse event, and controlling a display screen.
FIG. 2 is a two-dimensional planar image taken by the depth camera of an embodiment of the present invention showing a rectangular display screen and a hand gesturing to extend the index finger. In the embodiment of the present invention, a fingertip of a finger is taken as a target point for explanation, and in other embodiments, other detectable target objects may be selected as the target point.
Referring to fig. 2, the above step S100 includes the following sub-steps:
s110, automatically detecting three vertexes A, B, C of a display screen by using a target detection algorithm (such as a YOLO algorithm or an SSD algorithm) based on deep learning, selecting three vertexes A, B, C of the display screen to form a screen plane, and obtaining plane pixel coordinates A (ua, va), B (ub, vb) and C (uc, vc) of each point by taking the vertex A as an origin of a pixel coordinate system of the screen plane;
s120, with reference to fig. 3, establishing a two-stage target detection deep neural network structure, and automatically detecting the finger tip F, including:
s121, establishing a finger detection depth neural network for detecting fingers in the two-dimensional plane image, expanding the detected finger area and positioning the fingers according to the expanded area;
and S122, establishing a fingertip detection depth neural network for detecting the fingertips of the fingers and positioning the fingertips to obtain a plane pixel coordinate F (u0, v0) of the fingertips F.
In order to improve the positioning precision of the detection fingertip, the embodiment of the invention also establishes a neural network of a double-channel attention mechanism. The attention mechanism is a model for simulating the attention of the human brain, and important information is quickly screened out by using limited attention, so that the efficiency and the accuracy of the human brain in processing visual information are improved.
Referring to fig. 4, fig. 4 is a schematic diagram illustrating the positions of the depth camera, the finger tip and the display screen according to the embodiment of the present invention. The projected point F' of the finger tip F on the display screen is shown.
Referring to fig. 5, the above step S300 includes the following sub-steps:
s310, according to the vertex A, B, C, calculating a normal vector n ═ (a, b, c) of the screen plane passing through the vertex a, where:
a＝y1(z2-z3)+y2(z3-z1)+y3(z1-z2)，
b＝z1(x2-x3)+z2(x3-x1)+z3(x1-x2)，
c＝x1(y2-y3)+x2(y3-y1)+x3(y1-y2)；
s320, setting the three-dimensional coordinate of the projection point F 'in the depth camera coordinate system as F' (x ', y', z '), and obtaining an equation set according to the condition that the line segment FF' is parallel to the normal vector n:
s330, obtaining an equation according to the fact that the line segment AF' is perpendicular to the normal vector n:
a(x′-x1)+b(y′-y1)+c(z′-z1)＝0(**)；
and S340, simultaneously establishing an equation set (. +) and an equation (. +), obtaining t, and substituting the t into the equation set (. +), thereby obtaining the three-dimensional coordinates F ' (x ', y ', z ') of the projection point F '.
Specifically, the actions of the recognition target point F in the above step S500 include:
when the other vertex A, B, C is fixed, the user moves the finger and recalculates the three-dimensional coordinates of the finger tip F using the above steps S100 to S200. The actions of the target point F include clicking, double-clicking, holding a pressed state, being released after being pressed, and the like, and the correspondingly triggered mouse events include a mouse click event (click), a mouse double click event (dbclick), an event (mousedown) triggered when a mouse button is pressed, and an event (mouseup) triggered when the mouse button is released.
The following description will take the triggering of a mouse double-click event as an example.
Fig. 6 is a schematic diagram illustrating a relationship between a current position of a fingertip and an original position of the fingertip when a mouse double-click event is triggered in the embodiment of the present invention, where D1 and D2 are distances from the current position of the fingertip to the original position, respectively, and D1> D2. Referring to fig. 6, the definition of the double click action in the embodiment of the present invention is: the finger tip returns to the vicinity of the original position after leaving the original position, and the definition of the finger tip leaving the original position is as follows: the distance between the position of the finger tip and the original position is larger than D1, and the definition of the finger tip returning to the vicinity of the original position is as follows: the distance from the position of the finger tip to the original position is less than D2.
FIG. 7 is a flowchart of triggering a mouse double-click event in an embodiment of the present invention. Referring to fig. 7, since the mouse double-click event generally ends within 2 seconds, a click action occurs within the 50-frame image sequence. In the embodiment of the invention, the coordinates of the fingertip are detected by every 5 frames of images, the distance between the fingertip coordinate of the image with the mark FAR (TRUE) as the tn th frame and the original position is set to be larger than D1, and the distance between the fingertip coordinate of the image with the mark NEAR (TRUE) as the tm frame (0< tn < tm <50) and the original position is set to be smaller than D2; when FAR is TRUE and NEAR is TRUE, a mouse double-click event is triggered, and the display screen is manipulated. The specific process comprises the following steps:
firstly, reading an image, judging whether the distance FAR is equal to TRUE and no NEAR is equal to TRUE after 50 frames of images pass, if so, setting FAR equal to FALSE and NEAR equal to FALSE, namely, if the finger tip of the tn-th frame of image leaves the original position and the finger tip of the (tn +50) -th frame of image does not return to the vicinity of the original position, carrying out homing on the finger tip of the tn-th frame of image; if not, detecting the coordinates of the finger tip every 5 frames of images, when the detected coordinates of the finger tip meet FAR TRUE, continuing the detection, when the coordinates of the finger tip also meet NEAR TRUE, triggering a mouse double-click event, setting FAR FALSE and NEAR FALSE, and repeating the steps.
Let the three-dimensional coordinate of the finger tip at the original position be (x)0,y0,z0) The fingertip coordinate of the tn th frame image is (x)tn,ytn,ztn) The coordinate of the fingertip of the tm frame image is (x)tm,ytm,ztm) And then when simultaneously:
(xtn-x0)2+(ytn-y0)2+(ztn-z0)2＞D12and
(xtm-x0)2+(ytm-y0)2+(ztm-z0)2＜D22when so, a mouse double-click event is triggered.
The embodiment of the invention provides a non-contact human-computer interaction method, which is used for solving the problem that the content of a screen is controlled by a detectable target object such as a finger tip in a mouse-free environment. The novel non-contact human-computer interaction method does not need any calibration, is based on deep learning, obtains two-dimensional plane information and three-dimensional depth information through a depth camera, calculates to obtain the projection coordinates of a target object on a display screen, triggers a mouse event and realizes non-contact human-computer interaction. The method has the advantages of small calculation amount and simple hardware equipment.
Referring to fig. 8, an embodiment of the present invention further provides a new contactless human-computer interaction system, including: the computer is respectively connected with the depth camera and the display screen. The system adopts the method of the embodiment to carry out human-computer interaction. The computer is used for calculating the position information of the projection of the target point on the display screen according to the position information acquired by the depth camera, calling a related system mouse interface to trigger a mouse event, and controlling the display screen.
An embodiment of the present invention also provides a computer-readable storage medium storing a program for electronic data exchange, the program being used to execute the new contactless human-computer interaction method of the present invention.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit of the present invention are intended to be included therein.
Claims (8)
1. A novel non-contact man-machine interaction method comprises the following steps:
s100, automatically detecting three vertexes A, B, C and a target point F of a display screen by using a deep learning system in a two-dimensional plane image of a depth camera to obtain plane pixel coordinates of each point;
s200, combining the internal reference of the depth camera, and converting the plane pixel coordinates of the top point A, B, C and the target point F into three-dimensional coordinates under a depth camera coordinate system;
s300, calculating a three-dimensional coordinate of a projection point F' of the target point F on the display screen in a depth camera coordinate system;
s400, calculating the plane pixel coordinate of the projection point F' on the display screen;
and S500, identifying the action of the target point F, and calling a related system mouse interface to trigger a mouse event.
2. A new contactless human-computer interaction method according to claim 1, characterized in that said step S100 comprises the following sub-steps:
s110, automatically detecting a vertex A, B, C of the display screen by using a target detection algorithm based on deep learning to obtain plane pixel coordinates of each point;
s120, establishing a two-stage target detection deep neural network structure, and automatically detecting a target point F, wherein the two-stage target detection deep neural network structure comprises the following steps:
s121, establishing a target object detection depth neural network for detecting a target object in the two-dimensional plane image, expanding the detected target object area and positioning the target object according to the expanded area;
and S122, establishing a target point detection depth neural network for detecting a target point of the target object and positioning the target point to obtain a plane pixel coordinate of the target point F.
3. The new contactless human-computer interaction method according to claim 2, wherein the step S120 further comprises: and establishing a neural network with a dual-channel attention mechanism for improving the positioning precision of the detected target point.
4. The new contactless human-computer interaction method according to claim 1, wherein the step S300 includes:
from the vertex A, B, C, a normal vector n of the screen plane passing through the vertex a is calculated, and from the line segment FF ' parallel to the normal vector n and the line segment AF ' perpendicular to the normal vector n, the three-dimensional coordinates of the projected point F ' are calculated.
5. The new contactless human-computer interaction method according to claim 1, wherein the step S400 comprises:
and calculating the ratio of the abscissa u to the ordinate v in the plane pixel coordinate of the projection point F ', and then combining the screen resolution of the display screen to obtain the plane pixel coordinate of the projection point F'.
6. The new contactless human-computer interaction method according to claim 1, wherein the step S400 comprises:
respectively calculating the distance D from the projection point F' to the line segment ABFABAnd the distance D from the projection point F' to the line segment ACFACObtaining the ratio of the abscissa u to the ordinate v in the plane pixel coordinate of the projection point F', and then combining the screen resolution of the display screen according to the u ═ DFABV ═ screen horizontal pixels, (/ AC) ((D))FAC(AB) screen vertical pixels, resulting in the flatness of the projected point FThe plane pixel coordinates F' (u, v).
7. A novel non-contact human-computer interaction system, which comprises a computer, a depth camera and a display screen, wherein the computer is respectively connected with the depth camera and the display screen, and the system adopts the novel non-contact human-computer interaction method as claimed in any one of claims 1-6 for human-computer interaction.
8. A computer-readable storage medium storing a program for electronic data exchange, the program being for executing the new contactless human-computer interaction method according to any one of claims 1-6.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011395956.8A CN112363629B (en) | 2020-12-03 | 2020-12-03 | Novel non-contact man-machine interaction method and system |
| PCT/CN2020/137285 WO2022116281A1 (en) | 2020-12-03 | 2020-12-17 | New non-contact human-computer interaction method and system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011395956.8A CN112363629B (en) | 2020-12-03 | 2020-12-03 | Novel non-contact man-machine interaction method and system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN112363629A CN112363629A (en) | 2021-02-12 |
| CN112363629B true CN112363629B (en) | 2021-05-28 |
Family
ID=74536668
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011395956.8A Expired - Fee Related CN112363629B (en) | 2020-12-03 | 2020-12-03 | Novel non-contact man-machine interaction method and system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN112363629B (en) |
| WO (1) | WO2022116281A1 (en) |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113095243B (en) * | 2021-04-16 | 2022-02-15 | 推想医疗科技股份有限公司 | Mouse control method and device, computer equipment and medium |
| CN115885238A (en) * | 2021-07-26 | 2023-03-31 | 广州视源电子科技股份有限公司 | Implementation method and system of fingertip mouse |
| CN113807191B (en) * | 2021-08-23 | 2024-06-14 | 南京航空航天大学 | Non-invasive visual test script automatic recording method |
| CN115617178B (en) * | 2022-11-08 | 2023-04-25 | 润芯微科技(江苏)有限公司 | Method for completing key and function triggering by no contact between finger and vehicle |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102354345A (en) * | 2011-10-21 | 2012-02-15 | 北京理工大学 | Medical image browse device with somatosensory interaction mode |
| CN102749991A (en) * | 2012-04-12 | 2012-10-24 | 广东百泰科技有限公司 | Non-contact free space eye-gaze tracking method suitable for man-machine interaction |
| CN103345301A (en) * | 2013-06-18 | 2013-10-09 | 华为技术有限公司 | Depth information acquisition method and device |
| CN103914152A (en) * | 2014-04-11 | 2014-07-09 | 周光磊 | Recognition method and system for multi-point touch and gesture movement capturing in three-dimensional space |
| CN109683699A (en) * | 2019-01-07 | 2019-04-26 | 深圳增强现实技术有限公司 | The method, device and mobile terminal of augmented reality are realized based on deep learning |
| CN110782532A (en) * | 2019-10-23 | 2020-02-11 | 北京达佳互联信息技术有限公司 | Image generation method, image generation device, electronic device, and storage medium |
Family Cites Families (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011081480A (en) * | 2009-10-05 | 2011-04-21 | Seiko Epson Corp | Image input system |
| US20120242806A1 (en) * | 2011-03-23 | 2012-09-27 | Tk Holdings Inc. | Dynamic stereo camera calibration system and method |
| CN102968222A (en) * | 2012-11-07 | 2013-03-13 | 电子科技大学 | Multi-point touch equipment based on depth camera |
| CN103207709A (en) * | 2013-04-07 | 2013-07-17 | 布法罗机器人科技（苏州）有限公司 | Multi-touch system and method |
| CN103793060B (en) * | 2014-02-14 | 2017-07-28 | 杨智 | A kind of user interactive system and method |
| AU2019308228B2 (en) * | 2018-07-16 | 2021-06-03 | Accel Robotics Corporation | Autonomous store tracking system |
-
2020
- 2020-12-03 CN CN202011395956.8A patent/CN112363629B/en not_active Expired - Fee Related
- 2020-12-17 WO PCT/CN2020/137285 patent/WO2022116281A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102354345A (en) * | 2011-10-21 | 2012-02-15 | 北京理工大学 | Medical image browse device with somatosensory interaction mode |
| CN102749991A (en) * | 2012-04-12 | 2012-10-24 | 广东百泰科技有限公司 | Non-contact free space eye-gaze tracking method suitable for man-machine interaction |
| CN103345301A (en) * | 2013-06-18 | 2013-10-09 | 华为技术有限公司 | Depth information acquisition method and device |
| CN103914152A (en) * | 2014-04-11 | 2014-07-09 | 周光磊 | Recognition method and system for multi-point touch and gesture movement capturing in three-dimensional space |
| CN109683699A (en) * | 2019-01-07 | 2019-04-26 | 深圳增强现实技术有限公司 | The method, device and mobile terminal of augmented reality are realized based on deep learning |
| CN110782532A (en) * | 2019-10-23 | 2020-02-11 | 北京达佳互联信息技术有限公司 | Image generation method, image generation device, electronic device, and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN112363629A (en) | 2021-02-12 |
| WO2022116281A1 (en) | 2022-06-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112363629B (en) | Novel non-contact man-machine interaction method and system | |
| TWI690842B (en) | Method and apparatus of interactive display based on gesture recognition | |
| US6624833B1 (en) | Gesture-based input interface system with shadow detection | |
| Kollorz et al. | Gesture recognition with a time-of-flight camera | |
| Murthy et al. | Hand gesture recognition using neural networks | |
| O'Hagan et al. | Visual gesture interfaces for virtual environments | |
| NZ525717A (en) | A method of tracking an object of interest using multiple cameras | |
| Katz et al. | A multi-touch surface using multiple cameras | |
| CN110751097A (en) | Semi-supervised three-dimensional point cloud gesture key point detection method | |
| CN112657176A (en) | Binocular projection man-machine interaction method combined with portrait behavior information | |
| CN103761011B (en) | A kind of method of virtual touch screen, system and the equipment of calculating | |
| Simion et al. | Finger detection based on hand contour and colour information | |
| Schlattmann et al. | Markerless 4 gestures 6 DOF real‐time visual tracking of the human hand with automatic initialization | |
| Cheng et al. | Fingertip-based interactive projector–camera system | |
| Gope et al. | Interaction with Large Screen Display using Fingertip & Virtual Touch Screen | |
| Kolaric et al. | Direct 3D manipulation using vision-based recognition of uninstrumented hands | |
| Wickeroth et al. | Markerless gesture based interaction for design review scenarios | |
| Haubner et al. | Recognition of dynamic hand gestures with time-of-flight cameras | |
| Jia et al. | Tracking pointing gesture in 3d space for wearable visual interfaces | |
| Ukita et al. | Wearable Virtual Tablet: Fingertip Drawing Interface Using an Active-Infrared Camera. | |
| Wensheng et al. | Implementation of virtual mouse based on machine vision | |
| Ma et al. | Comparative Analysis on Few Typical Algorithms of Gesture Recognition for Human-robot Interaction | |
| Lokhande et al. | ML based Projector to Smart Board Conversion | |
| Xie et al. | Natural Bare-Hand Interaction for Remote Operating Large Touch Screen. | |
| Cong | Research on fingertip positioning and human-computer interaction technology based on stereo vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20210528 |
|
| CF01 | Termination of patent right due to non-payment of annual fee |