CN111522446A - Gesture recognition method and device based on multipoint TOF - Google Patents

Gesture recognition method and device based on multipoint TOF Download PDF

Info

Publication number
CN111522446A
CN111522446A CN202010517916.XA CN202010517916A CN111522446A CN 111522446 A CN111522446 A CN 111522446A CN 202010517916 A CN202010517916 A CN 202010517916A CN 111522446 A CN111522446 A CN 111522446A
Authority
CN
China
Prior art keywords
sensor
hand
time
measured
tof
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010517916.XA
Other languages
Chinese (zh)
Other versions
CN111522446B (en
Inventor
赵飞
陆小松
蒲天发
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangsu Thredim Photoelectric Co ltd
Original Assignee
Jiangsu Thredim Photoelectric Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangsu Thredim Photoelectric Co ltd filed Critical Jiangsu Thredim Photoelectric Co ltd
Priority to CN202010517916.XA priority Critical patent/CN111522446B/en
Publication of CN111522446A publication Critical patent/CN111522446A/en
Application granted granted Critical
Publication of CN111522446B publication Critical patent/CN111522446B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
    • Y02D30/00—Reducing energy consumption in communication networks
    • Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

The invention discloses a gesture recognition method and device based on a multipoint TOF, and belongs to the technical field of information processing. The main technical scheme comprises: starting an infrared time of flight (TOF) sensor; recording the entering time of the hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the TOF sensor, and calculating the signal phase calculation time according to the entering time and the exiting time; and calculating the motion deviation angle of the hand to be measured relative to the central coordinate of the plane of the TOF sensing detection device according to the signal phase calculation time, and determining the motion direction of the hand to be measured according to the motion deviation angle and the signal phase calculation time.

Description

Gesture recognition method and device based on multipoint TOF
Technical Field
The invention relates to the technical field of information processing, in particular to a gesture recognition method and device based on multi-point TOF.
Background
With the continuous development of social productivity and scientific technology, various industries pay more and more attention to gesture recognition methods in man-machine intelligent interaction scenes, such as intelligent interaction scenes, virtual reality interaction scenes, intelligent homes and the like, and gesture recognition in the scenes can bring about immersive control experience for operators.
At present, common gesture recognition is realized based on an image recognition algorithm, although the method can realize control of gesture recognition, the method has high requirements on the performance of a processor, high system power consumption and cost and is not suitable for application scenes with sensitive cost, low power consumption and equipment miniaturization.
Disclosure of Invention
In view of this, embodiments of the present invention provide a gesture recognition method and apparatus based on a multipoint TOF, and mainly aim to implement accurate recognition of gesture recognition on the premise of unchanged cost.
In order to solve the above problems, embodiments of the present invention mainly provide the following technical solutions:
in a first aspect, an embodiment of the present invention provides a gesture recognition method based on a multipoint TOF, including:
starting an infrared time of flight (TOF) sensor;
recording the entering time of the hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the TOF sensor, and calculating the signal phase calculation time according to the entering time and the exiting time;
and calculating the motion deviation angle of the hand to be measured relative to the central coordinate of the plane of the TOF sensor according to the signal phase calculation time, and determining the motion direction of the hand to be measured according to the motion deviation angle and the signal phase calculation time.
Optionally, the method further includes:
calculating a difference value between the exit time and the entry time, and determining whether the difference value is within a set threshold range;
and if the difference is determined to be within the set threshold range, continuing to perform gesture recognition.
Optionally, the method further includes:
if the difference is determined not to be within the set threshold range, measuring the intensity variation of the returned signal of the TOF sensor;
when the signal intensity variation is a positive number, determining that the hand to be measured is close;
when the signal intensity variation is negative, determining that the hand to be measured is far away;
and when the number of times of the approaching/departing continuous same movement direction exceeds a preset effective number threshold value, determining as one effective gesture recognition.
Optionally, before recording an entry time when the hand to be measured enters the TOF sensor and an exit time when the hand to be measured exits the sensor, the method further includes:
determining whether the signal strength is greater than a preset effective signal threshold;
and if the signal intensity is not larger than the preset effective signal threshold value, ignoring the gesture recognition.
Optionally, the TOF sensor includes at least one first-type sensor and at least two second-type sensors, where the first-type sensor is a receiving-type sensor, the second-type sensor is an emitting-type sensor, or the first-type sensor is an emitting-type sensor, and the second-type sensor is a receiving-type sensor;
when the hand to be measured passes through the TOF sensor, the difference value of the signal phase calculation time of the two second sensors with the X-axis distance is a horizontal motion offset angle d (X), and the difference value of the signal phase calculation time of the two second sensors with the Y-axis distance is a vertical motion offset angle d (Y);
ten (n) represents the entering time of the hand to be measured entering the signal of the second type sensor n, Tex (n) represents the exiting time of the hand to be measured leaving the signal of the second type sensor n, t (n) represents the phase calculation time of the signal of the sensor n, and n is a positive integer.
Optionally, calculating the signal phase calculation time according to the entry time and the exit time specifically includes:
Figure BDA0002530812630000021
if the TOF sensor comprises a receiving sensor and three transmitting sensors, calculating the motion offset angle of the to-be-measured hand relative to the central coordinate of the plane of the TOF sensing detection device according to the signal phase calculation time specifically comprises the following steps:
d(x)＝|t1-t2|；
d(y1)＝|t1-t3|：
d(y2)＝|t3-t2|。
optionally, determining the motion direction of the hand to be measured according to the motion offset angle and the signal phase calculation time includes:
when d (y1) > d (x)/2 and t (3) < t (1), or d (y2) > d (x)/2 and t (3) < t (2), determining that the hand to be measured moves from bottom to top;
when d (y1) > d (x)/2 and t (3) > t (1), or d (y2) > d (x)/2 and t (3) > t (2), determining that the hand to be measured moves from top to bottom;
when d (x) > d (y1) + d (y2) and t (2) > t (1) and t (3) > t (1), determining that the hand to be measured moves from left to right;
when d (x) > d (y1) + d (y2) and t (2) < t (1) and t (3) < t (1), it is determined that the hand to be measured moves from right to left.
In a second aspect, an embodiment of the present invention further provides a gesture recognition apparatus based on a multipoint TOF, including:
the starting unit is used for starting the infrared time of flight TOF sensor;
the recording unit is used for recording the entering time when the hand to be measured enters the TOF sensor and the exiting time when the hand to be measured exits the TOF sensor;
the first calculating unit is used for calculating the signal phase calculating time according to the entering time and the exiting time recorded by the recording unit;
the second calculation unit is used for calculating the motion offset angle of the hand to be measured relative to the central coordinate of the plane where the TOF sensing detection device is located according to the signal phase calculation time calculated by the first calculation unit;
and the first determining unit is used for determining the motion direction of the hand to be measured according to the motion deviation angle and the signal phase calculation time calculated by the second calculating unit.
Optionally, the apparatus further comprises:
a third calculation unit for calculating a difference between the exit time and the entry time;
a second determination unit configured to determine whether the difference calculated by the third calculation unit is within a set threshold range;
and the processing unit is used for continuing to perform gesture recognition when the second determining unit determines that the difference value is within the set threshold range.
Optionally, the apparatus further comprises:
the measuring unit is used for measuring the intensity variation of the returned signal of the TOF sensor when the second determining unit determines that the difference value is not in the set threshold range;
a third determination unit configured to determine that the hand to be measured is approaching when the signal intensity variation measured by the measurement unit is a positive number;
a fourth determination unit configured to determine that the hand to be measured is far away when the signal intensity variation measured by the measurement unit is a negative number;
and the fifth determining unit is used for determining as one effective gesture recognition when the times of the approaching/departing continuous same movement directions exceed a preset effective time threshold value.
Optionally, the apparatus further comprises:
a sixth determining unit, configured to determine whether the signal intensity is greater than a preset effective signal threshold before the recording unit records an entry time at which the hand to be measured enters the TOF sensor and an exit time at which the hand to be measured exits the sensor;
and the ignoring unit is used for ignoring the gesture recognition when the sixth determining unit determines that the signal intensity is not greater than the preset effective signal threshold.
Alternatively to this, the first and second parts may,
the TOF sensor comprises at least one first-class sensor and at least two second-class sensors, wherein the first-class sensor is a receiving-class sensor, the second-class sensor is an emitting-class sensor, or the first-class sensor is an emitting-class sensor, and the second-class sensor is a receiving-class sensor;
when the hand to be measured passes through the TOF sensor, the difference value of the signal phase calculation time of the two second sensors with the X-axis distance is a horizontal motion offset angle d (X), and the difference value of the signal phase calculation time of the two second sensors with the Y-axis distance is a vertical motion offset angle d (Y);
ten (n) represents the entering time of the hand to be measured entering the signal of the second type sensor n, Tex (n) represents the exiting time of the hand to be measured leaving the signal of the second type sensor n, t (n) represents the phase calculation time of the signal of the sensor n, and n is a positive integer.
Optionally, calculating the signal phase calculation time according to the entry time and the exit time specifically includes:
Figure BDA0002530812630000041
if the TOF sensor comprises a receiving sensor and three transmitting sensors, calculating the motion offset angle of the to-be-measured hand relative to the central coordinate of the plane of the TOF sensing detection device according to the signal phase calculation time specifically comprises the following steps:
d(x)＝|t1-t2|；
d(y1)＝|t1-t3|；
d(y2)＝|t3-t2|。
optionally, the first determination unit is further configured to determine that the hand to be measured is moving from bottom to top when d (y1) > d (x)/2 and t (3) < t (1), or d (y2) > d (x)/2 and t (3) < t (2);
when d (y1) > d (x)/2 and t (3) > t (1), or d (y2) > d (x)/2 and t (3) > t (2), determining that the hand to be measured moves from top to bottom;
when d (x) > d (y1) + d (y2) and t (2) > t (1) and t (3) > t (1), determining that the hand to be measured moves from left to right;
when d (x) > d (y1) + d (y2) and t (2) < t (1) and t (3) < t (1), it is determined that the hand to be measured moves from right to left.
By the technical scheme, the technical scheme provided by the embodiment of the invention at least has the following advantages:
the gesture recognition method and device based on the multi-point TOF provided by the embodiment of the invention start an infrared time of flight (TOF) sensor; recording the entering time of the hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the TOF sensor, and calculating the signal phase calculation time according to the entering time and the exiting time; the motion deviation angle of the hand to be measured relative to the central coordinate of the plane where the TOF sensing detection device is located is calculated according to the signal phase calculation time, and the motion direction of the hand to be measured is determined according to the motion deviation angle and the signal phase calculation time.
The foregoing description is only an overview of the technical solutions of the embodiments of the present invention, and the embodiments of the present invention can be implemented according to the content of the description in order to make the technical means of the embodiments of the present invention more clearly understood, and the detailed description of the embodiments of the present invention is provided below in order to make the foregoing and other objects, features, and advantages of the embodiments of the present invention more clearly understandable.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the embodiments of the invention. Also, like reference numerals are used to refer to like parts throughout the drawings. In the drawings:
fig. 1 illustrates a flowchart of a gesture recognition method based on a multipoint TOF according to an embodiment of the present invention;
FIG. 2 shows a schematic diagram of a TOF sensor layout provided by an embodiment of the present invention;
FIG. 3 is a block diagram illustrating components of an apparatus provided by an embodiment of the invention;
FIG. 4 is a diagram illustrating a waveform signal provided by an embodiment of the present invention;
FIG. 5 is a schematic diagram illustrating an offset angle of motion provided by an embodiment of the present invention;
FIG. 6 is a schematic diagram illustrating another angular offset of motion provided by an embodiment of the present invention;
FIG. 7 is a block diagram illustrating a multi-point TOF-based gesture recognition apparatus according to an embodiment of the present invention;
FIG. 8 is a block diagram illustrating another gesture recognition apparatus based on multi-point TOF according to an embodiment of the present invention
Fig. 9 is a schematic diagram of an electronic device according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
The embodiment of the invention provides a gesture recognition method based on a multipoint TOF, which comprises the following steps of:
101. the infrared time of flight TOF sensor is activated.
In the embodiment of the present invention, at least 1 single-point infrared Time of flight (TOF) sensor is used to implement gesture recognition, and the TOF sensor in the embodiment of the present invention includes at least one first-type sensor and at least two second-type sensors, where the first-type sensor is a receiving-type sensor, the second-type sensor is an emitting-type sensor, or the first-type sensor is an emitting-type sensor, and the second-type sensor is a receiving-type sensor. The TOF sensors described herein refer to a pair of receiving and transmitting, or a combination of transmitting and receiving, and the TOF sensors do not work completely independently, and it should be noted that the sensors of one type and the sensors of the second type are only differences in number, and their actual contents do not change.
Each TOF sensor comprises an infrared transmitting unit and an infrared receiving unit, and when the TOF sensor is determined to be a class II sensor, the TOF sensor can be adjusted to only start part of the infrared transmitting units or the infrared receiving units. The working combination between one TOF sensor or multiple TOF sensors can include, but is not limited to, a combination of multiple infrared transmitting units and one infrared receiving unit, a combination of one infrared transmitting unit and multiple infrared receiving units, and the like, and the specific combination can be determined according to different application scenarios.
In the following embodiments, 1 TOF sensor, a combination method of 3 infrared emitting units (sensors of the second type) and 1 infrared receiving unit (sensor of the first type) are taken as an example for description, but it should be noted that the description of the above method is not intended to limit the number and usage composition of TOF sensors, but is merely an exemplary description, and the number and combination usage of other TOF sensors are consistent with the methods of the following embodiments.
The layout of the TOF sensor is shown in fig. 2, fig. 2 is a schematic diagram of a layout of the TOF sensor provided by an embodiment of the invention, and an arrangement scheme of the TOF sensor including three infrared emitting units and one infrared receiving unit can be seen in the diagram. The three infrared transmitting units are arranged in an inverted 'pin' shape, the infrared receiving unit is placed between the two parallel infrared transmitting units, the above is only an exemplary illustration, other sensor layout modes are also feasible in practical application, and the arrangement of the TOF sensor is the same as the specific implementation process of the algorithm (namely the algorithm design principle is the same).
The multiple infrared transmitting units work in a time-sharing mode, for example, at the time 1, the infrared transmitting unit 1 is started, and reflected infrared light signals are measured through the infrared receiving unit; at the moment 2, the infrared transmitting unit 2 is started, and the reflected infrared light signal is measured through the infrared receiving unit; at the moment 3, the infrared transmitting unit 3 is started, and the reflected infrared light signal is measured through the infrared receiving unit; and sequentially and circularly measuring the infrared light signals reflected by the surface of the object and emitted by each infrared emission unit. The distance between the hand to be measured and the infrared transmitting unit can be judged according to the intensity of the received infrared light signal. The time-sharing working interval time of the infrared emission units is short (microsecond level), so the influence of the time-sharing interval on the measurement result can be ignored. When the hand to be measured is close to the infrared transmitting unit, the intensity of the infrared signal reflected by the hand to be measured is high, and when the hand to be measured is far from the infrared transmitting unit, the intensity of the infrared signal reflected by the hand to be measured is low. Note that the embodiment of the present invention does not use the intensity of the received infrared signal to implement distance measurement, the intensity of the received signal is only a reference amount for determining the relative movement (far or near) between the hand to be measured and the infrared transmitting unit, and the gesture recognition algorithm is implemented by receiving signal phases by multiple TOF sensors.
In practical application, the gesture recognition method based on the multi-point TOF is loaded in an electronic device, as shown in fig. 3, and mainly includes the following modules: the system comprises an infrared receiving unit, an infrared transmitting unit, a gesture recognition processing unit and a communication unit, wherein the infrared receiving unit is used for receiving infrared light signals and measuring the intensity, and is generally realized by an infrared receiving sensor, such as a si1153 sensor, so that the intensity measurement from ultraviolet light, visible light to near infrared light can be realized; the infrared emission unit is used for realizing the emission of infrared light signals, generally adopts an infrared LED, the wavelength can be 850nm or 940nm, and the emission angle is generally within 30 degrees for realizing longer distance measurement; the gesture recognition processing unit is used for realizing the processing of transmitting and receiving infrared signals, realizing a gesture recognition algorithm through echo signals of a plurality of received infrared transmitting signals, and can be realized by an MCU (microprogrammed control unit), wherein the infrared receiving unit is generally connected with the gesture recognition processing unit by an I2C interface; the communication unit is used for an external interface of the system, such as detected gesture command output, and the communication unit generally selects an RS232 interface, an RS485 interface or a USB interface.
102. Recording the entering time of the hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the TOF sensor, wherein the entering time is the starting time when the TOF sensor detects the gesture made by the hand to be measured, and the exiting time is the ending time when the TOF sensor detects the gesture, and calculating the signal phase calculating time according to the entering time and the exiting time.
With continued reference to fig. 2, it is assumed that when the hand to be measured moves from left to right, the waveform signals shown in fig. 4 can be obtained by measuring the reflected signals of the sensors. Since the infrared emission unit 1 is at the leftmost side, the reflected signal of the infrared emission unit 1 is firstly sensed by the hand, secondly by the infrared emission unit 3, and lastly by the infrared emission unit 2. On the contrary, if the hand moves from right to left, the phase of the received signal waveform is just opposite, and the gesture calculation can be realized by measuring the phase relation of the waveform through the three sensors.
In order to calculate the phase difference of the received signals of each sensor more accurately, the embodiment of the invention adopts a median method to calculate the signal phase calculation time. Namely the entry time ten (n) of the signal n when the hand enters each TOF sensor, the exit time tex (n) when the hand to be measured leaves the signal n of the sensor, t (n) representing the phase calculation time of the signal n of the sensor, n being a positive integer.
Calculating the signal phase calculation time according to the entry time and the exit time specifically comprises the following steps:
Figure BDA0002530812630000071
103. and calculating the motion deviation angle of the hand to be measured relative to the central coordinate of the plane of the TOF sensing detection device according to the signal phase calculation time, and determining the motion direction of the hand to be measured according to the motion deviation angle and the signal phase calculation time.
In the process of hand movement, the center coordinate of the plane where the hand is located moves along the X axis (transverse direction) and the Y axis (longitudinal direction) simultaneously, so the algorithm increases the calculation of the offset angle, when in practical application, the difference of the signal phase calculation time of two sensors with the X-axis distance is the horizontal movement offset angle d (X), and the difference of the signal phase calculation time of two sensors with the Y-axis distance is the vertical movement offset angle d (Y), wherein d (X), d (Y) are carried out according to the specific number of the infrared emission units, and if the TOF sensor is passed by the hand to be measured, the sensors with the second type have 3 numbers as shown in FIG. 2. In the above embodiment, since the infrared emitting units (1) and (2) are parallel, the motion offset angle between the infrared emitting units (1) and (2) relative to the Y axis is 0, and thus the motion offset angles between the two types of sensors according to the embodiment of the present invention relative to the Y axis are d (Y1) and d (Y2), respectively, as shown in fig. 5.
In the above embodiment, the infrared emission units (1) and (2) are parallel to each other, for an application scenario in which the infrared emission units (1) and (2) are not parallel to each other, a motion offset angle d (Y3) between the infrared emission unit (1) and the infrared emission unit (2) with respect to the Y axis and a motion offset angle d (X1) with respect to the X axis, a motion offset angle d (X2) between the infrared emission unit (1) and the infrared emission unit (3) with respect to the X axis and a motion offset angle d (X3) between the infrared emission unit (2) and the infrared emission unit (3) with respect to the X axis are calculated, that is, the motion offset angle between two sensors with respect to the X axis or the Y axis is calculated. And calculating and judging the direction of the gesture according to the X-axis movement offset angle and the Y-axis movement offset angle.
In the embodiment in the above steps, still three infrared emitting units and one infrared receiving unit are taken as an example for explanation, as shown in fig. 5, fig. 5 shows a schematic diagram of a movement offset angle provided by the embodiment of the present invention, a movement offset angle of a hand with respect to an x axis and a movement offset angle with respect to a y axis is realized by using d (x), d (y1) and d (y2), and whether a gesture is left or right or up or down is judged according to the magnitude relationship of d (x), d (y1) and d (y 2). In this case, d (x), d (y1), d (y2) are calculated as follows:
d(x)＝|t1-t2|；
d(y1)＝|t1-t3|；
d(y2)＝|t3-t2|；
when d (x) > d (y1) + d (y2) indicates that the gesture is a left-right motion, and when d (x) < d (y1) + d (y2) indicates that the gesture is an up-down motion.
According to the gesture recognition method and device based on the multi-point TOF and the electronic equipment, at least three single-point infrared TOF sensors are started; recording the entering time of the hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the TOF sensor, and calculating the signal phase calculation time according to the entering time and the exiting time; the motion deviation angle of the hand to be measured relative to the central coordinate of the plane where the TOF sensing detection device is located is calculated according to the signal phase calculation time, and the motion direction of the hand to be measured is determined according to the motion deviation angle and the signal phase calculation time.
The embodiment of the invention also provides a method, namely, the difference value between the exit time and the entry time is calculated, and whether the difference value is within the range of the set threshold value is determined; if the difference is determined to be within the set threshold range, continuing to perform gesture recognition; if the difference is determined not to be within the set threshold range, measuring the intensity variation of the returned signal of the TOF sensor; when the signal intensity variation is a positive number, determining that the hand to be measured is close; when the signal intensity variation is negative, determining that the hand to be measured is far away; and when the number of times of the approaching/departing continuous same movement direction exceeds a preset effective number threshold value, determining as one effective gesture recognition.
The method is used for identifying the application scene that the hand is far away from or close to the electronic equipment, and when the difference values of t1, t2 and t3 are small, the variation of the intensity of the return signals of the three transmitting units is measured according to the following formula:
ds(n，t)＝s(n，t)-s(n，t-1)，
where ds (n, t) represents the amount of change in the intensity of the return signal of the nth transmitting unit at time t from time t-1, s (n, t) is the intensity of the return signal of the nth transmitting unit at time t, and s (n, t-1) is the intensity of the return signal of the nth transmitting unit at time t-1. When ds (n, t) is positive, it means close, and when ds (n, t) is negative, it means far. To avoid interference, continuous judgment can be generally adopted, for example, if the movement is continued for 3 times in the same direction, the movement is regarded as effective movement. In addition, the consistency of the change directions of the returned signal strengths of the three transmitting units can be judged according to the consistency. The above-mentioned continuous times are merely exemplary illustrations, and may be adjusted in practical applications, for example, to 2 times or 4 times, and the specific times set in the embodiment of the present invention is not limited.
The above embodiment describes in detail the process of recognizing a distance or approach, and the following embodiment describes in detail the up-down recognition of gesture recognition, specifically: determining the motion direction of the hand to be measured according to the motion deviation angle and the signal phase calculation time comprises:
when d (y1) > d (x)/2 and t (3) < t (1), or d (y2) > d (x)/2 and t (3) < t (2), it is determined that the hand to be measured is moving from bottom to top;
when d (y1) > d (x)/2 and t (3) > t (1), or d (y2) > d (x)/2 and t (3) > t (2), it is determined that the hand to be measured moves from top to bottom;
when d (x) > d (y1) + d (y2) and t (2) > t (1) and t (3) > t (1), it is determined that the hand to be measured is moving from left to right;
when d (x) > d (y1) + d (y2) and t (2) < t (1) and t (3) < t (1), it is determined that the hand to be measured moves from right to left.
In order to prevent an application scenario in which a user may operate by mistake, such as a mistake touching an electronic device, in an actual application process, an embodiment of the present invention further provides a method before recording an entry time when a hand to be measured enters the TOF sensor and an exit time when the hand to be measured exits the TOF sensor, where the method further includes:
determining whether the signal strength is greater than a preset effective signal threshold; if the signal intensity is not larger than (smaller than or equal to) the preset effective signal threshold, ignoring the gesture recognition; and if the signal intensity is larger than the preset effective signal threshold value, recording the entering time of the hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the sensor.
And uploading the identification result through a communication interface.
The above embodiments are all described by taking an example that the second type of sensor includes 3 infrared emission units, and for an application scenario in which the second type of sensor includes 4 or more infrared emission units, the method is the same as the calculation method and principle that the second type of sensor includes 3 infrared emission units.
The following embodiment briefly describes an application scenario of one receiving sensor and four transmitting sensors. Fig. 6 is a schematic diagram of another motion offset angle provided by the embodiment of the present invention, and the horizontal layout is still illustrated in a parallel manner for convenience of illustration, but it should be noted that this illustration is not intended to limit the layout manner of the sensors. Different from three infrared emission units, there is a difference in calculating d (y), and the calculation method of the application scenario is dy1 ═ t1-t3|, dy2 ═ t2-t4|, dx1 ═ t1-t2|, dx2 ═ t3-t4 |.
In practical application, when four emission sensors exist, one sensor can be regarded as a redundant sensor, the redundant sensor has the function of auxiliary judgment, the increase of the auxiliary judgment can reduce the misjudgment rate, and the result of dynamic gesture recognition is more accurate.
Since the multi-point TOF based gesture recognition apparatus described in this embodiment is an apparatus capable of executing the multi-point TOF based gesture recognition method in the embodiment of the present invention, based on the multi-point TOF based gesture recognition method described in the embodiment of the present invention, a person skilled in the art can understand a specific implementation manner and various variations of the multi-point TOF based gesture recognition apparatus in this embodiment, so how the multi-point TOF based gesture recognition apparatus implements the multi-point TOF based gesture recognition method in the embodiment of the present invention is not described in detail herein. The device adopted by the person skilled in the art to implement the multi-point TOF-based gesture recognition method in the embodiments of the present invention is within the scope of the present application.
An embodiment of the present invention further provides a gesture recognition apparatus based on a multipoint TOF, as shown in fig. 7, including:
a start unit 21 that starts the infrared time of flight TOF sensor;
the recording unit 22 is used for recording the entering time when the hand to be measured enters the TOF sensor and the exiting time when the hand to be measured exits the TOF sensor;
a first calculating unit 23, configured to calculate a signal phase calculation time according to the entry time and the exit time recorded by the recording unit;
the second calculating unit 24 is configured to calculate a motion offset angle of the hand to be measured relative to a central coordinate of a plane where the TOF sensor is located according to the signal phase calculation time calculated by the first calculating unit 23;
a first determining unit 25, configured to determine a motion direction of the hand to be measured according to the motion offset angle and the signal phase calculation time calculated by the second calculating unit 24.
The gesture recognition device based on the multi-point TOF provided by the embodiment of the invention starts at least three single-point infrared time of flight TOF sensors; recording the entering time of the hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the TOF sensor, and calculating the signal phase calculation time according to the entering time and the exiting time; the motion deviation angle of the hand to be measured relative to the central coordinate of the plane where the TOF sensing detection device is located is calculated according to the signal phase calculation time, and the motion direction of the hand to be measured is determined according to the motion deviation angle and the signal phase calculation time.
Further, as shown in fig. 8, the apparatus further includes:
a third calculation unit 26 for calculating a difference between the exit time and the entry time;
a second determination unit 27 for determining whether the difference value calculated by the third calculation unit 26 is within a set threshold range;
a processing unit 28, configured to continue gesture recognition when the second determining unit 27 determines that the difference is within the set threshold range.
Further, as shown in fig. 8, the apparatus further includes:
a measuring unit 29 for measuring the amount of change in the TOF sensor return signal intensity when the second determining unit 27 determines that the difference is not within a set threshold range;
a third determination unit 210 configured to determine that the hand to be measured is approaching when the signal strength variation amount measured by the measurement unit 29 is a positive number;
a fourth determination unit 211 configured to determine that the hand to be measured is far away when the signal strength variation amount measured by the measurement unit 29 is a negative number;
a fifth determining unit 212, configured to determine as one valid gesture recognition when the number of consecutive same moving directions approaching/departing exceeds a preset valid number threshold.
Further, as shown in fig. 8, the apparatus further includes:
a sixth determining unit 213, configured to determine whether the signal intensity is greater than a preset effective signal threshold before the recording unit 22 records an entry time when the hand to be measured enters the TOF sensor and an exit time when the hand to be measured exits the sensor;
an ignoring unit 214, configured to ignore the gesture recognition of this time when the sixth determining unit 213 determines that the signal strength is not greater than the preset valid signal threshold.
Furthermore, the TOF sensor comprises at least one first-class sensor and at least two second-class sensors, wherein the first-class sensor is a receiving-class sensor, the second-class sensor is a transmitting-class sensor, or the first-class sensor is a transmitting-class sensor, and the second-class sensor is a receiving-class sensor;
when the hand to be measured passes through the TOF sensor, the difference value of the signal phase calculation time of the two second sensors with the X-axis distance is a horizontal motion offset angle d (X), and the difference value of the signal phase calculation time of the two second sensors with the Y-axis distance is a vertical motion offset angle d (Y);
ten (n) represents the entering time of the hand to be measured entering the signal of the second type sensor n, Tex (n) represents the exiting time of the hand to be measured leaving the signal of the second type sensor n, t (n) represents the phase calculation time of the signal of the sensor n, and n is a positive integer.
Further, calculating the signal phase calculation time according to the entry time and the exit time specifically includes:
Figure BDA0002530812630000111
if the TOF sensor comprises a receiving sensor and three transmitting sensors, calculating the motion offset angle of the to-be-measured hand relative to the central coordinate of the plane of the TOF sensing detection device according to the signal phase calculation time specifically comprises the following steps:
d(x)＝|t1-t2|；
d(y1)＝|t1-t3|；
d(y2)＝|t3-t2|。
further, the first determination unit is also used for determining that the hand to be measured moves from bottom to top when d (y1) > d (x)/2 and t (3) < t (1) or d (y2) > d (x)/2 and t (3) < t (2);
when d (y1) > d (x)/2 and t (3) > t (1), or d (y2) > d (x)/2 and t (3) > t (2), determining that the hand to be measured moves from top to bottom;
when d (x) > d (y1) + d (y2) and t (2) > t (1) and t (3) > t (1), determining that the hand to be measured moves from left to right;
when d (x) > d (y1) + d (y2) and t (2) < t (1) and t (3) < t (1), it is determined that the hand to be measured moves from right to left.
An embodiment of the present invention provides an electronic device, as shown in fig. 9, including: at least one processor (processor) 31; and at least one memory (memory)32, a bus 33, connected to the processor 31; wherein,
the processor 31 and the memory 32 complete mutual communication through the bus 33;
the processor 31 is configured to call program instructions in the memory 32 to perform the steps in the above-described method embodiments.
According to the electronic equipment provided by the embodiment of the invention, at least three single-point infrared time of flight (TOF) sensors are started; recording the entering time of the hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the TOF sensor, and calculating the signal phase calculation time according to the entering time and the exiting time; the motion deviation angle of the hand to be measured relative to the central coordinate of the plane where the TOF sensing detection device is located is calculated according to the signal phase calculation time, and the motion direction of the hand to be measured is determined according to the motion deviation angle and the signal phase calculation time.
The present embodiments provide a non-transitory computer-readable storage medium storing computer instructions that cause the computer to perform the methods provided by the method embodiments described above.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flow diagrams and/or block diagrams, and combinations of flows and/or blocks in the flow diagrams and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include forms of volatile memory in a computer readable medium, Random Access Memory (RAM) and/or non-volatile memory, such as Read Only Memory (ROM) or flash memory (flash RAM). The memory is an example of a computer-readable medium.
Computer-readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), Read Only Memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), Digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer readable medium does not include a transitory computer readable medium such as a modulated data signal and a carrier wave.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in the process, method, article, or apparatus that comprises the element.
As will be appreciated by one skilled in the art, embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The above are merely examples of the present application and are not intended to limit the present application. Various modifications and changes may occur to those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims (10)

1. A gesture recognition method based on multi-point TOF is characterized by comprising the following steps:
starting an infrared time of flight (TOF) sensor;
recording the entering time of the hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the TOF sensor, and calculating the signal phase calculation time according to the entering time and the exiting time;
and calculating the motion deviation angle of the hand to be measured relative to the central coordinate of the plane of the TOF sensor according to the signal phase calculation time, and determining the motion direction of the hand to be measured according to the motion deviation angle and the signal phase calculation time.
2. The method of claim 1, further comprising:
calculating a difference value between the exit time and the entry time, and determining whether the difference value is within a set threshold range;
and if the difference is determined to be within the set threshold range, continuing to perform gesture recognition.
3. The method of claim 2, further comprising:
if the difference is determined not to be within the set threshold range, measuring the intensity variation of the returned signal of the TOF sensor;
when the signal intensity variation is a positive number, determining that the hand to be measured is close;
when the signal intensity variation is negative, determining that the hand to be measured is far away;
and when the number of times of the approaching/departing continuous same movement direction exceeds a preset effective number threshold value, determining as one effective gesture recognition.
4. The method according to claim 2, characterized in that before recording the entry moment of the hand to be measured into the TOF sensor and the exit moment of the hand to be measured out of the sensor, the method further comprises:
determining whether the signal strength is greater than a preset effective signal threshold;
and if the signal intensity is not larger than the preset effective signal threshold value, ignoring the gesture recognition.
5. The method according to any one of claims 1 to 4,
the TOF sensor comprises at least one first-class sensor and at least two second-class sensors, wherein the first-class sensor is a receiving-class sensor, the second-class sensor is an emitting-class sensor, or the first-class sensor is an emitting-class sensor, and the second-class sensor is a receiving-class sensor;
when the hand to be measured passes through the TOF sensor, the difference of the signal phase calculation time of the two second-class sensors with the X-axis distance is a horizontal motion offset angle d (X), and the difference of the signal phase calculation time of the two second-class sensors with the Y-axis distance is a vertical motion offset angle d (Y).
6. The method according to claim 5, wherein Ten (n) represents the entry time of the hand to be measured into the signal of the sensor n of the second type, Tex (n) represents the exit time of the hand to be measured from the signal of the sensor n of the second type, t (n) represents the phase calculation time of the signal of the sensor n, n is a positive integer,
calculating the signal phase calculation time according to the entry time and the exit time specifically comprises the following steps:
Figure FDA0002530812620000021
7. the method of claim 6, wherein: if the TOF sensor comprises one receive type sensor and three transmit type sensors,
calculating the motion offset angle of the to-be-measured hand relative to the central coordinate of the plane of the TOF sensing detection device according to the signal phase calculation time specifically as follows:
d(x)＝|t1-t2|；
d(y1)＝|t1-t3|；
d(y2)＝|t3-t2|；
determining the motion direction of the hand to be measured according to the motion deviation angle and the signal phase calculation time comprises:
when d (y1) > d (x)/2 and t (3) < t (1), or d (y2) > d (x)/2 and t (3) < t (2), it is determined that the hand to be measured is moving from bottom to top;
when d (y1) > d (x)/2 and t (3) > t (1), or d (y2) > d (x)/2 and t (3) > t (2), it is determined that the hand to be measured moves from top to bottom;
when d (x) > d (y1) + d (y2) and t (2) > t (1) and t (3) > t (1), it is determined that the hand to be measured is moving from left to right;
when d (x) > d (y1) + d (y2) and t (2) < t (1) and t (3) < t (1), it is determined that the hand to be measured moves from right to left.
8. A gesture recognition apparatus based on multi-point TOF, comprising:
the starting unit is used for starting the infrared time of flight TOF sensor;
the recording unit is used for recording the entering time when the hand to be measured enters the TOF sensor and the exiting time when the hand to be measured exits the TOF sensor;
the first calculating unit is used for calculating the signal phase calculating time according to the entering time and the exiting time recorded by the recording unit;
the second calculation unit is used for calculating the motion offset angle of the hand to be measured relative to the central coordinate of the plane where the TOF sensing detection device is located according to the signal phase calculation time calculated by the first calculation unit;
and the first determining unit is used for determining the motion direction of the hand to be measured according to the motion deviation angle and the signal phase calculation time calculated by the second calculating unit.
9. The apparatus of claim 8,
the TOF sensor comprises at least one first-class sensor and at least two second-class sensors, wherein the first-class sensor is a receiving-class sensor, the second-class sensor is an emitting-class sensor, or the first-class sensor is an emitting-class sensor, and the second-class sensor is a receiving-class sensor;
when the hand to be measured passes through the TOF sensor, the difference of the signal phase calculation time of the two second-class sensors with the X-axis distance is a horizontal motion offset angle d (X), and the difference of the signal phase calculation time of the two second-class sensors with the Y-axis distance is a vertical motion offset angle d (Y).
10. The apparatus of claim 9, wherein Ten (n) represents an entry time when the hand to be measured enters the signal of the sensor n of the second type, Tex (n) represents an exit time when the hand to be measured exits the signal of the sensor n of the second type, t (n) represents a phase calculation time of the signal of the sensor n, n is a positive integer,
calculating the signal phase calculation time according to the entry time and the exit time specifically comprises the following steps:
Figure FDA0002530812620000031
CN202010517916.XA 2020-06-09 2020-06-09 Gesture recognition method and device based on multi-point TOF Active CN111522446B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010517916.XA CN111522446B (en) 2020-06-09 2020-06-09 Gesture recognition method and device based on multi-point TOF

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010517916.XA CN111522446B (en) 2020-06-09 2020-06-09 Gesture recognition method and device based on multi-point TOF

Publications (2)

Publication Number Publication Date
CN111522446A true CN111522446A (en) 2020-08-11
CN111522446B CN111522446B (en) 2023-10-03

Family

ID=71911752

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010517916.XA Active CN111522446B (en) 2020-06-09 2020-06-09 Gesture recognition method and device based on multi-point TOF

Country Status (1)

Country Link
CN (1) CN111522446B (en)

Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090077504A1 (en) * 2007-09-14 2009-03-19 Matthew Bell Processing of Gesture-Based User Interactions
US20100064261A1 (en) * 2008-09-09 2010-03-11 Microsoft Corporation Portable electronic device with relative gesture recognition mode
CN102439538A (en) * 2009-12-29 2012-05-02 摩托罗拉移动公司 Electronic device with sensing assembly and method for interpreting offset gestures
US20150241978A1 (en) * 2014-02-27 2015-08-27 Motorola Mobility Llc Electronic Device Having Multiple Sides with Gesture Sensors
CN105204649A (en) * 2015-10-15 2015-12-30 广东欧珀移动通信有限公司 Method and device for unlocking mobile terminal
US20160140384A1 (en) * 2014-11-17 2016-05-19 Wistron Corporation Gesture recognition method and gesture recognition apparatus using the same
CN106839290A (en) * 2017-01-16 2017-06-13 广东美的制冷设备有限公司 The control method and control device and air-conditioner of gesture identification
CN109710321A (en) * 2018-12-14 2019-05-03 云丁网络技术(北京)有限公司 Awakening method, device and the electronic equipment of electronic equipment
CN110031827A (en) * 2019-04-15 2019-07-19 吉林大学 A kind of gesture identification method based on ultrasonic distance measurement principle
US20190384387A1 (en) * 2018-06-15 2019-12-19 Lc Technologies, Inc. Area-of-Interest (AOI) Control for Time-of-Flight (TOF) Sensors Used in Video Eyetrackers

Patent Citations (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090077504A1 (en) * 2007-09-14 2009-03-19 Matthew Bell Processing of Gesture-Based User Interactions
US20100064261A1 (en) * 2008-09-09 2010-03-11 Microsoft Corporation Portable electronic device with relative gesture recognition mode
CN102439538A (en) * 2009-12-29 2012-05-02 摩托罗拉移动公司 Electronic device with sensing assembly and method for interpreting offset gestures
US20150241978A1 (en) * 2014-02-27 2015-08-27 Motorola Mobility Llc Electronic Device Having Multiple Sides with Gesture Sensors
US20160140384A1 (en) * 2014-11-17 2016-05-19 Wistron Corporation Gesture recognition method and gesture recognition apparatus using the same
CN105204649A (en) * 2015-10-15 2015-12-30 广东欧珀移动通信有限公司 Method and device for unlocking mobile terminal
CN106839290A (en) * 2017-01-16 2017-06-13 广东美的制冷设备有限公司 The control method and control device and air-conditioner of gesture identification
US20190384387A1 (en) * 2018-06-15 2019-12-19 Lc Technologies, Inc. Area-of-Interest (AOI) Control for Time-of-Flight (TOF) Sensors Used in Video Eyetrackers
CN109710321A (en) * 2018-12-14 2019-05-03 云丁网络技术(北京)有限公司 Awakening method, device and the electronic equipment of electronic equipment
CN110031827A (en) * 2019-04-15 2019-07-19 吉林大学 A kind of gesture identification method based on ultrasonic distance measurement principle

Also Published As

Publication number Publication date
CN111522446B (en) 2023-10-03

Similar Documents

Publication Publication Date Title
CN106767852B (en) A kind of method, apparatus and equipment generating detection target information
JP6839778B2 (en) Gesture detection methods and devices on the user-referenced spatial coordinate system
CN109059902A (en) Relative pose determines method, apparatus, equipment and medium
CN110047100B (en) Depth information detection method, device and system
CN108958490A (en) Electronic device and its gesture identification method, computer readable storage medium
KR20200005999A (en) Slam method and slam system using dual event camaer
US20170150118A1 (en) Method and Apparatus for Gross-Level User and Input Detection Using Similar or Dissimilar Camera Pair
US20150261373A1 (en) Determining User Handedness and Orientation Using a Touchscreen Device
US9207779B2 (en) Method of recognizing contactless user interface motion and system there-of
JP5802247B2 (en) Information processing device
KR102057531B1 (en) Mobile devices of transmitting and receiving data using gesture
JP2011138519A5 (en)
CN103503030A (en) Image processing device for specifying depth of object present in real space by performing image processing, stereoscopic viewing device, integrated circuit, and program
KR20140088545A (en) Optical tablet stylus and indoor navigation system
CN108027648A (en) The gesture input method and wearable device of a kind of wearable device
CN110782517A (en) Point cloud marking method and device, storage medium and electronic equipment
US11899848B2 (en) Method, mobile device, head-mounted display, and system for estimating hand pose
CN103399629A (en) Method and device for capturing gesture displaying coordinates
CN112445326B (en) Projection interaction method based on TOF camera, system thereof and electronic equipment
CN111522446A (en) Gesture recognition method and device based on multipoint TOF
RU2602829C2 (en) Assessment of control criteria from remote control device with camera
US9652081B2 (en) Optical touch system, method of touch detection, and computer program product
CN109032354A (en) Electronic device and its gesture identification method, computer readable storage medium
CN113537331B (en) Train wheel set positioning method, device and storage medium
CN104216560A (en) Mobile device, as well as system and control device for realizing aerial touch control in mobile device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant