CN111522446B - Gesture recognition method and device based on multi-point TOF - Google Patents
Gesture recognition method and device based on multi-point TOF Download PDFInfo
- Publication number
- CN111522446B CN111522446B CN202010517916.XA CN202010517916A CN111522446B CN 111522446 B CN111522446 B CN 111522446B CN 202010517916 A CN202010517916 A CN 202010517916A CN 111522446 B CN111522446 B CN 111522446B
- Authority
- CN
- China
- Prior art keywords
- sensor
- hand
- measured
- time
- tof
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D30/00—Reducing energy consumption in communication networks
- Y02D30/70—Reducing energy consumption in communication networks in wireless communication networks
Landscapes
- Engineering & Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The application discloses a gesture recognition method and device based on multi-point TOF, and belongs to the technical field of information processing. The main technical scheme comprises the following steps: activating an infrared time-of-flight TOF sensor; recording the entering time of a hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the sensor, and calculating the signal phase calculating time according to the entering time and the exiting time; and calculating a motion offset angle of the hand to be measured relative to the center coordinate of the plane of the TOF sensing detection device according to the signal phase calculation moment, and determining the motion direction of the hand to be measured according to the motion offset angle and the signal phase calculation moment.
Description
Technical Field
The present application relates to the field of information processing technologies, and in particular, to a gesture recognition method and apparatus based on multi-point TOF.
Background
Along with the continuous development of social productivity and scientific technology, various industries pay more and more attention to gesture recognition methods in human-computer intelligent interaction scenes, such as intelligent interaction scenes, virtual reality interaction scenes, intelligent houses and the like, and gesture recognition can bring immersive control experience to operators under the scenes.
At present, common gesture recognition is realized on the basis of an image recognition algorithm, and although the method can realize control of gesture recognition, the method has higher requirements on the performance of a processor, has higher system power consumption and cost, and is not suitable for application scenes of sensitive cost, low power consumption and miniaturized equipment.
Disclosure of Invention
In view of the above, the embodiment of the application provides a gesture recognition method and device based on multi-point TOF, which mainly aims to realize accurate gesture recognition on the premise of unchanged cost.
In order to solve the above problems, the embodiment of the present application mainly provides the following technical solutions:
in a first aspect, an embodiment of the present application provides a gesture recognition method based on multi-point TOF, including:
activating an infrared time-of-flight TOF sensor;
recording the entering time of a hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the TOF sensor, and calculating the signal phase calculating time according to the entering time and the exiting time;
and calculating a motion offset angle of the hand to be measured relative to the center coordinate of the plane of the TOF sensor according to the signal phase calculation moment, and determining the motion direction of the hand to be measured according to the motion offset angle and the signal phase calculation moment.
Optionally, the method further comprises:
calculating a difference value between the exiting time and the entering time, and determining whether the difference value is within a set threshold range;
if the difference value is determined to be within the set threshold range, continuing gesture recognition.
Optionally, the method further comprises:
if the difference value is not in the set threshold range, measuring the change quantity of the intensity of the return signal of the TOF sensor;
when the signal intensity variation is positive, determining that the hand to be measured is close;
when the signal intensity variation is negative, determining that the hand to be measured is far away;
and determining that the gesture is recognized once when the number of times of approaching/separating continuous identical motion directions exceeds a preset effective number threshold.
Optionally, before recording the time of entry of the hand to be measured into the TOF sensor and the time of exit of the hand to be measured from the sensor, the method further comprises:
determining whether the signal strength is greater than a preset valid signal threshold;
and if the signal strength is not greater than the preset effective signal threshold, ignoring the gesture recognition.
Optionally, the TOF sensor includes at least one type of sensor and at least two types of sensors, where the type of sensor is a receiving type of sensor, the type of sensor is an emitting type of sensor, or the type of sensor is an emitting type of sensor, and the type of sensor is a receiving type of sensor;
when a hand to be measured passes through the TOF sensor, the difference value of signal phase calculation moments of two kinds of sensors with X-axis spacing is a horizontal movement offset angle d (X), and the difference value of signal phase calculation moments of two kinds of sensors with Y-axis spacing is a vertical movement offset angle d (Y);
ten (n) represents the entering time of the hand to be measured into the n signals of the two types of sensors, tex (n) represents the exiting time of the hand to be measured out of the n signals of the two types of sensors, t (n) represents the phase calculation time of the n signals of the sensors, and n is a positive integer.
Optionally, the calculating time of the signal phase according to the entering time and the exiting time specifically includes:
if the TOF sensor includes a receiving sensor and three transmitting sensors, calculating a motion offset angle of a hand to be measured relative to a center coordinate of a plane of the TOF sensor according to the signal phase calculation time specifically includes:
d(x)＝|t(1)-t(2)|；
d(y1)＝|t(1)-t(3)|；
d(y2)＝|t(3)-t(2)|。
optionally, determining the motion direction of the hand to be measured according to the motion offset angle and the signal phase calculation time includes:
when d (y 1) > d (x)/2 and t (3) < t (1), or d (y 2) > d (x)/2 and t (3) < t (2), determining that the hand to be measured is moving from bottom to top;
when d (y 1) > d (x)/2 and t (3) > t (1), or d (y 2) > d (x)/2 and t (3) > t (2), it is determined that the hand to be measured moves from top to bottom;
when d (x) > d (y 1) +d (y 2) and t (2) > t (1) and t (3) > t (1), it is determined that the hand to be measured moves from left to right;
when d (x) > d (y 1) +d (y 2) and t (2) < t (1) and t (3) < t (1), it is determined that the hand to be measured moves from right to left.
In a second aspect, an embodiment of the present application further provides a gesture recognition apparatus based on multi-point TOF, including:
the starting unit starts the infrared time-of-flight TOF sensor;
the recording unit is used for recording the entering time of the hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the sensor;
the first calculating unit is used for calculating signal phase calculating time according to the entering time and the exiting time recorded by the recording unit;
a second calculating unit, configured to calculate a motion offset angle of a hand to be measured relative to a center coordinate of a plane where the TOF sensor is located according to the signal phase calculating time calculated by the first calculating unit;
and the first determining unit is used for determining the movement direction of the hand to be measured according to the movement offset angle calculated by the second calculating unit and the signal phase calculating moment.
Optionally, the apparatus further includes:
a third calculation unit for calculating a difference between the exit time and the entry time;
a second determining unit configured to determine whether the difference calculated by the third calculating unit is within a set threshold range;
and the processing unit is used for continuing gesture recognition when the second determining unit determines that the difference value is within the set threshold range.
Optionally, the apparatus further includes:
a measuring unit configured to measure the TOF sensor return signal intensity variation when the second determining unit determines that the difference value is not within a set threshold range;
a third determining unit configured to determine that the hand to be measured is close when the signal intensity variation measured by the measuring unit is a positive number;
a fourth determining unit configured to determine that the hand to be measured is far away when the signal intensity variation measured by the measuring unit is negative;
and a fifth determining unit for determining that one valid gesture recognition is performed when the number of times of the approaching/separating continuous identical movement directions exceeds a preset valid number of times threshold.
Optionally, the apparatus further includes:
a sixth determining unit, configured to determine, before the recording unit records an entry time when a hand to be measured enters the TOF sensor and an exit time when the hand to be measured exits the sensor, whether a signal strength is greater than a preset valid signal threshold;
and the neglecting unit is used for neglecting the gesture recognition when the sixth determining unit determines that the signal intensity is not greater than the preset effective signal threshold value.
Alternatively to this, the method may comprise,
the TOF sensor comprises at least one type of sensor and at least two types of sensors, wherein the type of sensor is a receiving type of sensor, the type of sensor is an emitting type of sensor, or the type of sensor is an emitting type of sensor, and the type of sensor is a receiving type of sensor;
when a hand to be measured passes through the TOF sensor, the difference value of signal phase calculation moments of two kinds of sensors with X-axis spacing is a horizontal movement offset angle d (X), and the difference value of signal phase calculation moments of two kinds of sensors with Y-axis spacing is a vertical movement offset angle d (Y);
ten (n) represents the entering time of the hand to be measured into the n signals of the two types of sensors, tex (n) represents the exiting time of the hand to be measured out of the n signals of the two types of sensors, t (n) represents the phase calculation time of the n signals of the sensors, and n is a positive integer.
Optionally, the calculating time of the signal phase according to the entering time and the exiting time specifically includes:
if the TOF sensor includes a receiving sensor and three transmitting sensors, calculating a motion offset angle of a hand to be measured relative to a center coordinate of a plane of the TOF sensor according to the signal phase calculation time specifically includes:
d(x)＝|t(1)-t(2)|；
d(y1)＝|t(1)-t(3)|；
d(y2)＝|t(3)-t(2)|。
optionally, the first determining unit is further configured to determine that the hand to be measured is moving from bottom to top when d (y 1) > d (x)/2 and t (3) < t (1), or d (y 2) > d (x)/2 and t (3) < t (2);
when d (y 1) > d (x)/2 and t (3) > t (1), or d (y 2) > d (x)/2 and t (3) > t (2), it is determined that the hand to be measured moves from top to bottom;
when d (x) > d (y 1) +d (y 2) and t (2) > t (1) and t (3) > t (1), it is determined that the hand to be measured moves from left to right;
when d (x) > d (y 1) +d (y 2) and t (2) < t (1) and t (3) < t (1), it is determined that the hand to be measured moves from right to left.
By means of the technical scheme, the technical scheme provided by the embodiment of the application has at least the following advantages:
the gesture recognition method and device based on the multi-point TOF provided by the embodiment of the application start an infrared time-of-flight TOF sensor; recording the entering time of a hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the sensor, and calculating the signal phase calculating time according to the entering time and the exiting time; according to the signal phase calculation moment, the motion offset angle of the hand to be measured relative to the center coordinate of the plane of the TOF sensor is calculated, and the motion direction of the hand to be measured is determined according to the motion offset angle and the signal phase calculation moment.
The foregoing description is only an overview of the technical solutions of the embodiments of the present application, and may be implemented according to the content of the specification, so that the technical means of the embodiments of the present application can be more clearly understood, and the following specific implementation of the embodiments of the present application will be more apparent.
Drawings
Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The drawings are only for purposes of illustrating the preferred embodiments and are not to be construed as limiting the application. Also, like reference numerals are used to designate like parts throughout the figures. In the drawings:
FIG. 1 shows a flowchart of a gesture recognition method based on multi-point TOF according to an embodiment of the present application;
FIG. 2 shows a schematic diagram of a TOF sensor layout provided by an embodiment of the present application;
FIG. 3 shows a block diagram of an apparatus provided by an embodiment of the application;
FIG. 4 is a schematic diagram of a waveform signal according to an embodiment of the present application;
FIG. 5 is a schematic view of a motion offset angle provided by an embodiment of the present application;
FIG. 6 illustrates a schematic view of another motion offset angle provided by an embodiment of the present application;
FIG. 7 shows a block diagram of a gesture recognition apparatus based on multi-point TOF according to an embodiment of the present application;
FIG. 8 is a block diagram illustrating another gesture recognition apparatus based on multi-point TOF according to an embodiment of the present application
Fig. 9 shows a schematic diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
An embodiment of the present application provides a gesture recognition method based on multi-point TOF, as shown in FIG. 1, the method includes:
101. an infrared time-of-flight TOF sensor is activated.
The embodiment of the application adopts at least 1 single-point infrared flight (TOF) sensor to realize gesture recognition, and the TOF sensor comprises at least one type of sensor and at least two types of sensors, wherein the type of sensor is a receiving type of sensor, the type of sensor is an emitting type of sensor, or the type of sensor is an emitting type of sensor, and the type of sensor is a receiving type of sensor. The TOF sensors described herein refer to a pair of one receiving and transmitting, or one receiving and two transmitting, or a combination of one transmitting and two receiving, etc., and the TOF sensors do not work completely independently, and it is noted that the first and second sensors only differ in number, and their actual contents do not change.
Each TOF sensor comprises an infrared emission unit and an infrared receiving unit, and when the type II sensor is determined, the adjustable TOF sensor only starts part of the infrared emission units or the infrared receiving units. The working combination of one TOF sensor or a plurality of TOF sensors may include, but is not limited to, a combination of a plurality of infrared emitting units and one infrared receiving unit, or a combination of one infrared emitting unit and a plurality of infrared receiving units, etc., and the specific combination mode may be determined according to different application scenarios.
In the following embodiments, 1 TOF sensor, a combination of 3 infrared emitting units (two types of sensors) and 1 infrared receiving unit (one type of sensor) will be described as an example, but it should be noted that the description of the above method is not intended to limit the number and usage composition of the TOF sensors, but is merely an exemplary description, and the number and combination of other TOF sensors are consistent with the methods of the following embodiments.
The layout of the TOF sensor is shown in fig. 2, fig. 2 shows a schematic diagram of the layout of the TOF sensor provided by the embodiment of the application, and the layout scheme of the TOF sensor including three infrared emitting units and one infrared receiving unit in the diagram can be seen. The three infrared emission units are arranged in an inverted-delta shape, the infrared receiving units are arranged between the two parallel infrared emission units, the above is only an exemplary illustration, other sensor layout modes are also feasible in practical application, and the arrangement of the TOF sensors is the same as the specific implementation process of the algorithm (i.e. the algorithm design principle is the same).
The multiple infrared emission units in the embodiment of the application work in a time-sharing manner, for example, at time 1, the infrared emission unit 1 is started, and reflected infrared light signals are measured through the infrared receiving unit; at time 2, the infrared emission unit 2 is started, and reflected infrared light signals are measured through the infrared receiving unit; at time 3, the infrared emission unit 3 is started, and reflected infrared light signals are measured through the infrared receiving unit; and sequentially and circularly measuring infrared light signals reflected by the surface of the object emitted by each infrared emission unit. The distance between the hand to be measured and the infrared emission unit can be judged according to the received infrared light signal intensity. The time-sharing operation interval time of the infrared transmitting units is very short (microsecond), so that the influence of the time-sharing interval on the measurement result can be ignored. When the hand to be measured is close to the infrared emission unit, the intensity of the infrared signal reflected by the hand to be measured, which is received and measured, is high, and when the hand to be measured is far from the infrared emission unit, the intensity of the infrared signal reflected by the hand to be measured, which is received and measured, is low. Note that in the embodiment of the present application, the intensity of the received infrared signal is not used to implement distance measurement, the intensity of the received signal is only a reference quantity, and is used to determine the relative motion (far and near) between the hand to be measured and the infrared emission unit, and the gesture recognition algorithm is implemented by using multiple TOF sensors to receive the signal phases.
In practical application, the gesture recognition method based on multi-point TOF is carried in an electronic device, as shown in fig. 3, and mainly includes the following modules: the infrared receiving unit is used for receiving infrared light signals and measuring the intensity, and is generally realized by an infrared sensor, such as si1153, so that the intensity measurement from ultraviolet light, visible light to near infrared light can be realized; the infrared emission unit is used for realizing the emission of infrared light signals, and is generally realized by an infrared LED, the wavelength of the infrared emission unit can be selected to be 850nm or 940nm, and the emission angle is generally within 30 degrees for realizing remote measurement; the gesture recognition processing unit is used for realizing the processing of transmitting and receiving infrared signals, realizing a gesture recognition algorithm through echo signals of a plurality of received infrared transmitting signals, and can be realized by an MCU (micro controller unit), wherein the infrared receiving unit and the gesture recognition processing unit are generally connected by adopting an I2C interface; the communication unit is used for outputting an external interface of the system, such as detected gesture commands, and the communication unit generally selects an RS232 interface, an RS485 interface or a USB interface.
102. Recording the entering time of a hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the sensor, wherein the entering time is the starting time of the TOF sensor detecting the gesture made by the hand to be measured, the exiting time is the ending time of the TOF sensor detecting the gesture, and calculating the signal phase calculating time according to the entering time and the exiting time.
With continued reference to fig. 2, it is assumed that when the hand to be measured moves from left to right, the reflected signals from the sensors can be measured to obtain the waveform signal shown in fig. 4. Since the infrared emission unit 1 is at the leftmost side, the reflected signal of the infrared emission unit 1 is perceived to the hand first, then the infrared emission unit 3, and finally the infrared emission unit 2. Otherwise, if the hand moves from right to left, the phases of the waveforms of the received signals are just opposite, and the gesture calculation can be realized by measuring the phase relation of the waveforms through three sensors.
In order to more accurately calculate the phase difference of signals received by each sensor, the embodiment of the application adopts a median method to calculate the signal phase calculation time. I.e. the entry moment Ten (n) of the hand into each TOF sensor n signal, the exit moment Tex (n) of the hand to be measured out of the sensor n signals, t (n) representing the sensor n signal phase calculation moment, n being a positive integer.
The signal phase calculation time calculated according to the entering time and the exiting time is specifically:
103. and calculating a motion offset angle of the hand to be measured relative to the center coordinate of the plane of the TOF sensor according to the signal phase calculation moment, and determining the motion direction of the hand to be measured according to the motion offset angle and the signal phase calculation moment.
Since the center coordinates of the plane of the TOF sensor move along the X-axis (transverse direction) and along the Y-axis (longitudinal direction) simultaneously during the hand movement, the algorithm increases the calculation of the offset angle, when the TOF sensor through which the hand passes is to be measured in practical application, the difference between the signal phase calculation moments of the two types of sensors with the X-axis spacing is the horizontal movement offset angle d (X), the difference between the signal phase calculation moments of the two types of sensors with the Y-axis spacing is the vertical movement offset angle d (Y), and the d (X) and the d (Y) are performed according to the specific number of infrared emission units, if the two types of sensors have 3 as shown in fig. 2. In the above embodiment, since the two infrared emission units (1) and (2) are parallel, the motion offset angle between the infrared emission units (1) and (2) relative to the Y axis is 0, so the motion offset angles between the two types of sensors according to the embodiment of the present application relative to the Y axis are d (Y1) and d (Y2), respectively, as shown in fig. 5.
In the above embodiment, the infrared emission units (1) and (2) are used as parallel application scenes, and for the application scenes of which the infrared emission units (1) and (2) are not parallel, the motion offset angle d (Y3) between the infrared emission unit (1) and the infrared emission unit (2) relative to the Y axis and the motion offset angle d (X1) between the infrared emission unit (1) and the infrared emission unit (3) relative to the X axis are required to be calculated respectively, and the motion offset angle d (X3) between the infrared emission unit (2) and the infrared emission unit (3) relative to the X axis, namely, the motion offset angle between two pairs of sensors relative to the X axis or the Y axis is required to be calculated if any two sensors are not in parallel relation. And calculating and judging the direction of the gesture according to the X-axis movement offset angle and the Y-axis movement offset angle.
In the embodiment of the above steps, three infrared emitting units and one infrared receiving unit are still taken as examples for explanation, as shown in fig. 5, fig. 5 shows a schematic diagram of a motion offset angle provided by the embodiment of the present application, and d (x), d (y 1) and d (y 2) are used to implement motion offset angles of hands relative to the x axis and relative to the y axis, and whether the gesture is left or right or up or down is determined according to the magnitude relation of d (x), d (y 1) and d (y 2). In this case, the calculations of d (x), d (y 1), d (y 2) are detailed in the following formula:
d(x)＝|t(1)-t(2)|；
d(y1)＝|t(1)-t(3)|；
d(y2)＝|t(3)-t(2)|
when d (x) > d (y 1) +d (y 2) indicates that the gesture is a left-right motion, and when d (x) < d (y 1) +d (y 2) indicates that the gesture is an up-down motion.
The gesture recognition method and device based on the multi-point TOF and the electronic equipment provided by the embodiment of the application start at least three single-point infrared time-of-flight TOF sensors; recording the entering time of a hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the sensor, and calculating the signal phase calculating time according to the entering time and the exiting time; according to the signal phase calculation moment, the motion offset angle of the hand to be measured relative to the center coordinate of the plane of the TOF sensor is calculated, and the motion direction of the hand to be measured is determined according to the motion offset angle and the signal phase calculation moment.
The embodiment of the application also provides a method, namely, calculating the difference value between the exit time and the entry time, and determining whether the difference value is within a set threshold range; if the difference value is determined to be within the set threshold range, continuing gesture recognition; if the difference value is not in the set threshold range, measuring the change quantity of the intensity of the return signal of the TOF sensor; when the signal intensity variation is positive, determining that the hand to be measured is close; when the signal intensity variation is negative, determining that the hand to be measured is far away; and determining that the gesture is recognized once when the number of times of approaching/separating continuous identical motion directions exceeds a preset effective number threshold.
The method is used for identifying the application scene that the hand is far away from or near to the electronic equipment, and when the difference value of t (1), t (2) and t (3) is small, the intensity variation of the return signals of the three transmitting units is measured according to the following formula:
ds(n,t)＝s(n,t)-s(n,t-1)，
where ds (n, t) represents the variation of the return signal strength of the nth transmitting unit at time t with respect to time t-1, s (n, t) is the return signal strength of the nth transmitting unit at time t, and s (n, t-1) is the return signal strength of the nth transmitting unit at time t-1. When ds (n, t) is positive, it indicates approaching, and when ds (n, t) is negative, it indicates separating. To avoid interference, a continuous judgment is generally adopted, such as 3 continuous movements in the same direction are considered to be one effective movement. In addition, the judgment can be enhanced according to the consistency of the change directions of the return signal strengths of the three transmitting units. The foregoing number of consecutive times is merely illustrative, and may be adjusted in practical applications, for example, to 2 times or 4 times, etc., and the embodiment of the present application is not limited to a specific number of times.
The above embodiments describe the far or near recognition process in detail, and the following embodiments describe the gesture recognition in detail, specifically: determining the motion direction of the hand to be measured according to the motion offset angle and the signal phase computing moment comprises the following steps:
when d (y 1) > d (x)/2 and t (3) < t (1), or d (y 2) > d (x)/2 and t (3) < t (2), determining that the hand to be measured is moving from bottom to top;
when d (y 1) > d (x)/2 and t (3) > t (1), or d (y 2) > d (x)/2 and t (3) > t (2), it is determined that the hand to be measured moves from top to bottom;
when d (x) > d (y 1) +d (y 2) and t (2) > t (1) and t (3) > t (1), it is determined that the hand to be measured moves from left to right;
when d (x) > d (y 1) +d (y 2) and t (2) < t (1) and t (3) < t (1), it is determined that the hand to be measured moves from right to left.
In an actual application process, an application scenario of misoperation of a user may occur, such as touching an electronic device by mistake, in order to prevent occurrence of such application scenario, an embodiment of the present application further provides a method, before recording an entry time when a hand to be measured enters the TOF sensor and an exit time when the hand to be measured exits the sensor, the method further includes:
determining whether the signal strength is greater than a preset valid signal threshold; if the signal strength is not greater than (smaller than or equal to) the preset effective signal threshold, ignoring the gesture recognition; if the signal intensity is determined to be larger than the preset effective signal threshold, recording the entering time of the hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the sensor.
And uploading the identification result through a communication interface.
The above embodiments are all described by taking the case that the two-class sensor includes 3 infrared emission units as an example, and for the application scenario that the two-class sensor includes 4 or more infrared emission units, the method and the principle are the same as those of the calculation method and the principle that the two-class sensor includes 3 infrared emission units.
The following embodiments briefly describe an application scenario of one receiving class sensor and four transmitting class sensors. As shown in fig. 6, fig. 6 is a schematic view showing another motion offset angle provided by the embodiment of the present application, and the layout in the horizontal direction is still illustrated as an example in a parallel manner for convenience of description, but it should be noted that the description is not intended to limit the layout manner of each sensor. Different from the three infrared emission units, differences exist in calculating d (y), and the calculation method of the application scene is dy 1= |t1-t3|, dy 2= |t2-t4|, dx 1= |t1-t2|, dx 2= |t3-t4|.
In practical application, when four emission sensors exist, one sensor can be regarded as a redundant sensor, and the redundant sensor has the function of assisting judgment, and the addition of the auxiliary judgment can reduce the misjudgment rate, so that the dynamic gesture recognition result is more accurate.
Since the multi-point TOF-based gesture recognition apparatus described in the present embodiment is an apparatus capable of executing the multi-point TOF-based gesture recognition method in the embodiment of the present application, those skilled in the art will be able to understand the specific implementation of the multi-point TOF-based gesture recognition apparatus and various modifications thereof, so how the multi-point TOF-based gesture recognition apparatus implements the multi-point TOF-based gesture recognition method in the embodiment of the present application will not be described in detail herein. The device used by those skilled in the art to implement the gesture recognition method based on multi-point TOF according to the embodiments of the present application is within the scope of the present application.
The embodiment of the application also provides a gesture recognition device based on multi-point TOF, as shown in FIG. 7, which comprises:
a starting unit 21 for starting the infrared time-of-flight TOF sensor;
a recording unit 22, configured to record an entry time when a hand to be measured enters the TOF sensor and an exit time when the hand to be measured exits the TOF sensor;
a first calculating unit 23, configured to calculate a signal phase calculating time according to the entering time and the exiting time recorded by the recording unit;
a second calculation unit 24 for calculating a motion offset angle of the hand to be measured with respect to the center coordinate of the plane of the TOF sensor according to the signal phase calculation time calculated by the first calculation unit 23;
a first determining unit 25, configured to determine a movement direction of the hand to be measured according to the movement offset angle calculated by the second calculating unit 24 and the signal phase calculating time.
The gesture recognition device based on the multi-point TOF provided by the embodiment of the application starts at least three single-point infrared time-of-flight TOF sensors; recording the entering time of a hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the sensor, and calculating the signal phase calculating time according to the entering time and the exiting time; according to the signal phase calculation moment, the motion offset angle of the hand to be measured relative to the center coordinate of the plane of the TOF sensor is calculated, and the motion direction of the hand to be measured is determined according to the motion offset angle and the signal phase calculation moment.
Further, as shown in fig. 8, the apparatus further includes:
a third calculation unit 26 for calculating a difference between the exit time and the entry time;
a second determining unit 27 for determining whether the difference value calculated by the third calculating unit 26 is within a set threshold range;
and a processing unit 28, configured to continue gesture recognition when the second determining unit 27 determines that the difference is within the set threshold range.
Further, as shown in fig. 8, the apparatus further includes:
a measuring unit 29 for measuring the TOF sensor return signal intensity variation when the second determining unit 27 determines that the difference is not within a set threshold range;
a third determination unit 210 for determining that the hand to be measured is close when the signal intensity variation measured by the measurement unit 29 is a positive number;
a fourth determination unit 211 for determining that the hand to be measured is far away when the signal intensity variation measured by the measurement unit 29 is negative;
the fifth determining unit 212 is configured to determine that one valid gesture recognition is performed when the number of times of the consecutive identical movement directions approaching/moving away exceeds a preset valid number of times threshold.
Further, as shown in fig. 8, the apparatus further includes:
a sixth determining unit 213, configured to determine, before the recording unit 22 records an entry time when the hand to be measured enters the TOF sensor and an exit time when the hand to be measured exits the sensor, whether the signal strength is greater than a preset valid signal threshold;
and an ignoring unit 214, configured to ignore the gesture recognition when the sixth determining unit 213 determines that the signal strength is not greater than the preset valid signal threshold.
Further, the TOF sensor includes at least one type of sensor and at least two types of sensors, where the type of sensor is a receiving type of sensor, the type of sensor is an emitting type of sensor, or the type of sensor is an emitting type of sensor, and the type of sensor is a receiving type of sensor;
when a hand to be measured passes through the TOF sensor, the difference value of signal phase calculation moments of two kinds of sensors with X-axis spacing is a horizontal movement offset angle d (X), and the difference value of signal phase calculation moments of two kinds of sensors with Y-axis spacing is a vertical movement offset angle d (Y);
ten (n) represents the entering time of the hand to be measured into the n signals of the two types of sensors, tex (n) represents the exiting time of the hand to be measured out of the n signals of the two types of sensors, t (n) represents the phase calculation time of the n signals of the sensors, and n is a positive integer.
Further, the signal phase calculating time calculated according to the entering time and the exiting time is specifically:
if the TOF sensor includes a receiving sensor and three transmitting sensors, calculating a motion offset angle of a hand to be measured relative to a center coordinate of a plane of the TOF sensor according to the signal phase calculation time specifically includes:
d(x)＝|t(1)-t(2)|；
d(y1)＝|t(1)-t(3)|；
d(y2)＝|t(3)-t(2)|
further, the first determining unit is further configured to determine that the hand to be measured is moving from bottom to top when d (y 1) > d (x)/2 and t (3) < t (1), or d (y 2) > d (x)/2 and t (3) < t (2);
when d (y 1) > d (x)/2 and t (3) > t (1), or d (y 2) > d (x)/2 and t (3) > t (2), it is determined that the hand to be measured moves from top to bottom;
when d (x) > d (y 1) +d (y 2) and t (2) > t (1) and t (3) > t (1), it is determined that the hand to be measured moves from left to right;
when d (x) > d (y 1) +d (y 2) and t (2) < t (1) and t (3) < t (1), it is determined that the hand to be measured moves from right to left.
An embodiment of the present application provides an electronic device, as shown in fig. 9, including: at least one processor (processor) 31; and at least one memory (memory) 32, bus 33 connected to the processor 31; wherein, the liquid crystal display device comprises a liquid crystal display device,
the processor 31 and the memory 32 complete communication with each other through the bus 33;
the processor 31 is arranged to invoke program instructions in the memory 32 for performing the steps of the method embodiments described above.
The electronic equipment provided by the embodiment of the application starts at least three single-point infrared time-of-flight TOF sensors; recording the entering time of a hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the sensor, and calculating the signal phase calculating time according to the entering time and the exiting time; according to the signal phase calculation moment, the motion offset angle of the hand to be measured relative to the center coordinate of the plane of the TOF sensor is calculated, and the motion direction of the hand to be measured is determined according to the motion offset angle and the signal phase calculation moment.
The present embodiment provides a non-transitory computer-readable storage medium storing computer instructions that cause a computer to perform the methods provided by the above-described method embodiments.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each flow and/or block of the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
In one typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory.
The memory may include volatile memory in a computer-readable medium, random Access Memory (RAM) and/or nonvolatile memory, etc., such as Read Only Memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.
Computer readable media, including both non-transitory and non-transitory, removable and non-removable media, may implement information storage by any method or technology. The information may be computer readable instructions, data structures, modules of a program, or other data. Examples of storage media for a computer include, but are not limited to, phase change memory (PRAM), static Random Access Memory (SRAM), dynamic Random Access Memory (DRAM), other types of Random Access Memory (RAM), read Only Memory (ROM), electrically Erasable Programmable Read Only Memory (EEPROM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital Versatile Discs (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices, or any other non-transmission medium, which can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, does not include transitory computer-readable media (transmission media), such as modulated data signals and carrier waves.
It should also be noted that the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article or apparatus that comprises an element.
It will be appreciated by those skilled in the art that embodiments of the present application may be provided as a method, system, or computer program product. Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, and the like) having computer-usable program code embodied therein.
The foregoing is merely exemplary of the present application and is not intended to limit the present application. Various modifications and variations of the present application will be apparent to those skilled in the art. Any modification, equivalent replacement, improvement, etc. which come within the spirit and principles of the application are to be included in the scope of the claims of the present application.
Claims (5)
1. A multi-point TOF-based gesture recognition method, comprising:
activating an infrared time-of-flight TOF sensor;
recording the entering time of a hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the TOF sensor, and calculating signal phase calculating time according to the entering time and the exiting time;
calculating a motion offset angle of a hand to be measured relative to the center coordinate of the plane of the TOF sensor according to the signal phase calculation moment, and determining the motion direction of the hand to be measured according to the motion offset angle and the signal phase calculation moment;
the TOF sensor comprises at least one type of sensor and at least two types of sensors, wherein the type of sensor is a receiving type of sensor, the type of sensor is an emitting type of sensor, or the type of sensor is an emitting type of sensor, and the type of sensor is a receiving type of sensor;
when a hand to be measured passes through the TOF sensor, the difference value of signal phase calculation moments of two kinds of sensors with X-axis spacing is a horizontal movement offset angle d (X), and the difference value of signal phase calculation moments of two kinds of sensors with Y-axis spacing is a vertical movement offset angle d (Y);
ten (n) represents the entering time of the hand to be measured into the n signals of the two types of sensors, tex (n) represents the exiting time of the hand to be measured out of the n signals of the two types of sensors, t (n) represents the phase calculation time of the n signals of the sensors, n is a positive integer,
the signal phase calculation time calculated according to the entering time and the exiting time is specifically:
if the TOF sensor comprises one receiving class sensor and three transmitting class sensors,
the motion offset angle of the hand to be measured relative to the center coordinate of the plane of the TOF sensor is calculated according to the signal phase calculation time, specifically:
d(x)＝|t(1)-t(2)|；
d(y1)＝|t(1)-t(3)|；
d(y2)＝|t(3)-t(2)|；
determining the motion direction of the hand to be measured according to the motion offset angle and the signal phase computing moment comprises the following steps:
when d (y 1) > d (x)/2 and t (3) < t (1), or d (y 2) > d (x)/2 and t (3) < t (2), determining that the hand to be measured is moving from bottom to top;
when d (y 1) > d (x)/2 and t (3) > t (1), or d (y 2) > d (x)/2 and t (3) > t (2), it is determined that the hand to be measured moves from top to bottom;
when d (x) > d (y 1) +d (y 2) and t (2) > t (1) and t (3) > t (1), it is determined that the hand to be measured moves from left to right;
when d (x) > d (y 1) +d (y 2) and t (2) < t (1) and t (3) < t (1), it is determined that the hand to be measured moves from right to left.
2. The method according to claim 1, wherein the method further comprises:
calculating a difference value between the exiting time and the entering time, and determining whether the difference value is within a set threshold range;
if the difference value is determined to be within the set threshold range, continuing gesture recognition.
3. The method according to claim 2, wherein the method further comprises:
if the difference value is not in the set threshold range, measuring the change quantity of the intensity of the return signal of the TOF sensor;
when the signal intensity variation is positive, determining that the hand to be measured is close;
when the signal intensity variation is negative, determining that the hand to be measured is far away;
and determining that the gesture is recognized once when the number of times of approaching/separating continuous identical motion directions exceeds a preset effective number threshold.
4. The method of claim 2, wherein prior to recording the entry time of a hand to be measured into the TOF sensor and the exit time of the hand to be measured out of the sensor, the method further comprises:
determining whether the signal strength is greater than a preset valid signal threshold;
and if the signal strength is not greater than the preset effective signal threshold, ignoring the gesture recognition.
5. A multi-point TOF-based gesture recognition apparatus, comprising:
the starting unit starts the infrared time-of-flight TOF sensor;
the recording unit is used for recording the entering time of the hand to be measured entering the TOF sensor and the exiting time of the hand to be measured exiting the sensor;
the first calculating unit is used for calculating signal phase calculating time according to the entering time and the exiting time recorded by the recording unit;
a second calculating unit, configured to calculate a motion offset angle of a hand to be measured relative to a center coordinate of a plane where the TOF sensor is located according to the signal phase calculating time calculated by the first calculating unit;
the first determining unit is used for determining the movement direction of the hand to be measured according to the movement offset angle calculated by the second calculating unit and the signal phase calculating moment;
the TOF sensor comprises at least one type of sensor and at least two types of sensors, wherein the type of sensor is a receiving type of sensor, the type of sensor is an emitting type of sensor, or the type of sensor is an emitting type of sensor, and the type of sensor is a receiving type of sensor;
when a hand to be measured passes through the TOF sensor, the difference value of signal phase calculation moments of two kinds of sensors with X-axis spacing is a horizontal movement offset angle d (X), and the difference value of signal phase calculation moments of two kinds of sensors with Y-axis spacing is a vertical movement offset angle d (Y);
ten (n) represents the entering time of the hand to be measured into the n signals of the two types of sensors, tex (n) represents the exiting time of the hand to be measured out of the n signals of the two types of sensors, t (n) represents the phase calculation time of the n signals of the sensors, n is a positive integer,
the signal phase calculation time calculated according to the entering time and the exiting time is specifically:
if the TOF sensor includes a receiving sensor and three transmitting sensors, calculating a motion offset angle of a hand to be measured relative to a center coordinate of a plane of the TOF sensor according to the signal phase calculation time specifically includes:
d(x)＝|t(1)-t(2)|；
d(y1)＝|t(1)-t(3)|；
d(y2)＝|t(3)-t(2)|；
the first determining unit is further used for determining that the hand to be measured moves from bottom to top when d (y 1) > d (x)/2 and t (3) < t (1), or d (y 2) > d (x)/2 and t (3) < t (2);
when d (y 1) > d (x)/2 and t (3) > t (1), or d (y 2) > d (x)/2 and t (3) > t (2), it is determined that the hand to be measured moves from top to bottom;
when d (x) > d (y 1) +d (y 2) and t (2) > t (1) and t (3) > t (1), it is determined that the hand to be measured moves from left to right;
when d (x) > d (y 1) +d (y 2) and t (2) < t (1) and t (3) < t (1), it is determined that the hand to be measured moves from right to left.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010517916.XA CN111522446B (en) | 2020-06-09 | 2020-06-09 | Gesture recognition method and device based on multi-point TOF |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010517916.XA CN111522446B (en) | 2020-06-09 | 2020-06-09 | Gesture recognition method and device based on multi-point TOF |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111522446A CN111522446A (en) | 2020-08-11 |
| CN111522446B true CN111522446B (en) | 2023-10-03 |
Family
ID=71911752
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010517916.XA Active CN111522446B (en) | 2020-06-09 | 2020-06-09 | Gesture recognition method and device based on multi-point TOF |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111522446B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102439538A (en) * | 2009-12-29 | 2012-05-02 | 摩托罗拉移动公司 | Electronic device with sensing assembly and method for interpreting offset gestures |
| CN105204649A (en) * | 2015-10-15 | 2015-12-30 | 广东欧珀移动通信有限公司 | Method and device for unlocking mobile terminal |
| CN106839290A (en) * | 2017-01-16 | 2017-06-13 | 广东美的制冷设备有限公司 | The control method and control device and air-conditioner of gesture identification |
| CN109710321A (en) * | 2018-12-14 | 2019-05-03 | 云丁网络技术(北京)有限公司 | Awakening method, device and the electronic equipment of electronic equipment |
| CN110031827A (en) * | 2019-04-15 | 2019-07-19 | 吉林大学 | A kind of gesture identification method based on ultrasonic distance measurement principle |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2009035705A1 (en) * | 2007-09-14 | 2009-03-19 | Reactrix Systems, Inc. | Processing of gesture-based user interactions |
| US20100064261A1 (en) * | 2008-09-09 | 2010-03-11 | Microsoft Corporation | Portable electronic device with relative gesture recognition mode |
| US20150241978A1 (en) * | 2014-02-27 | 2015-08-27 | Motorola Mobility Llc | Electronic Device Having Multiple Sides with Gesture Sensors |
| TWI540462B (en) * | 2014-11-17 | 2016-07-01 | 緯創資通股份有限公司 | Gesture recognition method and electronic apparatus using the same |
| US20190384387A1 (en) * | 2018-06-15 | 2019-12-19 | Lc Technologies, Inc. | Area-of-Interest (AOI) Control for Time-of-Flight (TOF) Sensors Used in Video Eyetrackers |
-
2020
- 2020-06-09 CN CN202010517916.XA patent/CN111522446B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102439538A (en) * | 2009-12-29 | 2012-05-02 | 摩托罗拉移动公司 | Electronic device with sensing assembly and method for interpreting offset gestures |
| CN105204649A (en) * | 2015-10-15 | 2015-12-30 | 广东欧珀移动通信有限公司 | Method and device for unlocking mobile terminal |
| CN106839290A (en) * | 2017-01-16 | 2017-06-13 | 广东美的制冷设备有限公司 | The control method and control device and air-conditioner of gesture identification |
| CN109710321A (en) * | 2018-12-14 | 2019-05-03 | 云丁网络技术(北京)有限公司 | Awakening method, device and the electronic equipment of electronic equipment |
| CN110031827A (en) * | 2019-04-15 | 2019-07-19 | 吉林大学 | A kind of gesture identification method based on ultrasonic distance measurement principle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111522446A (en) | 2020-08-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Wasenmüller et al. | Comparison of kinect v1 and v2 depth images in terms of accuracy and precision | |
| US10846864B2 (en) | Method and apparatus for detecting gesture in user-based spatial coordinate system | |
| US20160274732A1 (en) | Touchless user interfaces for electronic devices | |
| CN102508578B (en) | Projection positioning device and method as well as interaction system and method | |
| US9207779B2 (en) | Method of recognizing contactless user interface motion and system there-of | |
| CN108958490A (en) | Electronic device and its gesture identification method, computer readable storage medium | |
| US20120139907A1 (en) | 3 dimensional (3d) display system of responding to user motion and user interface for the 3d display system | |
| KR102057531B1 (en) | Mobile devices of transmitting and receiving data using gesture | |
| CN103970264A (en) | Gesture recognition and control method and device | |
| CN103677240A (en) | Virtual touch interaction method and equipment | |
| CN112015178A (en) | Control method, device, equipment and storage medium | |
| CN112214118A (en) | Touch pen, control method thereof and electronic equipment | |
| TW201911024A (en) | Electronic device, method and system for detecting fingers and non-transitory computer-readable medium | |
| CN104112112A (en) | Action identification method and device | |
| CN111522446B (en) | Gesture recognition method and device based on multi-point TOF | |
| CN102004583A (en) | Camera positioning touch device and method thereof | |
| US20130155313A1 (en) | Electronic device and method for focusing and measuring points of objects | |
| US9652081B2 (en) | Optical touch system, method of touch detection, and computer program product | |
| CN110837295A (en) | Handheld control equipment and tracking and positioning method, equipment and system thereof | |
| CN112416115A (en) | Method and equipment for man-machine interaction in control interaction interface | |
| KR20160140324A (en) | Touch recognition apparatus and control methods thereof | |
| CN109032354A (en) | Electronic device and its gesture identification method, computer readable storage medium | |
| US20170220133A1 (en) | Accurately positioning instruments | |
| CN103439127A (en) | Inertia inputting device motion mass measuring method, terminal and system | |
| US20150042621A1 (en) | Method and apparatus for controlling 3d object |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |