WO2018148951A1 - Robot control method and device based on gesture recognition, and robot - Google Patents

Abstract

Disclosed is a robot control method based on gesture recognition, comprising: when a distance measurement sensor detects that there is an object within a pre-set range of a robot, starting a hand-waving gesture recognition operation; after the hand-waving gesture recognition operation is started, if the distance measurement sensor currently detects that the duration for which the object is within the pre-set range of the robot is within a pre-set value range, recording one instance of a hand-waving operation; if the distance measurement sensor currently detects that the duration for which the object is within the pre-set range of the robot is not within the pre-set value range, or a time interval between the time when the distance measurement sensor currently detects that there is an object within the pre-set range of the robot and the time when the distance measurement sensor next detects that there is an object within the pre-set range of the robot is greater than a threshold value, stopping the hand-waving gesture recognition operation; and when the hand-waving gesture recognition operation is stopped, executing a corresponding operation on the robot according to the recorded number of hand-waving operations and a pre-set mapping table.

Description

Robot control method, device and robot based on gesture recognition Technical field

The invention relates to the technical field of smart home appliances, in particular to a robot control method, a control device and a robot based on gesture recognition.

Background technique

The currently used home robots usually require the user to bend over to press a button or use a mobile phone or a remote control to control. These modes of operation are not very convenient for the user to use, and the user experience of the product cannot be improved.

Summary of the invention

An object of the embodiments of the present invention is to provide a robot control method, a control device, and a robot based on gesture recognition, which can perform corresponding operations on the robot based on the gesture recognition of the user, facilitate user use, improve user experience of the product, and have low cost. .

To achieve the above objective, an embodiment of the present invention provides a robot control method based on gesture recognition, including the steps of:

When it is detected by the ranging sensor that an object exists within a preset range of the robot, the waving gesture recognition operation is initiated;

After the wave gesture recognition operation is initiated, if the ranging sensor currently detects that the duration of the object within the preset range of the robot is within a preset value range, recording is a one-shot operation; if the ranging sensor is currently Detecting that the duration of the object exists within the preset range of the robot is not within a preset value range, or the ranging sensor currently detects that an object exists within the preset range of the robot to the next time the ranging sensor is detected Stopping the waving gesture recognition operation when the time interval of the object exists in the preset range of the robot is greater than a threshold;

When the waving gesture recognition operation is stopped, a corresponding operation is performed on the robot according to the recorded number of the wave operation and the preset mapping table; wherein the mapping table records the correspondence between the number of wave operations and the execution operation.

As an improvement of the above solution, when the detection of the presence of an object within the preset range of the robot by the ranging sensor initiates the waving gesture recognition operation, the method further includes the steps of:

The user is prompted to perform a wave operation by controlling the robot to illuminate and/or vocalize.

As an improvement of the above scheme, the preset range is 50 cm to 120 cm.

As a modification of the above solution, the ranging sensor is a laser ranging sensor, an ultrasonic ranging sensor, or an infrared ranging sensor.

As an improvement of the above solution, after the wave gesture recognition operation is started, it is determined that the ranging sensor currently detects that an object exists in the preset range of the robot to the ranging sensor, and the next preset of the robot is detected by the following method. The time interval between objects in the range is greater than the threshold:

After the wave gesture recognition operation is started, when the ranging sensor detects for the first time that an object exists in the preset range of the robot, the first control signal is sent to start the first timer to start timing;

After the wave gesture recognition operation is initiated, each time the range sensor is detected within the preset range of the robot except for the first time When an object exists, a second control signal is issued to control the first timer to re-time;

When the timing time when the first timer receives the second control signal is greater than a threshold, determining that the ranging sensor currently detects that an object exists in the preset range of the robot to the ranging sensor next time It is detected that the time interval in which the object exists within the preset range of the robot is greater than the threshold.

As an improvement of the above solution, after the wave gesture recognition operation is started, it is determined by the following method that the ranging sensor currently detects whether the duration of the object existing in the preset range of the robot is within a preset value range:

After the wave gesture recognition operation is initiated, when the ranging sensor instantaneously detects that an object exists within the preset range of the robot, a third control signal is issued to start the second timer to start timing;

After starting the second timer to start timing, when the ranging sensor does not detect that an object exists within the preset range of the robot, the fourth control signal is issued to stop the second timer timing;

When the timing time when the second timer receives the fourth control signal falls within the preset value range, determining that the ranging sensor currently detects the presence of an object within a preset range of the robot The time is within the preset value range; otherwise, it is judged that the ranging sensor currently detects that the duration of the object existing in the preset range of the robot is not within the preset value range.

As an improvement of the above solution, the preset value range is greater than the threshold.

As a modification of the above solution, the ranging sensor is located at the top of the robot.

As an improvement of the above scheme, performing the corresponding operation on the robot includes starting work or stopping work.

As an improvement of the above scheme, the waving gesture recognition operation is started/stopped by a timing command, a control command, or a switch command.

The embodiment of the invention provides a robot control device based on gesture recognition, which comprises:

a waving gesture recognition operation starting module, configured to initiate a waving gesture recognition operation when detecting that an object exists within a preset range of the robot by the ranging sensor;

a wave operation judgment module, configured to record a wave operation if the distance sensor currently detects that the duration of the object within the preset range of the robot is within a preset value range after the wave gesture recognition operation is initiated; If the ranging sensor currently detects that the duration of the object existing in the preset range of the robot is not within the preset value range, or the ranging sensor currently detects that an object exists in the preset range of the robot to the measurement Stopping the waving gesture recognition operation when the distance sensor detects that the time interval of the object existing in the preset range of the robot is greater than the threshold value;

An execution module, configured to perform a corresponding operation on the robot according to the recorded number of wave operations and a preset mapping table when the waving gesture recognition operation is stopped; wherein the mapping table records the number of wave operations and the operation Correspondence relationship.

As an improvement of the above solution, the waving gesture recognition operation starting module is further configured to: when the wave gesture recognition operation is initiated by detecting that an object exists within a preset range of the robot by the ranging sensor, by controlling the robot to emit light and/or sound To prompt the user to wave.

As an improvement of the above scheme, the preset range is 50 cm to 120 cm.

As a modification of the above solution, the ranging sensor is a laser ranging sensor, an ultrasonic ranging sensor, or an infrared ranging sensor.

As an improvement of the above solution, the wave operation judgment module includes:

a first control signal generating unit, configured to detect the first time in the machine when the range sensor is activated after the wave gesture recognition operation is initiated When there is an object in the preset range of the person, the first control signal is sent to start the first timer to start timing;

a second control signal generating unit, configured to send a second control signal to control each time the ranging sensor detects an object within a preset range of the robot, except for the first time after the wave gesture recognition operation is initiated. The first timer is retimed;

a first determining unit, when the time when the first timer receives the second control signal is greater than a threshold, determining that the ranging sensor currently detects that an object exists in the preset range of the robot to the The next time the ranging sensor detects that there is an object within the preset range of the robot, the time interval is greater than the threshold.

As an improvement of the above solution, the wave operation judgment module further includes:

a third control signal generating unit, configured to: after the triggering gesture recognition operation is started, when the ranging sensor detects that an object exists within a preset range of the robot, issue a third control signal to start the second timer to start timing ;

a fourth control signal generating unit, configured to: after the start of the second timer starts timing, when the ranging sensor does not detect that an object exists in the preset range of the robot, the fourth control signal is sent to stop the first Two timers;

a second determining unit, configured to: when the timing time when the second timer receives the fourth control signal falls within the preset value range, determine that the ranging sensor currently detects a pre-mouse in the robot It is assumed that the duration of the object exists within the preset value range; otherwise, it is determined that the ranging sensor currently detects that the duration of the object exists within the preset range of the robot is not within the preset value range.

As an improvement of the above solution, the preset value range is greater than the threshold.

As a modification of the above solution, the ranging sensor is located at the top of the robot.

As an improvement of the above scheme, performing the corresponding operation on the robot includes starting work or stopping work.

As an improvement of the above aspect, the wave operation judgment module starts/stops the wave gesture recognition operation by issuing a timing command, a control command, or a switch command.

Compared with the prior art, the gesture recognition-based robot control method and the control device disclosed by the present invention detect that an object exists within a preset range of the robot by the ranging sensor, and then initiate a wave gesture recognition operation, and start waving After the gesture recognition operation, if the ranging sensor currently detects that the duration of the object existing in the preset range of the robot is within a preset value range, recording is a one-wave operation; if the ranging sensor is currently detected The duration of the object in the preset range of the robot is not within the preset value range, or the ranging sensor currently detects that an object exists in the preset range of the robot to the next time the ranging sensor detects the robot in advance When the time interval in which the object exists in the range is greater than the threshold, the waving gesture recognition operation is stopped; and when the waving gesture recognition operation is stopped, the corresponding execution is performed on the robot according to the recorded number of the wave operation and the preset mapping table. An operation; wherein the mapping table records a correspondence between the number of wave operations and the execution of the operation. Thereby, the corresponding operation (for example, starting work or stopping the work operation) can be performed on the robot based on the gesture recognition of the user, without the user bending over the button or using the mobile phone and the remote controller to control, facilitating the user to use, and improving the user's product. Experience.

The embodiment of the invention further provides a robot control method based on gesture recognition, comprising the steps of:

When it is recognized by the camera that there is an object within the shooting range of the robot, the waving gesture recognition operation is initiated;

After the wave gesture recognition operation is initiated, if the camera currently recognizes that the duration of the object within the shooting range of the robot is within a preset value range, recording is a one-shot operation; if the camera is currently recognized in the robot There is something in the shooting range The duration of the body is not within the preset value range, or the camera currently recognizes that there is an object within the shooting range of the robot until the next time the camera recognizes that the time interval exists within the shooting range of the robot is greater than a threshold, stopping The waving gesture recognition operation;

When the waving gesture recognition operation is stopped, a corresponding operation is performed on the robot according to the recorded number of the wave operation and the preset mapping table; wherein the mapping table records the correspondence between the number of wave operations and the execution operation.

As an improvement of the above solution, when the waving gesture recognition operation is initiated by the camera recognizing that an object exists within the shooting range of the robot, the method further includes the steps of:

The user is prompted to perform a wave operation by controlling the robot to illuminate and/or vocalize.

As an improvement of the above solution, after the wave gesture recognition operation is started, it is determined by the following method that the camera currently recognizes that the object exists within the shooting range of the robot and the time when the camera next recognizes that the object exists within the shooting range of the robot. The interval is greater than the threshold:

Based on the photographing frequency of the camera, by judging that the camera recognizes an instant from the presence or absence of an object in the shooting range of the robot, the camera recognizes that the object is present from nothing in the shooting range of the robot. The number of frames to calculate the time interval;

When the time interval is greater than the threshold, it is determined that the camera currently recognizes that the time interval between the object in the shooting range of the robot and the next time the camera recognizes that the object exists in the shooting range of the robot is greater than a threshold.

As an improvement of the above solution, after the wave gesture recognition operation is started, it is determined by the following method that the camera currently recognizes whether the duration of the object existing in the shooting range of the robot is within a preset value range:

Based on the shooting frequency of the camera, by determining that the camera recognizes that there is an instant of existence of the object within the preset range of the robot, the camera recognizes that there is an instant from the presence or absence of the object within the preset range of the robot. The number of frames passed to calculate the duration;

When the duration falls within the preset value range, determining that the ranging sensor currently detects that the duration of the object exists within the preset range of the robot is within a preset value range; otherwise determining the ranging The sensor currently detects that the duration of the object within the preset range of the robot is not within the preset range.

As an improvement of the above solution, it is possible to identify whether an object exists within the shooting range of the camera by:

When the robot is in a stopped state, the camera takes a picture according to the preset frequency, and compares the two photos before and after the camera is continuously shot. If the matching degree of the two photos before and after is greater than the set value, it is recognized that there is no object in the shooting range of the robot; If the matching degree of the two photos before and after is less than the set value, it is recognized that there is an object in the shooting range of the robot;

When the robot is in motion, the camera takes a picture according to the preset frequency, and compares the two photos before and after the continuous shooting of the camera. If the two photos are only slightly different from each other, it is recognized that there is no object in the shooting range of the robot; If there are not only translational differences between the two front and back, but also there is an instantaneous difference between the objects, it is recognized that there is an object in the shooting range of the robot.

As an improvement of the above scheme, the camera has a shooting frequency of 10 to 50 frames/second.

As an improvement of the above scheme, the camera has a shooting frequency of 20 to 30 frames/second.

As an improvement of the above solution, the preset value range is greater than the threshold.

As an improvement of the above solution, the camera is located at the top of the robot.

As an improvement of the above scheme, performing the corresponding operation on the robot includes starting work or stopping work.

As an improvement of the above scheme, the waving gesture recognition operation is started/stopped by a timing command, a control command, or a switch command.

The embodiment of the invention provides a robot control device based on gesture recognition, which comprises:

a waving gesture recognition operation starting module, configured to activate a waving gesture recognition operation when it is recognized by the camera that an object exists within a shooting range of the robot;

a wave operation judgment module, configured to record a wave operation if the camera currently recognizes that the duration of the object within the shooting range of the robot is within a preset value range after the wave gesture recognition operation is initiated; The camera currently recognizes that the duration of the object within the shooting range of the robot is not within the preset value range, or the camera currently recognizes that an object exists within the shooting range of the robot until the camera next recognizes the shooting range of the robot Stopping the waving gesture recognition operation when the time interval of the object in the object is greater than a threshold;

An execution module, configured to perform a corresponding operation on the robot according to the recorded number of wave operations and a preset mapping table when the waving gesture recognition operation is stopped; wherein the mapping table records the number of wave operations and the operation Correspondence relationship.

As an improvement of the above solution, the waving gesture recognition operation starting module is further configured to: when the gesture recognition operation is started by the camera to recognize that an object exists in the shooting range of the robot, prompting the user by controlling the robot to emit light and/or sounding Wave the operation.

As an improvement of the above solution, the wave operation judgment module includes:

a first frame number counting unit, based on the shooting frequency of the camera, by determining that the camera recognizes that there is no object in the shooting range of the robot, and the camera recognizes that the camera is within the shooting range of the robot Calculate the time interval by the number of frames through which the object exists;

a first determining unit, configured to: when the time interval is greater than a threshold, determine that the camera currently recognizes that an object exists in the shooting range of the robot until the camera next recognizes that an object exists in the shooting range of the robot The interval is greater than the threshold.

As an improvement of the above solution, the wave operation judgment module further includes:

a second frame number counting unit, based on the photographing frequency of the camera, by determining that the camera recognizes that there is an existing object in the shooting range of the robot, and the camera recognizes that the camera is within the shooting range of the robot Calculate the duration without the number of frames passed by the existing object;

a second determining unit, when the duration falls within the preset value range, determining that the ranging sensor currently detects that the duration of the object existing in the shooting range of the robot is within a preset value range; otherwise, determining The ranging sensor currently detects that the duration of the object within the shooting range of the robot is not within a preset value range.

As an improvement of the above solution, the waving gesture recognition operation starting module identifies whether an object exists in the shooting range of the camera by:

When the robot is in a stopped state, the camera takes a picture according to the preset frequency, and compares the two photos before and after the camera is continuously shot. If the matching degree of the two photos before and after is greater than the set value, it is recognized that there is no object in the shooting range of the robot; If the matching degree of the two photos before and after is less than the set value, it is recognized that there is an object in the shooting range of the robot;

When the robot is in motion, the camera takes a picture according to the preset frequency, and compares the two photos before and after the continuous shooting of the camera. If the two photos are only slightly different from each other, it is recognized that there is no object in the shooting range of the robot; If there are not only translational differences between the two front and back, but also there is an instantaneous difference between the objects, it is recognized that there is an object in the shooting range of the robot.

As an improvement of the above scheme, the camera has a shooting frequency of 10 to 50 frames/second.

As an improvement of the above scheme, the camera has a shooting frequency of 20 to 30 frames/second.

As an improvement of the above solution, the preset value range is greater than the threshold.

As an improvement of the above solution, the camera is located at the top of the robot.

As an improvement of the above scheme, performing the corresponding operation on the robot includes starting work or stopping work.

As an improvement of the above aspect, the wave operation judgment module starts/stops the wave gesture recognition operation by issuing a timing command, a control command, or a switch command.

Compared with the prior art, the gesture recognition-based robot control method and the control device disclosed by the present invention recognizes that there is an object in the shooting range of the robot through the camera, and then initiates a wave gesture recognition operation, and initiates a wave gesture recognition operation. Thereafter, if the camera currently recognizes that the duration of the object within the shooting range of the robot is within a preset value range, recording is a one-shot operation; if the camera currently recognizes that an object exists within the shooting range of the robot The duration is not within the preset value range, or the camera currently recognizes that there is an object within the shooting range of the robot until the next time the camera recognizes that the time interval exists within the shooting range of the robot is greater than a threshold, stopping the a wave gesture recognition operation; and when the wave gesture recognition operation is stopped, performing a corresponding operation on the robot according to the recorded number of wave operation operations and a preset mapping table; wherein the mapping table records the number of wave operations and performs an operation Correspondence. Therefore, it is realized that the corresponding operation (for example, starting work or stopping the work operation) can be performed on the robot based on the gesture recognition of the user, without the user bending over the button or using the mobile phone or the remote controller to control, facilitating the user to use, and improving the user to the product. Experience.

Embodiments of the present invention also provide a robot including the gesture recognition based robot control device as described above.

DRAWINGS

1 is a schematic flow chart of a robot control method based on gesture recognition in Embodiment 1 of the present invention.

2 is a flow chart showing a specific implementation of a robot control method based on gesture recognition in Embodiment 1 of the present invention.

FIG. 3 is a flowchart of a specific implementation of a robot control method based on gesture recognition in Embodiment 1 of the present invention.

4 is a schematic flow chart of a robot control method based on gesture recognition in Embodiment 2 of the present invention.

FIG. 5 is a flowchart of a specific implementation of a robot control method based on gesture recognition in Embodiment 2 of the present invention.

FIG. 6 is a flowchart of a specific implementation of a robot control method based on gesture recognition in Embodiment 2 of the present invention.

7a-7b are schematic diagrams of gesture swinging of a robot control method based on gesture recognition in Embodiment 2 of the present invention.

FIG. 8 is a structural block diagram of a robot control apparatus based on gesture recognition in Embodiment 3 of the present invention.

9 is a block diagram showing the structure of a wave operation judgment module of a gesture recognition-based robot control device according to Embodiment 3 of the present invention.

FIG. 10 is a block diagram showing the structure of a robot control apparatus based on gesture recognition in Embodiment 4 of the present invention.

11 is a block diagram showing the structure of a wave operation judgment module of a gesture recognition-based robot control device according to Embodiment 4 of the present invention.

detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, but not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

1 is a flowchart of a gesture recognition-based robot control method according to Embodiment 1 of the present invention. The control method includes steps S11-S13:

S11. When it is detected by the ranging sensor that an object exists in a preset range of the robot, the waving gesture recognition operation is started;

S12. After the trigger gesture recognition operation is started, if the ranging sensor currently detects that the duration of the object existing in the preset range of the robot is within a preset value range, recording is a one-wave operation; if the ranging is The sensor currently detects that the duration of the object within the preset range of the robot is not within a preset value range, or the ranging sensor currently detects that an object exists within the preset range of the robot to the next detection of the ranging sensor Stopping the waving gesture recognition operation when a time interval in which an object exists within a preset range of the robot is greater than a threshold;

S13. When the waving gesture recognition operation is stopped, performing a corresponding operation on the robot according to the recorded number of wave operation operations and a preset mapping table; wherein the mapping table records the correspondence between the number of wave operation operations and the execution operation.

In step S11, when the detection of the presence of an object within the preset range of the robot by the ranging sensor initiates the waving gesture recognition operation, the method further includes the steps of:

The user is prompted to perform a wave operation by controlling the robot to illuminate and/or vocalize.

It can be understood that the indicator light on the robot is off when there is no user operation, and only when an object (for example, a user's hand) is detected in the preset range by the ranging sensor, the control indicator lights up and can be accompanied by There is a voice prompting the user to perform a gesture operation on it.

Wherein, the preset range is 50 cm to 120 cm, preferably 60 cm to 100 cm. The ranging sensor is a laser ranging sensor, an ultrasonic ranging sensor or an infrared ranging sensor.

Wherein, the detecting by the ranging sensor that the object exists in the preset range of the robot means that the object passes through the detection area of the ranging sensor. The preset range is set according to a range in which the sensor can be effectively detected. Taking the user's wave operation as an example, the hand is regarded as an object, and the preset range is a relative preset between the hand and the distance measuring sensor by an experiment. A reasonable distance, this range determines the detection range of the device. When the distance value is within the range, it is determined that the hand is detected, and when the distance value exceeds the range, it is determined that the hand is not detected. There are some criteria for determining the reasonable range, but it is not limited to this: the preset range should be within the range of the sensor's ranging, and should be set with reference to the specific detection characteristics of the sensor, and the appropriate detection range should be set. When the detected distance is beyond the range, the algorithm determines that no object has passed; it is counted as valid detection only when there is a hand in the preset range.

In this embodiment, the ranging sensor is disposed on the top of the robot to facilitate detecting whether an object exists within a preset range.

In a specific implementation, when it is detected by the ranging sensor that an object exists in a preset range of the robot, the waving gesture recognition operation may be started/stopped by transmitting a timing instruction, a control instruction, or a switching instruction.

In step S12, as shown in FIG. 2, after the wave gesture recognition operation is started, it is determined by the following method that the ranging sensor currently detects that an object exists in the preset range of the robot to the next time the ranging sensor detects The time interval between objects in the preset range of the robot is greater than the threshold:

S1211: After the wave gesture recognition operation is started, when the ranging sensor detects for the first time that an object exists in a preset range of the robot, the first control signal is sent to start the first timer to start timing;

S1212: After the wave gesture recognition operation is started, when the ranging sensor detects that there is an object in the preset range of the robot every time before the first time, the second control signal is sent to control the first timer to be restarted. Timing

S1213. When the timing of receiving the second control signal by the first timer is greater than a threshold, determining that the ranging sensor currently detects that an object exists in the preset range of the robot to the ranging sensor. The next time it is detected that the time interval between objects in the preset range of the robot is greater than the threshold.

The threshold is smaller than the preset value range.

As shown in FIG. 3, in the step S12, after the wave gesture recognition operation is started, it is determined by the following method whether the duration of the object currently detected by the ranging sensor in the preset range of the robot is at a preset value. Within the scope:

S1221. After the wave gesture recognition operation is started, when the ranging sensor detects that an object exists in the preset range of the robot, the third control signal is sent to start the second timer to start timing;

S1222: After starting the second timer to start timing, when the ranging sensor does not detect that an object exists in the preset range of the robot, the fourth control signal is sent to stop the second timer timing;

S1223, when the timing time when the second timer receives the fourth control signal falls within the preset value range, it is determined that the ranging sensor currently detects that an object exists in a preset range of the robot. The duration of the object is within a preset range; otherwise, it is determined that the ranging sensor currently detects that the duration of the object within the preset range of the robot is not within a preset value range.

In step S13, performing the corresponding operation on the robot includes starting a work or stopping a work operation.

A range sensor is mounted on the top of the robot. A range sensor is used to scan the top of the robot for reflected signals. If there is a reflected signal at the top, the robot gives an acousto-optic indication, and the operator can perform the operation of the robot while waving the gesture. Wherein, the ranging sensor includes but is not limited to a laser ranging sensor/infrared ranging sensor/ultrasonic ranging sensor/TOF ranging sensor).

In this implementation, a single ranging sensor is used to realize user gesture recognition, and a single ranging sensor is disposed on the top of the robot. After the robot is powered on, the ranging sensor always detects whether there is an object above the robot, and when the object is sensed to be in a certain range (for example, 60 cm-100 cm) When the sound and light indication signal is emitted and the detection continues. If the top is the operator waving the hand directly above the sensor, the range sensor will detect the hand - the hand is not detected, so cycle until the wave is stopped.

In addition, the present embodiment uses the distance measured by the distance measuring sensor (preferably 60 cm - 100 cm) as an anti-interference measure to avoid robot recognition errors.

In addition, the robot will have other objects such as furniture at the top of the running, in order to avoid the robot recognition error (not gestures, but other objects). In this embodiment, an anti-interference measure is added, as follows: monitoring the dwell time of the hand (object) above the ranging sensor (because the hand can swing faster on the upper side, so the dwell time is less than a certain value and greater than another specific value, ie The duration of the object is within the preset range, and the hand is swiping above). If the interval is greater than a certain defined time (threshold), it is considered to be other interference, not a gesture. If the switching time between the detected object and the undetected object is short (ie, the time interval is less than the threshold), and the number of waved times is a specified number of times, it is considered to be a normal operation (ie, the number of wave hands reaches the corresponding executable operation in the preset mapping table) . The content of the user's operation (for example, starting work, stopping work, etc.) can be judged by detecting the number of wave times.

It can be seen that the gesture control-based robot control method disclosed in this embodiment detects that an object exists within a preset range of the robot by the ranging sensor, and then initiates a wave gesture recognition operation, after starting the wave gesture recognition operation, If the ranging sensor currently detects that the duration of the object existing within the preset range of the robot is within a preset value range, recording is a one-shot operation; if the ranging sensor currently detects the preset range of the robot The duration of the in-vivo object is not within the preset value range, or the ranging sensor currently detects that an object exists within the preset range of the robot to the ranging sensor, and the next time an object is detected within the preset range of the robot When the time interval is greater than the threshold, the waving gesture recognition operation is stopped; and when the waving gesture recognition operation is stopped, the corresponding operation is performed on the robot according to the recorded number of the wave operation and the preset mapping table; The mapping table records the correspondence between the number of wave operations and the execution of the operation. Thereby, the corresponding operation (for example, starting work or stopping the work operation) can be performed on the robot based on the gesture recognition of the user, without the user bending over the button or using the mobile phone and the remote controller to control, facilitating the user to use, and improving the user's product. Experience.

4 is a schematic flowchart of a robot control method based on gesture recognition according to Embodiment 2 of the present invention. The control method includes steps S21 to S23:

S21. When it is recognized by the camera that an object exists in the shooting range of the robot, the waving gesture recognition operation is started;

S22. After the wave gesture recognition operation is started, if the camera currently recognizes that the duration of the object existing in the shooting range of the robot is within a preset value range, recording is a wave operation; if the camera currently recognizes The duration of the object in the shooting range of the robot is not within the preset value range, or the camera currently recognizes that there is an object in the shooting range of the robot until the next time the camera recognizes that there is an object in the shooting range of the robot Stopping the waving gesture recognition operation when the interval is greater than the threshold;

S23. When the waving gesture recognition operation is stopped, performing a corresponding operation on the robot according to the recorded number of wave operation operations and a preset mapping table; wherein the mapping table records the correspondence between the number of wave operation operations and the execution operation.

In the step S21, the camera recognizes that an object is present in the shooting range of the robot, and an object passes through the imaging area of the camera. Specifically, it is possible to identify whether an object (for example, a hand) exists in the shooting range of the camera:

Mode A: The robot is in a stopped state, and the camera takes a picture according to the preset frequency, and the two photos before and after the camera are continuously shot. For comparison, if the matching degree between the two photos is relatively high (greater than the set value), it is recognized that there is no object in the shooting range of the robot; if there are obvious differences between the two front and back (the matching degree is less than the set value) ), identified as an object within the shooting range of the robot.

Mode B: The robot is in motion, the camera takes a picture according to the preset frequency, and compares the two photos before and after the camera is continuously shot. If the two photos are only slightly shifted, the recognition is not in the shooting range of the robot. Object; if there are not only translational differences between the two front and back, but also there is a momentary difference between the objects, it is recognized that there is an object in the shooting range of the robot. It can be understood that the weak translation difference can be a preset translation reference value.

In step S21, when the waving gesture recognition operation is initiated by the camera recognizing that an object exists within the shooting range of the robot, the method further includes the steps of:

The user is prompted to perform a wave operation by controlling the robot to illuminate and/or vocalize.

It can be understood that the indicator light on the robot is off when there is no user operation, and only when an object (for example, a user's hand) is detected in the preset range by the ranging sensor, the control indicator lights up and can be accompanied by There is a voice prompting the user to perform a gesture operation on it.

In this embodiment, the camera is disposed on the top of the robot to facilitate detecting whether an object exists in the shooting range. In a specific implementation, when it is recognized by the camera that an object exists in the shooting range of the robot, the waving gesture recognition operation can be started/stopped by transmitting a timing command, a control command, or a switch command.

In step S22, as shown in FIG. 5, after the wave gesture recognition operation is started, it is determined by the following method that the camera currently recognizes that an object exists in the shooting range of the robot and the camera recognizes the shooting range of the robot next time. The time interval between objects in the object is greater than the threshold:

S2211, based on the imaging frequency of the camera, by determining that the camera recognizes an instant from the presence or absence of an object in the shooting range of the robot, and the camera recognizes that the object is present from nothing in the shooting range of the robot. The number of frames passed to calculate the time interval;

S2212: When the time interval is greater than the threshold, it is determined that the camera currently recognizes that the time interval between the object existing in the shooting range of the robot and the next time the camera recognizes that the object exists in the shooting range of the robot is greater than a threshold.

In step S22, as shown in FIG. 6, after the wave gesture recognition operation is started, it is determined by the following method that the camera currently recognizes whether the duration of the object existing in the shooting range of the robot is within a preset value range:

S2221, based on the shooting frequency of the camera, by determining that the camera recognizes that there is an instantaneous moment in the preset range of the robot, and the camera recognizes that there is no presence in the preset range of the robot. Calculating the duration of the number of frames that the object passes in an instant;

S2222, when the duration falls within the preset value range, determining that the ranging sensor currently detects that the duration of the object exists within the preset range of the robot is within a preset value range; otherwise determining the The range sensor is currently detected in the robot's pre- The duration of the object within the range is not within the preset range.

The shooting frequency of the camera is 10 to 50 frames/second, preferably 20 to 30 frames/second. The preset value range is greater than the threshold.

Hereinafter, the present embodiment will be described in detail with reference to Figs. 7a to 7b. Taking the user's wave as an example, the hand is regarded as an object. As long as the object is swung on the camera, a series of pictures will be formed, which is achieved by comparing the multiple photos of consecutive shots. As shown in Fig. 7a to Fig. 7b, when the hand moves over the camera, the CCD/CMOS continuously captures 5 photos (or 5 frames captured in the video stream). It can be understood that 5 photos are only examples, It can be 3 or 8 sheets. When the hand is waving on the camera, the CCD/CMOS will move an object inside when it is static. It can be judged that there is an object moving fast by comparing with the picture taken at the time of static.

The five photos shown in Figures 7a to 7b are a series of photos of the hand moving from left to right. The object (hand) can be judged from left to right in CCD/CMOS, and can be judged from right to left. Once. This allows you to record the number of swings.

For the duration mentioned above, it can be calculated as follows: the camera photographing frequency is known (software setting), assuming that 24 photos are taken in 1 second (corresponding to the video stream is 24 frames/second), then two consecutive photos are taken. The time interval is 1/24 second, which is 0.042 seconds. That can be calculated by judging the number of photos taken by the object (hand) in the CCD/CMOS to disappear. The example is as follows: Suppose the hand enters the CCD from the left. From the middle to the right to take 5 photos, then the hand stays in the CCD for 0.21 seconds, then the duration is 0.21 seconds.

For the above-mentioned time interval, it can be calculated as follows: the camera photographing frequency is known (software setting), assuming that 24 photos are taken in 1 second (corresponding to the video stream is 24 frames/second), then two consecutive photos are taken. The time interval is 1/24 second, which is 0.042 seconds. Then, the time interval can be calculated by judging the number of photographs taken by the object (hand) in the CCD/CMOS from disappearing to appearing. For example: Assume that the hand enters the CCD from the left to take 5 photos from the right, then the time spent in the CCD is 0.21 seconds, and the hand waving to the right will not stop just out of the CCD field of view, and may still Continue to move to the right, assuming another 3 photos, then the time is 0.126 seconds, from the right back to the right CCD field of view and took 3 more photos, the time is 0.126 seconds, from the right to go out to take 5 photos from the left The time is 0.21 seconds. Then you can follow two 3 photos (that is, the time when the hand is not on the CCD), which is 0.126*2=0.252 seconds.

In this embodiment, the recognition method of the gesture recognition by the camera effectively solves the problem of gesture recognition (because the shape of the hand is many, such as fingers close together, fingers open, fists, etc.), this method identifies the hand as an object, therefore, It is not the hand that is continuously waving on the top, and it is also possible to recognize the gesture. In order to prevent interference, the duration, interval time and number of waves are introduced to prevent interference. Usually, if it is interference, it will not form a continuous projection and there will be multiple wave times in the segment time interval.

It can be seen that the gesture control-based robot control method disclosed in the embodiment detects that an object exists within the shooting range of the robot by the camera, and then initiates a wave gesture recognition operation, after the wave gesture recognition operation is started, if the camera is currently It is recognized that the duration of the object existing in the shooting range of the robot is within a preset value range, and is recorded as a wave operation; if the camera currently recognizes that the duration of the object exists in the shooting range of the robot is not in the preset value range Inside, or the camera currently recognizes Stopping the waving gesture recognition operation when an object exists within the shooting range of the robot until the camera recognizes that the time interval in which the object exists within the shooting range of the robot is greater than the threshold; and when the waving gesture recognition operation is stopped Corresponding operations are performed on the robot according to the recorded number of wave operations and the preset mapping table; wherein the mapping table records the correspondence between the number of wave operations and the execution operation. Therefore, it is realized that the corresponding operation (for example, starting work or stopping the work operation) can be performed on the robot based on the gesture recognition of the user, without the user bending over the button or using the mobile phone or the remote controller to control, facilitating the user to use, and improving the user to the product. Experience.

FIG. 8 is a structural block diagram of a gesture control-based robot control apparatus according to Embodiment 3 of the present invention, the control apparatus including:

The waving gesture recognition operation starting module 31 is configured to initiate a wave gesture recognition operation when detecting that an object exists within a preset range of the robot by the ranging sensor;

The wave operation determination module 32 is configured to record a wave operation if the distance sensor currently detects that the duration of the object within the preset range of the robot is within a preset value range after the wave gesture recognition operation is initiated. If the ranging sensor currently detects that the duration of the object exists within the preset range of the robot is not within a preset value range, or the ranging sensor currently detects that an object exists within the preset range of the robot to the The next time the ranging sensor detects that the time interval of the object exists in the preset range of the robot is greater than a threshold, the waving gesture recognition operation is stopped;

The execution module 33 is configured to perform a corresponding operation on the robot according to the recorded number of the wave operation and the preset mapping table when the wave gesture recognition operation is stopped; wherein the mapping table records the number of wave operations and performs an operation Correspondence.

The waving gesture recognition operation starting module is further configured to: when the detection of the presence of an object in the preset range of the robot by the ranging sensor, initiate the wave gesture recognition operation, prompt the user to perform the illumination by controlling the robot to emit light and/or sound. Waved.

As shown in FIG. 9, the wave operation determining module 32 specifically includes:

The first control signal generating unit 321 is configured to: after the triggering gesture recognition operation is started, when the ranging sensor detects that an object exists in the preset range of the robot for the first time, issue a first control signal to start the first timer. Timing

The second control signal generating unit 322 is configured to: after the triggering gesture recognition operation is started, when the ranging sensor detects that there is an object in the preset range of the robot every time before the first time, the second control signal is sent to Controlling the first timer to re-time;

The first determining unit 323, when the timing when the first timer receives the second control signal is greater than the threshold, determining that the ranging sensor currently detects that an object exists in the preset range of the robot The time interval sensor detects that the time interval of the object existing in the preset range of the robot is greater than a threshold;

The third control signal generating unit 324 is configured to: after the triggering gesture recognition operation is started, when the ranging sensor instantaneously detects that an object exists in the preset range of the robot, issue a third control signal to start the second timer. Timing

a fourth control signal generating unit 325, configured to: after the start of the second timer starts timing, when the ranging sensor does not detect that an object exists in the preset range of the robot, the fourth control signal is sent to stop the Second timer timing;

The second determining unit 326 is configured to determine that the time measuring time when the second timer receives the fourth control signal falls within the preset value range, and determine that the ranging sensor is currently detected in the robot The duration of the object within the preset range is within a preset value range; otherwise, it is determined that the ranging sensor currently detects that the duration of the object within the preset range of the robot is not within the preset value range.

In a specific implementation, the wave operation judgment module starts/stops the wave gesture recognition operation by issuing a timing command, a control command, or a switch command.

Preferably, the preset value range is greater than the threshold. The ranging sensor is located at the top of the robot. The above-described operations performed on the robot include starting work or stopping work.

The working principle and the working process of the gesture recognition-based robot control device disclosed in this embodiment are the same as those in the first embodiment. For details, refer to the first embodiment, and no further details are provided herein.

10 is a structural block diagram of a robot control device based on gesture recognition in Embodiment 4 of the present invention, the control device includes:

The waving gesture recognition operation starting module 41 is configured to initiate a waving gesture recognition operation when it is recognized by the camera that an object exists within the shooting range of the robot;

The wave operation determining module 42 is configured to record, as a wave-handling operation, if the camera currently recognizes that the duration of the object within the shooting range of the robot is within a preset value range after the wave gesture recognition operation is initiated; The camera currently recognizes that the duration of the object within the shooting range of the robot is not within the preset value range, or the camera currently recognizes that an object exists within the shooting range of the robot until the camera next recognizes that the robot is photographed Stopping the waving gesture recognition operation when the time interval of the object exists in the range is greater than the threshold;

The execution module 43 is configured to perform a corresponding operation on the robot according to the recorded number of the wave operation and the preset mapping table when the wave gesture recognition operation is stopped; wherein the mapping table records the number of wave operations and performs an operation Correspondence.

In the waving gesture recognition operation starting module 41, it is recognized by the camera that an object is present in the shooting range of the robot, and an object that has passed the imaging area of the camera instantaneously. Specifically, it is possible to identify whether an object (for example, a hand) exists in the shooting range of the camera:

Mode A: The robot is in a stopped state, and the camera takes a picture according to the preset frequency, and compares the two photos before and after the camera is continuously shot. If the matching degree of the two photos is relatively high (greater than the set value), it is recognized as being in the robot. There is no object in the shooting range; if there is a significant difference between the two front and back (the matching degree is less than the set value), it is recognized that there is an object in the shooting range of the robot.

Mode B: The robot is in motion, the camera takes a picture according to the preset frequency, and compares the two photos before and after the camera is continuously shot. If the two photos are only slightly shifted, the recognition is not in the shooting range of the robot. Object; if there are not only translational differences between the two front and back, but also there is a momentary difference between the objects, it is recognized that there is an object in the shooting range of the robot.

The waving gesture recognition operation starting module 41 is further configured to prompt the user to perform a wave operation by controlling the robot to emit light and/or vocalize when the gesture of recognizing the presence of an object within the shooting range of the robot is detected by the camera. .

As shown in FIG. 11, the wave operation determining module 42 specifically includes:

The first frame number counting unit 421 determines, based on the photographing frequency of the camera, that the camera recognizes that there is no object in the shooting range of the robot, and the camera recognizes that the camera has never been within the shooting range of the robot. Calculate the time interval by the number of frames that have passed through the existing object;

The first determining unit 422 is configured to: when the time interval is greater than the threshold, determine that the camera currently recognizes that an object exists in the shooting range of the robot, and the next time the camera recognizes that an object exists in the shooting range of the robot The time interval is greater than the threshold;

The second frame number counting unit 423 determines, based on the photographing frequency of the camera, that the camera recognizes that there is an existing object in the shooting range of the robot, and the camera recognizes that there is a target within the shooting range of the robot. Calculate the duration by the number of frames passed by no object; and

a second determining unit 424, when the duration falls within the preset value range, determining that the ranging sensor currently detects that the duration of the object exists in the shooting range of the robot is within a preset value range; otherwise It is judged that the ranging sensor currently detects that the duration of the object existing in the shooting range of the robot is not within the preset value range.

The shooting frequency of the camera is 10 to 50 frames/second, preferably 20 to 30 frames/second.

In a specific implementation, the wave operation judgment module starts/stops the wave gesture recognition operation by issuing a timing command, a control command, or a switch command.

As a modification of the embodiment, the preset value range is greater than the threshold. The camera is located on top of the robot. The above-described operations performed on the robot include starting work or stopping work.

The working principle and working process of the gesture recognition-based robot control device disclosed in this embodiment are the same as those in the above-mentioned Embodiment 2. For details, refer to the foregoing Embodiment 2, and details are not described herein again.

Another embodiment of the present invention also discloses a robot comprising the gesture recognition based robot control device as described in the above embodiments.

It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical. Units can be located in one place or distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the embodiment. In addition, in the drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, and specifically, one or more communication buses or signal lines can be realized. Those of ordinary skill in the art can understand and implement without any creative effort.

Through the description of the above embodiments, those skilled in the art can clearly understand that the present invention can be added by software. The implementation of the general hardware means, of course, can also be realized by dedicated hardware including an application specific integrated circuit, a dedicated CPU, a dedicated memory, a dedicated component, and the like. In general, functions performed by computer programs can be easily implemented with the corresponding hardware, and the specific hardware structure used to implement the same function can also be various, such as analog circuits, digital circuits, or dedicated circuits. Circuits, etc. However, for the purposes of the present invention, software program implementation is a better implementation in more cases. Based on the understanding, the technical solution of the present invention, which is essential or contributes to the prior art, can be embodied in the form of a software product stored in a readable storage medium, such as a floppy disk of a computer. , U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., including a number of instructions to make a computer device (may be A personal computer, server, or network device, etc.) performs the methods described in various embodiments of the present invention.

The above is a preferred embodiment of the present invention, and it should be noted that those skilled in the art can also make several improvements and retouchings without departing from the principles of the present invention. It is the scope of protection of the present invention.

Claims (41)

1. A robot control method based on gesture recognition, comprising the steps of:

   When it is detected by the ranging sensor that an object exists within a preset range of the robot, the waving gesture recognition operation is initiated;

   After the wave gesture recognition operation is initiated, if the ranging sensor currently detects that the duration of the object within the preset range of the robot is within a preset value range, recording is a one-shot operation; if the ranging sensor is currently Detecting that the duration of the object exists within the preset range of the robot is not within a preset value range, or the ranging sensor currently detects that an object exists within the preset range of the robot to the next time the ranging sensor is detected Stopping the waving gesture recognition operation when the time interval of the object exists in the preset range of the robot is greater than a threshold;

   When the waving gesture recognition operation is stopped, a corresponding operation is performed on the robot according to the recorded number of the wave operation and the preset mapping table; wherein the mapping table records the correspondence between the number of wave operations and the execution operation.
2. The gesture recognition-based robot control method according to claim 1, wherein when the range sensor detects that an object exists within a preset range of the robot and initiates a wave gesture recognition operation, the method further includes the steps of:

   The user is prompted to perform a wave operation by controlling the robot to illuminate and/or vocalize.
3. The gesture recognition-based robot control method according to claim 1, wherein the preset range is 50 cm to 120 cm.
4. The gesture recognition-based robot control method according to claim 1, wherein the ranging sensor is a laser ranging sensor, an ultrasonic ranging sensor, a TOF ranging sensor, or an infrared ranging sensor.
5. The gesture recognition-based robot control method according to claim 1, wherein after the triggering gesture recognition operation is started, it is determined that the ranging sensor currently detects that an object exists in the preset range of the robot to the The next time the ranging sensor detects that there is an object within the preset range of the robot, the time interval is greater than the threshold:

   After the wave gesture recognition operation is started, when the ranging sensor detects for the first time that an object exists in the preset range of the robot, the first control signal is sent to start the first timer to start timing;

   After the wave gesture recognition operation is started, when the ranging sensor detects that there is an object in the preset range of the robot every time before the first time, the second control signal is sent to control the first timer to re-time;

   When the timing time when the first timer receives the second control signal is greater than a threshold, determining that the ranging sensor currently detects that an object exists in the preset range of the robot to the ranging sensor next time It is detected that the time interval in which the object exists within the preset range of the robot is greater than the threshold.
6. The gesture recognition-based robot control method according to claim 1, wherein after the wave gesture recognition operation is started, it is determined by the following method that the ranging sensor currently detects the duration of the object existing in the preset range of the robot. Whether it is within the preset range:

   After the wave gesture recognition operation is initiated, when the ranging sensor instantaneously detects that an object exists within the preset range of the robot, a third control signal is issued to start the second timer to start timing;

   After starting the second timer to start timing, when the ranging sensor does not detect that an object exists within the preset range of the robot, the fourth control signal is issued to stop the second timer timing;

   When the timing time when the second timer receives the fourth control signal falls within the preset value range, determining that the ranging sensor currently detects the presence of an object within a preset range of the robot The time is within the preset value range; otherwise, it is judged that the ranging sensor currently detects that the duration of the object existing in the preset range of the robot is not within the preset value range.
7. The gesture recognition based robot control method according to claim 1, wherein the preset value range is greater than the threshold.
8. A gesture recognition based robot control method according to claim 1, wherein said ranging sensor is located at the top of said robot.
9. The gesture recognition-based robot control method according to claim 1, wherein the performing the corresponding operation on the robot comprises starting a work or stopping the work.
10. The gesture recognition-based robot control method according to claim 1, wherein the waving gesture recognition operation is started/stopped by a timing command, a control command, or a switch command.
11. A robot control device based on gesture recognition, comprising:

    a waving gesture recognition operation starting module, configured to initiate a waving gesture recognition operation when detecting that an object exists within a preset range of the robot by the ranging sensor;

    a wave operation judgment module, configured to record a wave operation if the distance sensor currently detects that the duration of the object within the preset range of the robot is within a preset value range after the wave gesture recognition operation is initiated; If the ranging sensor currently detects that the duration of the object existing in the preset range of the robot is not within the preset value range, or the ranging sensor currently detects that an object exists in the preset range of the robot to the measurement Stopping the waving gesture recognition operation when the distance sensor detects that the time interval of the object existing in the preset range of the robot is greater than the threshold value;

    An execution module, configured to perform a corresponding operation on the robot according to the recorded number of wave operations and a preset mapping table when the waving gesture recognition operation is stopped; wherein the mapping table records the number of wave operations and the operation Correspondence relationship.
12. The gesture recognition-based robot control apparatus according to claim 11, wherein the waving gesture recognition operation starting module is further configured to: when the distance measuring sensor detects that an object exists in a preset range of the robot, start waving In the gesture recognition operation, the user is prompted to perform a wave operation by controlling the robot to emit light and/or sound.
13. A gesture recognition-based robot control apparatus according to claim 11, wherein said preset range is 50 cm to 120 cm.
14. The gesture recognition-based robot control device according to claim 11, wherein the ranging sensor is a laser ranging sensor, an ultrasonic ranging sensor, a TOF ranging sensor, or an infrared ranging sensor.
15. The gesture recognition-based robot control apparatus according to claim 11, wherein the wave operation determination module comprises:

    a first control signal generating unit, configured to: after the triggering gesture recognition operation is started, when the ranging sensor detects that an object exists within a preset range of the robot for the first time, issue a first control signal to start the first timer to start timing ;

    a second control signal generating unit, configured to send a second control signal to control each time the ranging sensor detects an object within a preset range of the robot, except for the first time after the wave gesture recognition operation is initiated. The first timer is retimed;

    a first determining unit, when the time when the first timer receives the second control signal is greater than a threshold, determining that the ranging sensor currently detects that an object exists in the preset range of the robot to the The next time the ranging sensor detects that there is an object within the preset range of the robot, the time interval is greater than the threshold.
16. The gesture recognition-based robot control apparatus according to claim 11, wherein the wave operation determination module further comprises:

    a third control signal generating unit, configured to: after the triggering gesture recognition operation is started, when the ranging sensor detects that an object exists within a preset range of the robot, issue a third control signal to start the second timer to start timing ;

    a fourth control signal generating unit, configured to: after the start of the second timer starts timing, when the ranging sensor does not detect that an object exists in the preset range of the robot, the fourth control signal is sent to stop the first Two timers;

    a second determining unit, configured to: when the timing time when the second timer receives the fourth control signal falls within the preset value range, determine that the ranging sensor currently detects a pre-mouse in the robot It is assumed that the duration of the object exists within the preset value range; otherwise, it is determined that the ranging sensor currently detects that the duration of the object exists within the preset range of the robot is not within the preset value range.
17. The gesture recognition-based robot control apparatus according to claim 11, wherein the preset value range is greater than the threshold.
18. A gesture recognition based robot control apparatus according to claim 11, wherein said ranging sensor is located at the top of said robot.
19. The gesture recognition-based robot control apparatus according to claim 11, wherein the performing the corresponding operation on the robot comprises starting a work or stopping the work.
20. The gesture recognition-based robot control apparatus according to claim 11, wherein said waving operation judgment module starts/stops said waving gesture recognition operation by issuing a timing command, a control command, or a switch command.
21. A robot control method based on gesture recognition, comprising the steps of:

    When it is recognized by the camera that there is an object within the shooting range of the robot, the waving gesture recognition operation is initiated;

    After the wave gesture recognition operation is initiated, if the camera currently recognizes that the duration of the object within the shooting range of the robot is within a preset value range, recording is a one-shot operation; if the camera is currently recognized in the robot The duration of the object within the shooting range is not within the preset value range, or the camera currently recognizes that there is an object within the shooting range of the robot until the next time the camera recognizes that the object exists within the shooting range of the robot is greater than At the threshold, the waving gesture recognition operation is stopped;

    When the waving gesture recognition operation is stopped, a corresponding operation is performed on the robot according to the recorded number of the wave operation and the preset mapping table; wherein the mapping table records the correspondence between the number of wave operations and the execution operation.
22. The gesture recognition-based robot control method according to claim 1, wherein when the gesture of recognizing the presence of an object within the shooting range of the robot by the camera is initiated, the method further comprises the steps of:

    The user is prompted to perform a wave operation by controlling the robot to illuminate and/or vocalize.
23. The gesture recognition-based robot control method according to claim 21, wherein after the wave gesture recognition operation is activated, it is determined by the following method that the camera currently recognizes that an object exists in the shooting range of the robot and the camera next time It is recognized that the time interval between objects in the shooting range of the robot is greater than the threshold:

    Based on the photographing frequency of the camera, by judging that the camera recognizes an instant from the presence or absence of an object in the shooting range of the robot, the camera recognizes that the object is present from nothing in the shooting range of the robot. The number of frames to calculate the time interval;

    When the time interval is greater than the threshold, it is determined that the camera currently recognizes that the time interval between the object in the shooting range of the robot and the next time the camera recognizes that the object exists in the shooting range of the robot is greater than a threshold.
24. The gesture recognition-based robot control method according to claim 21, wherein after the wave gesture recognition operation is activated, it is determined by the following method whether the duration of the object existing in the shooting range of the robot is currently recognized by the camera. Within the set value range:

    Based on the shooting frequency of the camera, by determining that the camera recognizes that there is an instant of existence of the object within the preset range of the robot, the camera recognizes that there is an instant from the presence or absence of the object within the preset range of the robot. The number of frames passed to calculate the duration;

    When the duration falls within the preset value range, determining that the ranging sensor currently detects that the duration of the object exists within the preset range of the robot is within a preset value range; otherwise determining the ranging The sensor currently detects that the duration of the object within the preset range of the robot is not within the preset range.
25. The gesture recognition-based robot control method according to claim 21, wherein the object is recognized in the shooting range of the camera by:

    When the robot is in a stopped state, the camera takes a picture according to the preset frequency, and compares the two photos before and after the camera is continuously shot. If the matching degree of the two photos before and after is greater than the set value, it is recognized that there is no object in the shooting range of the robot; If the matching degree of the two photos before and after is less than the set value, it is recognized that there is an object in the shooting range of the robot;

    When the robot is in motion, the camera takes a picture according to the preset frequency, and compares the two photos before and after the camera is continuously shot. If the two photos are only slightly different between the two shots, it is recognized that there is no object in the shooting range of the robot; if there are no translation differences between the two front and back, and there is an instantaneous difference between the objects, it is recognized as There is an object in the shooting range.
26. The gesture recognition-based robot control method according to claim 25, wherein the camera has a shooting frequency of 10 to 50 frames/second.
27. The gesture recognition-based robot control method according to claim 21, wherein the preset value range is greater than the threshold.
28. A gesture recognition based robot control method according to claim 21, wherein said camera is located at the top of said robot.
29. The gesture recognition-based robot control method according to claim 21, wherein the performing the corresponding operation on the robot comprises starting a work or stopping the work.
30. A gesture recognition-based robot control method according to claim 21, wherein said waving gesture recognition operation is started/stopped by a timing command, a control command or a switch command.
31. A robot control device based on gesture recognition, comprising:

    a waving gesture recognition operation starting module, configured to activate a waving gesture recognition operation when it is recognized by the camera that an object exists within a shooting range of the robot;

    a wave operation judgment module, configured to record a wave operation if the camera currently recognizes that the duration of the object within the shooting range of the robot is within a preset value range after the wave gesture recognition operation is initiated; The camera currently recognizes that the duration of the object within the shooting range of the robot is not within the preset value range, or the camera currently recognizes that an object exists within the shooting range of the robot until the camera next recognizes the shooting range of the robot Stopping the waving gesture recognition operation when the time interval of the object in the object is greater than a threshold;

    An execution module, configured to perform a corresponding operation on the robot according to the recorded number of wave operations and a preset mapping table when the waving gesture recognition operation is stopped; wherein the mapping table records the number of wave operations and the operation Correspondence relationship.
32. A gesture recognition-based robot control apparatus according to claim 31, wherein said waving gesture recognition operation The startup module is further configured to prompt the user to perform a wave operation by controlling the robot to emit light and/or vocalize when the gesture of recognizing the presence of an object within the shooting range of the robot is detected by the camera.
33. The gesture recognition-based robot control apparatus according to claim 31, wherein the wave operation determination module comprises:

    a first frame number counting unit, based on the shooting frequency of the camera, by determining that the camera recognizes that there is no object in the shooting range of the robot, and the camera recognizes that the camera is within the shooting range of the robot Calculate the time interval by the number of frames through which the object exists;

    a first determining unit, configured to: when the time interval is greater than a threshold, determine that the camera currently recognizes that an object exists in the shooting range of the robot until the camera next recognizes that an object exists in the shooting range of the robot The interval is greater than the threshold.
34. The gesture recognition-based robot control apparatus according to claim 31, wherein the wave operation determination module further comprises:

    a second frame number counting unit, based on the photographing frequency of the camera, by determining that the camera recognizes that there is an existing object in the shooting range of the robot, and the camera recognizes that the camera is within the shooting range of the robot Calculate the duration without the number of frames passed by the existing object;

    a second determining unit, when the duration falls within the preset value range, determining that the ranging sensor currently detects that the duration of the object existing in the shooting range of the robot is within a preset value range; otherwise, determining The ranging sensor currently detects that the duration of the object within the shooting range of the robot is not within a preset value range.
35. The gesture recognition-based robot control apparatus according to claim 31, wherein the waving gesture recognition operation starting module identifies whether an object exists in a shooting range of the camera by:

    When the robot is in a stopped state, the camera takes a picture according to the preset frequency, and compares the two photos before and after the camera is continuously shot. If the matching degree of the two photos before and after is greater than the set value, it is recognized that there is no object in the shooting range of the robot; If the matching degree of the two photos before and after is less than the set value, it is recognized that there is an object in the shooting range of the robot;

    When the robot is in motion, the camera takes a picture according to the preset frequency, and compares the two photos before and after the continuous shooting of the camera. If the two photos are only slightly different from each other, it is recognized that there is no object in the shooting range of the robot; If there are not only translational differences between the two front and back, but also there is an instantaneous difference between the objects, it is recognized that there is an object in the shooting range of the robot.
36. A gesture recognition-based robot control apparatus according to claim 35, wherein said camera shooting frequency It is 10 to 50 frames/second.
37. The gesture recognition-based robot control apparatus according to claim 31, wherein said preset value range is greater than said threshold value.
38. A gesture recognition based robot control apparatus according to claim 31, wherein said camera is located at the top of said robot.
39. A gesture recognition-based robot control apparatus according to claim 31, wherein performing the corresponding operation on the robot includes starting work or stopping work.
40. The gesture recognition-based robot control apparatus according to claim 31, wherein said waving operation judgment module starts/stops said waving gesture recognition operation by issuing a timing command, a control command, or a switch command.
41. A robot comprising the gesture recognition-based robot control device according to any one of claims 11 to 20 and 31 to 40.

Images