WO2018170713A1

WO2018170713A1 - Gesture recognition-based robot car control method and device

Info

Publication number: WO2018170713A1
Application number: PCT/CN2017/077425
Authority: WO
Inventors: 梅杰
Original assignee: 深圳市欸阿技术有限公司
Priority date: 2017-03-21
Filing date: 2017-03-21
Publication date: 2018-09-27

Abstract

A gesture recognition-based robot car control method, comprising: determining a starting position and an ending position of a palm, recognizing the motion trajectory of the palm on the basis of the starting position and the ending position of the palm, and determining a gesture signal of the palm according to the motion trajectory of the palm (S10); invoking a gesture control command corresponding to the gesture signal on the basis of the gesture signal (S20); and controlling the motion of a robot car according to the gesture control command (S30). According to the method, a gesture of the palm is recognized simply and accurately to control the motion of a robot car, and the problem of low recognition rate easily caused by interference of factors such as illumination and background of the environment during existing dynamic gesture detection is resolved.

Description

Trolley control method and device based on gesture recognition

Technical field

The invention is based on the field of image information processing and control of human-computer interaction, and particularly relates to a method and device for controlling a car based on gesture recognition.

Background technique

The image recognition technology of gestures can be roughly divided into static gesture recognition and dynamic gesture recognition. Static gesture recognition is characterized by learning the mapping relationship from the graphical feature space to the hand shape space. Specifically, firstly, a set of gestures is predefined, and a unique gesture feature description is extracted from each gesture, and then the image feature space is directly mapped to the gesture space to obtain an estimation of the gesture, that is, by image features (points, lines, An angle or texture area, etc.) to identify the gesture or gesture state. This method requires learning and training based on data templates representing various gestures. Commonly extracted gesture features include skin color features, Haar features, HOG features, shape features, and the like. The dynamic gesture recognition is mainly based on the motion detection algorithm, and the predefined dynamic gesture is detected according to the feature description of the motion region. At present, dynamic gesture detection research is relatively mature, but if considering the interference factors such as illumination and background, the recognition rate is greatly reduced.

The above content is only used to assist in understanding the technical solutions of the present invention, and does not constitute an admission that the above is prior art.

Summary of the invention

The main object of the present invention is to provide a car control method based on gesture recognition, which aims to solve the problem of lowering the recognition rate caused by interference of ambient light, background and the like in the existing dynamic gesture detection.

To achieve the above object, the present invention provides a car control method based on gesture recognition, the control method comprising the following steps:

Determining a starting position and an ending position of the palm, and identifying a movement trajectory of the palm based on the starting position and the ending position of the palm, and determining a gesture signal of the palm according to the movement trajectory of the palm;

And acquiring, according to the gesture signal, a gesture control command corresponding to the gesture signal;

The cart motion is controlled according to the gesture control command.

Preferably, the determining the starting position of the palm includes:

Determining the position of the palm according to the joint point of the palm, and determining whether the range of the palm of the palm is within a preset value within a preset time;

When the jitter range of the palm is within the preset value, and is recognized as an effective gesture, the center point of the region where the palm is located within the preset time is identified as the starting position of the palm.

Preferably, the determining the end position of the palm includes:

After identifying the starting position of the palm, delaying the first preset time;

Determining, in the second preset time, whether the first jitter range of the palm is within the first preset value after the first preset time is counted;

When the first jitter range of the palm is within the first preset value, the center point of the region where the palm is located in the second preset time is identified as the end position of the palm.

Preferably, the judging of the shaking of the palm is achieved by determining a change in the displacement of the palm according to the gradation value of the palm.

Preferably, the identifying the motion trajectory of the palm by the starting position and the ending position of the palm center specifically includes:

A coordinate system is established based on the starting position of the palm, and the motion trajectory of the palm is recognized by determining the coordinates of the starting position and the ending position.

In order to achieve the above object, the present invention further provides a car control device based on gesture recognition, and the car control device based on gesture recognition includes:

Determining a module for determining a starting position and an ending position of the palm;

a judging module, configured to identify a motion trajectory of the palm based on the starting position and the ending position of the palm, and determine a gesture signal of the palm according to the motion trajectory of the palm;

a command generating module, configured to retrieve a gesture control command corresponding to the gesture signal according to the gesture signal;

And a control module, configured to control the movement of the trolley according to the gesture control command.

Preferably, the determining module is specifically configured to determine a position of the palm according to the joint point of the palm, and determine whether the range of the jitter of the palm is within a preset value within a preset time; when the jitter range of the palm is in the preset When the value is set, it is recognized as a valid gesture, and the center point of the region where the palm is located within the preset time is identified as the starting position of the palm.

Preferably, the determining module is configured to delay the first preset time after the initial position of the palm is recognized; and determine the second preset time after the first preset time is counted Whether the first jitter range of the palm is within the first preset value; when the first jitter range of the palm is within the first preset value, identifying the area where the palm is located in the second preset time The center point is the end position of the palm.

Preferably, the judgment of the shaking of the palm is performed by determining a change in the displacement of the palm according to the gradation value of the palm.

Preferably, the determining module is specifically configured to establish a coordinate system based on a starting position of the palm, and identify a motion trajectory of the palm by determining coordinates of the starting position and the ending position.

The invention recognizes the start position and the end position of the palm, and determines the gesture signal of the palm according to the start position and the end position, and retrieves the gesture control command corresponding to the gesture signal based on the gesture signal, and finally according to the gesture The command controls the movement of the trolley, realizes the gesture of recognizing the palm of the hand in a simple and accurate manner to control the movement of the trolley, and solves the problem of the reduction of the recognition rate caused by the interference of the ambient light, the background and the like in the existing dynamic gesture detection, and the present invention By judging the jitter range based on the gray value, continuous gesture recognition can be realized within a certain distance range.

DRAWINGS

1 is a schematic flow chart of a first embodiment of a method for controlling a car based on gesture recognition according to the present invention;

2 is a schematic diagram of functional modules of a first embodiment of a vehicle control device based on gesture recognition according to the present invention.

The implementation, functional features, and advantages of the present invention will be further described in conjunction with the embodiments.

detailed description

It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

A method and apparatus for controlling a car based on gesture recognition according to an embodiment of the present invention will be described below with reference to the accompanying drawings.

First, a gesture recognition based car control method proposed by an embodiment of the present invention will be described. 1 is a flow chart showing a method of controlling a car based on gesture recognition according to a first embodiment of the present invention. As shown in FIG. 1 , the gesture recognition based car control method of the embodiment of the present invention includes the following steps:

Step S10, determining a starting position and an ending position of the palm, and identifying a movement trajectory of the palm based on the starting position and the ending position of the palm, and determining a gesture signal of the palm according to the movement trajectory of the palm.

When the solution adopted by the present invention recognizes a gesture by an image sensor such as Kinect, unlike the conventional method for analyzing gesture recognition by recognizing the image of the palm, the present invention only needs to recognize the start position and the end position of the palm, and according to the start. The position and end position can determine the gesture direction signal of the palm.

Specifically, when performing gesture recognition, the starting point of the palm is recognized by the image of the palm, and the starting point is the origin, and the plane rectangular coordinate system is established, and the end position of the ending point in the coordinate system is determined by analyzing the end point. The position, combined with the origin, can determine the current trajectory of the palm, and then draw the basic gesture of the palm according to the trajectory of the palm, including the palm to the left, to the right, up, down or clenched.

Step S20: Acquire a gesture control command corresponding to the gesture signal based on the gesture signal.

After the gesture action is recognized by the Kinect sensor, the car controller retrieves the gesture control command corresponding to the gesture signal through the memory, for example, by acquiring five kinds of gesture signals: the palm moves to the left, right, up, and down. Or the fist movements correspond to the five types of movements of the car's forward, backward, left turn, right turn, and stop, which can be easily obtained by means of a memory lookup table.

In step S30, the car motion is controlled according to the gesture control command.

After the car controller obtains the gesture control command based on the gesture signal, the car controller controls the car to perform the corresponding action according to the control command. The two wheels of a specific car have two motor drive runs respectively, and the car can be driven to perform corresponding actions by controlling the direction of motion of the two motors.

In this embodiment, by recognizing the start position and the end position of the palm, and determining the gesture signal of the palm according to the start position and the end position, and acquiring the gesture control command corresponding to the gesture signal based on the gesture signal, and finally The gesture command controls the movement of the trolley, realizes the gesture of recognizing the palm of the hand in a simple and accurate manner to control the movement of the trolley, and solves the problem of the recognition rate reduction caused by the interference of ambient lighting, background and the like in the existing dynamic gesture detection. The present invention can realize continuous gesture recognition within 2 to 20 meters by judging the jitter range based on the gray value.

Further, the second embodiment of the gesture recognition-based car control method is based on the first embodiment of the above-described gesture recognition-based car control method of the present invention. In this embodiment, the determining the starting position of the hand center specifically includes: The palm joint point determines the palm position, and determines whether the jitter range of the palm is within a preset value within a preset time; when the jitter range of the palm is within the preset value, the recognition is a valid gesture, and the recognition The center point of the jitter range of the palm is the starting position of the palm.

The position of the palm of the present embodiment is specifically: identifying the joint point of the palm (which can be identified based on the ready-made library function), and determining the position of the palm (one point) according to the joint point of the palm. Before identifying the starting position of the palm, it is also necessary to perform a gesture recognition processing, that is, de-jitter processing, specifically, determining whether the jitter range of the palm is within a preset value within a preset time, if the preset time The jitter range of the palm is within the preset value and is recognized as a valid gesture. For example, it is determined whether the jitter range does not exceed the preset value of 5 cm in a predetermined time of 0.5 seconds, and if so, it is recognized as a valid gesture, and the center point of the region where the palm is located within the preset time is identified as a palm. starting point.

Further, the third embodiment of the gesture recognition-based trolley control method is based on the first embodiment of the above-described gesture recognition-based trolley control method of the present invention. In this embodiment, the determining the end position of the palm includes: After the initial position of the palm, delaying the first preset time; determining whether the first jitter range of the palm is in the first pre-predetermined time after the first preset time is counted When the first jitter range of the palm is within the first preset value, the center point of the region where the palm is located in the second preset time is identified as the end position of the palm.

The third embodiment is different from the second embodiment in that after the initial position of the palm is recognized, a first preset time needs to be delayed, and no recognition processing is performed in the first preset time. After the first preset time is counted, it is determined whether the first jitter range of the palm is within the first preset value by the second preset time, and if it is within the first preset value, it is recognized as a valid gesture. For example, after the initial position of the palm is recognized, the first preset time is delayed by 0.5 seconds, and after 0.5 seconds, the first jitter range of the palm is judged to not exceed the first preset within 0.5 second of the second preset time. A value of 5 cm is shaken, and if so, the center point of the area where the palm is located within 0.5 of the second preset time is identified as the end position of the palm. Here, the purpose of increasing the delay of the first preset time is to recognize the start position of the palm, and since the gesture of the subsequent palm is still in continuous motion, the action of whether the gesture ends or not can not be immediately followed, according to the gesture of different people. Select a suitable delay time for the speed, such as 0.5 seconds. After 0.5 seconds, the jitter of the palm can be judged so that the end position of the palm can be judged accurately.

Further, the fourth embodiment of the gesture recognition-based trolley control method is based on the second or third embodiment of the above-described gesture recognition-based trolley control method of the present invention. In this embodiment, the jitter of the palm is determined by the gray scale according to the palm. The value determines the change in the displacement of the palm.

Since the depth information of the object can be acquired by the Kinect infrared sensor, when the preset motion of the user recognizes the area where the palm is located, the gray value parameter of the image of the area where the palm is located can be further recognized, that is, the depth information, that is, the palm and the Kinect sensor. The distance information can be determined according to the depth information of the user's palm position and the approximate distance of the Kinect sensor, such as different distance intervals, such as 2-3 meters, 3-4 meters, ..., 19- A distance range of 20 meters, according to which distance, a preset distance for recognizing the movement of the image in the palm screen when the palm is moved by a preset distance such as 5 cm can be converted. After the preset gray value parameter, that is, the depth information parameter, is obtained in the area where the user's palm is located, when the gesture motion of the user's palm is recognized, the depth image of the gesture may be separated from the depth image of the user by the depth information of the area where the palm of the user is located. The information, and further based on the depth information combined with the range of the palm's jitter in the screen, can identify the range value of the palm time jitter. For example, after the palm of the hand is determined by the palm joint (one point), the palm is judged to be 10 meters away from the lens according to the depth information of the palm. If the distance of the palm in the screen is within 0.1 cm within 0.5 s, the actual palm shake can be judged. Within 5cm, it is recognized as a valid gesture according to its actual palm jitter within the preset value.

Further, the fifth embodiment of the gesture recognition based car control method is based on the first embodiment of the above-described gesture recognition based car control method according to the present invention. In this embodiment, the palm movement is recognized based on the starting position and the ending position of the palm. The trajectory specifically includes: establishing a coordinate system based on the starting position of the palm, and identifying a motion trajectory of the palm by determining coordinates of the starting position and the ending position.

By judging the starting position of the palm and starting from the starting position, the coordinate system is established, the positive direction of the y-axis is downward, and the positive direction of the x-axis is to the right, and the difference between the horizontal and vertical coordinates and the ordinate of the end point and the starting point is calculated ( End point minus starting point), the trajectory of the palm is recognized by the difference between the ordinate and the ordinate of the end point and the starting point. If the ordinate difference is positive and the absolute value is greater than the absolute value of the abscissa difference, it is recognized as palm up. Movement; the ordinate difference is negative, and the absolute value is greater than the absolute value of the abscissa difference, which is recognized as the palm moving downward; the abscissa difference is positive, and the absolute value is greater than the absolute value of the ordinate coordinate, which is recognized as the palm moving to the right; The difference in the abscissa is negative, and the absolute value is greater than the absolute value of the ordinate difference, which is recognized as the palm moving to the left.

It should be noted that when the coordinate system is established based on the starting position of the palm, and the motion trajectory of the palm is recognized by determining the coordinates of the starting position and the ending position, in order to identify more accurately, when determining the starting position of the palm, it is necessary to delay one. The end position is recognized only after the first preset time is 0.5 s. Recognizing the movement trajectory of the palm as the palm moves upward, the palm moves downward, the palm moves to the right, and the palm moves to the left, since the initial position of the palm is recognized as continuous motion, the first preset time is The gesture is not finished at the end, and the jitter range of the palm is not detected within the first preset value, that is, the end position of the palm cannot be detected; but the second preset time after the first preset time When the range of the jitter of the palm is detected within the first preset value, that is, the center point of the region where the palm is located in the second preset time is the end position of the palm, and the end of the second preset time is the gesture. End time; for the hand to master the boxing action, since the palm has no motion, the end of the first preset time can immediately recognize that the range of the palm of the hand is within the first preset value, that is, the end position of the palm is recognized, This recognizes that the hand grasps the punching action, that is, whether or not the palm is a clenching action based on whether or not the end position of the palm is immediately recognized at the end of the first preset time.

The invention also provides a trolley control device based on gesture recognition.

Referring to FIG. 2, FIG. 2 is a schematic diagram of functional modules of a first embodiment of a vehicle control device based on gesture recognition according to the present invention.

In this embodiment, the gesture recognition based car control device includes:

Determining module 1 for determining a starting position and an ending position of the palm;

The determining module 2 is configured to identify a motion track of the palm based on the starting position and the ending position of the palm, and determine a gesture signal of the palm according to the motion track of the palm;

The command generating module 3 is configured to retrieve a gesture control command corresponding to the gesture signal according to the gesture signal;

The control module 4 is configured to control the movement of the trolley according to the gesture control command.

In this embodiment, by identifying the start position and the end position of the palm, and determining the gesture signal of the palm according to the start position and the end position, and acquiring the gesture control command corresponding to the gesture signal based on the gesture signal, and finally The gesture command controls the movement of the trolley, realizes the gesture of recognizing the palm of the hand in a simple and accurate manner to control the movement of the trolley, and solves the problem of the recognition rate reduction caused by the interference of ambient lighting, background and the like in the existing dynamic gesture detection. The present invention can realize continuous gesture recognition within 2 to 20 meters by judging the jitter range based on the gray value.

Further, the second embodiment of the gesture recognition-based car control device is based on the first embodiment of the above-described gesture recognition-based car control device according to the present invention. In the embodiment, the determining module is specifically configured to be based on the palm joint point. Determining a position of the palm, and determining whether the range of the palm of the hand is within a preset value within a preset time; and identifying the effective gesture when the range of the jitter of the palm is within the preset value, identifying the pre- Set the center point of the area where the palm is located to the starting position of the palm.

In the embodiment, the determining module 1 recognizes that the joint point of the palm can be identified based on the ready-made library function, and the position of the palm (one point) can be determined according to the joint point of the palm. Before the starting position of the palm is recognized, whether It is effective gesture recognition processing, that is, de-jitter processing, specifically determining whether the jitter range of the palm is within a preset value within a preset time. If the jitter range of the palm is within a preset value within a preset time, it is recognized as Effective gestures. For example, it is determined whether the jitter range does not exceed the preset value of 5 cm in a predetermined time of 0.5 seconds, and if so, it is recognized as a valid gesture, and the center point of the region where the palm is located within the preset time is identified as a palm. starting point.

Further, the third embodiment of the gesture recognition-based car control device is based on the first embodiment of the above-described gesture recognition-based car control device according to the present invention. In the embodiment, the determining module is specifically configured to recognize the palm of the hand. After the initial position, delaying the first preset time; determining whether the first jitter range of the palm is at the first preset value within a second preset time after the first preset time is counted When the first jitter range of the palm is within the first preset value, the center point of the region where the palm is located in the second preset time is identified as the end position of the palm.

Further, the fourth embodiment of the gesture recognition-based car control device is based on the second or third embodiment of the above-described gesture recognition-based car control device of the present invention. In this embodiment, the jitter of the palm is judged by the gray scale according to the palm. The value determines the change in the displacement of the palm.

Further, the fifth embodiment of the gesture recognition-based car control device is based on the first embodiment of the above-described gesture recognition-based car control device according to the present invention. In this embodiment, the palm movement is recognized based on the starting position and the ending position of the palm. The track details include:

In the above embodiment of the vehicle control device based on gesture recognition, combined with a specific car control system, each module of the device can be specifically set as follows:

The car control system detects the image information of the palm of the user through the Kinect sensor and can further detect the image gray value information of the position of the palm, and further recognizes the starting position and the ending position of the palm, so the determining module 1 is set in the Kinect sensor, Kinect After the sensor recognizes the starting position and the ending position of the palm, the position information is sent to the terminal device such as the PC end, and the Kinect sensor can be connected to the terminal device by wire, and the terminal device judges the gesture of the palm according to the starting position and the ending position. Signal, so the judging module 2 is set in the terminal device, or the Kinect sensor can directly determine the gesture signal and send it to the terminal device based on the initial position and the end position analysis of the palm, and the judging module 2 is also set in the Kinect sensor. The device retrieves the gesture control command corresponding to the gesture signal according to the gesture signal, so the command generation module 3 is set in the terminal device, and finally the terminal device sends the gesture control command to the controller on the trolley by wireless transmission, such as Bluetooth, the car Controller according to hand Car operation control command, the control module 4 is provided on the carriage.

The above are only the preferred embodiments of the present invention, and are not intended to limit the scope of the invention, and the equivalent structure or equivalent process transformations made by the description of the present invention and the drawings are directly or indirectly applied to other related technical fields. The same is included in the scope of patent protection of the present invention.

Claims

A trolley control method based on gesture recognition, characterized in that the control method comprises the following steps:

Determining a starting position and an ending position of the palm, and identifying a movement trajectory of the palm based on the starting position and the ending position of the palm, and determining a gesture signal of the palm according to the movement trajectory of the palm;

And acquiring, according to the gesture signal, a gesture control command corresponding to the gesture signal;

The cart motion is controlled according to the gesture control command.
The gesture recognition-based car control method according to claim 1, wherein the determining the starting position of the palm comprises:

Determining the position of the palm according to the joint point of the palm, and determining whether the range of the palm of the palm is within a preset value within a preset time;

When the jitter range of the palm is within the preset value, and is recognized as an effective gesture, the center point of the region where the palm is located within the preset time is identified as the starting position of the palm.
The gesture recognition-based trolley control method according to claim 1, wherein the determining the end position of the palm of the hand comprises:

After identifying the starting position of the palm, delaying the first preset time;

Determining, in the second preset time, whether the first jitter range of the palm is within the first preset value after the first preset time is counted;

When the first jitter range of the palm is within the first preset value, the center point of the region where the palm is located in the second preset time is identified as the end position of the palm.
A gesture recognition-based car control method according to claim 2 or 3, characterized in that

The judgment of the jitter of the palm is achieved by judging the change in the displacement of the palm according to the gradation value of the palm.
The gesture recognition-based trolley control method according to claim 1, wherein the motion trajectory of the palm of the hand based on the starting position and the ending position of the palm center comprises:

A coordinate system is established based on the starting position of the palm, and the motion trajectory of the palm is recognized by determining the coordinates of the starting position and the ending position.
A gesture recognition-based car control device, characterized in that the gesture recognition-based car control device comprises:

Determining a module for determining a starting position and an ending position of the palm;

a judging module, configured to identify a motion trajectory of the palm based on the starting position and the ending position of the palm, and determine a gesture signal of the palm according to the motion trajectory of the palm;

a command generating module, configured to retrieve a gesture control command corresponding to the gesture signal according to the gesture signal;

And a control module, configured to control the movement of the trolley according to the gesture control command.
A gesture recognition-based car control device according to claim 6, wherein

The determining module is specifically configured to determine a position of the palm according to the palm joint point, and determine whether the jitter range of the palm is within a preset value within a preset time; when the jitter range of the palm is within the preset value When it is recognized as a valid gesture, the center point of the area where the palm is located within the preset time is identified as the starting position of the palm.
A gesture recognition-based car control device according to claim 6, wherein

The determining module is configured to delay a first preset time after the initial position of the palm is recognized; and determine the first of the palms in the second preset time after the first preset time is counted Whether a jitter range is within the first preset value; when the first jitter range of the palm is within the first preset value, identifying a center point of a region where the palm is located in the second preset time The end position of the palm.
The gesture recognition-based car control device according to claim 7 or 8, wherein the judging of the palm of the hand is performed by determining a change in the displacement of the palm according to the gradation value of the palm.
A gesture recognition-based car control device according to claim 6, wherein

The determining module is specifically configured to establish a coordinate system based on a starting position of the palm, and identify a motion track of the palm by determining coordinates of the starting position and the ending position.