WO2021082376A1

WO2021082376A1 - Dual-mode control method for table tennis ball retriever robot

Info

Publication number: WO2021082376A1
Application number: PCT/CN2020/087354
Authority: WO
Inventors: 范青武; 杨凯; 王迪; 马帅
Original assignee: 北京工业大学
Priority date: 2019-10-31
Filing date: 2020-04-28
Publication date: 2021-05-06
Also published as: CN110681133A

Abstract

The present invention relates to a dual-mode control method for table tennis ball retriever robot. Working modes of the robot can be switched by means of an upper computer App. In an automatic cruise mode, the robot obtains the position information of a table tennis ball within the field of view by a vision system, and adjusts the posture according to coordinates of the ball to find an optimal collection angle. In a manual remote control mode, the upper computer App sends a motion instruction to the robot by means of Bluetooth to control the work of the robot. The robot also has an obstacle avoidance function and is able to automatically avoid obstacles in two working modes. The advantages of the present invention are as follows: switch can be performed between the two working modes, the automatic cruise and manual remote control modes, as needed; the robot is applicable in a wide range of usage scenarios due to the obstacle avoidance function; during collecting table tennis balls in the automatic cruise mode, the robot can slightly adjust its own posture according to the coordinates of the table tennis ball, and the influence of the field of view and distance on the coordinates is eliminated; and in addition, the structure of the whole algorithm is clear and rigorous, and different functions correspond to different threads, thus facilitating daily maintenance and secondary development.

Description

A dual-mode control method for picking up table tennis robot

Technical field

The invention relates to the technical field of robots, in particular to a dual-mode control method for a table tennis robot.

Background technique

With the deepening of people's understanding of the intelligent nature of robotics, robotics has begun to continuously penetrate into all areas of human activities. Among them, service robots, as an important branch, have received widespread attention in research fields at home and abroad. Service robots have a wide range of applications, mainly engaged in maintenance, repair, transportation, cleaning, security, rescue, monitoring and other tasks. The ball picking robot involved in the present invention is an autonomous mobile service robot applied to table tennis sports.

The training ground for table tennis is relatively open. Athletes need to use a lot of table tennis during training. As a result, table tennis will be scattered all over the venue. Picking up the table tennis manually will waste a lot of time and energy.

There are already many auxiliary tools on the market that can help people collect ping pong balls, but these tools still need to rely on manpower to collect balls, and they have not liberated people from this boring work.

The invention patent with application number 201810308608.9 discloses a ball picking machine, which can shovel table tennis balls into the collecting bucket and finally into the collecting basket by turning and resetting the collecting bucket, but because it cannot automatically find and collect table tennis The ball is still completely dependent on human remote control.

The invention patent with application number 201910121639.8 discloses a robot that can automatically cruise and collect table tennis balls, but it is not equipped with a range-finding sensor. It cannot avoid obstacles during the cruise, and it is not easy to avoid obstacles such as wall roots and table corners. The collection of table tennis balls is very difficult, and on the whole cannot meet all the needs of daily use.

In view of the above situation, the present invention has designed a dual-mode control method for picking up table tennis robots. It adopts dual modes of automatic cruise and manual remote control. Under normal scenes, the automatic cruise mode is used to cruise and collect table tennis in the stadium. ; When the ball is in a special position or you want to reclaim the robot, you can switch to the manual remote control mode, and send motion instructions through the upper computer App to control the robot's work, which greatly improves the robot's work efficiency.

Summary of the invention

The purpose of the present invention is to provide a bimodal control method for picking up a table tennis robot. The specific technical solutions adopted are as follows:

A working schematic diagram of a dual-mode control method for picking up a table tennis robot is shown in Figure 2. The robot has two working modes: automatic cruise and manual remote control, which can be switched by the upper computer App. The upper computer App is developed based on the Android platform. Through Bluetooth communication with the robot, it can realize the four functions of the robot's mode switching, obstacle avoidance function switch, wireless remote control and operating data monitoring.

Further, in the automatic cruise mode: the robot cruises the entire field, recognizes and locates the ping pong ball in the field of view through the vision system, and when it recognizes the ping pong ball, the robot fine-tunes its posture and performs the collection operation;

Further, in the manual remote control mode: the upper computer App connects to the robot via Bluetooth and sends motion instructions. After receiving the instructions, the robot translates them into serial commands, which are processed by the coprocessor and drive the vehicle body to complete the specified motion.

Further, when the obstacle avoidance function is turned on: the program receives the distance information from the ultrasonic distance sensor in real time, and if the distance from the obstacle is less than 15cm, the obstacle avoidance operation is executed.

The present invention has the following advantages:

1) It adopts two working modes of automatic cruise and manual remote control and can be switched at will, with the obstacle avoidance function, the use scene is very wide;

2) When collecting table tennis balls in automatic cruise mode, the algorithm is optimized so that the robot can fine-tune its own posture according to the table tennis coordinates, and eliminate the influence of the field of view and distance on the coordinates;

3) The structure of the whole algorithm is clear and rigorous, and different functions correspond to different threads, which is easy for daily maintenance and secondary development.

Description of the drawings

Figure 1 is a schematic diagram of the structure of the table tennis robot in the embodiment;

Figure 2 is a schematic diagram of the work of the table tennis robot;

Figure 3 is a flowchart of the upper computer App in the present invention;

Figure 4 is the upper computer App interface of the present invention;

Figure 5 is a flowchart of manual remote control in the present invention;

Figure 6 is a flowchart of the automatic cruise mode in the present invention;

Figure 7 is a flowchart of the obstacle avoidance function of the present invention;

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the following further describes the present invention in detail with reference to the accompanying drawings and specific embodiments, but it is not intended to limit the present invention.

Detailed ways:

As shown in Figure 1, the table tennis robot in the embodiment is composed of a ball picking device and a car body. The ball picking device is a fan blade made of ball bearings, long screws, hard plates, and soft brushes. The drum structure is driven by the combination of motor, belt and fixed pulley to rotate the fan blades. When picking up the ball, the brush drives the ping-pong ball into the storage device along the slope, thus realizing the collection of the ping-pong ball. In addition, the ball picking device is also Installed with visual sensor and ultrasonic distance sensor; the car body part has a two-layer structure, the upper layer is placed with the main and sub-level processors, lithium batteries and the motor that drives the rotation of the ball picking device blades; the lower layer is the ball storage device, It is an open cuboid box shape with an opening facing the front end of the car body to store the table tennis balls collected by the ball picking device. The lower chassis is equipped with four mecanum wheels and their control motors, allowing the car body to achieve any 360° Move in direction.

As shown in Figure 2, the working mode of the ping-pong robot can be switched by the upper computer App. In the automatic cruise mode, the main control program of the robot can identify and locate the ping-pong ball in the field of view according to the image information returned by the sensor, and drive it. The robot adjusts its posture and collects table tennis balls; in the manual remote control mode, the App communicates with the robot via Bluetooth, sends movement instructions, and drives the vehicle body to complete the specified movement after program translation; the switch of the obstacle avoidance function is also controlled by the upper computer App, which is turned on When the time, the program monitors the distance between the robot and the obstacle through the ultrasonic distance sensor, and executes the obstacle avoidance operation when it is less than the preset value; the important data generated when the robot is working will be sent to the App via Bluetooth in real time and displayed on its main page. On the interface, so that users can monitor the running status of the robot.

App design:

As shown in Figure 3, the upper computer App is developed based on Android. When it is working, it first obtains the BluetoothAdapter object to determine whether the device supports Bluetooth and whether the Bluetooth is turned on. If it is not turned on, the openBlueSync method is called to prompt the user to turn on Bluetooth. Call after Bluetooth is turned on The startLeScan and connectGatt methods scan and connect to the Bluetooth serial port module. After successfully pairing with the module and establishing a stable connection, the communication can be started. The entire communication read and write process is completed in the asynchronous callback function of BluetoothGattCallback. After all the communication is completed, the software will call BluetoothGatt The disconnect and close methods in disconnect the connection with the Bluetooth module of the robot.

As shown in Figure 4, in the main interface of the upper computer App, the uppermost display is the connection status between the App and the robot. The two

buttons

2 and 3 below it correspond to the switching of the robot mode and the obstacle avoidance function. You can change the working status of the robot by clicking the corresponding button; the robot remote control components 4 and 5 are located in the middle of the interface. Drag the joystick 4 in the center to control the robot to move in eight directions. There are two arrow-shaped buttons below the joystick. 5.1, 5.2 The robot can rotate in the corresponding direction; the gray frame 6 at the bottom of the main interface is the data monitoring column, and the current working data of the robot will be updated in the column in real time, including the information of turning on (off) the machine and the table tennis in the field of view. , The received motion instructions and the distance to obstacles, etc.

Design of manual remote control mode:

As shown in Figure 5, in the manual remote control mode, the upper computer App needs to establish a connection with the Bluetooth module of the robot and transfer data to each other; in the main control program, each function runs on different threads without interfering with each other. The change mode sent by the App or The switch function command can be realized by blocking and activating the corresponding thread; if the App sends a remote control command, the two-stage processor will translate the command into a serial port command and finally generate a PWM wave to control the encoder to drive the car body to complete the specified movement.

The serial port command format of the robot is shown in Table 1:

Table 1

A complete serial port command contains ten bytes, of which the first three bytes are the fixed command header, the fourth byte is the high 8 bits of the X-axis speed, and the fifth byte is the low 8 bits of the X-axis speed. , The next four bytes correspond to Y-axis and Z-axis speeds respectively. The last byte is used to specify the positive direction of each axis, and the final motion of the robot is the vector combination of the three-axis motion. During the process, the user's remote control information will be transmitted to the upper computer App for display in real time.

Design of automatic cruise mode:

As shown in Figure 6, there are two states in automatic cruise mode: cruise state and ball picking state;

When there is no table tennis in the field of view, the robot is in a cruising state. During the cruising process, the robot moves forward in the current direction. When encountering an obstacle, it moves back a short distance and turns 75 degrees in the direction where there is no obstacle. If there are obstacles on both sides Object, repeat the above operation, when the vision system recognizes the ball, the robot switches to the ball picking state;

The ball picking state is divided into adjustment phase and collection phase:

Adjustment stage: Because the robot's field of view is a fan shape, coupled with the perspective principle and the vibration of the image sensor, the coordinates that are recognized for the first time are directly used for processing, which will reduce the collection success rate. Therefore, after the ball is recognized, it is necessary to enter the adjustment stage to calibrate the position of the ball and the orientation of the robot. The principle is as follows. In the embodiment, the resolution of the camera used by the robot is 640*480, and the field of view is about 60 degrees. And experimental demonstrations prove that when the distance between the robot and the ping-pong ball is 20-40 cm, the x-coordinate of the ball is between 180 and 370, and the best collection effect is between the robot and the ball 40-60 cm. 340. At work, when the robot recognizes the ping-pong ball, it will pause to move, and then accurately recognize the ball’s coordinates again, and then estimate the distance to the robot based on its radius, select the appropriate best collection interval, and based on the ping-pong ball position coordinates and the best collection The posture adjustment is continuously carried out on the difference of the interval coordinates until the table tennis ball is in the best collection interval and enters the collection stage;

Collection stage: According to the estimated distance, get the approximate moving time and drive the car body to move. If the ping pong ball can be recognized in the field of view after the collection operation is performed once, the robot returns to the adjustment stage of picking up the ball and continues to pick up the ball. If there is no ping pong ball in the field of view, it changes to the cruise state.

In the automatic cruise mode, the robot's working data will be uploaded to the upper computer App in real time, which is convenient for users to monitor and maintain the robot.

Design of obstacle avoidance function:

As shown in Figure 7, the obstacle avoidance function can be turned on or off on the upper computer App; when the function is turned on, the program receives the distance information from the ultrasonic distance sensor in real time. If the distance between the obstacle and the obstacle is less than the preset value of 15cm, it will execute Obstacle avoidance operation; the robot rotates to the left and right to confirm whether there are obstacles on the left and right. After the confirmation is completed, the robot rotates 75 degrees in the direction where there are no obstacles. If there are obstacles on both sides, the robot retreats and performs the above confirmation operation again until There is no obstacle nearby, the robot turns to this direction and continues to work; the data in the obstacle avoidance state will be uploaded to the upper computer App in real time.

The communication protocol involved in the method:

The main control program of the robot runs on the development board. The vision sensor, Bluetooth module, ultrasonic ranging sensor and the main control program communicate through the serial port; the Bluetooth module uses the HC-08 Bluetooth serial communication module, and its wireless working frequency band is 2.4 GHz ISM, the modulation method is GFSK. The maximum transmit power of the module is 4dBm, and the receiving sensitivity is -93dBm. It can realize long-distance communication of 40 meters in an open environment. It uses Bluetooth Specification V4.0 BLE Bluetooth protocol when connecting with App and data transmission, and supports Android 4.3 and above systems.

Claims

A dual-mode control method for a robot for picking up table tennis is characterized in that: the picking-up table tennis robot is composed of a ball picking device and a car body. The ball picking device uses ball bearings, long screws, hard plates, and soft The roller structure with fan blades composed of high-quality brushes and other materials drives the fan blades to rotate through the combination of motors, belts, and fixed pulleys. When picking up the ball, the brush drives the ping-pong ball into the storage device along the slope, thereby realizing the ping-pong For ball collection, in addition, visual sensors and ultrasonic distance sensors are installed on the ball picking device; the car body has a two-layer structure, and the upper layer is equipped with two-stage processors, lithium batteries and driving the ball picking device to rotate. The motor; the lower layer is the ball storage device, which is an open rectangular box shape, the opening faces the front of the car body, and stores the table tennis balls collected by the ball picking device. The lower chassis is equipped with four mecanum wheels and their The motor can be controlled to make the car body move in any direction of 360°; the robot has two working modes: automatic cruise and manual remote control, which can be switched through the upper computer App. The upper computer App is developed based on the Android platform and communicates with the robot through Bluetooth. There are four functions for robot mode switching, obstacle avoidance function switch, wireless remote control and running data monitoring.
The dual-mode control method of a robot for picking up ping-pong balls according to claim 1, wherein the main control program of the robot in the automatic cruise mode is based on the image information returned by the sensor to control the objects in the field of view. The table tennis is identified and positioned, drives the robot to adjust its posture and collects table tennis balls; in the manual remote control mode, the upper computer App communicates with the robot via Bluetooth, sends motion instructions, and drives the vehicle body to complete the specified motion after program translation; the avoidance The switch of the obstacle function is controlled by the upper computer App. When it is turned on, the program monitors the distance between the robot and the obstacle through the ultrasonic distance sensor, and executes the obstacle avoidance operation when it is less than the preset value; the important data generated when the robot is working, It will be sent to the App via Bluetooth in real time and displayed on its main interface so that the user can monitor the running status of the robot; when the upper computer APP is working, first obtain the BluetoothAdapter object to determine whether the device supports Bluetooth and whether Bluetooth is turned on. If it is not turned on, Call the openBlueSync method to prompt the user to turn on Bluetooth. After Bluetooth is turned on, call the startLeScan and connectGatt methods to scan and connect to the Bluetooth serial port module. After successfully pairing with the module and establishing a stable connection, communication can begin. The entire communication read and write process is in the asynchronous callback function of BluetoothGattCallback After all the communication is completed, the software will call the disconnect and close methods in BluetoothGatt to disconnect from the Bluetooth module of the robot; in the main interface of the upper computer App, the connection status between the App and the robot is displayed at the top 1, The two buttons 2 and 3 below it correspond to the switch of robot mode and the switch of obstacle avoidance function. You can change the working state of the robot by clicking the corresponding button; the robot remote control component 4 and 5 are located in the middle of the interface, drag the center The joystick 4 can control the robot to move in eight directions. The two arrow-shaped buttons 5.1 and 5.2 below the joystick can make the robot rotate in the corresponding direction; the gray box 6 at the bottom of the main interface is the data monitoring bar, the current robot The work data will be updated in real time in the column, including the machine on (off), the information of the table tennis in the field of view, the received movement instructions and the distance to the obstacle.
The dual-mode control method of a robot for picking up ping-pong balls according to claim 1-2, wherein: in the manual remote control mode, the upper computer App needs to establish a connection with the Bluetooth module of the robot to communicate with each other Data; In the main control program, each function runs on different threads without interfering with each other. The command to change the mode or switch function sent by the App can be realized by blocking and activating the corresponding thread; if the App sends a remote control command, two levels The processor will translate the instruction into a serial port instruction and finally generate a PWM wave to control the encoder to drive the car body to complete the specified movement; the serial port instruction format of the robot is shown in Table 1:

Table 1

A complete serial port command contains ten bytes, of which the first three bytes are the fixed command header, the fourth byte is the high 8 bits of the X-axis speed, and the fifth byte is the low 8 bits of the X-axis speed. , The next four bytes correspond to Y-axis and Z-axis speeds respectively. The last byte is used to specify the positive direction of each axis. The final motion of the robot is the vector combination of the three-axis motion; the information of the user's remote control will be transmitted to the upper computer App for display in real time.
A dual-mode control method for a robot for picking up ping pong according to claim 1-2, characterized in that: there are two states in the automatic cruise mode: cruise state and ball picking state; when there is no ping pong in the field of view When the ball, the robot is in a cruise state. During the cruise, the robot moves forward in the current direction. When encountering an obstacle, it moves back a short distance and turns 75 degrees in the direction where there is no obstacle. If there are obstacles on both sides, repeat the above Operation, when the vision system recognizes the ball, the robot switches to the state of picking up the ball;

The ball picking state is divided into an adjustment phase and a collection phase:

Adjustment stage: After the ball is recognized, it is necessary to first enter the adjustment stage to calibrate the position of the ball and the orientation of the robot; at work, when the robot recognizes the ping-pong ball, it will pause to move, and then accurately identify the coordinates of the ball, and then estimate it based on its radius Select the appropriate optimal collection interval for the distance from the robot, and continuously adjust the posture according to the difference between the coordinates of the table tennis position and the coordinates of the optimal collection interval until the table tennis is in the optimal collection interval and enter the collection stage;

Collection stage: According to the estimated distance, get the approximate moving time and drive the car body to move. If the ping pong ball can be recognized in the field of view after the collection operation is performed once, the robot returns to the adjustment stage of picking up the ball and continues to pick up the ball. If there is no ping pong ball in the field of view, it will switch to the cruise state; in the automatic cruise mode, the robot's work data will be uploaded to the upper computer App in real time, which is convenient for the user to monitor and maintain the robot.
The bimodal control method of a robot for picking up table tennis according to claim 1-2, characterized in that: the obstacle avoidance function can be turned on or off on the upper computer App; when the function is turned on, the program Receive the distance information from the ultrasonic distance sensor in real time. If the distance between the obstacle and the obstacle is less than the preset value of 15cm, the obstacle avoidance operation will be executed; the robot rotates to the left and right to confirm whether there are obstacles on the left and right sides. After the confirmation is completed, the robot is facing no obstacles Turn the direction of the object by 75 degrees. If there are obstacles on both sides, the robot will retreat and perform the above confirmation operation again. Until there are no obstacles nearby, the robot will turn to this direction and continue to work; the data in the obstacle avoidance state will be uploaded in real time. Host computer App.