AU2018102036A4 - A search-and-rescue hexapod robot with a tele-operable mechanical arm - Google Patents

Abstract

A search-and-rescue hexapod robot with a tele-operable mechanical arm and multiple methods for controlling the same are disclosed. The hexapod robot for searching and rescuing on rough terrain by remote operation is composed of: six "C" shaped legs (1) powered by one motor each (2); two binocular vision sensor (3) and one infrared distance sensor(4) disposed on the front of the body for reflecting the real environment to the remote operator; a main control board(8) , two power supply board (7) providing 3.8 volt and 12 volt power and three motor driver (6) controlling six leg motors; a 6-battery module(5) fixed on the back of the bottom plate providing power for all the needed accessories; three signal antenna (9) transmit Wi-Fi, Bluetooth and GPS signal; a six degree of freedom mechanical arm (10) which is tele-operable. Figure 1

Description

FIELD OF THE INVENTION

The present invention relates to a kind of hexapod robot with a new control method, specifically, a hexapod robot manipulated by wearable tele-controller with higher accessibility, accuracy and information-providing capability.

BACKGROUND OF THE INVENTION

As robotics has become the most promising and innovative field of science nowadays, new industries and applications inspired by it has been emerging unprecedentedly fast. The first generation of robots were implemented in 1962 in the United States, as a more efficient replacement of human laborers, taking part in repeated factory works. In recent decades, intelligent robots, with their ability of situation judgement and information transmission, are considered a significant method to execute tasks in dangerous or inaccessible areas instead of human with even higher efficiency.

Current mechanical structure of robots is mainly wheeled, tracked, and legged. Wheeled robots have the highest speed among all kinds of land movement robot structures, but obstacles can be highly restrictive to them.

2018102036 09 Dec 2018

Tracked robots have stronger ability to cross barriers, but their low motility due to heavy weight and tracked framework lead to inefficiency. Legged structure is considered one of the most practical robot structures for urgent navigation tasks on unstructured terrains for its balance between robustness and motility. Moreover, the six-legged structure, or the hexapod structure, imitated the behavior of insect locomotion and is proved to be more efficient than conventional structures in navigation and loading capacity.

Additionally, present robots’ executive arms are in lack of a new and more precise real-time control method than key-operations (e.g., keyboard joystick) for more complex assignment. Wearable remote controller controls a machine by accord its actions with human body movements. The arm of this innovation is controlled by controller attaching to human hand in order to increase the accuracy of arm movement and in the meantime simplify the controlling process, increasing the invention’s accessibility for untrained users.

Current development of hexapod robot has been considerable, yet many problems and possible improvements exist actual applications. One rescue hexapod called “Octopus III” is designed to carry emergency supplies could bear load as heavy as 300kg, the number is impressive, but the robot is too cumbersome to finish other tasks it is designed for as a rescue robot. Apart from the mechanical problems, controlling methods

2018102036 09 Dec 2018 of robots is also a point that can be enhanced. An under-water robot utility model was advanced by Zhejiang Ocean University aiming precise under-water tasks. However, the robot’s mechanical arm was remotely controlled by touch screens and joysticks, and led to insufficient precision than intended.

In this invention, we designed and developed a new kind of hexapod robot with high navigation and mobilization capacity on unstructured surface with the utilization of biotic structure that can accomplish more precise tasks with a tele-operable mechanic arm controlled by wearable controller.

SUMMARY OF THE INVENTION

The innovation is based on a robot with gesture recognition control teleoperation that can move quickly in the wild. Most of the traditional robots with similar ability to work in the field are characterized by small size and streamlining, and the functions are mainly for reconnaissance and detection. We improve on the basis of these detection robots. On the one hand, we adopt the hexapod eccentric half-leg structure to make it stable, fast and have certain climbing ability. On the other hand, combined with the theory of teleoperation, a set of teleoperated gesture-controlled robotic arms is added on the robot to mimic human motion.

2018102036 09 Dec 2018

In terms of mechanical structure, we used Solidworks simulation software to simulate the structure, assembly and behavior of the robot. On the hardware side, we developed the development board of stm32f429 with Keli uvision5 MDK, which cooperates with the gyroscope to control the travel and gait of the whole robot. Algorithm and data reception. The robot is equipped with a robot arm with six degrees of freedom. The torso and wrist can be rotated freely. The upper arm, arms, wrists and fingers can be moved around the axis, and Epson is installed at the wrist and finger positions. The force sense and tactile sensors feedback the interaction force and touch of the remote robot and the remote environment to the operator in time, generate virtual reality about the remote environment mapping, and enable the operator to effectively control the robot to complete the operation task.

In remote control, the MEMS sensor is used to collect the arm motion information captured by the inertial motion on the master-slave teleoperation terminal. On the other hand, the motion information is converted into the joint angle information to directly control the arm joint; the Leap Motion is used to capture the operator's hand. The action of the part, in conjunction with the data glove, transmits the acquired joint posture parameters to the computer through the USB3.0 interface, and the computer sends the data to the controller of the robot through the serial port to control the three-finger part of the robot. On the other hand, this

2018102036 09 Dec 2018 design uses the MYO armband to complete the acquisition of the forearm EMG signal. MYO has built in 8 EMG sensors and an inertial measurement unit. The MYO armband presets 5 gestures: left swing, right swing, double click (thumb and middle), fist and hand. Combined with the force sensitive sensor FSR, 14 gestures were designed to control the operation mode of the robot. In this way, the operator's right hand makes the action of the robot arm, and the left hand makes a corresponding gesture to control the gait of the robot, so the teleoperation is completed in multiple points.

The robot improves the current popular wheeled type or caterpillar track of travel, with reference to the structure of the bionic insect. And for ease of control, the leg freedom is reduced to unidimensional, using a six-foot eccentric half-wheel structure; six wheels pass can be driven separately by the CAN bus to work at different speeds, phases and even directions, which enable the robot capable for the three-foot gait, the four-foot gait and the six-foot gait and the switching between them. The above three gaits enable the robot to complete the proceeding, turning, retreating, fast forwarding, climbing, stepping over the obstacles, so basically can handle most of the terrains encountering in the wild. At present, we are still trying to work on a new gait that allows the robot to turn over under special conditions when it is upside down. Because the six wheels can be operated at different speeds, it can adjust the speed and center of gravity

2018102036 09 Dec 2018 fluctuation of the robot. The relationship makes it possible to adjust its emphasis between speed and stability.

Operators may complete dangerous operations such as rescue and disaster relief, explosion through our robot, in the light of remote location, thereby ensuring the safety of the operator.

DESCRIPTION OF THE DRAWINGS

The appended drawings are only for the purpose of description and explanation but not for limitation, wherein:

Fig.l is an overall view of the present invention;

Fig.2 is the trajectory of the “C” shaped leg while moving showing a complete, smooth path;

Fig.3 is a three-view of the six degree of freedom mechanical arm;

Fig.4 is a flow chart of the control process including binocular vision cameras, GPS module, WIFI module, Bluetooth module, Sensor module, Motor drive module, pneumatic manipulator air pressure execution system, etc.

2018102036 09 Dec 2018

Fig.5 is a flow chart showing the control loop of the motors with the scheme to monitor and report the precise position of the motors with the help of a transducer, or an encoder.

Fig.6 is a picture of Zedboard.

Fig.7 is a picture of MPU-9250 9-axis attitude sensor.

Fig.8 is a picture of pressure sensors.

Fig.9 is a picture of HC-08 Bluetooth model and ESP8266 WIFI Module.

Fig. 10 is a picture of TFmini infrared distance sensor HNY-CV-001 binocular camera.

Fig.l 1 is a picture of GPS Module.

Fig. 12 is a graph showing the support phase and swing phase in each operating cycle of the wheel leg.

Fig. 13 is a schematic diagram showing two groups of six legs while traveling in a three-legged gait.

2018102036 09 Dec 2018

Fig. 14 is a schematic diagram showing four-foot gait whiling crossing obstacles.

Fig. 15 is a picture of data gloves which are used to detect the posture of fingers and joints during the movement of the user's fingers.

Fig. 16 is a picture of Leap Motion.

DESCRIPTION OF PREFERRED EMBODIMENT

In order that the present invention can be more readily understood, reference will now be made to the accompanying drawing to illustrate the embodiments of the present invention.

Referring to Fig.l, 1 is one of the “C” shaped legs; 2 is one of the motors;

is one of the binocular vision sensors; 4 is one infrared distance sensor;

is a 6-battery module; 6 is one of the motor drivers; 7 is one of the power supply boards; 8 is the main control board; 9 is one of the signal antennas transmit Wi-Fi> Bluetooth and GPS signal; 10 is a six degree of freedom mechanical arm.

The invention is designed with “C” shaped legs to advance the hexapod robot’s mobility and the adaptation ability. This specially designed shape reduces the robot’s momentum loss when contacting with hard surfaces, giving the robot more dynamics. As shown in Fig.2, the trajectory of the

2018102036 09 Dec 2018 “C” shaped leg while moving is a complete, smooth path. Compared to the conventional “L” shaped leg with segmented path, the new design largely reduces the mechanical wear of the robot joint. Moreover, the curved shape can spread the force exerted on the foot more evenly thus the robot’s resistance is further improved. On the country, the sharp-turned foot design depends majorly on the end of the structure, resulting in higher damage possibility. Moreover, the design integrated the foot structure into a whole piece, thus reduces the use of joints (servos) on the leg. Although this limits the flexibility of the robot’s leg movement, it increases the robot’s stability significantly, for joints are highly possible to be stuck with small dregs when walking through unstructured ground. The “C” shaped leg design is a crucial improvement of our invention’s stability and adaptability.

The three-view of our invention’s multipurpose mechanical arm is shown in Fig.3. It consists of a rotatable pedestal, a main arm and a two-fingered end-effector. Three stepping motors are used in total: one on the pedestal to control its rotation and two at the main arm’s joints. The end-effector is designed to be pneumatic, but electric motors can also be applied. The mechanical arm is the most significant improvement we made on the hexapod robot design. Most multi-legged robots perform search and transport tasks, with highly limited ability to interact with environment and accidents. However, this ability is crucial for this robot since its

2018102036 09 Dec 2018 purpose is to navigate under dangerous and unpredictable situations, hence an interactive mechanical arm is necessary. With this structure, the robot’s ability is extended in a considerable scale: instead of just searching for resources or people, it can remove obstacles, transfer supplies and even fundamentally repair and operate other devices with artificial teleoperation. To prevent the invention from over-weighting and stabilize its center of mass, the whole arm is designed to be fiber-reinforced-plastic, a strong, light and relatively cheap material. Three-fingered or Five-fingered end-effector can also be applied in the invention.

Fig.4 is an overall view of the control process. Referring to Fig.5, to form a negative feedback loop which is capable to obtain more precise results, a transducer, or an encoder is needed in order to monitor and report the precise position of the motors.

Zedboard (as shown in Fig.6) is used to connect with other peripherals. Zedboard includes Zynq™-7000 processor which is based on the Xilinx® All Programmable SoC (AP SoC) architecture which integrates a feature-rich dual-core ARM® Cortex™-A9 MPCore™ based processing system(PS) and Xilinx programmable logic (PL) in a single device. DDR3 memory (512MB) is connected to the hard memory controller in the Processor Subsystem (PS), the SPI Flash connects to the Zynq-7000 AP SoC supporting up to Quad-I/O SPI interface and the Zynq PS io

2018102036 09 Dec 2018

SD/SDIO peripheral controls communication with the ZedBoard SD Card. There are 33.33333 MHz clock source for PS and 100 MHz oscillator for PL. Users can use the Zedboard communicated with other devices by 10/100/1000 Ethernet or USB OTG 2.0 (universal serial bus) and configure and debug it by Xilinx Platform Cable JTAG connector.

An attitude sensor is an integration of a three-axis gyroscope, a three-axis accelerometer, and a three-axis electronic compass. Used for the detection of the finger motions of the user in this invention, the sensor have to be reliable and precise. The sensor chosen is MPU-9250 9-axis attitude sensor (as shown in Fig. 7).

The pressure sensors are used on the data glove responsible for gait control as well as the pneumatic mechanical hand (as shown in Fig.8 left). In this control of movement and gait selection of the hexapod robot, a teleoperation system based on gesture recognition technology is designed. Force - sensitive sensor FSR (as shown in Fig. 8 right) is selected to collect the hand back pressure signal. Force - sensitive sensor (FSR) is a thick film polymer sensor whose resistance will decrease with the increase of pressure applied to the sensor surface. Moreover, the force-sensitive characteristic of FSR sensor optimizes human touch, so it is very suitable for collecting touch signals.

The Bluetooth model and WIFI Module (as shown in Fig.9) are used to transmit information between the teleoperation system and the robot

2018102036 09 Dec 2018 bidirectionally, which is crucial for the realization of adequate teleoperation performance. For the WIFI module, we chose ESP8266 model for its universality in different micro controllers. The chosen Bluetooth module is HC-08 Bluetooth module, with Bluetooth V2.0 protocol.

One of the most important prerequisites of teleoperation is that the user can feel the slave machine’s condition, especially, the machine’s view. The use of two cameras from different angles improves the visual feedback from 2d to 3d. Thus, the model structure and texture information can be reconstructed. We find that HNY-CV-001 binocular camera (as shown in Fig. 10 below) is competent and accessible. Considering the dangerous and unpredictable working condition of the robot, a distance-ranging module can assist the robot avoid unnecessary damages due to collision or entering in extreme environment (fires, combustible gases, etc.). Taking all aspects in consideration, the chosen ranging module is TFmini (as shown in Fig. 10 up).

The hexapod robot with rocking operation special service perceives its own location through GPS (as shown in Figs. 11), and can complete map construction, positioning, motion control and path planning in its work, realizing autonomous movement in a complex environment, thus completing a specific task.

2018102036 09 Dec 2018

The separate control of the six wheels of the robot base is realized by the CAN control system. The CAN-BUS system is a network system that connects different devices and uses code for data transmission. It can connect all the devices with the least number of lines, can be used for the data transfer of various wheels on the car and the decoder and engine that control them, so that each subsystem can work independently. The use of CAN-BUS makes it possible for the robot to complete the asynchronous state and the way of travel.

As shown in Fig. 12, we divide each operating cycle of the wheel leg into two phases: the phase that is in contact with the ground and has the effect of force is called the “ support phase ” , which is not in contact with or touching the ground but does not exist. The stage of action is called the swing phase. The angular velocity of the rotation of the wheel legs in the supporting phase should be smaller than the angular velocity of the wheel legs in the swinging phase. This is to enable the wheel and leg structure of the robot to balance the speed and stability: the smaller the angular velocity ratio between the supporting phase and the swinging phase, the smaller the angle of the supporting phase. The smaller the center of gravity of the robot is.

In order to make the robot flexible, we have three basic gaits: three-legged gait, four-legged gait and six-legged gait.

2018102036 09 Dec 2018

When traveling in a three-legged gait, the six legs of the robot are divided into two groups, as shown in Fig. 13. When one group is in the supporting phase and the other group is in the swinging phase, the ground is alternately supported. The speed of the robot in this gait can be very fast, but because of the three-legged support, the stability is lower compared to other gaits.

The steering of the robot is also done in three-legged gait. The other operating conditions remain unchanged. For the left turn, for example, the left three legs rotate in the opposite direction, and the right three legs rotate in the forward direction to complete the left turn.

When traveling in three-legged gait, the six feet of the robot are in the swing phase or the support phase at the same time. The robot travels slowly in this gait, but it has the most power and is suitable for complex terrain such as muddy land and sand.

The four-foot gait is suitable for situations where obstacles are crossed.

As shown in Fig. 14, the forefoot, midfoot, and hindfoot are synchronized respectively; at any time, the robot has four feet in the support phase. The forefoot leads the phase ahead of the hind foot, and when it encounters an obstacle, it relies on the forefoot to reach the obstacle and complete the overstep.

The selection of the asynchronous state is controlled by the gesture recognition module, the gesture recognition device is made into a

2018102036 09 Dec 2018 wearable device, and the application based on the algorithm is studied. The algorithm uses FSR to collect back pressure changes and fixes the FSR on the gloves in the form of data gloves. The FSR acquisition circuit is integrated into the collection watch and the forearm EMG is collected using an inexpensive wearable MYO armband. The signal is changed, and then the collected gesture signal is analyzed and processed, and the extracted gesture feature value is used as an input of kNN, thereby completing fine gesture recognition. The recognition of the gait selection and motion teleoperation gestures of the commonly used hexapod robot is realized.

The control of pneumatic manipulator is mainly divided into two parts: forward control and feedback. Forward control means that the user completes the grasping control of the flexible manipulator at the hexapod robot end by inputting instructions from the attitude detection equipment such as data gloves. Feedback refers to the use of the attitude data acquisition system located in the part of the manipulator to collect attitude data information such as the joint angle of the manipulator, fingertip pressure, etc. and then feed back to the upper computer located at the user end for processing and display, so as to specifically deal with practical problems in the grabbing process.

These two parts will be described at following part separately. The first part is forward control.

2018102036 09 Dec 2018

The realization of forward control is mainly achieved by two systems: the attitude detection system at the user side and the pneumatic execution system at the robot side.

The attitude detection system is mainly composed of data gloves (as shown in Fig. 15) and Leap Motion (as shown in Fig. 16), which are used to detect the posture of fingers and joints during the movement of the user's fingers, so as to convert the actions the user wants the robot to do into commands input.

Data glove can be obtained through the installation of attitude sensor on the gloves. Attitude sensor is the motion measurement system with high performance on 3D motion detection based on MEMS technology. Another other component is Leap Motion. Leap Motion is a virtual motion-sensing device based on image recognition technology. It is equipped with two infrared cameras, which can take pictures at high speed, and a high frequency of GPU that can track the movement of the hand with the speed of 200 tons per second.

Pneumatic execution system mainly consists of electric proportional valve, electromagnetic directional valve, pressure reducing valve, throttle valve, air pump, etc. Each finger of the airway is controlled independently.

The basic process of the systems above is as follows. The attitude acquisition system detects the posture of the user's finger, and after the

2018102036 09 Dec 2018 processing of the core control module of the upper computer, the position and movement information of the finger are obtained, and the display function of the upper computer will display these information in real time. At the same time, these signals will be transmitted via Bluetooth to the control module of the lower computer of the hexapod robot. After processing, these signals will be transformed into control instructions for controlling the pneumatic execution system, so as to realize the function of reappearing the user's finger posture on the manipulator.

The main part functioning in the feedback process is the data acquisition system located in the robot end, which mainly includes the functions of pressure data collection, contact pressure collection and attitude sensor parameter collection.

The feedback signals will be passed to the next bit machine control module for processing. After that, the processed signals are transmitted to the PC and local control system via Bluetooth. Local control system is located in the six-legged robot end, according to the feedback signals, it can do some negative control to the pneumatic manipulator, achieving the reliable and stable operation of a single joint, the whole finger, and a few.

Claims (2)

1. A search-and-rescue hexapod robot with a tele-operable mechanical arm, which is equipped with a six degree of freedom mechanical arm on the top of the hexapod robot, it is a multifunctional utility robot; the extremity of the mechanical arm can equip force sensor, cutting machine or electric welder, which can be used in flexible crawling, moving obstacle, and even fundamentally repairing and operating other devices.

2. A search-and-rescue hexapod robot with a tele-operable mechanical arm mentioned as claim 1, the movement of which is controlled by remote operator, its motion instruction is generated by perceiving arms posture with wearable devices, the robot’s gait is controlled by one arm, and the mechanical arm’s action is controlled by the other arm, the arm posture controlling instructions are pre-designed, and while feeling the change of arm posture the robot will take the relevant action.