CN111283680B - System and method for wireless remote control robot - Google Patents

Abstract

The invention relates to a system for wireless remote control of a robot, comprising: the system comprises an upper computer client, a server comprising a communication module and a robot entity; the remote communication is carried out between the client of the upper computer and the communication module by adopting a Socket network protocol, and communication data is packaged by adopting a self-defined Json data format; and the robot entity and the communication module are in wireless communication by adopting WIFI, and communication data are packaged by adopting a self-defined serial data format. The invention adopts the server end containing the communication module as the middleware of the upper computer client and the robot to forward the control command and the data message, and the communication module adopts a mode of processing tasks simultaneously by multithreading, so that the functions of establishing connection, sending the control command, receiving the feedback data and the like are mutually independent and do not interfere with each other, and the real-time performance of controlling the robot is improved.

Description

System and method for wireless remote control robot

technical field

The invention relates to the technical field of robot control, in particular to a system and method for a wireless remote control robot.

Background technique

OpenCV is a cross-platform computer vision library released under the BSD license (open source), which can run on Linux, Windows, Android and Mac OS operating systems. It is lightweight and efficient-consisting of a series of C functions and a small number of C++ classes, while providing interfaces for languages such as Python, Ruby, and MATLAB, and implementing many general-purpose algorithms in image processing and computer vision.

Socket is an intermediate software abstraction layer for communication between the application layer and the TCP/IP protocol family, and it is a set of interfaces. In the design mode, Socket is actually a facade mode, which hides the complex TCP/IP protocol family behind the Socket interface. For users, a set of simple interfaces is all, and Socket organizes data to meet the specified protocol. Therefore, developers do not need to deeply understand the tcp/udp protocol, the socket has been packaged for the user, and the user only needs to follow the socket regulations to program, and the written program naturally follows the tcp/udp standard.

With the advancement of science and technology, the development of robot technology has also entered a new stage. A large number of robots have been applied to different fields, especially in environments where it is difficult for humans to explore or is likely to cause harm to personal safety, different types of robots are required. robots to replace humans in real work. Most of the existing robots are controlled by wired connections. When robots work in these scenarios, the control method based on wired connections has great limitations on mobile robots, and their exploration range will be limited. Therefore, there is an urgent need in the industry to develop a stable system or method for wirelessly controlling a mobile robot that can support long-distance communication.

Contents of the invention

Aiming at the disadvantage that most existing robots in the prior art are controlled by wired connections, the present invention designs a system and method for wireless remote control of robots.

The specific scheme of this application is as follows:

A system for wireless remote control of a robot, comprising: a host computer client, a server including a communication module, and a robot entity; The data of the entity, and perform data analysis; the server end is used to process the robot control instructions, and output the control instructions to the robot entity; the robot entity is used to move according to the control instructions; the host computer client and The communication modules use the Socket network protocol for long-distance communication, and the communication data is encapsulated in a custom Json data format; the robot entity and the communication module use WIFI for wireless communication, and the communication data uses a custom serial port data format. encapsulation.

Preferably, the robot entity is a multi-joint series robot, and the robot entity includes a main controller, a WIFI module, a camera, a bus, M driving steering gears and a power supply unit; the driving steering gears are connected to the main controller through the bus ; The driving steering gear moves according to the control command output by the main controller; the power supply unit is used for power supply.

Preferably, the server end and the host computer client are located in a local area network, and the server end includes a wireless network card driver connected to the WIFI module of the robot entity.

Preferably, there are N number of host computer clients, 1024≥N≥1.

A method for a wireless remote control robot, comprising:

S1, as the server side of network communication, initializes the socket network connection, and monitors the connection from the host computer client;

S2, the server end receives the network connection request sent by the host computer client, and establishes a Socket network connection between the server end and the host computer client;

S3, create and run the sub-thread module of WIFI communication, and establish the communication between the server end and the WIFI module of the robot entity;

S4, create and run the sub-thread module of TCP communication, the server side receives the control command from the client, and analyzes the control command, and sends the corresponding control command to the robot entity according to the result of the analysis;

S5, the robot entity moves according to the control instruction.

Preferably, step S2 includes:

S21, when the server side receives the network connection request from the client of the upper computer, process the network connection request, and generate a socket identification variable corresponding to the client;

S22. Save the client socket identification variable into the FD set of the current process, and the FD set is used to save all host computer client sockets that have established connections with the current server. When there is a current connection between the host computer and the server When the communication module is disconnected, its corresponding socket will also be removed from the FD collection;

S23 , judging whether the number of currently connected host computer clients reaches the upper limit, if it has reached the upper limit, wait for the rest of the host computers to disconnect, and if it has not reached the upper limit, continue to execute S21 .

Preferably, step S3 includes:

S31, the server side cyclically monitors the data from the WIFI module, the data includes video data, picture data, audio data, serial port data, etc.; if the data is video data, then perform step S32; if the data is picture data, Then execute step S33; if the data is serial data, then execute step S34;

S32, process the video data frame by frame, convert it into BGR format, and then display the parsed video frame data frame by frame through the opencv function library to form continuous video images;

S33, displaying the picture in the program window through the opencv function library;

S34, analyze according to the self-defined serial port data format, then repackage according to the self-defined Json data format into information to be sent, then poll to obtain writable client sockets in the FD collection, pass the information to be sent through The TCP communication is written into the writable client socket and sent to the corresponding host computer.

Preferably, step S4 includes:

S41: Use the select function in socket network programming to regularly poll the FD set to detect whether there is a control command from a certain host computer client; if so, execute step S42;

S42: Clear the temporary buffer of the received message, write the control command from the client into the temporary buffer of the received message, take out the control command from the buffer and analyze it, and send a corresponding control command to the robot entity according to the result of the analysis;

S43: The robot entity returns a corresponding status code to the client sending the control instruction according to the execution result of the control instruction, and continues to execute S41.

Compared with the prior art, the present invention has the following beneficial effects:

In this solution, the server end including the communication module is used as the middleware between the host computer client and the robot, and forwards control commands and data messages. The communication module adopts a multi-threaded simultaneous task processing method to establish a connection, send control commands and receive feedback. Data and other functions are independent of each other and do not interfere with each other, which improves the real-time performance of robot control. details as follows:

(1) The present invention adopts the method of Socket network communication, which solves the one-to-one communication problem in which a host computer directly controls a robot in the previous robot control, so that the robot can accept control commands from multiple host computers and realize multiple Machine control, and various data information fed back by the robot, such as angle information, sensor information, collected image data, etc., can also be sent back to multiple host computers for processing at the same time;

(2) In the method for the wireless remote control robot disclosed by the present invention, the communication module adopts the mode of multithreading to process tasks simultaneously, so that functions such as establishing a connection, sending control commands and receiving return data are independent of each other and do not interfere with each other, which improves the efficiency of Real-time performance of robot control;

(3) The present invention adopts the communication method combining LAN and WIFI, and the communication function is more stable than Bluetooth control, and also expands the communication range of robot control, and can also break the limitation of wired control at the same time, and realize the control of the robot. remote control.

Description of drawings

Fig. 1 is a schematic structural diagram of a system of a wireless remote control robot of an embodiment;

Fig. 2 is a schematic flowchart of a method for a wireless remote control robot according to an embodiment;

FIG. 3 is a flowchart of a method for wireless remote controlling a robot according to an embodiment.

Detailed ways

In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments It is a part of embodiments of the present invention, but not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

Referring to Fig. 1, a system of a wireless remote control robot, comprising: a host computer client, a server end comprising a communication module and a robot entity; the host computer client is used to send robot control instructions to realize motion control of the robot entity , receiving data from the robot entity and performing data analysis; the server end is used for processing robot control instructions and outputting control instructions to the robot entity; the robot entity is used for moving according to the control instructions; the upper The Socket network protocol is used for long-distance communication between the machine client and the communication module, and the communication data is encapsulated in a custom Json data format; WIFI is used for wireless communication between the robot entity and the communication module, and the communication data is in a custom Serial port data format for encapsulation.

The server side including the communication module uses the communication module program as the middleware between the upper computer and the robot, and forwards control commands and data messages. Wherein, the communication module may be a software program, which is developed by using C++ language combined with the socket network.

In this embodiment, the robot entity is a multi-joint series robot, and the robot entity includes a main controller, a WIFI module, a camera, a bus, M driving servos and a power supply unit; connected with the controller; the driving steering gear moves according to the control command output by the main controller; the power supply unit is used for power supply.

In this embodiment, the server end and the host computer client are in the same local area network, and the server end includes a wireless network card driver connected to the WIFI module of the robot entity.

In this embodiment, there are N number of host computer clients, 1024≥N≥1.

Referring to Fig. 2-3, the wireless remote control robot method based on the above-mentioned wireless remote control robot system includes:

S1, as the server end of network communication, initializes the socket network connection, and monitors the connection from the host computer client; specifically, step S1 includes: using the current process as the server end of socket network communication, creating the server socket of the current process, using The TCP protocol communicates, and this socket is bound with the IP address of the computer where the process is located, and a computer port number for communication is designated for it; the server socket is set to non-blocking mode, and the listening mode is turned on, waiting for the host The connection request of the machine client;

S2, the server end receives the network connection request sent by the host computer client, and establishes a Socket network connection between the server end and the host computer client; step S2 is the main thread of program operation, specifically, step S2 includes:

S21. When the server receives a network connection request from the client of the upper computer, it processes the network connection request, and generates a socket identification variable corresponding to the client, that is, obtains the socket identification variable and the ip address of the upper computer client.

S22. Save the client socket identification variable into the FD set of the current process, and the FD set is used to save all host computer client sockets that have established connections with the current server. When there is a current connection between the host computer and the server When the communication module is disconnected, its corresponding socket will also be removed from the FD collection;

S23 , judging whether the number of currently connected host computer clients reaches the upper limit, if it has reached the upper limit, wait for the rest of the host computers to disconnect, and if it has not reached the upper limit, continue to execute S21 .

S3, create and run the sub-thread module of WIFI communication, establish the communication between the server end and the WIFI module of the robot entity; specifically, step S3 includes:

S31, the server side cyclically monitors the data from the WIFI module, the data includes video data, picture data, audio data, serial port data, etc.; if the data is video data, then perform step S32; if the data is picture data, Then execute step S33; if the data is serial data, then execute step S34;

S32, process the video data frame by frame, convert it into BGR format, and then display the parsed video frame data frame by frame through the opencv function library to form continuous video images;

S33, displaying the picture in the program window through the opencv function library;

S34, if the serial port data is received, it means that there is data feedback from the robot, analyze according to the custom serial port data format, and then repackage according to the custom Json data format into information to be sent, and then poll to obtain the FD set In the writable client socket, write the information to be sent into the writable client socket through TCP communication, write the information to be sent into the writable client socket through TCP communication, and send it to the corresponding host In the machine, the custom serial port data format is as follows:

masthead command type ID Data length message data CRC check digit

In the format, the length of the header is two bytes, which are fixed two characters, and are used to identify the starting position of the serial port data; the command type ID is an enumerated value, indicating whether the message type is a control command sent to the robot or a robot Feedback message data; the data length is used to record the length of the following message data, so that the program can read the content of the message data correctly; the message data part is used to store the data, instructions and parameters to be transmitted; the CRC check digit is used to judge the data Whether the transmission result is correct.

S4, create and run the sub-thread module of TCP communication, the server receives the control instruction from the client, and analyzes the control instruction, and sends a corresponding control instruction to the robot entity according to the analysis result; specifically, step S4 includes:

S41: Use the select function in socket network programming to regularly poll the FD set to detect whether there is a control command from a certain host computer client; if so, execute step S42;

S42: Clear the temporary buffer of the received message, write the control command from the client into the temporary buffer of the received message, take out the control command from the buffer and analyze it, and send a corresponding control command to the robot entity according to the result of the analysis; The custom Json format control commands are as follows:

The MessageType field is used to store an enumerated value, such as 0, 1, 2; 0 indicates that the message type sent is a top-level command, that is, the UpperLayerMessage field has data, and 1 indicates that the message type sent is bottom-level data, that is, the LowerLayerMessage field has data ;2 indicates that the message type sent is a mixed command, that is, both the UpperLayerMessage field and the LowerLayerMessage field have data;

The UpperLayerMessage field is used to store top-level commands, which are also enumerated values. The commands include: open serial communication, open camera, take pictures and other robot functions;

The LowerLayerMessage field is used to store the underlying data, the format is a custom serial port data format, and controls the specified steering gear to rotate at a certain angle;

S43: The robot entity returns a corresponding status code to the client sending the control instruction according to the execution result of the control instruction, and continues to execute S41.

S5, the robot entity moves according to the control instruction. After step S5, it also includes: if the communication module program is closed, all socket network connections established by the process are disconnected.

The above-mentioned embodiments only express several implementation modes of the present invention, and the descriptions thereof are relatively specific and detailed, but should not be construed as limiting the patent scope of the invention. It should be noted that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention, and these all belong to the protection scope of the present invention. Therefore, the protection scope of the patent for the present invention should be based on the appended claims.

Claims (3)

1. A system for wireless remote control of a robot, comprising: the system comprises an upper computer client, a server comprising a communication module and a robot entity;

the upper computer client is used for sending a robot control instruction to realize motion control of a robot entity, receiving data from the robot entity and analyzing the data;

the server side is used for processing the robot control instruction and outputting the control instruction to the robot entity;

the robot entity is used for moving according to the control instruction;

the remote communication is carried out between the client of the upper computer and the communication module by adopting a Socket network protocol, and communication data is packaged by adopting a self-defined Json data format;

the robot entity and the communication module are in wireless communication by adopting WIFI, and communication data are packaged by adopting a self-defined serial data format;

the robot entity comprises a WIFI module;

the server end and the client end of the upper computer are in a local area network, and the server end comprises a wireless network card driver connected with a WIFI module of the robot entity;

the number of the upper computer clients is N, and N is more than or equal to 1024 and more than or equal to 1;

the method comprises the steps of using a select function in socket network programming to poll an FD set at regular time, and detecting whether a control instruction from a certain upper computer client exists or not; if so, the temporary buffer area of the received message is emptied, the control instruction from the client is written into the temporary buffer area of the received message, the control instruction is taken out from the buffer area for analysis, and the corresponding control instruction is sent to the robot entity according to the analysis result.

2. The system of wireless remote control robots of claim 1, wherein the robot entity is a multi-joint serial robot, the robot entity further comprising a master controller, a camera, a bus, M drive steering engines, and a power supply unit; the driving steering engine is connected with the main controller through a bus; the driving steering engine moves according to a control instruction output by the main controller; the power supply unit is used for supplying power.

3. A method of wirelessly remotely controlling a robot, comprising:

s1, initializing socket network connection as a server side of network communication, and monitoring connection from an upper computer client side;

s2, the server side receives a network connection request sent by the client side of the upper computer, and Socket network connection between the server side and the client side of the upper computer is established;

s3, creating and running a sub-thread module of WIFI communication, and establishing communication between a server side and the WIFI module of the robot entity;

s4, creating a sub-thread module of TCP communication and running, receiving a control instruction of a client by a server, analyzing the control instruction, and sending out a corresponding control instruction to a robot entity according to an analysis result;

s5, the robot entity moves according to the control instruction;

the server end and the client end of the upper computer are in a local area network, and the server end comprises a wireless network card driver connected with a WIFI module of the robot entity;

the step S2 comprises the following steps:

s21, when a server side receives a network connection request sent by a client side of an upper computer, the network connection request is processed, and a socket identification variable corresponding to the client side is generated;

s22, storing the client socket identification variable into an FD set of a current process, wherein the FD set is used for storing all the clients of the upper computer which are connected with a current server, and when the upper computer is disconnected with a current communication module of the server, the corresponding sockets are removed from the FD set;

s23, judging whether the number of the clients of the upper computer which are connected currently reaches an upper limit, if so, waiting for the rest upper computers to disconnect, and if not, continuing to execute S21;

the step S3 comprises the following steps:

s31, the server side circularly monitors data from the WIFI module, wherein the data comprise video data, picture data, audio data and serial port data; if the data is video data, executing step S32; if the data is picture data, executing step S33; if the data is serial data, executing step S34;

s32, processing the video data frame by frame, converting the processed video data into a BGR format, and then displaying the analyzed video frame data frame by frame through an opencv function library to form continuous video images;

s33, displaying the pictures in a program window through an opencv function library;

s34, analyzing according to a self-defined serial port data format, packaging again according to a self-defined Json data format to form information to be sent, then polling to obtain writable client sockets in the FD set, writing the information to be sent into the writable client sockets through TCP communication, and sending the information to be sent to a corresponding upper computer;

the step S4 includes:

s41: using a select function in socket network programming to poll the FD set regularly, and detecting whether a control instruction from a certain upper computer client exists; if yes, go to step S42;

s42: the temporary buffer area of the received message is emptied, a control instruction from a client is written into the temporary buffer area of the received message, the control instruction is taken out from the buffer area to be analyzed, and a corresponding control instruction is sent to a robot entity according to an analysis result;

s43: and the robot entity returns a corresponding state code to the client side sending the control instruction according to the execution result of the control instruction, and continues to execute S41.