CN103499972A - Subsumption hierarchical structure-based universal robot multi-scene control method - Google Patents

Abstract

The invention discloses an inclusive hierarchical multi-scenario service robot control method, which is used in multiple scenarios, using a priority mechanism and a prohibition mechanism, through keyboard instructions, voice instructions, socket (Socket) instructions and mail instructions Control the service robot and have simple obstacle avoidance walking function. The remarkable advantage of the present invention is that each control method is executed in parallel in a hierarchical structure, and a simple priority mechanism and a prohibition mechanism are used for coordination between the layers, so the real-time performance is good; this control method is software design and has nothing to do with the robot platform. Therefore, the portability is good; there is no main control module, and the addition of new functions does not require rewriting existing modules, so the scalability is good; the coordination mechanism is simple, and the processor requirements are lower than the complex main control module, so the hardware cost is reduced. , easy to popularize.

Description

A Multi-scenario Control Method for General Robots Based on Inclusive Hierarchy

technical field

The invention relates to the field of robot control, in particular to a multi-scene control method for a general robot based on an inclusive hierarchical structure.

Background technique

my country's "National Medium and Long-Term Science and Technology Development Plan (2006-2020)" mentioned that "intelligent service robots are intelligent equipment integrated with various high technologies that provide necessary services for humans in an unstructured environment."

With the development of robot technology, the method of robot control is also constantly evolving and changing. The traditional control method adopts the framework of "Sense→Plan→Act" ("Sense→Plan→Act"). First, the sensor perceives the surrounding environment, receives different types of instructions, and then sends the information to the main control module for comprehensive analysis. The next action is handed over to the corresponding module for execution. For example, Felipe Trujillo-Romero of Technological University of the Mixteca and others proposed a multimodal robot control system. The system first obtains image and voice information from the camera and microphone, and sends them to the main control module after identification, and then the main control module The control module uses a voting algorithm to determine the next behavior, and finally drives the motor to execute. In recent years, HERB, ARMAR-III, PengPengII and other robots proposed by famous research institutions at home and abroad all use this framework, and have achieved good intelligent results.

But the framework of "Sense→Plan→Act" ("Sense→Plan→Act") is not suitable as a control system for service robots. There are three main reasons:

1. When adding or deleting sensors and auxiliary function modules, it is necessary to redesign the comprehensive analysis algorithm in the main control module. This makes updating existing robotic systems expensive.

2. The service robot is mainly used to receive and execute user commands. It does not need to have strong autonomy, but requires good real-time performance. However, with the continuous increase of robot sensors and auxiliary function modules, the comprehensive analysis algorithm of the main control module will become very complicated, which will affect the real-time performance of the service robot.

3. The complex main control module will have high requirements on the processor, which will increase the hardware cost of the service robot and affect the promotion of the service robot.

Brooks of MIT once proposed an inclusive hierarchy (subsumption) to overcome "perception→plan→action"

Drawbacks of the (“Sense → Plan → Act”) framework. In the inclusive hierarchical architecture (subsumption), the robot control system is divided into layers to decompose tasks into parallel processing, and a simple suppression mechanism (suppression and inhibition) is used to coordinate between layers without a main control module. Robots using an inclusive hierarchical architecture include Myrmix, etc., which have high real-time performance.

But subsumption has two major disadvantages:

1. When the system level increases, it is difficult to design a suppression mechanism to decompose and parallelize tasks well.

2. The robot is controlled from the circuit level, so the robot control system is not portable.

In recent years, many robot companies (such as Japan's Vstone Co., Ltd., Beijing Bochuang Xingsheng Technology Co., Ltd., etc.) have abstracted the control at the hardware level into a C/C++ language interface, which separates the robot control system from the robot hardware platform and realizes robot control. System portability provides technical support.

Contents of the invention

Purpose of the invention: The technical problem to be solved by the present invention is to provide a multi-scene control method for a general robot based on an inclusive hierarchical structure for the deficiencies of the prior art.

In order to solve the above technical problems, the present invention discloses an inclusive and hierarchical multi-scenario service robot control method, which is used in multiple scenarios, using a priority mechanism and a prohibition mechanism, through keyboard commands, voice commands, sockets ( Socket) command and mail command to control the service robot, and has a simple obstacle avoidance walking function, including the following steps:

(1) Start the robot platform, start the external or embedded computer, and connect the camera and microphone to the computer;

(2) Connect the computer to the network, and establish a socket connection with the client as the server;

(3) Create four command queues for storing control commands in four scenarios;

(4) Define the priority of the four instruction queues;

(5) Start four control modules to receive instructions;

(6) The control module loads the received command into the corresponding command queue;

(7) Take out the instruction with the highest priority from the four instruction queues;

(8) Identification instructions;

(9) Execute the command and return to step (6).

Wherein step (8) further comprises the following steps:

(81) If the command is unrecognizable, give an "Unknown Command (unrecognized command)" prompt on the control interface and discard it;

(82) If the instruction can be identified, then enter step (9)

Wherein step (9) further comprises the following steps:

(91) If the command cannot be executed, "Obstacle ahead" will be given on the control interface

prompt and discard;

(92) If the instruction can be executed, return to step (6) after the execution is completed.

In the above method, the robot platform refers to all robots that provide C/C++ language interface for hardware level control.

In the above method, an external or embedded computer refers to a computer with a Windows operating system, camera and microphone drivers, Microsoft voice recognition software, and a network connection function installed.

In the above method, the four control scenarios refer to: direct operation of an external or embedded computer (denoted as scene-0), being in the same room with the robot (denoted as scene-1), and being in the same house with the robot (denoted as scene-0). Scenario-2), all parts of the world with Internet access outside the house (denoted as Scenario-3).

In the above method, the four control modes respectively refer to: keyboard commands, voice commands, socket commands, and mail commands.

In the above method, the command with the highest priority is taken out by the priority mechanism, and the achieved effect is: the priority between command queues is defined by the user, and the priority of the command queue defined by the user is high; The priority of the command is determined by the order of time, and the command that enters the queue first has the highest priority.

In the above method, the four control modules respectively refer to: a keyboard receiving module, a speech recognition module, a socket (Socket) communication module, and a mail receiving module.

In the above method, the control module is started in the form of independent threads and executed concurrently or in parallel.

In the above method, identifying the instruction refers to finding the processing method of the current instruction in the system. If the processing method is not defined, it cannot be recognized.

In the above method, executing the instruction includes calling service software to execute the instruction and driving a motor to execute the instruction. Service software includes: weather forecast, email sending, and visitor notification. The drive motor includes: forward, go left, go right, turn left, turn right, back.

In the above method, the obstacle avoidance walking function is completed by the obstacle detection module and the prohibition mechanism. When the obstacle detection module outputs obstacle information, the prohibition mechanism will be triggered, so that the motor cannot move.

The present invention overcomes the shortcomings of the traditional "Sense→Plan→Act" architecture in the application of service robots, introduces subsumption technology, and designs the robot control system in a hierarchical manner, which is easy to use The coordination mechanism of the subsumption cancels the setting of the main control module. At the same time, it also overcomes the shortcomings of difficult design and poor portability of the suppression mechanism in the inclusive hierarchy (subsumption). Prohibition mechanism, two simple coordination mechanisms, a service robot control method with good real-time performance, portability and scalability.

Beneficial effect: the significant advantage of the present invention is that each control method is executed in parallel in a hierarchical structure, and a simple priority mechanism and a prohibition mechanism are used for coordination between the layers, so the real-time performance is good; this control method is a software design, which is similar to that of a robot The platform is independent, so the portability is good; there is no main control module, adding new functions does not need to rewrite the existing modules, so the scalability is good; the coordination mechanism is simple, and the processor requirements are lower than the complex main control module, so The cost of hardware is reduced, and it is easy to popularize.

Description of drawings

The advantages of the above and/or other aspects of the present invention will become clearer as the present invention will be further described in detail in conjunction with the accompanying drawings and specific embodiments.

Fig. 1 is a structure diagram of the present invention.

Fig. 2 is a working flow chart of the present invention.

Fig. 3 is a schematic diagram of the speech recognition module.

Fig. 4 is a schematic diagram of the socket communication module.

Fig. 5 is a schematic diagram of the mail receiving module.

Fig. 6 is a schematic diagram of an obstacle detection module.

Figure 7 is a working diagram of the priority mechanism.

Figure 8 is a working diagram of the prohibition mechanism.

Detailed ways

Fig. 1 is a structure diagram of the present invention. (Taking priority setting as scene-0 higher than scene-1 higher than scene-2 higher than scene-3 as an example)

The present invention designs the control mode into four levels corresponding to the scene:

(1) Keyboard control corresponds to scene-0 of directly operating the robot. The keyboard receives the information, and puts the keyboard command into the command queue-0; the implementation of the keyboard receiving module is: use the scanf() function to read the user's keyboard input from the console, encapsulate it into a text format command, and call the command queue-0 The push function queues keyboard commands.

(2) Voice control corresponds to scene-1 where the robot is in the same room. The information is received by the microphone, and the voice command is loaded into the command queue-1 in the voice recognition module; as shown in Figure 3, the implementation of the voice recognition module is: install Microsoft Speech SDK5.1 and Chinese language pack, use Microsoft Speech SDK5.1 The built-in speech recognition function writes a speech listener, calls the GetText() function in CComPtr<ISpRecoResult> to convert the monitored speech information into text information, extracts keywords from the text information, encapsulates them into text format instructions, and calls the instruction queue The push function of -1 puts voice commands into the queue.

(3) Socket control corresponds to scene-2 where the robot is in the same house. The socket (Socket) receives information, and the socket (Socket) communication module loads the socket (Socket) instruction into the instruction queue-2; as shown in Figure 4, the implementation of the socket (Socket) communication module Yes: The computer communicates as a socket (Socket) server by setting the server address, binding the socket (bind function), and listening to the port (listen function) to establish a connection with the client (accept function), and then receive the message from the client Socket (Socket) information (recv function), then extract keywords from the socket (Socket) information, encapsulate it into a text format command, call the push function of command queue-2 to load the Socket command into the queue.

(4) Mail control corresponds to scenario-3 around the world that can be networked. Receiving information by the mail system, the mail command is loaded into the command queue-3 by the mail control module; as shown in Figure 5, the implementation mode of the mail control module is: install and register the Jmail4. ) function connects to the mail server, and from the mailing list of the Messages member, according to the difference between the number of last mail and the number of this time, obtains the unread mail of the specified mailbox, and then extracts keywords from the mail content, and encapsulates it into an instruction in text format. Load instruction queue-3.

In the robot control system shown in Figure 1, there are three priority mechanisms. The first is the blocking of command queue-0 to command queue-1. As long as command queue-0 has command output, command queue-1 will be blocked, and the commands of command queue-0 will be prioritized for identification. The second is the blockage of instruction queue-0 and 1 to instruction queue-2. As long as there are instructions output from instruction queue-0 or 1, instruction queue-2 will be blocked, and the instructions of instruction queue 0 or 1 will be prioritized for identification. The third place is the blocking of instruction queue-0, 1 and 2 for instruction queue-3, as long as there are instructions output in instruction queue-0, 1 or 2, instruction queue 3 will be blocked, and the instructions of instruction queue 0, 1 or 2 will be blocked. Priority feed into identification.

Fig. 2 is a working flow chart of the present invention.

Step S01: Start the robot platform, start the external or embedded computer, and connect the camera and microphone to the computer.

In this step, the external or embedded computer of the robot needs to install the windows operating system, camera and microphone drivers, and a computer with Microsoft speech recognition software.

Step S02: Connect the computer to the network, and establish a socket connection with the client as the server.

In this step, the computer is used as the server, and other control terminals in the house are used as clients to establish a one-to-many socket connection.

Step S03: Create four command queues corresponding to the control modes in the four control scenarios.

In this step, the instruction queue is created by creating four C++ template class queues, and the elements of the queue templates are instructions in text format.

Step S04: Define the priorities of the four instruction queues.

In this step, the priority is determined as follows: the priority among the command queues is defined by the user, and the priority of the command queue defined by the user is high.

Step S05: Start four control modules for receiving instructions.

In this step, the starting method of the control module is: use the pthread_create function to create four threads, and each control module is executed concurrently or in parallel in the form of an independent thread.

Step S06: The control module loads the received command into the corresponding command queue.

In this step, the way of enqueueing the instruction is: the control module uses the push function of the queue template to load the instruction into the corresponding instruction queue.

Step S07: Take out the instruction with the highest priority from the four instruction queues.

In this step, as shown in Figure 7, under the coordination of the priority mechanism of the four instruction queues, the instruction with the highest priority is the first fetched instruction, and the pop function of the queue template is used to fetch the instruction.

Step S08: Identify the instruction.

In this step, search the processing mode of the current command in the system, if no processing mode is defined, go to step S09; otherwise, go to step S10.

Step S09: Give the prompt of "Unknown Command" on the control interface, discard it, and return to step S07.

Step S10: Determine whether the instruction can be executed.

In this step, if the current command is to drive the motor, and the obstacle detection module outputs information that cannot be executed, trigger the prohibition mechanism shown in Figure 8 to prohibit the drive of the motor, and proceed to step S11; otherwise, proceed to step S12.

Step S11: Give an "Obstacle ahead" prompt on the control interface, discard it, and return to step S07.

Step S12: execute the instruction, and return to step S07.

As shown in Figure 1, the robot behavior can include service software modules, including: weather forecast, email sending, and visitor notification.

The way to implement the weather forecast is: use the gethostbyname() function to obtain the address of the weather forecast website named www.webxml.com.cn, establish a Socket connection with the weather forecast website, and use the send() function to send a request to the weather forecast website Message, the city code is passed in as a parameter, and the offset of the required weather information in the returned message (recv function) is calculated to obtain the weather forecast, and then the Speak function in Microsoft SDK5.1 is used to read the weather forecast.

The way to send mail is: install Jmail4.4 component, use the function SendMail in Jmail4.4 to quickly send mail, and pass in the sender's mailbox, recipient's mailbox, mail subject, body, and SMTP server as parameters.

The way to realize the visitor notification is: install the Jmail4.4 component and the OpenCV2.3.1 library, use the camera function cvCaptureFromCAM() in the OpenCV2.3.1 library to take photos of the visitors, and use the function Send() in Jmail4.4 to send emails with attachments, Pass in the sender's mailbox, recipient's mailbox, email subject, text, and SMTP server as member variables, and pass in the photo of the visitor as an attachment (AddAttachment function).

The function of the drive motor module is to shield the details of the robot platform and realize the portability of the system. The specific implementation method is: converting each motor action into an interface call sequence of a specific robot platform. The motor actions of the robot system we have implemented include: forward, walk left, walk right, turn left, turn right, and back. Each action is implemented by calling the motor interface of the C/C++ language provided by the robot platform. Taking the Robovie_X robot as an example, the implementation of each action is to first use the VSRC003_LoadMotion() function to load the motor interface, and then use the VSRC003_PlayMotion() function to execute the action. The command-action-interface correspondence table is:

instruction action interface forward go ahead Go_Forward(advanced).txt go left turn left Side step left(1step).txt go right go right Side step right(1step).txt turn left Turn left Turn in place left.txt turn right Turn right Turn in place right.txt

back step back Walk backwards.txt

The transplantation mode of the present invention is as follows:

The porting condition provides C/C++ language interface for the robot platform.

When the present invention is transplanted to the new robot platform R, the existing motor actions: forward, walk to the left, walk to the right, turn left, turn right, back, respectively converted into the motor interface call sequence of the R robot. For example, when the R robot is a humanoid robot, the "forward" action is converted into an alternate call of the R robot's "leg lift" and "leg release" interfaces; when the R robot is a spider-type robot, the "forward" action is converted into R Multiple calls of the robot's "crawling" interface.

After experimental verification, the transplantation of the system can be realized after replacing the calling interface in the drive motor module. Figure 9 shows the two robot platforms used in the experiment (the left picture is Robovie-X, and the right picture is VstoneTichno). The following table shows the hardware combinations of the two groups of experiments.

The invention also designs an obstacle-avoiding walking function to prevent the robot from colliding with obstacles when it moves. As shown in Figure 6, the implementation of the obstacle detection module is: install the OpenCV2.3.1 library, obtain the video through the OpenCV2.3.1 library function cvCreateCameraCapture(), then use the cvQueryFrame() function to capture a frame of image, and use the cvCvtColor() function to capture the video Convert the photo into a grayscale image, after smoothing (cvSmooth function) and extracting the contour (extractContourImg function), then convert the grayscale image into a binary image (cvThreshold function), and calculate the proportion of black pixels (obstacles) to the total pixels of the picture Percentage, if the calculation result is greater than the threshold (6% is more appropriate after multiple experiments), it is judged that there is an obstacle, the obstacle information is output, the prohibition mechanism is activated, all the instructions sent to the drive motor module are discarded, and execution is prohibited.

Figure 8 is a working diagram of the prohibition mechanism. The function of the prohibition mechanism is to prohibit the execution of instructions. If no information is output by the high-level module, the inhibit mechanism is not activated and the current instruction can be sent for execution. If the high-level module outputs information, the prohibition mechanism is activated, the current instruction is discarded, and execution is prohibited.

In the robot control system shown in Figure 1, there is a total of prohibition mechanisms. When the obstacle detection module outputs obstacle information, all instructions sent to the drive motor module are discarded and prohibited from being executed.

The present invention provides a multi-scenario control method for a universal robot based on an inclusive hierarchical structure. There are many methods and approaches to specifically realize the technical solution. The above description is only a preferred embodiment of the present invention. Those of ordinary skill may make some improvements and modifications without departing from the principle of the present invention, and these improvements and modifications shall also be regarded as the protection scope of the present invention. All components that are not specified in this embodiment can be realized by existing technologies.

Claims (6)

1. many scenes of the all-purpose robot control method based on the containment type hierarchical structure, is characterized in that, comprises the steps:

(1) the people's platform that starts the machine, start external or Embedded computing machine, and camera, microphone are connected to computing machine;

(2) by the computing machine access network, and set up socket as server end and client and be connected;

(3) create four instruction queues, for depositing the steering order of keyboard, microphone, socket and mail;

(4) define the priority of four instruction queues, be followed successively by from high to low the steering order of keyboard, microphone, socket and mail;

(5) start four control modules for receiving instruction;

(6) four control modules are by the instruction the received corresponding instruction queue of packing into;

(7) take out the highest instruction of priority from four instruction queues;

(8) robot platform recognition instruction;

(9) the robot platform control is carried out instruction.

2. a kind of many scenes of all-purpose robot control method based on the containment type hierarchical structure according to claim 1, is characterized in that, step (8) is further comprising the steps:

(81) if the instruction None-identified provides prompting and abandons instruction at the control interface;

(82) if instruction can be identified, enter step (9).

3. a kind of many scenes of all-purpose robot control method based on the containment type hierarchical structure according to claim 2, is characterized in that, wherein step (9) is further comprising the steps:

(91) if instruction can't be carried out, at the control interface, provide prompting and abandon instruction;

(92) if instruction can be carried out, execution is returned to step (6) after finishing.

4. a kind of many scenes of all-purpose robot control method based on the containment type hierarchical structure according to claim 1, it is characterized in that, take out the highest instruction of priority and completed by priority mechanism, the priority of same instruction queue inside determines by time sequencing, and the instruction priority of first joining the team is high.

5. a kind of many scenes of all-purpose robot control method based on the containment type hierarchical structure according to claim 1, it is characterized in that, four control modules refer to respectively: keyboard receiver module, sound identification module, socket Socket communication module, mail reception module.

6. a kind of many scenes of all-purpose robot control method based on the containment type hierarchical structure according to claim 1, is characterized in that, the Starting mode of control module is: with form the concurrent or executed in parallel of separate threads.

Images