KR20230135548A - Communication-based robot control system and method thereof - Google Patents

Abstract

A robot control method is provided in which a robot control system generates control commands for controlling the driving part of the robot based on raw data received from the robot, and transmits the generated control command to the robot to control the driving part of the robot. Control commands generated by the robot control system include low-level control commands for controlling the driving part of the robot.

Description

Communication-based robot control method and system {COMMUNICATION-BASED ROBOT CONTROL SYSTEM AND METHOD THEREOF}

The description below relates to a method and system for controlling a robot, which controls the robot by generating control commands for controlling the driving part of the robot based on raw data received from the robot. .

Interest in robots that provide services such as manufacturing, logistics, transportation, cleaning and navigation is growing. A robot implemented to provide such services can, for example, move to the destination along an optimal route through autonomous driving and provide services at the destination.

In order for a robot to provide these services, it is necessary to properly control the robot's driving parts. The driving part of the robot can be controlled based on the state of the robot or the surrounding environment. Configuring the robot so that the calculation for controlling the robot's driving part is performed on the robot side not only complicates the implementation of the robot, but also increases the maintenance cost of the robot and reduces the scalability of the robot.

Therefore, in order to lower the implementation cost of the robot and facilitate maintenance and expansion of the robot, the calculations required to control the driving part of the robot and the generation of commands to perform low-level control of the driving part It is required to build a robot and robot control system so that it can be performed on the server side.

Korean Patent Publication No. 10-2005-0024840 is a technology related to a path planning method for autonomous mobile robots. It provides an optimal path for mobile robots that move autonomously at home or in the office to move safely and quickly to the target point while avoiding obstacles. Describes how to plan.

The information described above is for illustrative purposes only and may include content that does not form part of the prior art and may not include what the prior art would suggest to a person skilled in the art.

Provides a robot control method in which a robot control system generates control commands for controlling the driving part of the robot based on raw data received from the robot, and transmits the generated control command to the robot to control the driving part of the robot.

The robot control system generates low-level control commands to control the robot's driving parts based on raw sensing data received through wireless communication with the robot, and recognizes and interprets commands based on the sensing data on the robot side. Provides a control method that can directly control the actuators and motors included in the driving part of the robot through the low-level control command without the need for tasks to be processed.

In one aspect, a method of controlling a robot performed by a robot control system includes receiving raw data from the robot, and based on the received raw data, a control command for controlling a driving unit of the robot. Generating and transmitting the generated control command to the robot, wherein the control command is input to the driving unit and includes a low-level control command for controlling the driving unit. A method for controlling is provided.

The driving unit includes at least one of an actuator and a motor, and the control command includes a command for controlling at least one of the speed, position, and torque of the actuator, or a command for controlling at least one of the speed and torque of the motor. May contain commands.

The control command may include a command for feedback control of the actuator or the motor.

The raw data may include raw sensing data acquired by the sensor unit of the robot.

Generating the control command includes preprocessing the raw sensing data, wherein the raw sensing data is an image acquired by a camera of the sensor unit or a plurality of sensing data acquired by a plurality of sensors of the sensor unit. data, and the pre-processing includes performing refine processing to enable recognition of an object within the image, or determining the state of the robot or the state of the environment in which the robot travels. It may include fusion of sensing data.

Generating the control command may include obtaining at least one of state information of the robot and environmental information of an environment in which the robot travels based on the raw sensing data, and based on the obtained information, It may include generating the control command for controlling the driving unit.

The obtaining step includes at least the steps of i) generating a local map around the robot based on the raw sensing data, and ii) recognizing objects around the robot based on the raw sensing data. It includes one, and the at least one piece of information can be obtained based on at least one of the local map and a recognition result of the object.

The control command is a control command for controlling the movement of the robot for autonomous driving of the robot, and the step of generating the control command includes a motion plan associated with the robot, the state information, and the environment information. Based on, it is possible to generate control commands for the drive unit to cause the robot to follow the movement plan and avoid obstacles in the environment.

The movement plan includes information about the path the robot should travel, the state information includes the location of the robot, the environment information includes information about objects around the robot, and the control command may include a command for feedback control of at least one of a motor and an actuator included in the driving unit so that the robot moves along the path and avoids the obstacle.

The raw sensing data is the sensing data initially collected from the sensor unit and initially preprocessed on the robot side, and the control command may be input to the driving unit after correction processing on the robot side.

According to communication between the wireless communication unit of the robot and the wireless communication unit of the robot control system, the raw data may be received from the robot, and the control command may be transmitted to the robot.

The robot may be a brainless robot that controls the driving unit by executing the low-level control command received from the robot control system without performing a process of interpreting the received control command.

The method of controlling the robot includes, when control of another robot is requested, generating a robot control model for controlling the other robot based on predefined information about the other robot, based on the robot control model. The method may further include estimating the state of the other robot and generating a control command for controlling a driving unit of the other robot based on the estimated state of the other robot.

Predefined information about the other robot used to create the robot control model includes specification information of the sensor unit and driving unit included in the other robot, physical shape information of the other robot, and dynamics associated with the other robot. A model (dynamics model) may be included.

The robot is controlled by a first robot brain of the robot control system, and the receiving, generating, and transmitting steps are performed by the first robot brain, and the method of controlling the robot When control of another robot is requested, it further includes configuring a second robot brain that is logically distinct from the first robot brain for controlling the other robot, and receiving raw data from the other robot. , Generation of a control command for controlling the driving unit of the other robot and transmission of a control command for controlling the driving unit of the other robot may be performed through the second robot brain.

The first robot brain is initially connected to the robot through a gateway of the robot control system, and after the initial connection, communicates with the robot through a communication unit of the first robot brain, and the second robot brain communicates with the other robot. and is initially connected through the gateway, and after the initial connection, can communicate with the other robot through the communication unit of the second robot brain.

If the control command is not received by the robot within a first predetermined time or the control command is not transmitted to the robot for more than a predetermined second time, the robot may be switched to standby mode.

When an object is detected within a predetermined distance from the robot, the robot may be switched to standby mode regardless of the control command.

In another aspect, a robot control system for controlling a robot includes a wireless communication unit and at least one processor implemented to execute commands readable by a computer, wherein the at least one processor is configured to communicate with the wireless communication unit and the robot. According to communication between the wireless communication units, raw data is received from the robot, and based on the received raw data, a control command for controlling the driving unit of the robot is generated, and between the wireless communication unit and the wireless communication unit of the robot According to communication, the generated control command is transmitted to the robot, and the control command is input to the driving unit, and includes a low-level control command for controlling the driving unit. A robot control system is provided. .

The raw data includes raw sensing data acquired by a sensor unit of the robot, and the at least one processor includes a preprocessor that preprocesses the raw sensing data, based on the preprocessed raw sensing data, A robot state and environment recognition processing unit that acquires at least one of the status information of the robot and environmental information of the environment in which the robot runs, and a robot that generates the control command for controlling the driving unit based on the obtained information. It may include a control unit.

The control command is a control command for controlling the movement of the robot for autonomous driving of the robot, and the at least one processor includes a travel planning unit that manages a movement plan associated with the robot, and the robot control unit, Based on the state information and the environment information, a control command may be generated for the driving unit to cause the robot to follow the movement plan and avoid obstacles in the environment.

The at least one processor is configured to generate a robot control model for controlling the other robot based on predefined information about the other robot when control of the other robot is requested, and the robot control model is configured to generate the robot control model for controlling the other robot. The state of the other robot may be estimated, and a control command for controlling the driving unit of the other robot may be generated based on the estimated state of the other robot.

The at least one processor includes a first robot brain, and the first robot brain is configured to receive the raw data, generate the control command, and transmit the control command to the robot, and the at least one processor is configured to receive the raw data, generate the control command, and transmit the control command to the robot. The processor is configured to generate a second robot brain logically distinct from the first robot brain for controlling the other robot when control of the other robot is requested, receiving raw data from the other robot, Generation of control commands for controlling the driving unit of the other robot and transmission of the control command for controlling the driving unit of the other robot may be performed through the second robot brain.

In another aspect, in a robot controlled by a robot control system, a wireless communication unit, a sensor unit including at least one sensor, a driving unit including at least one of a motor and an actuator, a sensor driver for the sensor unit, and the A control unit including a drive unit driver for the drive unit, wherein the control unit transmits raw data including raw sensing data acquired by the sensor unit to the robot control system through the wireless communication unit, The robot control system receives a control command for controlling the driving unit generated based on the raw data through the wireless communication unit, executes the received control command to control the driving unit, and controls the received control. Commands are input to the driving unit, and a robot is provided, including low-level control commands for controlling the driving unit.

The robot control system generates low-level control commands to control the driving part of the robot based on raw sensing data received through wireless communication with the robot, and the actuator included in the driving part of the robot through the low-level control command. and by being able to directly control the motors, the robot can be configured to not include a complex on-board computer system or to include an on-board computer system that includes minimal hardware.

Since the calculations required to generate low-level control commands to control the driving part of the robot are performed on the robot control system, tasks such as recognition and interpretation of commands based on sensing data do not need to be processed on the robot side, thereby improving the service. The complexity of robots can be reduced and maintenance of service robots can be streamlined.

Since the robot is simply implemented so that the driving part of the robot is controlled according to control commands from the robot control system, the possibility of changing the robot's functions and expandability can be increased depending on the configuration of the robot control system.

1 shows a method of controlling a robot, according to one embodiment.
Figure 2 is a block diagram showing a robot, according to one embodiment.
Figure 3 is a block diagram showing a robot control system for controlling a robot, according to one embodiment.
Figure 4 shows a method in which the robot control system controls the driving unit of the robot through a low-level control command based on wireless communication between the robot and the robot control system, according to one embodiment.
Figure 5 is a flowchart showing a method for controlling a robot, according to one embodiment.
Figure 6 is a flowchart showing a method of generating a control command for controlling a driving unit of a robot in a robot control system, according to an example.
FIG. 7 illustrates a method in which a robot drives autonomously by controlling a driving part of the robot through a low-level control command based on wireless communication between the robot and the robot control system, according to an example.
Figure 8 is a flowchart showing a method of controlling a robot by creating a robot control model, according to an embodiment.
Figure 9 is a flowchart showing a method of controlling a robot by creating a robot brain, according to an example.
Figure 10 shows a multi-robot control method for controlling a plurality of robots, according to an example.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

1 shows a method of controlling a robot, according to one embodiment.

The robot 100 shown in FIG. 1 may be a service robot configured to provide services within a space, such as a building, indoor, or other open area.

The robot 100 may be configured to perform a specific function or perform a task related to the provision of a service according to control by the robot control system 120.

The space where the robot 100 runs is a place where the robot 100 provides services, and may represent, for example, a building. This space is a space where a plurality of people (hereinafter referred to as users) work or reside, and may include a plurality of partitioned spaces. A space can represent a part of a building (a specific floor or a partial space within that floor). Robot 100, a service robot, may be configured to provide services on at least one floor of space.

The robot control system 120 may be configured to control each of a plurality of robots. At this time, each of the robots can move within the space to provide a service to an appropriate location in the space or to an appropriate user.

Services provided by the robot 100 may include, for example, at least one of a parcel delivery service, an ordered beverage (coffee, etc.) delivery service, a cleaning service, and other information/content provision services.

The robot 100 can provide services at a certain location in space or to a certain user through autonomous driving. The robot 100 can move to a specific location under control by the robot control system 120 or perform other tasks or functions required to provide a service.

In an embodiment, the robot control system 120 generates a control command for controlling the driving part of the robot 100 based on raw data received from the robot 100, and transmits the generated control command to the robot to The driving part of (100) can be controlled.

For example, as shown, the robot control system 120 may receive raw data including raw sensing data collected by the sensor unit of the robot 100 from the robot 100 (①). The robot control system 120 may generate a control command to control the driving unit of the robot 100 at a low level based on the received raw data (②). The robot control system 120 can transmit the generated control command to the robot 100 (③), and the robot 100 can control the driving unit by executing the received control command. In other words, the robot control system 120 does not generate abstracted high-level commands as control commands for controlling the driving part of the robot 100, but rather controls the motor and/or actuator of the driving part (at a low level). You can create low-level control commands that allow direct control. Therefore, there is no need to perform a process of interpreting the received control command on the robot 100 side, and the driving unit can be operated according to the control command simply by executing the received low-level control command (e.g., inputting it to the driving unit). .

In this way, the robot 100 may be implemented as a brainless robot that controls the driving unit by executing low-level control commands from the robot control system 120 without performing a process of interpreting received control commands. In other words, the robot control system 120 can be implemented as a robot brain (brain system) that controls such a brainless robot.

The robot 100 may include only a configuration for transmitting the collected raw sensing data to the robot control system 120, receiving low-level control commands from the robot control system 120, and operating the driving unit, and may include a complex onboard It may not contain components such as computer systems.

The structure and specific operations of the robot 100 and the robot control system 120 will be described in more detail with reference to FIGS. 2 to 4, which will be described later.

Figure 2 is a block diagram showing a robot, according to one embodiment.

As described above, the robot 100 may be a service robot used to provide services within a space. The robot 100 may be configured to provide services at a certain location in space or to a certain user through autonomous driving.

The robot 100 may be a physical device and, as shown, may include a control unit 104, a driving unit 108, a sensor unit 106, and a communication unit 102.

The control unit 104 may be a physical processor built into the robot 100 or an on-board computer system. The control unit 104 allows the robot 100, implemented as a brainless robot, to communicate with the robot control system 120, transmit data to the robot control system 120, and execute commands received from the robot control system 120. It may include only the components necessary for processing (eg, delivery to the driving unit 108 and/or sensor unit 106).

For example, the control unit 104 transmits raw sensing data collected through the sensor unit 106 to the robot control system 120, receives low-level control commands from the robot control system 120, and operates the drive unit 108. It can only include configuration for doing so. In other words, the control unit 104 may not include a complex configuration (eg, GPU, etc.) for interpreting and processing sensing data and control commands.

The control unit 104 may include a sensor driver for the sensor unit 106 and a driving unit driver for the driving unit 108.

The communication unit 102 may be a component that allows the robot 100 to communicate with another device (robot control system 120, etc.). That is, the communication unit 102 includes antennas, data buses, network interface cards, network interface chips, and networking interfaces of the robot 100, which transmit/receive data and/or information to and from other devices, such as the robot control system 120. It may be a hardware module such as a port, or a software module such as a network device driver or networking program.

For example, the communication unit 102 is a wireless communication unit for communicating with the robot control system 120, and transmits raw data including (raw) sensing data to the robot control system 120 and receives information from the robot control system 120. A control command for the driver 108 may be received.

The sensor unit 106 may be configured to collect data required for autonomous driving and service provision of the robot 100. The sensor unit 106 may not include expensive sensing equipment, but may only include sensors such as low-cost ultrasonic sensors and/or low-cost cameras.

The sensor unit 106 may include a sensor for identifying objects such as other robots, people, obstacles, etc. in the front and/or rear. For example, other robots, people, and other objects can be identified through the camera of the sensor unit 106. Alternatively, the sensor unit 106 may include an infrared sensor (or infrared camera). In addition to the camera, the sensor unit 106 may further include a sensor for recognizing/identifying nearby users, other robots, or objects. Additionally, the sensor unit 106 may include at least one distance sensor for identifying the distance to object(s) existing in the surrounding area. In addition, the sensor unit may include sensors for determining the state of the robot 100 and recognizing the environment, including an odometer.

(Raw) sensing data from the sensors of the sensor unit 106 may be transmitted to the robot control system 120 through the communication unit 102. For example, sensing data may be transmitted to the robot control system 120 through the communication unit 102 via the sensor driver (or sensor hub) of the control unit 104.

The driving unit 108 controls the movement of the robot 100 and may include equipment (hardware) to enable movement. Additionally, the driving unit 108 may include equipment (hardware) to perform functions necessary for the robot 100 to perform tasks related to the requested service.

For example, the driving unit 108 may include at least one motor and/or at least one actuator for operating wheels, caterpillar wheels, legs, etc. for movement of the robot 100.

Additionally, the driving unit 108 may include equipment related to services provided by the robot 100. For example, in order to perform a food/delivery delivery service, the driving unit 108 of the robot 100 is configured to load food/delivery or deliver food/delivery to the user (e.g., a robot arm). may include. Additionally, the robot 100 may further include speakers and/or displays for providing information/content.

The driving unit 108 may be controlled according to control commands from the robot control system 120. The driver 108 may execute a low-level control command received from the robot control system 120 and perform an operation corresponding to the control command. For example, when a low-level control command from the robot control system 120 is input to the driver 108, the driver 108 can perform the operation indicated by the corresponding control command.

Control commands from the robot control system 120 may be transmitted to the driving unit 108 through the communication unit 102. For example, control commands from the robot control system 120 are received through the communication unit 102 and transmitted to each component of the drive unit 108 (e.g., each motor and/or actuator) by the drive unit driver of the control unit 104. It can be.

As described, the robot 100 is controlled only by transmitting sensing data from the sensor unit 106 to the robot control system 120 and receiving control commands from the robot control system 120 (corresponding to the brain) It may be a brainless robot (controlled by the robot control system 120).

Meanwhile, each of the robots 100 may have different sizes and shapes (i.e., different types of sensor unit 106 and/or drive unit 108) depending on the model or service provided.

The configuration and operation of the robot control system 120 that controls the robot 100 will be described in more detail with reference to FIGS. 3 and 4, which will be described later.

The description of the technical features described above with reference to FIG. 1 can also be applied to FIG. 2 , so overlapping descriptions will be omitted.

Figure 3 is a block diagram showing a robot control system for controlling a robot composed of a plurality of modular robots, according to an embodiment.

The robot control system 120 may be a device that controls the movement (i.e., running) of the above-described robot 100 within a space and the provision of services within the space by the robot 100. When there are multiple robots 100, the robot control system 120 can control the movement of each robot 100 and the provision of services for each robot 100.

Through communication with the robot 100, the robot control system 120 can plan and set the path that the robot 100 must travel to provide a service, and sends control commands for movement along this path to the robot ( 100). The robot 100 can move to a predetermined location or a predetermined user according to the received control command. Additionally, the robot 100 may provide a service (perform a service-related task) at a predetermined location or to a predetermined user under the control of the robot control system 120.

Robot control system 120 may include at least one computing device.

As described above, the robot control system 120 may be a device that plans and sets a path for the robot 100 to travel and controls the movement of the robot 100. The robot control system 120 may include at least one computing device and may be implemented with at least one server located within or outside the space in which the robot 100 travels.

As shown, the robot control system 120 may include a memory 330, a processor 320, a communication unit 310, and an input/output interface 340.

The memory 330 is a computer-readable recording medium and may include a non-permanent mass storage device such as random access memory (RAM), read only memory (ROM), and a disk drive. Here, the ROM and the non-perishable mass recording device may be separated from the memory 330 and included as a separate permanent storage device. Additionally, an operating system and at least one program code may be stored in the memory 330. These software components may be loaded from a computer-readable recording medium separate from the memory 330. Such separate computer-readable recording media may include computer-readable recording media such as floppy drives, disks, tapes, DVD/CD-ROM drives, and memory cards. In another embodiment, software components may be loaded into the memory 330 through the communication unit 310 rather than a computer-readable recording medium.

The processor 320 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. Commands may be provided to the processor 320 by the memory 330 or the communication unit 310. For example, the processor 320 may be configured to execute instructions received according to program code loaded into the memory 330.

The communication unit 310 may be a component that allows the robot control system 120 to communicate with another device (such as the robot 100 or another server). In other words, the communication unit 310 is a hardware module such as an antenna, a data bus, a network interface card, a network interface chip, and a networking interface port of the robot control system 120, which transmits/receives data and/or information to or from other devices. It may be a software module such as a network device driver or networking program.

For example, the communication unit 310 is a wireless communication unit for communicating with the robot 100, receives raw data including (raw) sensing data from the robot 100, and transmits raw data including (raw) sensing data to the robot 100 for the drive unit 108. Control commands can be sent. In other words, the robot 100 and the robot control system 120 can transmit and receive data and commands by communicating through their respective wireless communication units 102 and 310.

The input/output interface 340 may be a means for interfacing with an input device such as a keyboard or mouse and an output device such as a display or speaker.

Additionally, in other embodiments, the robot control system 120 and processor 320 may include more components than those shown. For example, processor 320 may include components (eg, 410 to 440) as shown in FIG. 4 .

In an embodiment, raw data including raw sensing data collected by the sensor unit 106 of the robot 100 may be received from the robot 100 through the communication unit 310 to the processor 320, and the processor ( 320 may generate a control command (low-level control command) for the driving unit 108 of the robot 100 based on the sensing data received from the robot 100. The processor 320 can transmit a control command to the robot 100 through the communication unit 310, and thus the driver 108 can be controlled according to the transmitted control command. The communication unit 310 may utilize socket communication, streams, message queues, etc. to communicate with the robot 100.

An operation in which the processor 320 generates a control command for the driver 108 may be performed by components 410 to 440.

Each of the components 410 to 440 of the processor 320 may be a software and/or hardware module as part of the processor 320, and may represent a function (functional block) implemented by the processor. Configurations 410 to 440 of the processor 320 will be described later with reference to FIG. 4 .

The description of the technical features described above with reference to FIGS. 1 and 2 can also be applied to FIG. 3 , so overlapping descriptions will be omitted.

Figure 4 shows a method in which the robot control system controls the driving unit of the robot through a low-level control command based on wireless communication between the robot and the robot control system, according to one embodiment.

As shown, the robot control system 120 may include a pre-processing unit 410, a robot state and environment recognition processing unit 420, a robot control unit 430, and a travel planning unit 440. These components 410 to 440 may be part of the processor 320 described above or may be functions implemented by the processor 320. The components included in the processor 320 are different functions performed by at least one processor included in the processor 320 according to control instructions according to the code of the operating system or the code of at least one computer program ( These may be expressions of different functions.

Meanwhile, the control unit 104 of the robot 100 may include a sensor driver 450 and a driving unit driver 460. The sensor unit 106 is shown to include a plurality of sensors 470-1 to 4, and the drive unit 108 is shown to include an actuator 482 and a motor 484.

With reference to FIG. 4, the robot 100, which is a brainless robot, and the robot control system 120, which is a brain system that controls the robot 100, will be described in more detail.

The control unit 104 of the robot 100 sends raw data including raw sensing data acquired (collected) by the sensor unit 106 (i.e., sensors 470-1 to 4) to the robot control system 120. It can be sent to . Raw data including raw sensing data can be transferred from the sensors 470-1 to 4 to the sensor driver 450 (sensor hub) and transmitted to the robot control system 120 through the wireless communication unit 102. there is.

The robot control system 120 may receive raw data including raw sensing data from the robot 100 according to communication between its wireless communication unit 320 and the wireless communication unit 102 of the robot 100.

The processor 320 of the robot control system 120 may generate a control command for controlling the driving unit 108 of the robot 100 based on the received raw data. The robot control system 120 may transmit the generated control command to the robot 100 according to communication between its wireless communication unit 320 and the wireless communication unit 102 of the robot 100. Control commands may be received by the robot 100 through the wireless communication unit 102, and the robot 100 may control the driving unit 108 by executing these control commands.

As shown, the control command from the robot control system 120 is received by the drive unit driver 104 through the wireless communication unit 102 to control the elements (actuator 482 and/or motor 484) that require control of the drive unit 108. )). The drive unit driver 104 may determine an element among the actuator 482 and the motor 484 to be controlled by the received control command and transmit a control command to the determined element to control the determined element.

The control command received by the robot 100 from the robot control system 120 is input to the drive unit 108 (actuator 482 and/or motor 484), and is input to the drive unit 108 (actuator 482 and/or motor 484). It may include low-level control commands for controlling the motor 484). In other words, the robot control system 120 does not generate an abstracted high-level command as a control command for controlling the drive unit 108 and transmits it to the robot 100, but rather the motor of the drive unit 108 ( 482) and/or low-level control commands that allow direct control (at a low level) of the actuator 484 may be generated and transmitted to the robot 100.

Accordingly, feedback control of the robot 100 of the robot control system 120 may be performed.

Below, the method for generating control commands on the robot control system 120 side will be described in more detail.

Raw sensing data (or raw data) received by the robot communication unit 310 may first be preprocessed by the preprocessor 410. The pre-processed raw sensing data may be processed by the robot state and environment recognition processor 420. The robot state and environment recognition processing unit 420 provides status information indicating the (real-time or near-real-time) state of the robot 100 and the driving condition of the robot 100, based on the raw sensing data preprocessed by the preprocessor 410. It is possible to obtain (generate) at least one piece of environmental information about the environment.

The robot control unit 430 may generate a (low-level) control command for controlling the driving unit 108 based on the robot state and information acquired by the environment recognition processing unit 420. The robot control unit 430 may calculate commands to specifically control the actuator 482 and motor 484 of the driving unit 108. For example, the robot control unit 430 may generate a command for controlling at least one of the speed, position, and torque of the actuator 482, or a command for controlling at least one of the speed and torque of the motor 484. Actuator 482 may consist of at least one motor, slider, or cylinder.

Meanwhile, the robot control unit 430 may further use reference information when generating control commands for the driving unit 108.

For example, in the case where the control command generated by the robot control unit 430 is a control command for controlling the movement of the robot 100 for autonomous driving of the robot 100, the robot control unit 430 A control command for the driving unit 108 may be generated by referring to a motion plan from the travel planning unit 450, which manages the movement plan associated with the driving unit 108. At this time, the robot control unit 430 determines the movement plan of the robot 100 based on the status information of the robot 100 acquired (generated) from raw sensing data and the environmental information of the environment in which the robot 100 drives. Control commands may be generated for the actuator 108 to follow and avoid obstacles in the environment.

Therefore, according to the control command from the robot control system 120, the actuator 482 and the motor 484 can be appropriately controlled so that the robot 100 can avoid obstacles while traveling according to the movement plan.

Meanwhile, the robot control unit 430 may further consider dynamics information of the robot 100 when generating a control command. Epidemiological information may be an example of the aforementioned reference information.

In addition, the robot control unit 430 generates control commands to ensure real-time control of the robot 100, delay (e.g., delay in wireless communication between the robot control system 120 and the robot 100). The control command can be corrected by taking this into account.

Through the embodiment, autonomous driving of the robot 100 can be effectively controlled through remote control from the robot control system 120, which is a server, without performing complex calculations on the robot 100 side. Meanwhile, the processor 320 may further include components not shown for providing services through autonomous driving of the robot 100.

Although not shown, the processor 320 may include a map creation module, a localization processing module, a route planning processing module, and a service operation module.

The map generation module may be a component for generating an indoor map of a target facility using sensing data generated about the target facility (e.g., the interior of the space) by a mapping robot (not shown) autonomously driving inside the space. . At this time, the localization processing module may determine the location of the robot 100 inside the target facility using the sensing data from the robot 100 and the indoor map of the target facility generated through the map generation module. The localization processing module may be included in the robot state and environment recognition processing unit 420 described above.

The path planning processing module may generate a control signal for controlling the indoor autonomous driving of the robot 100 using the sensing data received from the above-described robot and the generated indoor map. The path planning processing module may be included in the robot control unit 430 or the travel planning unit 440 described above. For example, the path planning processing module may generate a path (ie, path data) of the robot 100. The generated path (path data) may be set for the robot 100 to drive the robot 100 along the corresponding path. The path on which the robot 100 is set to travel may be included in the movement plan used by the robot control unit 430 to generate commands for the robot 100.

The service operation module of the processor 320 may include a function for controlling services provided by the robot 100 within the space. For example, a service provider that operates the robot control system 120 or space provides an Integrated Development Environment (IDE) for services (e.g., cloud services) provided by the robot control system 120 to users or producers of the robot 100. ) can be provided. At this time, the user or producer of the robot 100 can create software for controlling the services provided by the robot 100 in the space through the IDE and register it in the robot control system 120. In this case, the service operation module can control the service provided by the robot 100 using software registered in association with the robot 100. As a specific example, assuming that the robot 100 provides a service that delivers items requested by the user (e.g., food or parcels) to the user's location, the robot control system 120 enables the indoor autonomous driving of the robot 100. Commands related to controlling the robot 100 not only to move to the user's location, but also to provide a series of services such as delivering an object to the user when it arrives at the target location and outputting a user response voice. can be transmitted to the robot 100. The robot control unit 430 may obtain reference information referred to when generating a control command for the drive unit 108 (i.e., a control command for processing a service-related task) through the service operation module.

More detailed operations for controlling the robot 100 of the robot control system 120 will be described in more detail with reference to FIGS. 5 to 10 to be described later.

The description of the technical features described above with reference to FIGS. 1 to 3 can also be applied to FIG. 4 , so overlapping descriptions will be omitted.

In the detailed description to be described later, operations performed by the components of the robot control system 120 and the robot 100 may be described as being performed by the robot control system 120 and the robot 100 for convenience of explanation.

Figure 5 is a flowchart showing a method for controlling a robot, according to one embodiment.

At step 510 , robot control system 120 may receive raw data from robot 100 . The raw data may include raw sensing data acquired by the sensor unit 106 of the robot 100. For example, raw sensing data is data collected or acquired from each of the sensors included in the sensor unit 106 and the camera, and may be data that has not been processed through preprocessing or the like.

Alternatively, raw sensing data may be data obtained by performing only minimal pre-processing or correction processing on the robot 100 side of the sensing data initially collected from the sensor unit 106. In other words, raw sensing data may be primarily preprocessed sensing data initially collected from the sensor unit 106. For example, this primary preprocessing may be minimal noise removal from the sensing data, cropping of the image (corresponding to the sensing data), or simply processing the sensing data into a format for transmission through the wireless communication unit 102. You can.

Accordingly, there is no need to provide a separate configuration for processing or pre-processing the raw sensing data on the robot 100 side, or only a configuration for minimal pre-processing can be provided.

In step 520, the robot control system 120 may generate a control command for controlling the driving unit 108 of the robot 100 based on raw data received from the robot 100.

In step 530, the robot control system 120 may transmit the generated control command to the robot 100.

The control command generated by the robot control system 120 and transmitted to the robot 100 provides feedback control of the driving unit 108 of the robot 100 (based on raw data including sensing data from the robot 100). It may be for this purpose. That is, the control command may include a command for feedback control of at least one of the actuator and motor included in the driving unit 108.

The control command is input to the driver 108 and may include low-level control commands for controlling the driver 108. In other words, the robot control system 120 does not generate abstracted high-level commands as control commands for controlling the drive unit 108, but directly controls the motor and/or actuator of the drive unit (at a low level). You can create low-level control commands that allow you to do this.

For example, the control command may include a command for controlling at least one of the speed, position, and torque of the actuator included in the driving unit 108. Or/additionally, the control command may include a command for controlling at least one of the speed and torque of the motor included in the driving unit 108.

Receiving raw data from the robot 100 in the above-described step 510 and transmitting a control command to the robot 100 in step 530 are performed by the wireless communication unit 102 of the robot 100 and the robot control. It may be performed according to communication between the wireless communication units 310 of the system 120.

Therefore, in the embodiment, in implementing a control loop for controlling the robot 100, a controller that generates low-level control commands for controlling the robot 100 is installed at a remote location of the robot 100, such as the robot control system 120. By placing and adding wireless communication to the control loop, most of the computation for generating low-level control commands can be performed on the robot control system 120 side (rather than the robot 100). That is, in the embodiment, the control loop consisting of the sensor unit 106 - the controller - the driver 108 may be implemented outside the robot 100.

In an embodiment, localization of the robot 100, sensing data fusion, image processing and (object) recognition, and other sensing data processing (processing to extract environmental information and determine the state (including position) of the robot) are performed by the robot control system. (120) It can be performed on the side. In addition, the planning process - establishing a movement plan based on the map, information on the robot 100, task information (related to the service) - and the control process - generating control commands based on the processed sensing data and the movement plan - all control the robot. It may be performed on the system 120 side. In the embodiment, speed control, position control, torque control, etc. can be performed in various ways depending on the type of driving unit 108 through low-level control commands.

A more specific method of generating low-level control commands for controlling the robot 100 will be described in more detail with reference to FIG. 6, which will be described later.

Meanwhile, from the perspective of the robot 100, if the control command is not received by the robot 100 within a predetermined first time or the control command is not transmitted to the robot 100 for more than a predetermined second time, the robot ( 100) can be switched to standby mode. The first time and the second time may be the same or may be set differently from each other, and may be set by the robot control system 120 or the manager of the robot 100, respectively. Alternatively, when an object is detected within a predetermined distance from the robot 100, the robot 100 may be switched to standby mode. At this time, the robot 100 may ignore the control command from the robot control system 120 (i.e., regardless of the control command) and switch to standby mode.

According to the operation of the robot 100, minimum safety in the robot 100 and around the robot 100 can be secured. That is, in order to ensure the safety of the robot 100 and the surroundings of the robot 100, if a control command is not received periodically (or over a certain period of time), the robot 100 stops operation (or switches to standby mode). )can do.

For example, if a control command is not received for the robot 100 within an allowable delay time range, the robot 100 may stop operating.

As another example, when an object is detected (sensing of an object is detected) within a predetermined distance in the sensor (e.g., distance sensor) of the sensor unit 106 included in the robot 100, the robot 100 is controlled. Operation can be stopped preemptively regardless of commands. At this time, the control unit 104 of the robot 100 may include a minimum configuration (filter, etc.) for detecting objects existing within a predetermined distance.

In the embodiment as described above, when controlling the robot 100 through the robot control system 120, the minimum safety of the robot 100 and the surroundings of the robot 100 can be ensured.

The description of the technical features described above with reference to FIGS. 1 to 4 can also be applied to FIG. 5 , so overlapping descriptions will be omitted.

Figure 6 is a flowchart showing a method of generating a control command for controlling a driving unit of a robot in a robot control system, according to an example.

In step 610, the robot control system 120 may preprocess the raw sensing data received from the robot 100. The preprocessor 410 described above with reference to FIG. 4 may perform preprocessing of such raw sensing data.

For example, raw sensing data may include an image acquired by a camera of the sensor unit 106 or a plurality of sensing data acquired by a plurality of sensors (eg, distance sensors) of the sensor unit. Preprocessing for raw sensing data may include performing refine processing to enable recognition of objects within the image. For example, preprocessing of an image may include improving the image so that objects such as obstacles, people, robots, etc. can be recognized from the image. Alternatively, preprocessing for raw sensing data may be performed on the state of the robot 100 (e.g., position, posture, operation state, etc.) or the state of the environment in which the robot 100 runs (e.g., location of surrounding objects, distribution of surrounding objects) etc.) may include fusion of multiple sensing data (i.e., combining, processing, and synthesizing multiple sensing data).

The preprocessor 410 may preprocess the raw sensing data by filtering the received raw sensing data to remove noise or outliers.

In addition, the preprocessor 410 based on the time-related metadata (e.g., timestamp, round-trip time, etc.) of the sensing data included in the sensing data, (robot 100) Time-based sensing data can be corrected (using past state information and sensing data or history of state information). Additionally, the pre-processing unit 410 may perform processing on the sensing data to compensate for latency caused by wireless communication between the wireless communication unit 310 and the wireless communication unit 102.

In step 620, the robot control system 120 may obtain at least one of status information of the robot 100 and environmental information of the environment in which the robot 100 travels based on the raw sensing data. The robot state and environment recognition processing unit 420 described above with reference to FIG. 4 may perform step 620. Raw sensing data used to acquire state information and/or environmental information may represent raw sensing data preprocessed in step 610. The robot state and environment recognition processing unit 420 may obtain real-time (or nearly real-time) state information and/or environment information of the robot 100 based on the sensing data from the robot 100.

The robot state and environment recognition processing unit 420 may be responsible for creating local information around the robot 100 using sensor fusion (that is, fusion of sensing data collected from multiple sensors).

The status information of the robot 100 may include, for example, at least one of the position, posture, and operating status (speed, torque, etc.) of the robot 100 (or the driving unit 108). The environmental information may include at least one of the location of objects around the robot 100 and the distribution of objects around the robot 100, for example, in relation to the environment in which the robot 100 drives.

As in step 622, the robot state and environment recognition processing unit 420 generates a local map of the surroundings of the robot 100 based on sensing data from the robot 100 and/or (from a camera or distance sensor). ) Based on the sensing data, it is possible to recognize objects around the robot 100. In generating a local map, the robot state and environment recognition processing unit 420 may further use the path or motion plan along which the robot 100 travels as reference data.

The robot state and environment recognition processing unit 420 may obtain state information and/or environment information based on at least one of the generated local map and/or the object recognition result.

In step 630, the robot control system 120 may generate a control command for the drive unit 108 based on the information obtained in step 620. The robot control unit 430 described above with reference to FIG. 4 may perform step 630. The robot control unit 430 may generate low-level control commands to appropriately control the driving unit 108 of the robot 100 based on the status information and environmental information of the robot 100.

Below, the case of generating a control command to control the movement of the robot 100 for autonomous driving of the robot 100 will be described in more detail.

The robot control unit 430 allows the robot 100 to avoid obstacles identified in the environment while following the movement plan based on the motion plan associated with the robot 100, robot 100 status information, and environmental information. Control commands for the driver 108 can be generated.

For example, the movement plan may include information about the path along which the robot 100 should travel. The robot control unit 430 may obtain this movement plan from the travel planning unit 440. The travel planning unit 440 is a global travel planning unit that serves to create a global path plan for the robot 100, and can also create a path using a node map constructed for space. That is, when generating a control command, the robot control unit 430 may refer to a path (movement) plan including the path along which the robot 100 will travel. The status information may include the location of the robot 100. The location of the robot 100 may correspond to the current location (at the time the sensing data is collected). The environmental information may include information about objects around the robot 100. Information about the object may include location information of people, robots, or obstacles around the robot 100.

The robot control unit 430 uses a control command to cause the robot 100 to move along the path (indicated by the movement plan) and avoid surrounding obstacles (based on environmental information and state information). A command for feedback control of at least one of a motor and an actuator may be generated.

Through a control command, the robot controller 430 can move the robot 100 from the current location to the next waypoint (waypoint) based on the global path plan, and remove obstacles around the robot 100 detected from the sensing data. You can control the robot to avoid it. Additionally, the robot control unit 430 can establish an optimal speed plan for the robot 100 and transmit a control command for this to the driving unit 108 of the robot 100.

Meanwhile, the generated control command is a low-level control command that can be directly input to the driving unit 108 of the robot 100 to control the driving unit 108. Alternatively, it may be input to the drive unit 108 after correction processing (eg, minimal in the drive unit driver) on the robot 100 side. The correction process may be processing the control command received from the robot control system 120 into a format that allows the driver 108 to recognize it.

Below, with reference to FIG. 7, based on wireless communication between the robot 100 and the robot control system 120, the robot control system 120 controls the driving unit 108 of the robot 100 through low-level control commands. By doing so, it shows how the robot 100 drives autonomously.

As shown, the sensor unit 106 of the robot 100 can collect sensing data to determine the status of the robot 100 and recognize the environment (①), and the collected raw sensing data can be used to control the robot through wireless communication. It can be transmitted to the system 120 (②). Based on the raw sensing data (after preprocessing the raw sensing data), the robot control system 120 can determine the state of the robot 100 and recognize the environment in which the robot 100 runs (③). The robot control system 120 may generate a control command to control the driving unit 108 with reference to the path plan of the robot 100, based on the status and environmental information of the robot 100 (④). The robot control system 120 can generate and transmit control commands to the robot 100 that allow the robot 100 to travel a planned path and avoid surrounding obstacles (⑤). The robot 100 can appropriately control the motor and/or actuator of the driving unit 108 according to the received control command (⑥). Accordingly, the robot 100 can drive autonomously while avoiding obstacles along the path to provide services.

The description of the technical features described above with reference to FIGS. 1 to 5 can also be applied to FIGS. 6 and 7, so overlapping descriptions will be omitted.

Figure 8 is a flowchart showing a method of controlling a robot by creating a robot control model, according to an embodiment.

Referring to FIG. 8, a method of constructing a robot control model associated with the robot 100 and controlling the robot 100 using the robot will be described.

In step 810, the robot control system 120 (processor 320) controls the robot 100 based on predefined information about the robot 100 when control of the robot 100 is requested. You can create a robot control model to do this.

In step 820, the robot control system 120 may estimate the state of the robot 100 based on the generated robot control model.

In step 830, the robot control system 120 may generate a control command to control the driving unit 108 of the robot 100 based on the state of the robot 100 estimated by the robot control model. . The generated control commands may include the low-level control commands described above.

That is, the robot control model may be configured to estimate the state of the robot 100 and generate a control command for controlling the driving unit of the robot 100 based on the estimated state of the robot 100.

Here, the robot controlled by steps 810 to 830 is a different robot from the robot 100 described above with reference to FIGS. 1 to 7, or in addition to the robot 100 described above with reference to FIGS. 1 to 7. It may be an additional controlled robot.

In other words, the robot control system 120 can not only control the robot 100 using the feedback control method described above, but also build a robot control model as described above for the robot 100 and/or other robots. You can also control the robot 100 and/or other robots.

The robot control system 120 may receive a request for control of a new robot from (control system or external service interface), and accordingly obtain information about the robot from the robot control model information DB to control the robot. You can configure (or dynamically create) a robot control model for the robot.

The robot control model information DB may contain predefined information about the robot for which control has been requested. For example, predefined information about the robot (stored and managed in the robot control model information DB) used to create a robot control model includes specification information of the sensor unit and driving unit included in the robot, and the physical shape of the robot. It may include information and dynamics models associated with the robot.

The robot control system 120 can estimate the state of the robot using predefined information about the robot for which control has been requested, and create a robot control model for controlling the robot according to the estimated state. there is.

When controlling a robot using this robot control model, errors due to latency due to communication between the robot and the robot control system 120 accompanying the feedback control described above can be reduced. That is, the robot can be controlled based on the state estimated by the constructed robot control model.

Meanwhile, in the robot control model, a virtual robot can be instantiated in order to estimate the real-time state of the robot, and by executing this instantiated virtual robot, the state of the robot that is the control target can be estimated. In other words, the state of the robot being controlled can be estimated according to the simulation of the virtual robot.

Meanwhile, the robot control system 120 may further create a model for the environment in which the robot travels. This environment model can be built based on predefined information about the environment in which the robot drives (eg, data collected in the past, latest environmental information, etc.).

The robot control system 120 generates appropriate control commands for controlling the robot within the environment and transmits them to the robot, based on the robot control model corresponding to the requested robot and the environment model corresponding to the environment in which the robot travels. You can.

Control of a robot using such a robot control model can be performed when the task performed by the robot is predictable, such as repetitive or simple. In other words, robot control using a robot control model can be used to control a robot operating in a stable and deterministic situation in a space with a high level of predictability, such as an unmanned warehouse or factory.

The description of the technical features described above with reference to FIGS. 1 to 7 can also be applied to FIG. 8 , so overlapping descriptions will be omitted.

Figure 9 is a flowchart showing a method of controlling a robot by creating a robot brain, according to an example.

The robot 100 described above with reference to FIGS. 1 to 7 may be controlled by the first robot brain of the robot control system 120. That is, the above-described steps 510 to 530 may be performed by the first robot brain.

In step 910, the robot control system 120 (processor 320) generates a robot brain to control the robot when the addition of a robot is requested (i.e., when control of an additional robot is requested). You can. For example, when control of another robot is requested, the robot control system 120 (processor 320) provides a brain logically separate from the first robot brain (for controlling the robot 100) for controlling the other robot. 2 You can configure (create) a robot brain. In controlling other robots, receiving raw data from other robots, generating control commands for controlling the driving parts of the other robot, and transmitting control commands for controlling the driving parts of the other robots will be performed through the second robot brain. You can.

In this way, the robot control system 120 can dynamically generate a robot brain for controlling the robot for which control is requested.

In step 920, the robot control system 120 can individually control each robot using the generated robot brain, and can control a plurality of robots (multi-robot control such as coordinated control and cooperative control). there is.

The robot brain may be a control unit defined within the robot control system 120 to control an individual robot, for example, the preprocessor 410, the robot state and environment recognition processor 420, and the robot control unit 420 described above with reference to FIG. It may be configured to include a control unit 430.

In relation to this, FIG. 10 shows a multi-robot control method for controlling a plurality of robots when controlling a robot through a robot brain, according to an example.

In Figure 10, robots 100-1 to 100-4 controlled by the robot control system 120 are shown. The robot control system 120 can generate a robot brain in 1:1 correspondence with each robot, and control the robots individually through each of the robot brains 1020. Each of the robot brains 1020 may correspond to a layer that controls one robot. One robot brain may be logically connected to the corresponding robot brain and may correspond to the brain of a brainless robot.

Robot brains 1020 may be managed by a multi-robot management unit 1030. For example, the multi-robot management unit 1030 may request a robot or control to perform the requested service according to a request (e.g., a service provision request or a robot control request) from the control system 1040 (e.g., an external service interface). You can create a robot brain to control the robot. Alternatively, the multi-robot manager 1030 may remove a robot brain corresponding to a robot that does not require control among the robot brains 1020.

As such, in the embodiment, robot brains 1020 for controlling a plurality of robots 100-1 to 100-4 may be dynamically created and removed according to the need for controlling each robot.

The multi-robot management unit 1030 can transmit an abstracted command (global command) to the robot brain corresponding to the requested robot among the robot brains 1020 in response to a request from the control system 1040, and the robot brain is global. By analyzing the command, a (low-sequence) control command can be generated to control the corresponding robot based on the sensing data received from the corresponding robot.

As shown, one robot brain can be connected 1:1 to the robot corresponding to the gateway 1050. At this time, the gateway 1050 may include the communication unit 310 described above or may be a part of the communication unit 310.

Alternatively, each of the robot brains 1020 may have a communication unit 310 or a communication function, and may be directly (or directly) connected to the corresponding robot. In this case, gateway 1050 may only be involved in the initial connection between the robot brain and the corresponding robot. That is, the gateway 1050 can manage the initial connection between the robot brain and the corresponding robot.

For example, the first robot brain (one of the robot brains 1020) for controlling the above-described robot 100 is initially connected to the robot 100 and the gateway 1050 of the robot control system 120, After the initial connection, it is possible to communicate with the robot 100 through the communication unit (communication function) of the first robot brain. Meanwhile, the second robot brain (one of the robot brains 1020) for controlling another robot described above is initially connected to another robot through the gateway 1050, and after the initial connection, the communication unit of the second robot brain You can communicate with other robots through (communication function).

In other words, the robot control system 120 may be configured to have a communication unit corresponding to each of the robot brains 1020, and the robot brains 1020 may be configured to share one communication unit.

Meanwhile, in FIG. 10, the robot control model 1010 described above with reference to FIG. 8 is further illustrated. As an example, the robot control model 1010 can be built for the robot 100-1, generates a control command to control the driving part of the robot 100-1, and transmits it to the robot 100-1. You can. Control commands may be transmitted to the robot 100-1 through the gateway 1050. The robot control model 1010 may be constructed using a robot control model information DB, not shown. A plurality of robot control models 1010 may also be constructed corresponding to the robots 100-1 to 100-4, and the plurality of robot control models 1010 may be multi-functional (in a similar manner to the robot brains 1020). It can be managed by the robot management unit 1030.

According to a global command by the multi-robot management unit 1030, the robot brains 1020 control the robots 100-1 to 100-4, thereby controlling at least two of the robots 100-1 to 100-4. can be controlled in conjunction with each other (cooperative control). For example, two or more robots may be controlled to operate in cooperation with each other and may be controlled to perform tasks related to each other in providing services.

As such, in the embodiment, a plurality of brainless robots 100-1 to 100-4 may be controlled individually or in conjunction with each other by the robot control system 120.

The description of the technical features described above with reference to FIGS. 1 to 8 can also be applied to FIGS. 9 and 10, so overlapping descriptions will be omitted.

As such, in the embodiment, analysis and processing of sensing data for controlling the robot 100 and generation of control commands may all be performed in the robot control system 120, which is a server. Accordingly, the control unit 104 of the robot 100 may not include a complex configuration such as a GPU, and a component such as the GPU may be mounted only on the processor 320 of the robot control system 120.

Meanwhile, depending on the embodiment, the robot control system 120 establishes an optimal speed plan for the robot 100, but the robot 100 and the robot control system 120 ensure that low-level control according to the plan is performed on the robot. ) could also be implemented.

The system or device described above may be implemented with hardware components, software components, or a combination of hardware components and software components. For example, devices and components described in embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), etc. , may be implemented using one or more general-purpose or special-purpose computers, such as a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. It can be embodied permanently or temporarily. Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

As described above, although the embodiments have been described with limited examples and drawings, various modifications and variations can be made by those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the claims described below.

Claims (15)

In a method of controlling a robot, performed by a robot control system,
Receiving raw sensing data from the robot;
Generating a control command to control a driving unit of the robot based on the raw sensing data; and
Transmitting the generated control command to the robot
Including,
The control command is input to the driving unit and includes a low-level control command for controlling the driving unit,
According to communication between the wireless communication unit of the robot and the wireless communication unit of the robot control system, the raw sensing data is received from the robot, and the control command is transmitted to the robot,
The robot control system is configured to control a multi-robot including the robot and other robots,
The above method is,
When control of the other robot is requested, estimating the state of the other robot based on a robot control model established in advance to control the other robot based on predefined information about the other robot; and
Based on the environment model pre-built based on predefined information about the environment in which the robot and the other robot travel, and the estimated state of the other robot, generate another control command to control the driving unit of the other robot. steps to do
It further includes,
The control command and the other control command allow the robot and the other robot to perform tasks associated with each other as a global command for controlling the multi-robot is received, so that the robot and the other robot cooperate with each other. A method of controlling a robot, which is a command to control the robot to do something. According to paragraph 1,
The driving unit includes at least one of an actuator and a motor,
The control command includes a command for controlling at least one of speed, position, and torque of the actuator, or
A method of controlling a robot, including commands for controlling at least one of speed and torque of the motor. According to paragraph 2,
The control command includes a command for feedback controlling the actuator or the motor. According to paragraph 1,
The step of generating the control command is,
Preprocessing the raw sensing data
Including,
The raw sensing data includes an image acquired by a camera of a sensor unit included in the robot or a plurality of sensing data acquired by a plurality of sensors of the sensor unit,
The preprocessing is,
Including performing refine processing to enable recognition of objects within the image, or
A method of controlling a robot, comprising fusion of the plurality of sensing data to determine the state of the robot or the state of the environment in which the robot travels. According to paragraph 1,
The step of generating the control command is,
Based on the raw sensing data, acquiring at least one of information about the state of the robot and environmental information of an environment in which the robot drives; and
Generating the control command for controlling the driving unit based on the obtained information
A method of controlling a robot, including. According to clause 5,
The obtaining step is,
i) generating a local map around the robot based on the raw sensing data; and ii) recognizing objects around the robot based on the raw sensing data.
Contains at least one of
A method of controlling a robot, acquiring the at least one piece of information based on at least one of the local map and a recognition result of the object. According to clause 5,
The control command is a control command for controlling the movement of the robot for autonomous driving of the robot,
The step of generating the control command is,
Based on a motion plan associated with the robot, the state information, and the environment information, the robot generates control commands for the drive unit to cause the robot to follow the movement plan and avoid obstacles in the environment. How to control it. In clause 7,
The movement plan includes information about the path the robot should travel,
The status information includes the location of the robot,
The environmental information includes information about objects around the robot,
The control command includes a command for feedback controlling at least one of a motor and an actuator included in the driving unit so that the robot moves along the path and avoids the obstacle. According to paragraph 1,
The raw sensing data is the sensing data initially collected from the sensor unit included in the robot and initially preprocessed on the robot side,
The control command is input to the driving unit after correction processing on the robot side,
The primary preprocessing includes i) removal of noise included in the raw sensing data, ii) cropping processing for the image included in the raw sensing data, and iii) from the wireless communication unit of the robot to the wireless communication unit of the robot control system. A method for controlling a robot, comprising at least one of processing the raw sensing data into a format for transmission. According to paragraph 1,
The robot is a brainless robot that controls the driving unit by executing the low-level control command received from the robot control system without performing a process of interpreting the received control command. . According to paragraph 1,
Predefined information about the other robot used to create the robot control model is,
A method of controlling a robot, including specification information of a sensor unit and a driving unit included in the other robot, physical form information of the other robot, and a dynamics model associated with the other robot. According to paragraph 1,
The robot is controlled by a first robot brain of the robot control system,
The receiving step, the generating step, and the transmitting step are performed by the first robot brain,
When control of another robot included in the multi-robot is requested, configuring a second robot brain that is logically distinct from the first robot brain for controlling the other robot.
It further includes,
Receiving raw data from the other robot, generating a control command for controlling the driving unit of the other robot, and transmitting another control command for controlling the driving unit of the other robot are performed through the second robot brain. ,
The control command, the other control command, and the another control command, as the global command for controlling the multi-robot is received, cause the robot, the other robot, and the another robot to each perform tasks associated with each other. A method of controlling a robot, which is a command for controlling the robot, the other robot, and the other robot to operate in cooperation with each other. According to clause 12,
The first robot brain is initially connected to the robot through a gateway of the robot control system, and after the initial connection, communicates with the robot through a communication unit of the first robot brain,
The second robot brain is initially connected to the other robot through the gateway, and after the initial connection, communicates with the other robot through a communication unit of the second robot brain. According to paragraph 1,
If the control command is not received by the robot within a first predetermined time, or the control command is not transmitted to the robot for more than a predetermined second time,
A method of controlling a robot, wherein the robot is switched to standby mode. In a robot control system that controls a robot,
Department of Wireless Communications; and
At least one processor implemented to execute computer readable instructions
Including,
The at least one processor,
According to communication between the wireless communication unit and the wireless communication unit of the robot, raw sensing data is received from the robot,
Based on the raw sensing data, generate a control command to control the driving unit of the robot,
According to communication between the wireless communication unit and the wireless communication unit of the robot, transmitting the generated control command to the robot,
The control command is input to the driving unit and includes a low-level control command for controlling the driving unit,
The robot control system is configured to control a multi-robot including the robot and other robots,
The at least one processor,
When control of the other robot is requested, the state of the other robot is estimated based on a robot control model established in advance to control the other robot based on predefined information about the other robot,
Based on the environment model pre-built based on predefined information about the environment in which the robot and the other robot travel, and the estimated state of the other robot, generate another control command to control the driving unit of the other robot. do,
The control command and the other control command allow the robot and the other robot to perform tasks associated with each other as a global command for controlling the multi-robot is received, so that the robot and the other robot cooperate with each other. A robot control system, which is a command to control something to do.