WO2022234923A1

WO2022234923A1 - Communication-based robot control method and system, and building in which robot is positioned

Info

Publication number: WO2022234923A1
Application number: PCT/KR2022/000145
Authority: WO
Inventors: 윤영환; 박경식
Original assignee: 네이버랩스 주식회사
Priority date: 2021-05-06
Filing date: 2022-01-05
Publication date: 2022-11-10
Also published as: KR20220151460A; KR102578127B1; KR20230135548A

Abstract

Provided is a robot control method in which a robot control system: generates a control command for controlling a driving unit of the robot, on the basis of raw data received from the robot; and transmits the generated control command to the robot to control the driving unit of the robot. The control command generated by the robot control system comprises a low-level control command for controlling the driving unit of the robot.

Description

Communication-based robot control method and system, and a building in which the robot is placed

The following description relates to a method and system for controlling a robot, and to a method and system for controlling a robot by generating a control command for controlling a driving unit of the robot based on raw data received from the robot. .

There is growing interest in robots that provide services such as manufacturing, logistics, transportation, cleaning and navigation. A robot implemented to provide such a service may move to a destination on an optimal route through autonomous driving, for example, and provide a service at the destination.

In order for the robot to provide these services, it is necessary to properly control the robot's driving unit. The driving unit of the robot may be controlled based on the state of the robot or the surrounding environment. Configuring the robot so that the operation for controlling the robot's driving unit is performed on the robot side not only complicates the implementation of the robot, but also increases the maintenance cost of the robot and reduces the scalability of the robot.

Accordingly, in order to lower the implementation cost of the robot and facilitate maintenance and expansion of the robot, calculations required to control the driving unit of the robot and the generation of commands for performing low-level control of the driving unit It is required to build a robot and a robot control system so that it can be performed on the server side.

Korean Patent Laid-Open No. 10-2005-0024840 is a technology related to a path planning method for an autonomous mobile robot, and it is a technology for a mobile robot that moves autonomously at home or in an office to avoid obstacles and to find an optimal path that can move safely and quickly to a target point. How to plan is disclosed.

The information described above is for understanding only, and may include content that does not form a part of the prior art, and may not include what the prior art can present to a person skilled in the art.

Provided is a robot control method in which a robot control system generates a control command for controlling a driving unit of the robot based on raw data received from the robot, and transmits the generated control command to the robot to control the driving unit of the robot.

The robot control system generates a low-level control command to control the driving unit of the robot based on the raw sensing data received through wireless communication with the robot, and recognition and command interpretation based on the sensing data from the robot side. To provide a control method capable of directly controlling an actuator and a motor included in a driving unit of a robot through the low-level control command without having to process the operation of the robot.

In one aspect, in a method of controlling a robot, performed by a robot control system, receiving raw data from the robot, based on the received raw data, a control command for controlling a driving unit of the robot and transmitting the generated control command to the robot, wherein the control command is input to the driving unit and includes a low-level control command for controlling the driving unit. A method of controlling is provided.

The driving unit includes at least one of an actuator and a motor, and the control command includes a command for controlling at least one of speed, position, and torque of the actuator, or for controlling at least one of the speed and torque of the motor It can contain commands.

The control command may include a command for feedback control of the actuator or the motor.

The raw data may include raw sensing data obtained by the sensor unit of the robot.

The generating of the control command includes preprocessing the raw sensing data, wherein the raw sensing data is an image acquired by a camera of the sensor unit or a plurality of sensing units acquired by a plurality of sensors of the sensor unit. data, wherein the pre-processing includes performing refine processing to enable recognition of an object in the image, or the plurality of It may include fusion (fusion) of the sensed data of.

The generating of the control command may include, based on the raw sensing data, obtaining at least one information of the state information of the robot and environment information of an environment in which the robot is driven, and based on the obtained information, the It may include generating the control command for controlling the driving unit.

The acquiring may include at least one of i) generating a local map around the robot based on the raw sensing data, and ii) recognizing an object around the robot based on the raw sensing data. one, and the at least one piece of information may be acquired based on at least one of the local map and the recognition result of the object.

The control command is a control command for controlling movement of the robot for autonomous driving of the robot, and generating the control command includes a motion plan associated with the robot, the state information and the environment information. based on , it is possible to generate a control command for the driving unit to cause the robot to follow the movement plan and avoid obstacles in the environment.

The movement plan includes information on a path through which the robot should travel, the status information includes a position of the robot, the environment information includes information about objects around the robot, and the control command may include a command for feedback-controlling at least one of a motor and an actuator included in the driving unit so that the robot moves along the path and avoids the obstacle.

The raw sensing data is that the sensing data first collected from the sensor unit is pre-processed by the robot side, and the control command may be input to the driving unit after correction processing at the robot side.

According to communication between the wireless communication unit of the robot and the wireless communication unit of the robot control system, the raw data may be received from the robot, and the control command may be transmitted to the robot.

The robot may be a brainless robot that controls the driving unit by executing the low-level control command received from the robot control system without performing a process of interpreting the received control command.

The method for controlling the robot includes generating a robot control model for controlling the other robot based on predefined information about the other robot when control of the other robot is requested, based on the robot control model The method may further include estimating the state of the other robot and generating a control command for controlling the driving unit of the other robot based on the estimated state of the other robot.

The predefined information about the other robot used to generate the robot control model includes specification information of a sensor unit and a driving unit included in the other robot, physical shape information of the other robot, and dynamics associated with the other robot. It may include a dynamics model.

The robot is controlled by a first robot brain of the robot control system, the receiving step, the generating step, and the transmitting step are performed by the first robot brain, the method of controlling the robot Further comprising the step of configuring a second robot brain logically separated from the first robot brain for controlling the other robot when the control of the other robot is requested, and receiving the raw data from the other robot , generation of a control command for controlling the driving unit of the other robot and transmission of a control command for controlling the driving unit of the other robot may be performed through the second robot brain.

The first robot brain is first connected to the robot through a gateway of the robot control system, and after the initial connection, communicates with the robot through the communication unit of the first robot brain, and the second robot brain is the other robot It is connected to the gateway for the first time, and after the initial connection, it can communicate with the other robot through the communication unit of the second robot brain.

If the control command is not received by the robot within a first predetermined time period or the control command is not transmitted to the robot for a second predetermined time period or more, the robot may be switched to a standby mode.

When an object is detected within a predetermined distance from the robot, the robot may be switched to a standby mode regardless of the control command.

In another aspect, in a robot control system for controlling a robot, comprising at least one processor implemented to execute a command readable by a wireless communication unit and a computer, wherein the at least one processor includes the wireless communication unit and the robot According to the communication between the wireless communication unit of the receiving raw data from the robot, based on the received raw data, generates a control command for controlling the driving unit of the robot, between the wireless communication unit and the wireless communication unit of the robot According to communication, the generated control command is transmitted to the robot, and the control command is input to the driving unit, and includes a low-level control command for controlling the driving unit, a robot control system is provided. .

The raw data includes raw sensing data obtained by a sensor unit of the robot, and the at least one processor, a preprocessor preprocessing the raw sensing data, based on the preprocessed raw sensing data, A robot generating the control command for controlling the driving unit based on the robot state and environment recognition processing unit obtaining at least one information of the state information of the robot and the environment information of the environment in which the robot is driven, and the obtained information It may include a control unit.

The control command is a control command for controlling the movement of the robot for autonomous driving of the robot, and the at least one processor includes a travel planning unit for managing a movement plan associated with the robot, the robot control unit comprising: Based on the state information and the environment information, it is possible to generate a control command for the driving unit to cause the robot to follow the movement plan and avoid obstacles in the environment.

The at least one processor is configured to generate a robot control model for controlling the other robot based on predefined information about the other robot when control of the other robot is requested, wherein the robot control model is configured to: It is possible to estimate the state of the other robot, and generate a control command for controlling the driving unit of the other robot based on the estimated state of the other robot.

wherein the at least one processor comprises a first robot brain, wherein the first robot brain is configured to receive the raw data, generate the control command, and send the control command to the robot, wherein the at least one The processor is configured to generate a second robot brain logically separated from the first robot brain for controlling the other robot when the control of the other robot is requested, and receiving raw data from the other robot, The generation of a control command for controlling the driving unit of the other robot and transmission of the control command for controlling the driving unit of the other robot may be performed through the second robot brain.

In another aspect, in a robot controlled by a robot control system, a wireless communication unit, a sensor unit including at least one sensor, a driving unit including at least one of a motor and an actuator, and a sensor driver for the sensor unit and the and a control unit including a driving unit driver for the driving unit, wherein the control unit transmits raw data including raw sensing data obtained by the sensor unit to the robot control system through the wireless communication unit, The robot control system receives a control command for controlling the driving unit generated based on the raw data through the wireless communication unit, executes the received control command to control the driving unit, and the received control A command is input to the driving unit, and includes a low-level control command for controlling the driving unit, the robot is provided.

The robot control system generates a low-level control command to control the driving unit of the robot based on the raw sensing data received through wireless communication with the robot, and the actuator included in the driving unit of the robot through the low-level control command And by being able to directly control the motor, the robot can be configured to include an on-board computer system that does not include a complex on-board computer system or includes minimal hardware.

Since the calculation required to generate a low-level control command for controlling the driving unit of the robot is performed on the robot control system side, tasks such as recognition and command interpretation based on the sensing data on the robot side do not need to be processed, so that the service The complexity of the robot can be reduced and the maintenance of the service robot can be streamlined.

Since the robot is simply implemented so that the driving unit of the robot is controlled according to a control command from the robot control system, the possibility of changing the function and scalability of the robot can be increased according to the configuration of the robot control system.

1 shows a method of controlling a robot, according to an embodiment.

2 is a block diagram illustrating a robot, according to an embodiment.

3 is a block diagram illustrating a robot control system for controlling a robot, according to an embodiment.

4 illustrates a method for the robot control system to control the driving unit of the robot through a low-level control command based on wireless communication between the robot and the robot control system, according to an embodiment.

5 is a flowchart illustrating a method of controlling a robot, according to an embodiment.

6 is a flowchart illustrating a method of generating a control command for controlling a driving unit of a robot in a robot control system, according to an example.

7 illustrates a method in which the robot autonomously travels by controlling the driving unit of the robot through a low-level control command by the robot control system based on wireless communication between the robot and the robot control system, according to an example.

8 is a flowchart illustrating a method of controlling a robot by generating a robot control model, according to an embodiment.

9 is a flowchart illustrating a method of controlling a robot by generating a robot brain, according to an example.

10 illustrates a multi-robot control method for controlling a plurality of robots, according to an example.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

1 shows a method of controlling a robot, according to an embodiment.

The robot 100 illustrated in FIG. 1 may be, for example, a service robot configured to provide a service in a space such as a building, indoors, or other open area.

The robot 100 may be configured to perform a specific function or perform a task related to provision of a service according to control by the robot control system 120 .

The space in which the robot 100 travels is a place where the robot 100 provides a service, and may represent, for example, a building. Such a space is a space where a plurality of persons (hereinafter, referred to as users) work or reside, and may include a plurality of partitioned spaces. A space may represent a part of a building (a specific floor or subspace within that floor). The robot 100, which is a service robot, may be configured to provide a service on at least one floor of a space.

The robot control system 120 may be configured to control each of a plurality of robots. At this time, each of the robots may move in the space to provide a service to an appropriate location in the space or an appropriate user.

The service provided by the robot 100 may include, for example, at least one of a delivery service delivery service, a beverage (coffee, etc.) delivery service according to an order, a cleaning service, and other information/content providing services.

The robot 100 may provide a service to a predetermined user or at a predetermined location in space through autonomous driving. The robot 100 may move to a specific location according to the control by the robot control system 120 or may execute other tasks or functions required to provide services.

In the embodiment, the robot control system 120 generates a control command for controlling the driving unit of the robot 100 based on the raw data received from the robot 100, and transmits the generated control command to the robot to It is possible to control the driving unit of (100).

For example, as shown, the robot control system 120 may receive raw data including raw sensing data collected by the sensor unit of the robot 100 from the robot 100 (①). The robot control system 120 may generate a control command for controlling the driving unit of the robot 100 to a low-level based on the received raw data (②). The robot control system 120 may transmit the generated control command to the robot 100 (③), and the robot 100 may control the driving unit by executing the received control command. In other words, the robot control system 120 does not generate a high-level command abstracted as a control command for controlling the driving unit of the robot 100, but operates the motor and/or actuator of the driving unit (lower level). You can create low-level control commands that give you direct control. Therefore, on the robot 100 side, there is no need to perform a process of interpreting the received control command, and only by executing the received low-level control command (eg, input to the driving unit), the driving unit can be operated according to the corresponding control command. .

As such, the robot 100 may be implemented as a brainless robot that controls the driving unit by executing a low-level control command from the robot control system 120 without performing a process of interpreting the received control command. That is, the robot control system 120 may be implemented as a robot brain (brain system) that controls such a brainless robot.

The robot 100 transmits the collected raw sensing data to the robot control system 120 and receives a low-level control command from the robot control system 120 to operate the driving unit. It may not contain components such as computer systems.

The structure and specific operation of the robot 100 and the robot control system 120 will be described in more detail with reference to FIGS. 2 to 4 to be described later.

2 is a block diagram illustrating a robot, according to an embodiment.

As described above, the robot 100 may be a service robot used to provide a service in a space. The robot 100 may be configured to provide a service to a predetermined user or at a predetermined location in space through autonomous driving.

The robot 100 may be a physical device, and as shown, may include a control unit 104 , a driving unit 108 , a sensor unit 106 , and a communication unit 102 .

The control unit 104 may be a physical processor built into the robot 100 or an onboard computer system. In the control unit 104 , the robot 100 implemented as a brainless robot communicates with the robot control system 120 , transmits data to the robot control system 120 , and receives a command received from the robot control system 120 . It may include only the components necessary for processing (eg, transmission to the drive unit 108 and/or the sensor unit 106 ).

For example, the control unit 104 transmits the raw sensing data collected through the sensor unit 106 to the robot control system 120 , and receives a low-level control command from the robot control system 120 to operate the driving unit 108 . It may include only a configuration to make it happen. In other words, the control unit 104 may not include a complex configuration (eg, GPU, etc.) for interpreting and processing the sensed data and control commands.

The control unit 104 may include a sensor driver for the sensor unit 106 and a driver driver for the driving unit 108 .

The communication unit 102 may be configured for the robot 100 to communicate with other devices (such as the robot control system 120 ). In other words, the communication unit 102 transmits/receives data and/or information to/from other devices such as the robot control system 120 , an antenna of the robot 100 , a data bus, a network interface card, a network interface chip, and a networking interface. It may be a hardware module, such as a port, or a software module, such as a network device driver or networking program.

For example, the communication unit 102 is a wireless communication unit for communicating with the robot control system 120 , and transmits raw data including (raw) sensing data to the robot control system 120 , and from the robot control system 120 . A control command for the driving unit 108 may be received.

The sensor unit 106 may be configured to collect data required for autonomous driving and service provision of the robot 100 . The sensor unit 106 may not include expensive sensing equipment, and may only include a sensor such as a low-cost ultrasonic sensor and/or a low-cost camera.

The sensor unit 106 may include sensors for identifying objects such as other robots, people, obstacles, etc. in front and/or behind. For example, other robots, people, and other features may be identified through the camera of the sensor unit 106 . Alternatively, the sensor unit 106 may include an infrared sensor (or an infrared camera). In addition to the camera, the sensor unit 106 may further include a sensor for recognizing/identifying a nearby user, another robot, or a feature. In addition, the sensor unit 106 may include at least one distance sensor for identifying a distance to an object(s) existing in the vicinity. In addition, the sensor unit may include a sensor for detecting the state of the robot 100 and recognizing the environment, including an odometer.

The (raw) sensing data from the sensors of the sensor unit 106 may be transmitted to the robot control system 120 through the communication unit 102 . For example, the sensed data may be transmitted to the robot control system 120 through the communication unit 102 via a sensor driver (or sensor hub) of the control unit 104 .

The driving unit 108 controls the movement of the robot 100 and may include equipment (hardware) for performing this as a configuration that enables movement. In addition, the driving unit 108 may include equipment (hardware) for performing a function necessary for the robot 100 to perform a task related to the requested service.

For example, the driving unit 108 may include at least one motor and/or at least one actuator for operating wheels, caterpillar wheels, legs, etc. for movement of the robot 100 .

Also, the driving unit 108 may include equipment related to a service provided by the robot 100 . For example, in order to perform a food/delivery delivery service, the driving unit 108 of the robot 100 is configured to load food/delivery or deliver food/delivery to a user (eg, robot arm). may include In addition, the robot 100 may further include a speaker and/or a display for providing information/content.

The driving unit 108 may be controlled according to a control command from the robot control system 120 . The driving unit 108 may execute a low-level control command received from the robot control system 120 to perform an operation corresponding to the control command. For example, when a low-level control command from the robot control system 120 is input to the driving unit 108 , the driving unit 108 may perform an operation indicated by the corresponding control command.

A control command from the robot control system 120 may be transmitted to the driving unit 108 through the communication unit 102 . For example, a control command from the robot control system 120 is received via the communication unit 102 and transmitted to each component of the driving unit 108 (eg, each motor and/or actuator) by a driving unit driver of the control unit 104 . can be

As described, the robot 100 transmits the sensed data from the sensor unit 106 to the robot control system 120 and receives a control command from the robot control system 120 to control the bar, (corresponding to the brain). It can be a brainless robot (controlled by the robot control system 120 ).

Meanwhile, each of the robots 100 may have different sizes and shapes (ie, different types of sensor units 106 and/or driving units 108 ) according to models or services provided.

The configuration and operation of the robot control system 120 for controlling the robot 100 will be described in more detail with reference to FIGS. 3 and 4 to be described later, respectively.

The description of the technical features described above with reference to FIG. 1 may be applied to FIG. 2 as it is, and thus a redundant description will be omitted.

3 is a block diagram illustrating a robot control system for controlling a robot composed of a plurality of modular robots, according to an embodiment.

The robot control system 120 may be a device for controlling the movement (ie, driving) in the space of the robot 100 and the provision of services in the space by the robot 100 . When there are a plurality of robots 100 , the robot control system 120 may control movement of each of the plurality of robots and the provision of services of each of the robots 100 .

The robot control system 120 can plan and set a path to be moved by the robot 100 in order to provide a service through communication with the robot 100, and transmit a control command for movement according to this path to the robot ( 100) can be forwarded. The robot 100 may move to a predetermined position or to a predetermined user according to the received control command. In addition, the robot 100 may provide a service (perform a service-related task) to a predetermined location or to a predetermined user according to the control of the robot control system 120 .

The robot control system 120 may include at least one computing device.

The robot control system 120 may be a device for planning and setting a route for the traveling of the robot 100 and controlling the movement of the robot 100 as described above. The robot control system 120 may include at least one computing device, and may be implemented as at least one server (eg, a cloud server) located in or outside the space in which the robot 100 travels.

As shown, the robot control system 120 may include a memory 330 , a processor 320 , a communication unit 310 , and an input/output interface 340 .

The memory 330 is a computer-readable recording medium and may include a random access memory (RAM), a read only memory (ROM), and a permanent mass storage device such as a disk drive. Here, the ROM and the non-volatile mass storage device may be separated from the memory 330 and included as separate permanent storage devices. Also, an operating system and at least one program code may be stored in the memory 330 . These software components may be loaded from a computer-readable recording medium separate from the memory 330 . The separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, and a memory card. In another embodiment, the software components may be loaded into the memory 330 through the communication unit 310 rather than a computer-readable recording medium.

The processor 320 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. The command may be provided to the processor 320 by the memory 330 or the communication unit 310 . For example, the processor 320 may be configured to execute a received instruction according to a program code loaded into the memory 330 .

The communication unit 310 may be configured for the robot control system 120 to communicate with other devices (such as the robot 100 or another server). In other words, the communication unit 310 transmits/receives data and/or information to/from other devices, such as an antenna of the robot control system 120, a data bus, a network interface card, a network interface chip, and a hardware module such as a networking interface port or the like. It may be a software module such as a network device driver or a networking program.

As an example, the communication unit 310 is a wireless communication unit for communicating with the robot 100 , and receives raw data including (raw) sensing data from the robot 100 , and for the driving unit 108 to the robot 100 . Control commands can be sent. In other words, the robot 100 and the robot control system 120 may transmit and receive data and commands by communicating through the respective

wireless communication units

102 and 310 .

The input/output interface 340 may be a means for interfacing with an input device such as a keyboard or mouse and an output device such as a display or speaker.

Further, in other embodiments, the robot control system 120 and the processor 320 may include more components than the illustrated components. For example, the processor 320 may include components such as those shown in FIG. 4 (eg, 410 to 440 ).

In the embodiment, raw data including the raw sensing data collected by the sensor unit 106 of the robot 100 from the robot 100 through the communication unit 310 may be received by the processor 320, and the processor ( The 320 may generate a control command (low-level control command) for the driving unit 108 of the robot 100 based on the sensed data received from the robot 100 . The processor 320 may transmit a control command to the robot 100 through the communication unit 310 , and thus the driving unit 108 may be controlled according to the transmitted control command. The communication unit 310 may utilize socket communication, a stream, a message queue, and the like for communication with the robot 100 .

The operation of the processor 320 generating a control command for the driving unit 108 may be performed by the components 410 to 440 .

Each of the components 410 to 440 of the processor 320 may be a software and/or hardware module as a part of the processor 320 , and may represent a function (functional block) implemented by the processor. Configurations 410 to 440 of the processor 320 will be described later with reference to FIG. 4 .

The description of the technical features described above with reference to FIGS. 1 and 2 can be applied to FIG. 3 as it is, and thus a redundant description will be omitted.

As shown, the robot control system 120 may include a preprocessor 410 , a robot state and environment recognition processor 420 , a robot controller 430 , and a travel planner 440 . These components 410 to 440 may be a part of the above-described processor 320 or a function implemented by the processor 320 . Components included in the processor 320 may perform different functions ( different functions).

Meanwhile, the control unit 104 of the robot 100 may include a sensor driver 450 and a driving unit driver 460 . The sensor unit 106 includes a plurality of sensors 470 - 1 to 4 , and the driving unit 108 is illustrated as including an actuator 482 and a motor 484 .

With reference to FIG. 4 , the robot 100 as a brainless robot and the robot control system 120 as a brain system for controlling the robot 100 will be described in more detail.

The control unit 104 of the robot 100 transmits raw data including the raw sensing data acquired (collected) by the sensor unit 106 (ie, the sensors 470-1 to 4) to the robot control system 120 . can be sent to Raw data including raw sensing data may be transmitted from the sensors 470-1 to 4 to the sensor driver 450 (sensor hub) and may be transmitted to the robot control system 120 through the wireless communication unit 102 . have.

The robot control system 120 may receive raw data including raw sensing data from the robot 100 according to communication between its own wireless communication unit 320 and the wireless communication unit 102 of the robot 100 .

The processor 320 of the robot control system 120 may generate a control command for controlling the driving unit 108 of the robot 100 based on the received raw data. The robot control system 120 may transmit the generated control command to the robot 100 according to communication between its own wireless communication unit 320 and the wireless communication unit 102 of the robot 100 . The control command may be received by the robot 100 through the wireless communication unit 102 , and the robot 100 may control the driving unit 108 by executing the control command.

As shown, the control command from the robot control system 120 is received by the driving unit driver 104 through the wireless communication unit 102, and the elements that require control of the driving unit 108 (actuator 482 and/or motor 484) )) can be entered. The driving unit driver 104 may determine an element to be controlled by the received control command among the actuator 482 and the motor 484 , and transmit the control command to the determined element for controlling the determined element.

The control command received by the robot 100 from the robot control system 120 is input to the drive unit 108 (actuator 482 and/or motor 484 ), and the drive unit 108 (actuator 482 and/or motor 484 ). low-level control commands for controlling the motor 484). In other words, the robot control system 120 does not generate and transmit a high-level command abstracted as a control command for controlling the driving unit 108 to the robot 100, but rather the motor ( 482 ) and/or actuators 484 may be generated and sent to the robot 100 to allow direct (low-level) control of the actuators 484 .

Accordingly, feedback control of the robot 100 of the robot control system 120 may be performed.

Hereinafter, a method of generating a control command on the robot control system 120 side will be described in more detail.

The raw sensing data (or raw data) received by the robot communication unit 310 may be preprocessed first by the preprocessor 410 . The preprocessed raw sensing data may be processed by the robot state and environment recognition processing unit 420 . The robot state and environment recognition processing unit 420 is based on the raw sensing data preprocessed by the preprocessing unit 410, the robot 100 is driven and the state information indicating the (real-time or near real-time) state of the robot 100. It is possible to acquire (generate) at least one piece of environment information of the environment to be used.

The robot control unit 430 may generate a (low level) control command for controlling the driving unit 108 based on the information obtained by the robot state and environment recognition processing unit 420 . The robot control unit 430 may calculate a command for specifically controlling the actuator 482 and the motor 484 of the driving unit 108 . For example, the robot controller 430 may generate a command for controlling at least one of the speed, position, and torque of the actuator 482 or a command for controlling at least one of the speed and torque of the motor 484 . Actuator 482 may be comprised of at least one motor, slider, or cylinder.

Meanwhile, the robot control unit 430 may further use the reference information in generating a control command for the driving unit 108 .

For example, when the control command generated by the robot control unit 430 is a control command for controlling the movement of the robot 100 for autonomous driving of the robot 100 , the robot control unit 430 may A control command for the driving unit 108 may be generated with reference to a motion plan from the driving planner 450 that manages the related movement plan. At this time, the robot control unit 430, based on the state information of the robot 100 obtained (generated) from the raw sensing data, and the environment information of the environment in which the robot 100 travels, the robot 100 determines the movement plan. It can generate control commands for the drive unit 108 to follow and avoid obstacles in the environment.

Accordingly, according to a control command from the robot control system 120 , the actuator 482 and the motor 484 can be appropriately controlled so that the robot 100 can avoid obstacles while traveling according to the movement plan.

Meanwhile, the robot controller 430 may further consider information on dynamics of the robot 100 in generating a control command. The dynamics information may be an example of the aforementioned reference information.

In addition, the robot control unit 430 is delayed (eg, a delay in wireless communication between the robot control system 120 and the robot 100) in order to ensure real-time control of the robot 100 in generating the control command. It is possible to correct the control command in consideration of

In the embodiment, through remote control from the robot control system 120 that is a server, autonomous driving of the robot 100 can be effectively controlled without performing complex calculations on the robot 100 side. Meanwhile, the processor 320 may further include components not shown for providing a service through autonomous driving of the robot 100 .

Although not shown, the processor 320 may include a map generation module, a localization processing module, a route planning processing module, and a service operation module.

The map generation module may be a component for generating an indoor map of a target facility using sensing data generated by a mapping robot (not shown) autonomously driving inside a space for a target facility (eg, inside a space). . In this case, the localization processing module may determine the position of the robot 100 inside the target facility by using the sensing data from the robot 100 and the indoor map of the target facility generated through the map generation module. The localization processing module may be included in the robot state and environment recognition processing unit 420 described above.

The path planning processing module may generate a control signal for controlling the indoor autonomous driving of the robot 100 by using the sensing data received from the above-described robot and the generated indoor map. The path planning processing module may be included in the above-described robot control unit 430 or travel planning unit 440 . For example, the path planning processing module may generate a path (ie, path data) of the robot 100 . The generated path (path data) may be set for the robot 100 for the robot 100 to travel along the corresponding path. The path that the robot 100 is set to travel may be included in a movement plan used by the robot controller 430 to generate a command for the robot 100 .

The service operation module of the processor 320 may include a function for controlling the service provided by the robot 100 in the space. For example, the robot control system 120 or a service provider operating the space provides an IDE (Integrated Development Environment) for a service (eg, cloud service) provided by the robot control system 120 to a user or a manufacturer of the robot 100 . ) can be provided. At this time, the user or the manufacturer of the robot 100 may create software for controlling the service provided in the space by the robot 100 through the IDE and register it in the robot control system 120 . In this case, the service operation module may control the service provided by the robot 100 using software registered in association with the robot 100 . As a specific example, assuming that the robot 100 provides a service that delivers a user-requested item (eg, food or parcel) to the user's location, the robot control system 120 controls the indoor autonomous driving of the robot 100 . In addition to controlling the robot 100 to move to the user's location by controlling, commands related to the robot 100 to provide a series of services for delivering goods to the user and outputting user response voices when arriving at the target location may be transmitted to the robot 100 . The robot control unit 430 may obtain reference information referenced in generating a control command for the driving unit 108 (ie, a control command for processing a service-related task) through the service operation module.

A more detailed operation for controlling the robot 100 of the robot control system 120 will be described in more detail with reference to FIGS. 5 to 10 to be described later.

The description of the technical features described above with reference to FIGS. 1 to 3 can be applied to FIG. 4 as it is, and thus a redundant description will be omitted.

In the detailed description to be described later, operations performed by the components of the robot control system 120 and the robot 100 may be described as being performed by the robot control system 120 and the robot 100 for convenience of description.

In step 510 , the robot control system 120 may receive raw data from the robot 100 . The raw data may include raw sensing data acquired by the sensor unit 106 of the robot 100 . For example, the raw sensing data is data collected or acquired from each of the sensors and the camera included in the sensor unit 106, and may be data that has not been processed through pre-processing or the like.

Alternatively, the raw sensing data may be data obtained by performing minimal pre-processing or correction processing on the sensing data first collected from the sensor unit 106 on the robot 100 side. In other words, the raw sensing data may be the first preprocessing of the sensing data first collected from the sensor unit 106 . For example, this primary pre-processing is a minimal noise removal from the sensed data or cropping of an image (corresponding to the sensed data), or just processing the sensed data into a format for transmission through the wireless communication unit 102 . can

Accordingly, there is no need to separately provide a configuration for processing or pre-processing the raw sensing data on the robot 100 side, or only a configuration for minimal pre-processing may be provided.

In operation 520 , the robot control system 120 may generate a control command for controlling the driving unit 108 of the robot 100 based on the raw data received from the robot 100 .

In step 530 , the robot control system 120 may transmit the generated control command to the robot 100 .

The control command transmitted to the robot 100 generated by the robot control system 120 controls the driving unit 108 of the robot 100 (based on raw data including the sensed data from the robot 100) feedback control. it may be for That is, the control command may include a command for feedback control of at least one of an actuator and a motor included in the driving unit 108 .

The control command may be input to the driving unit 108 and include a low-level control command for controlling the driving unit 108 . In other words, the robot control system 120 does not generate a high-level command abstracted as a control command for controlling the drive unit 108, but directly (low-level) the motor and/or actuator of the drive unit. You can create low-level control commands that allow you to do this.

For example, the control command may include a command for controlling at least one of a speed, a position, and a torque of an actuator included in the driving unit 108 . Or/additionally, the control command may include a command for controlling at least one of a speed and a torque of a motor included in the driving unit 108 .

The reception of the raw data from the robot 100 in the above-described step 510 and the transmission of the control command to the robot 100 in the step 530 are performed with the wireless communication unit 102 of the robot 100 and the robot control. It may be performed according to communication between the wireless communication units 310 of the system 120 .

Accordingly, in the embodiment, in implementing a control loop for controlling the robot 100 , a controller that generates a low-level control command for controlling the robot 100 is used as a remote location of the robot 100 , the robot control system 120 . , and adding wireless communication to the control loop, it is possible to allow most of the operations to generate low-level control commands to be performed on the side of the robot control system 120 (rather than on the robot 100). That is, in the embodiment, a control loop consisting of the sensor unit 106 - the controller - the driving unit 108 may be implemented outside the robot 100 .

In the embodiment, localization of the robot 100, sensing data fusion, image processing and (object) recognition, and other sensing data processing (processing for extracting environmental information and determining the state (including position) of the robot) are performed by the robot control system (120) can be performed on the side. In addition, the planning process - establishment of a movement plan based on the map, information of the robot 100, work information (related to the service), etc. - and the control process - generation of a control command based on the processed sensing data and movement plan - are both robot control This may be performed on the system 120 side. In the embodiment, speed control, position control, torque control, etc. may be variously performed according to the type of the driving unit 108 through a low-level control command.

A more specific method of generating a low-level control command for controlling the robot 100 will be described in more detail with reference to FIG. 6 to be described later.

On the other hand, from the standpoint of the robot 100, when the control command is not received by the robot 100 within the first predetermined time period, or when the control command is not transmitted to the robot 100 for the second predetermined time period or longer, the robot ( 100) can be switched to a standby mode. The first time and the second time may be the same, may be set differently, and may be set by the robot control system 120 or the manager of the robot 100, respectively. Alternatively, when an object is detected within a predetermined distance from the robot 100 , the robot 100 may be switched to a standby mode. At this time, the robot 100 may ignore the control command from the robot control system 120 (ie, regardless of the control command) and switch to the standby mode.

According to the operation of the robot 100 as described above, the robot 100 and the minimum safety around the robot 100 may be secured. That is, in order to promote safety around the robot 100 and the robot 100, the robot 100 stops operation (or switches to a standby mode) when a control command is not received periodically (or over a certain period of time). )can do.

For example, if a control command is not received over the allowable delay time range for the robot 100 , the robot 100 may stop operating.

As another example, when the sensor (eg, distance sensor) of the sensor unit 106 included in the robot 100 detects an object within a predetermined distance (sensing of the object), the robot 100 controls The operation can be stopped preemptively regardless of the command. In this case, the control unit 104 of the robot 100 may include a minimum configuration (filter, etc.) for detecting an object existing within a predetermined distance.

In the embodiment as described above, in controlling the robot 100 through the robot control system 120 , the robot 100 and the minimum safety around the robot 100 can be guaranteed.

The description of the technical features described above with reference to FIGS. 1 to 4 can be applied to FIG. 5 as it is, and thus a redundant description will be omitted.

In step 610 , the robot control system 120 may pre-process the raw sensing data received from the robot 100 . The preprocessor 410 described above with reference to FIG. 4 may preprocess the raw sensing data.

For example, the raw sensing data may include an image acquired by a camera of the sensor unit 106 or a plurality of sensing data acquired by a plurality of sensors (eg, distance sensors) of the sensor unit. The pre-processing for the raw sensing data may include performing a refine process to enable recognition of an object in an image. For example, the pre-processing of the image may include improving the image so that an object such as an obstacle, a person, a robot, or the like can be recognized from the image. Alternatively, the pre-processing for the raw sensing data is the state of the robot 100 (eg, position, posture, operation state, etc.) or the state of the environment in which the robot 100 travels (eg, the position of the surrounding object, the surrounding object distribution situation). and the like) of fusion (ie, combining, processing, and synthesizing a plurality of sensing data) of a plurality of sensed data to determine.

The preprocessor 410 may preprocess the raw sensed data by filtering the received raw sensed data to remove noise or outliers.

In addition, the preprocessor 410 is based on the time-related metadata (eg, timestamp, round-trip time, etc.) of the sensing data included in the sensing data, (robot 100) Time-based sensing data can be corrected (using the past state information and sensed data or the history of state information). In addition, the pre-processing unit 410 may perform a process to compensate for the latency caused by wireless communication between the wireless communication unit 310 and the wireless communication unit 102 for the sensed data.

In operation 620 , the robot control system 120 may acquire at least one of the state information of the robot 100 and the environment information of the environment in which the robot 100 travels based on the raw sensing data. The robot state and environment recognition processing unit 420 described above with reference to FIG. 4 may perform step 620 . The raw sensed data used to obtain the state information and/or the environment information may represent the raw sensed data preprocessed in step 610 . The robot state and environment recognition processing unit 420 may acquire real-time (or near real-time) state information and/or environment information of the robot 100 based on the sensed data from the robot 100 .

The robot state and environment recognition processing unit 420 may play a role of creating local information around the robot 100 using sensor fusion (ie, fusion of sensing data collected from a plurality of sensors).

The state information of the robot 100 may include, for example, at least one of a position, an attitude, and an operation state (speed, torque, etc.) of the robot 100 (or the driving unit 108 ). The environment information may include, for example, at least one of a location of an object around the robot 100 and a distribution situation of a surrounding object in relation to an environment in which the robot 100 is driven.

As in step 622 , the robot state and environment recognition processing unit 420 generates a local map around the robot 100 based on the sensed data from the robot 100 and/or (from a camera or distance sensor). ) based on the sensing data, it is possible to recognize an object around the robot 100 . In generating the local map, the robot state and environment recognition processing unit 420 may further use a path or a motion plan on which the robot 100 travels as reference data.

The robot state and environment recognition processing unit 420 may acquire state information and/or environment information based on at least one of the generated local map and/or the recognition result of the object.

In step 630 , the robot control system 120 may generate a control command for the driving unit 108 based on the information obtained in step 620 . The robot controller 430 described above with reference to FIG. 4 may perform step 630 . The robot control unit 430 may generate a low-level control command for appropriately controlling the driving unit 108 of the robot 100 based on the state information and the environment information of the robot 100 .

Hereinafter, a case of generating a control command for controlling the movement of the robot 100 for autonomous driving of the robot 100 will be described in more detail.

The robot control unit 430 is configured to avoid obstacles identified in the environment while the robot 100 follows the movement plan, based on a motion plan associated with the robot 100, state information of the robot 100, and environment information. A control command for the driving unit 108 may be generated.

For example, the movement plan may include information on a path through which the robot 100 should travel. The robot control unit 430 may obtain such a movement plan from the travel planning unit 440 . The travel planner 440 is a global travel planner, and serves to create a global route plan for the robot 100 , and may create a route by utilizing the node map constructed for the space. That is, when generating the control command, the robot controller 430 may refer to a path (movement) plan including a path for the robot 100 to travel. The state information may include a position of the robot 100 . The position of the robot 100 may correspond to a current position (at the time the sensing data is collected). The environment information may include information about objects around the robot 100 . The information about the object may include position information of a person, a robot, or an obstacle in the vicinity of the robot 100 .

The robot control unit 430 controls the robot 100 as a control command so that the robot 100 moves along the path (represented by the movement plan) and avoids the surrounding obstacles (based on the environment information and status information). A command for feedback control of at least one of the motor and the actuator may be generated.

Through the control command, the robot control unit 430 can move the robot 100 from the current position to the next waypoint (waypoint) from the global route plan, and the obstacles around the robot 100 detected from the sensing data. You can control the robot to avoid it. In addition, the robot control unit 430 may establish an optimal speed plan to be taken by the robot 100 , and transmit a control command for this to the driving unit 108 of the robot 100 .

Meanwhile, the generated control command may be directly input to the driving unit 108 of the robot 100 as a low-level control command to control the driving unit 108 . Alternatively, the driving unit 108 may be input after correction processing (eg, minimal in the driving unit driver) on the robot 100 side. The correction processing may be processing into a format that allows the driving unit 108 to recognize the control command received from the robot control system 120 .

Below, with reference to FIG. 7 , based on wireless communication between the robot 100 and the robot control system 120 , the robot control system 120 controls the driving unit 108 of the robot 100 through a low-level control command. This shows how the robot 100 autonomously travels.

As shown, the sensor unit 106 of the robot 100 can collect sensing data for understanding the state of the robot 100 and recognizing the environment (①), and control the robot through wireless communication with the collected raw sensing data. It can be transmitted to the system 120 (②). The robot control system 120 may determine the state of the robot 100 based on the raw sensing data (after preprocessing the raw sensing data) and recognize the environment in which the robot 100 runs (③). The robot control system 120 may generate a control command for controlling the driving unit 108 with reference to the path plan of the robot 100 based on the state and environment information of the robot 100 (④). The robot control system 120 may generate a control command to allow the robot 100 to travel on a planned path and to avoid surrounding obstacles and transmit it to the robot 100 (⑤). The robot 100 may appropriately control the motor and/or actuator of the driving unit 108 according to the received control command (⑥). Accordingly, the robot 100 may autonomously travel while avoiding obstacles along a path to provide a service.

The description of the technical features described above with reference to FIGS. 1 to 5 can be applied to FIGS. 6 and 7 as it is, and thus a redundant description will be omitted.

Referring to FIG. 8 , a method of constructing a robot control model associated with the robot for the robot 100 and controlling the robot 100 using the model will be described.

In step 810 , the robot control system 120 (processor 320 ) controls the robot 100 based on information predefined for the robot 100 when the control of the robot 100 is requested. You can create a robot control model for

In step 820 , the robot control system 120 may estimate the state of the robot 100 based on the generated robot control model.

In step 830 , the robot control system 120 may generate a control command for controlling the driving unit 108 of the robot 100 based on the state of the robot 100 estimated by the robot control model. . The generated control command may include the aforementioned low-level control command.

That is, the robot control model may be configured to estimate the state of the robot 100 and generate a control command for controlling the driving unit of the robot 100 based on the estimated state of the robot 100 .

Here, the robot controlled by steps 810 to 830 is a robot different from the robot 100 described above with reference to FIGS. 1 to 7 , or in addition to the robot 100 described above with reference to FIGS. 1 to 7 . It may further be a controlled robot.

In other words, the robot control system 120 can not only control the robot 100 in the above-described feedback control method, but also build a robot control model as described above for the robot 100 and/or other robots to create a robot control model. It is also possible to control the robot 100 and / or other robots through.

The robot control system 120 may receive a request for control of a new robot from (control system or external service interface), and accordingly, obtain information about the corresponding robot from the robot control model information DB to control the robot You can construct (or dynamically create) a robot control model for

The robot control model information DB may include predefined information about the robot for which control is requested. For example, the predefined information about the robot (stored and managed in the robot control model information DB) used to create the robot control model includes specification information of the sensor unit and the driving unit included in the robot, and the physical shape of the robot. information and a dynamics model associated with the robot.

The robot control system 120 may use predefined information about the robot for which the control is requested as described above, estimate the state of the robot, and generate a robot control model for controlling the robot according to the estimated state. have.

When the robot is controlled using such a robot control model, it is possible to reduce errors due to latency due to communication between the robot and the robot control system 120 involved in the feedback control described above. That is, the robot may be controlled based on the state estimated by the built robot control model.

Meanwhile, in the robot control model, a virtual robot may be instantiated in order to estimate the real-time state of the robot, and the state of the robot to be controlled may be estimated as the instantiated virtual robot is executed. That is, the state of the robot to be controlled may be estimated according to the simulation of the virtual robot.

Meanwhile, the robot control system 120 may further generate a model for an environment in which the robot travels. Such an environment model may be built based on predefined information (eg, data collected in the past, latest environmental information, etc.) about the environment in which the robot runs.

The robot control system 120 generates and transmits an appropriate control command for controlling the robot in the environment based on the robot control model corresponding to the requested robot and the environment model corresponding to the environment in which the robot travels. can

Control of the robot using such a robot control model may be performed when the task performed by the robot is repetitive or predictable, such as simple. In other words, in a space with a high degree of predictability, such as an unmanned warehouse or factory, the robot control using the robot control model can be utilized to control the robot operating in a stable and deterministic situation.

The description of the technical features described above with reference to FIGS. 1 to 7 can be applied to FIG. 8 as it is, and thus a redundant description will be omitted.

The robot 100 described above with reference to FIGS. 1 to 7 may be controlled by the first robot brain of the robot control system 120 . In other words, the above-described steps 510 to 530 may be performed by the first robot brain.

In step 910, the robot control system 120 (processor 320) generates a robot brain for controlling the robot when the addition of the robot is requested (that is, when the control of the additional robot is requested). can For example, the robot control system 120 (processor 320) is a first robot brain logically separated from the first robot brain for controlling the other robot (for controlling the robot 100) when the control of the other robot is requested. 2 You can configure (create) a robot brain. In the control of another robot, reception of raw data from another robot, generation of a control command for controlling a driving unit of another robot, and transmission of a control command for controlling a driving unit of another robot may be performed through the second robot brain. can

As such, the robot control system 120 may dynamically generate a robot brain for controlling the robot for which control is requested.

In step 920, the robot control system 120 may control each robot individually, using the generated robot brain, and control a plurality of robots (multi-robot control such as coordination control and cooperative control). have.

The robot brain may be a control unit defined in the robot control system 120 to control an individual robot, for example, the pre-processing unit 410, the robot state and environment recognition processing unit 420 and the robot described above with reference to FIG. 4 . The controller 430 may be included.

In relation to FIG. 10, FIG. 10 shows a multi-robot control method for controlling a plurality of robots in controlling a robot through a robot brain, according to an example.

In FIG. 10 , robots 100 - 1 to 100 - 4 controlled by the robot control system 120 are shown. The robot control system 120 may generate a robot brain in 1:1 correspondence to each robot, and may individually control the robot through each of the robot brains 1020 . Each of the robot brains 1020 may correspond to a layer controlling one robot. One robot brain may be logically connected to a corresponding robot brain, and may correspond to a brain of a brainless robot.

The robot brains 1020 may be managed by the multi-robot management unit 1030 . For example, the multi-robot management unit 1030 is a robot for performing a requested service or a control request according to a request (eg, a service provision request or a robot control request) from the control system 1040 (eg, an external service interface). You can create a robot brain to control the robot. Alternatively, the multi-robot management unit 1030 may remove a robot brain corresponding to a robot whose control is unnecessary among the robot brains 1020 .

As such, in the embodiment, the robot brains 1020 for controlling the plurality of robots 100-1 to 100-4 may be dynamically created and removed according to the need for control of each robot.

The multi-robot management unit 1030 may transmit an abstract command (global command) according to a request from the control system 1040 to the robot brain corresponding to the requested robot among the robot brains 1020, and the robot brain Based on the sensing data received from the corresponding robot by analyzing the command, it is possible to generate a (low-order) control command for controlling the corresponding robot.

One robot brain may be 1:1 connected to a robot corresponding to the gateway 1050 as shown. In this case, the gateway 1050 may include the above-described communication unit 310 or may be a part of the communication unit 310 .

Alternatively, each of the robot brains 1020 may have a communication unit 310 or a communication function, and may be directly (or directly) connected to a corresponding robot. In this case, the gateway 1050 may be involved only in the initial connection between the robot brain and the corresponding robot. That is, the gateway 1050 may manage the initial connection between the robot brain and the corresponding robot.

For example, the first robot brain (which is one of the robot brains 1020) for controlling the above-described robot 100 is first connected through the gateway 1050 of the robot 100 and the robot control system 120, After the initial connection, it is possible to communicate with the robot 100 through the communication unit (communication function) of the first robot brain. On the other hand, the second robot brain (which is one of the robot brains 1020) for controlling the other robot described above is first connected to another robot through the gateway 1050, and after the initial connection, the communication unit of the second robot brain (Communication function) can communicate with other robots.

In other words, the robot control system 120 may be configured to have a communication unit corresponding to each of the robot brains 1020 , and the robot brains 1020 may be configured to share one communication unit.

Meanwhile, in FIG. 10 , the robot control model 1010 described above with reference to FIG. 8 is further illustrated. As an example, the robot control model 1010 may be built for the robot 100-1, generate a control command for controlling the driving unit of the robot 100-1, and transmit it to the robot 100-1. can The control command may be transmitted to the robot 100 - 1 through the gateway 1050 . The robot control model 1010 may be built by a robot control model information DB (not shown). In response to the robots 100-1 to 100-4, a plurality of robot control models 1010 may also be built, and a plurality of robot control models 1010 may be multiplied (in a similar manner to the robot brains 1020). It may be managed by the robot manager 1030 .

According to a global command by the multi-robot management unit 1030, the robot brains 1020 control the robots 100-1 to 100-4, so that at least two or more of the robots 100-1 to 100-4 may be interlocked with each other (cooperative control). For example, two or more robots may be controlled to operate cooperatively with each other, and may be controlled to perform tasks related to each other in providing a service.

As such, in the embodiment, the plurality of brainless robots 100 - 1 to 100 - 4 may be controlled individually or in conjunction with each other by the robot control system 120 .

The description of the technical features described above with reference to FIGS. 1 to 8 can be applied to FIGS. 9 and 10 as it is, and thus overlapping descriptions will be omitted.

As described above, in the embodiment, both the analysis and processing of sensing data for controlling the robot 100 and the generation of control commands may be performed in the robot control system 120 serving as a server. Accordingly, the control unit 104 of the robot 100 may not include a complex configuration such as a GPU, and such a configuration as the GPU may be mounted only on the processor 320 of the robot control system 120 .

On the other hand, according to the embodiment, the robot control system 120 side so that the optimal speed plan for the robot 100 is established, the robot 100 and the robot control system 120 so that the low-level control according to the plan is performed by the robot ) could be implemented.

The system or apparatus described above may be implemented as a hardware component, a software component, or a combination of a hardware component and a software component. For example, devices and components described in the embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions, may be implemented using one or more general purpose or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For convenience of understanding, although one processing device is sometimes described as being used, one of ordinary skill in the art will recognize that the processing device includes a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that may include For example, the processing device may include a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as parallel processors.

The software may comprise a computer program, code, instructions, or a combination of one or more thereof, which configures a processing device to operate as desired or is independently or collectively processed You can command the device. The software and/or data may be any kind of machine, component, physical device, virtual equipment, computer storage medium or device, to be interpreted by or to provide instructions or data to the processing device. may be permanently or temporarily embody in The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored in one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and available to those skilled in the art of computer software. Examples of the computer-readable recording medium include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floppy disks. - includes magneto-optical media, and hardware devices specially configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine language codes such as those generated by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

As described above, although the embodiments have been described with reference to the limited embodiments and drawings, various modifications and variations are possible by those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or the described components of the system, structure, apparatus, circuit, etc. are combined or combined in a different form than the described method, or other components Or substituted or substituted by equivalents may achieve an appropriate result.

Therefore, other implementations, other embodiments, and equivalents to the claims are also within the scope of the following claims.

Claims

A method of controlling a robot, performed by a robot control system, comprising:

receiving raw data from the robot;

generating a control command for controlling a driving unit of the robot based on the received raw data; and

transmitting the generated control command to the robot

including,

The control command is input to the driving unit, and includes a low-level control command for controlling the driving unit.
According to claim 1,

The driving unit includes at least one of an actuator and a motor,

The control command includes a command for controlling at least one of speed, position, and torque of the actuator;

A method of controlling a robot, comprising instructions for controlling at least one of a speed and a torque of the motor.
3. The method of claim 2,

The method of controlling a robot, wherein the control command includes a command for feedback control of the actuator or the motor.
According to claim 1,

The raw data includes raw sensing data obtained by a sensor unit of the robot, the method of controlling a robot.
5. The method of claim 4,

The step of generating the control command comprises:

pre-processing the raw sensing data

including,

The raw sensing data includes an image obtained by the camera of the sensor unit or a plurality of sensing data obtained by a plurality of sensors of the sensor unit,

The pretreatment is

Including performing refinement processing to enable recognition of an object in the image, or

A method of controlling a robot, comprising fusion of the plurality of sensed data to determine the state of the robot or the state of the environment in which the robot travels.
5. The method of claim 4,

The step of generating the control command comprises:

acquiring at least one information of the state information of the robot and environmental information of an environment in which the robot is driven, based on the raw sensing data; and

generating the control command for controlling the driving unit based on the obtained information

A method of controlling a robot, comprising:
7. The method of claim 6,

The obtaining step is

i) generating a local map around the robot based on the raw sensing data; and ii) recognizing an object around the robot based on the raw sensing data.

comprising at least one of

Based on at least one of the local map and the recognition result of the object, the method of controlling a robot to obtain the at least one piece of information.
7. The method of claim 6,

The control command is a control command for controlling the movement of the robot for autonomous driving of the robot,

The step of generating the control command comprises:

Based on a motion plan associated with the robot, the status information and the environment information, the robot generates a control command for the drive unit to cause the robot to follow the movement plan and avoid obstacles in the environment. how to control it.
9. The method of claim 8,

The movement plan includes information about the path the robot must travel,

The state information includes the position of the robot,

The environment information includes information about objects around the robot,

The control command includes a command for feedback control of at least one of a motor and an actuator included in the driving unit so that the robot moves along the path and avoids the obstacle.
5. The method of claim 4,

The raw sensing data is that the sensing data first collected from the sensor unit is primarily pre-processed on the robot side,

The method of controlling the robot, wherein the control command is input to the driving unit after correction processing on the robot side.
According to claim 1,

According to the communication between the wireless communication unit of the robot and the wireless communication unit of the robot control system,

receiving the raw data from the robot;

The method for controlling a robot, wherein the control command is transmitted to the robot.
According to claim 1,

The robot is a brainless robot that controls the driving unit by executing the low-level control command received from the robot control system without performing a process of interpreting the received control command. .
According to claim 1,

generating a robot control model for controlling the other robot when the control of the other robot is requested, based on the information predefined for the other robot;

estimating the state of the other robot based on the robot control model; and

generating a control command for controlling the driving unit of the other robot based on the estimated state of the other robot

Further comprising, a method of controlling a robot.
14. The method of claim 13,

The predefined information about the other robot used to generate the robot control model is,

A method for controlling a robot, including specification information of a sensor unit and a driving unit included in the other robot, physical shape information of the other robot, and a dynamics model associated with the other robot.
According to claim 1,

The robot is controlled by the first robot brain of the robot control system,

The receiving step, the generating step, and the transmitting step are performed by the first robot brain,

When control of another robot is requested, configuring a second robot brain logically separated from the first robot brain for controlling the other robot

further comprising,

Receiving the raw data from the other robot, generating a control command for controlling the driving unit of the other robot, and transmitting the control command for controlling the driving unit of the other robot are performed through the second robot brain. How to control.
16. The method of claim 15,

The first robot brain is first connected to the robot through the gateway of the robot control system, and after the initial connection, communicates with the robot through the communication unit of the first robot brain,

The second robot brain is first connected to the other robot through the gateway, after the initial connection, communicating with the other robot through the communication unit of the second robot brain, a method of controlling a robot.
According to claim 1,

If the control command is not received by the robot within a first predetermined time period, or if the control command is not transmitted to the robot for a second predetermined time period or longer,

The method of controlling a robot, wherein the robot is switched to a standby mode.
According to claim 1,

When an object is detected within a predetermined distance from the robot, the robot is switched to a standby mode regardless of the control command.
In the robot control system for controlling the robot,

wireless communication unit; and

at least one processor implemented to execute computer-readable instructions

including,

the at least one processor,

In accordance with communication between the wireless communication unit and the wireless communication unit of the robot, receiving raw data from the robot,

Based on the received raw data, generating a control command for controlling the driving unit of the robot,

Transmitting the generated control command to the robot according to communication between the wireless communication unit and the wireless communication unit of the robot,

The control command is input to the driving unit, and includes a low-level control command for controlling the driving unit.
20. The method of claim 19,

The raw data includes raw sensing data obtained by the sensor unit of the robot,

the at least one processor,

a preprocessor for preprocessing the raw sensing data;

a robot state and environment recognition processing unit configured to acquire at least one of state information of the robot and environment information of an environment in which the robot travels based on the pre-processed raw sensing data; and

A robot control unit that generates the control command for controlling the driving unit based on the obtained information

Including, robot control system.
21. The method of claim 20,

The control command is a control command for controlling the movement of the robot for autonomous driving of the robot,

the at least one processor,

A travel planning unit that manages a movement plan associated with the robot

including,

The robot control unit, based on the status information and the environment information, generates a control command for the driving unit for the robot to follow the movement plan and avoid obstacles in the environment, the robot control system.
20. The method of claim 19,

the at least one processor,

and generate a robot control model for controlling the other robot based on information predefined for the other robot when the control of the other robot is requested,

The robot control model estimates the state of the other robot, and based on the estimated state of the other robot, generates a control command for controlling the driving unit of the other robot.
20. The method of claim 19,

the at least one processor,

A first robot brain comprising;

the first robot brain is configured to receive the raw data, generate the control command, and send the control command to the robot;

the at least one processor,

configured to generate a second robot brain logically separated from the first robot brain for controlling the other robot when the control of the other robot is requested,

Receiving raw data from the other robot, generating a control command for controlling the driving unit of the other robot, and transmitting a control command for controlling the driving unit of the other robot are performed through the second robot brain, robot control system.
A robot controlled by a robot control system, comprising:

wireless communication unit;

a sensor unit including at least one sensor;

a driving unit including at least one of a motor and an actuator; and

A control unit including a sensor driver for the sensor unit and a driving unit driver for the driving unit

including,

The control unit is

Transmitting raw data including raw sensing data obtained by the sensor unit to the robot control system through the wireless communication unit,

Receiving a control command for controlling the driving unit generated based on the raw data on the robot control system side through the wireless communication unit,

Control the driving unit by executing the received control command,

The received control command is input to the driving unit, and includes a low-level control command for controlling the driving unit.
in the building,

at least one robot moving through space within the building

is placed,

The robot is controlled by a robot control system,

The robot control system,

wireless communication unit; and

at least one processor implemented to execute computer-readable instructions

including,

the at least one processor,

In accordance with communication between the wireless communication unit and the wireless communication unit of the robot, receiving raw data from the robot,

Based on the received raw data, generating a control command for controlling the driving unit of the robot,

Transmitting the generated control command to the robot according to communication between the wireless communication unit and the wireless communication unit of the robot,

The control command is input to the driving unit and includes a low-level control command for controlling the driving unit.
in the building,

at least one robot moving through space within the building and controlled by a robot control system

is placed,

The robot is

wireless communication unit;

a sensor unit including at least one sensor;

a driving unit including at least one of a motor and an actuator; and

A control unit including a sensor driver for the sensor unit and a driving unit driver for the driving unit

including,

The control unit is

Transmitting raw data including raw sensing data obtained by the sensor unit to the robot control system through the wireless communication unit,

Receiving a control command for controlling the driving unit generated based on the raw data on the robot control system side through the wireless communication unit,

Control the driving unit by executing the received control command,

The received control command is input to the driving unit and includes a low-level control command for controlling the driving unit.