WO2023027252A1

WO2023027252A1 - Device and method for controlling mobile robot capable of moving between buildings by using traveling and working plans based on semantic map

Info

Publication number: WO2023027252A1
Application number: PCT/KR2021/017202
Authority: WO
Inventors: 엄태영; 배기덕; 이종득; 이정우; 최영호
Original assignee: 한국로봇융합연구원
Priority date: 2021-08-26
Filing date: 2021-11-22
Publication date: 2023-03-02
Also published as: KR20230030954A

Abstract

A control device is provided. The control device may comprise: a planning unit for generating robot traveling and working plans; a positioning unit for identifying the current position of the robot; and a control unit for controlling the robot positioned inside a first building so as to move into a second building by using the traveling plan, the working plan, and the current position.

Description

Apparatus and method for controlling a mobile robot capable of moving between buildings using semantic map-based driving and work planning

The present invention relates to an apparatus and method for establishing a travel plan and work plan of a robot and controlling the robot.

Common methods of controlling robots performing set-up tasks are limited to movement within the same building or fail to replan. Therefore, an environment in which the robot performs setting work while moving around the building is not prepared, and due to this phenomenon, the usability of the robot is reduced. This may soon act as a factor limiting the popularization of autonomous robots.

Therefore, there is a need for a framework technology capable of autonomously moving between floors and moving between outdoor buildings through communication between a planner and an elevator monitoring team to overcome this problem.

The present invention is to provide a control device and control method capable of autonomous movement between floors and movement between outdoor buildings.

The control device of the present invention includes a planning unit generating a travel plan and a work plan of the robot; a location unit that determines the current location of the robot; and a control unit for controlling the robot located inside the first building to move to the inside of the second building using the travel plan, the work plan, and the current location.

The control method of the present invention includes acquiring model information of a robot, indoor/outdoor map information of a first building, indoor/outdoor map information of a second building, and job information; determining the size and locomotion of the robot using the model information; Establishing a work plan for the robot using the work information; establishing a traveling plan for the robot using the work plan, the size of the robot, the loco motion, indoor/outdoor map information of the first building, and indoor/outdoor map information of the second building; generating a movement sequence of the robot according to the traveling plan; determining whether the robot exists inside the first building; analyzing whether interfloor movement is necessary if it is determined that the robot exists indoors of the first building; If it is determined that the robot needs to move between floors, moving the robot in front of an elevator door capable of boarding on the first current floor; generating a control protocol of an elevator monitoring panel of the first building when it is confirmed that the robot has arrived in front of the elevator door; controlling the elevator to board the elevator when the elevator monitoring panel moves the elevator to the floor where the robot is waiting and when it is confirmed that the elevator door is open; controlling an elevator monitoring panel of the first building so that the elevator moves to a first destination floor when the robot boarding of the elevator is completed; controlling the robot to get off the elevator when the elevator arrives at the first destination floor by the elevator monitoring panel; determining whether outdoor movement is necessary when the robot is completely removed from the elevator; controlling the robot to enter a second building after leaving the first building when it is determined that outdoor movement of the robot is necessary; When the robot enters the second building, it determines whether or not to use the elevator in the second building, and depending on the determination result, moves the robot to the work place on the second current floor or uses the elevator in the second building. It may include; moving the robot to the work place of the second destination floor after use.

According to the present invention, a robot controls a robot that uses a robot and an elevator monitoring panel to establish a plan related to the operation and driving of the robot, and performs various tasks by moving between a plurality of buildings through movement between buildings including indoor floor-to-floor movement and outdoor movement. is possible

1 is a schematic diagram showing a control device of the present invention.

Figure 2 is a schematic diagram showing the loco motion of the robot.

3 is a schematic diagram showing a movement sequence.

4 and 5 are flowcharts showing the control method of the present invention.

6 is a diagram illustrating a computing device according to an embodiment of the present invention.

Hereinafter, with reference to the accompanying drawings, embodiments of the present invention will be described in detail so that those skilled in the art can easily carry out the present invention. However, the present invention may be embodied in many different forms and is not limited to the embodiments described herein. And in order to clearly explain the present invention in the drawings, parts irrelevant to the description are omitted, and similar reference numerals are attached to similar parts throughout the specification.

In this specification, redundant descriptions of the same components are omitted.

In addition, in this specification, when a component is referred to as being 'connected' or 'connected' to another component, it may be directly connected or connected to the other component, but another component in the middle It should be understood that may exist. On the other hand, in this specification, when a component is referred to as 'directly connected' or 'directly connected' to another component, it should be understood that no other component exists in the middle.

In addition, terms used in this specification are only used to describe specific embodiments and are not intended to limit the present invention.

Also, in this specification, a singular expression may include a plurality of expressions unless the context clearly indicates otherwise.

In addition, in this specification, terms such as 'include' or 'having' are only intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more It should be understood that the presence or addition of other features, numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Also in this specification, the term 'and/or' includes a combination of a plurality of listed items or any item among a plurality of listed items. In this specification, 'A or B' may include 'A', 'B', or 'both A and B'.

Also, in this specification, detailed descriptions of well-known functions and configurations that may obscure the subject matter of the present invention will be omitted.

Recently, with the 4th revolution, the demand for the use of robots is increasing. At this time, the request for use of the robot may include an activity in which the robot enters a human living area and an activity in which the robot performs various tasks while autonomously traveling through a plurality of buildings. To this end, the control device communicates with the robot and the elevator monitoring panel, establishes a plan suitable for the robot, and proposes a method capable of performing various tasks through movement between buildings, including indoor movement between floors and outdoor movement.

1 is a schematic diagram showing a control device of the present invention.

The control device shown in FIG. 1 may include a planning unit 110, a location unit 130, and a control unit 150.

The planning unit 110 may generate a travel plan and a work plan of the robot 10 .

The planning unit 110 may include an acquisition unit 111 , a planning unit 113 , and a generation unit 115 .

The acquisition unit 111 may acquire model information of the robot 10, indoor/outdoor map information of the first building, indoor/outdoor map information of the second building, and job information.

The acquisition unit 111 may be provided with a first communication module that communicates with the robot 10, a second communication module that communicates with a map server or a management server of each building, and a third communication module that communicates with a job server. Model information of the robot 10 may be obtained through the first communication module. Indoor/outdoor map information of the first building and indoor/outdoor map information of the second building may be obtained through the second communication module. Job information to be performed by the robot may be obtained through the third communication module. Types of tasks included in the task information may include delivery, cleaning, guidance, crime prevention, and the like.

The planning unit 113 may determine the size and locomotion of the robot 10 using model information. Loco motion may refer to a method in which the robot 10 moves the body.

Figure 2 is a schematic diagram showing the loco motion of the robot.

The first robot shown in (a) of FIG. 2 may be manufactured to be able to rotate (rotate in place) about an imaginary axis p parallel to the direction of gravity in place.

The second robot shown in (b) of FIG. 2 cannot rotate in place and can be manufactured to rotate via a set radius of rotation.

Rotation in place as shown in (a) of FIG. 2 may correspond to one of the loco motions of the first robot. Rotation via a set radius as shown in (b) of FIG. 2 may correspond to one of the loco motions of the second robot.

Returning to FIG. 1 again, the planning unit 113 may establish a work plan using the work information.

The task plan may include setting the type of task given to the robot, the task location, the start point of the task, the end point of the task, the start time of the task, and the end time of the task.

The planning unit 113 may establish a driving plan using the work plan, the size of the robot, loco motion, indoor/outdoor map information of the first building, and indoor/outdoor map information of the second building. The driving plan may include setting the order and passing time of passing through various terrain features, rooms, corridors, etc. inside and outside the building.

The generation unit 115 may generate a movement sequence of the robot according to a driving plan. The movement sequence may include a sequence via location coordinates or identification numbers indicating a feature, room, corridor, or the like.

3 is a schematic diagram showing a movement sequence.

In FIG. 3, in the driving sequence (movement sequence) determined as corridor 36-32-35-door 317-elevator 2, the robot moves while sending a confirmation message including the current location to the control device, and moves in front of the elevator to move between floors. The arrived message can be transmitted to the control device.

Returning to FIG. 1 again, the location unit 130 can determine the current location of the robot. For example, the location unit 130 may determine the current location of the robot using real-time location information obtained from the robot.

The control unit 150 may control the robot located inside the first building to move to the inside of the second building using the travel plan, the work plan, and the current location of the robot.

If the robot exists inside the first building, the control unit 150 may analyze whether movement between floors is necessary. In the case of a robot that moves on wheels, etc., it may be difficult to move between floors using stairs. Accordingly, the robot may move between floors using the elevator.

When the robot needs to move between floors, the control unit 150 may move the robot from the first current floor corresponding to the current floor of the first building to the front of an elevator door capable of boarding.

A building equipped with an elevator may be equipped with an elevator monitoring panel that controls the elevation of the elevator. When it is confirmed that the robot has arrived in front of the elevator door, the control unit 150 may generate a control protocol for the elevator monitoring panel of the first building. The corresponding control protocol may include instructions for the elevator monitoring panel to move the elevator to the floor where the robot is waiting.

The control unit 150 may control the robot to board the elevator when the elevator monitoring panel moves the elevator to the floor where the robot is waiting and confirms that the elevator door is open according to the control protocol.

When boarding of the elevator is completed, the control unit 150 may control the elevator monitoring panel of the first building so that the elevator moves to the first destination floor corresponding to the destination floor of the first building.

The control unit 150 may control the robot to get off the elevator when the elevator arrives at the first destination floor by the elevator monitoring panel.

It may be difficult for a robot riding an elevator to independently determine whether or not the first destination floor has arrived. This is because communication with the outside is cut off in the elevator, and it is difficult to use a GPS or the like that can determine the current location.

When the elevator is used only by a single robot, when the elevator stops after moving and the elevator door is opened, the corresponding floor may be estimated as the destination floor. However, in the case of an elevator used with other robots and passengers, it is virtually difficult to estimate the destination floor based on whether or not the door is open. This is because the elevator may stop and the door may be opened by a robot or a passenger located on a floor different from the target floor.

On the other hand, the control unit 150 communicating with the elevator monitoring panel can accurately determine the current location of the elevator through the elevator monitoring panel. When the control unit 150 obtains the fact that the elevator has reached the first destination floor from the elevator monitoring panel, the control unit 150 may transmit a command to get off the robot to the target. The corresponding get-off command is transmitted to the robot when the door is opened at the first destination floor, and the robot receiving the get-off command can get off the elevator. In order to specify a specific robot to get off at the first destination floor among a plurality of robots, identification information of the specific robot may be included in the getting off command. Due to the identification information, the corresponding get-off command can only be used to control a specific robot.

When the robot gets off the elevator, the control unit 150 may determine that outdoor movement is not necessary.

The control unit 150 may control the robot to enter the second building after leaving the first building when outdoor movement of the robot is required.

When the robot enters the inside of the second building, the control unit 150 determines whether the elevator in the second building is used, and according to the determination result, the second current floor corresponding to the current floor of the second building or the elevator of the second building After using , the robot can be moved from the second destination floor (the destination floor of the second building) to the work place.

4 and 5 are flowcharts showing the control method of the present invention.

The control method may be performed by the control device shown in FIG. 1 .

A mission request including tasks such as delivery, cleaning, and crime prevention may be obtained (S510). The control device that has obtained the mission request can control the robot and the elevator monitoring panel to complete the mission.

A step of acquiring model information of the robot, indoor/outdoor map information of the first building, indoor/outdoor map information of the second building, and job information may be performed (S520).

A step of determining the size and locomotion of the robot using the model information may be performed (S520).

A step of establishing a work plan for the robot using the work information may be performed (S530).

A step of establishing a driving plan for the robot may be performed using the work plan, the size of the robot, loco motion, indoor/outdoor map information of the first building, and indoor/outdoor map information of the second building (S530).

A step of generating a movement sequence of the robot according to the travel plan may be performed (S540). Steps S520 to S540 may be performed by the planning unit 110.

A step of acquiring the position of the robot in real time may be performed (S550). The control device may receive a current location of the robot from a robot that performs SLAM (Simultaneous Localization and Mapping). This step may be performed by the location unit 130.

Subsequent processes may be performed by the control unit 150 as a control step (S501).

A step of determining whether the robot exists inside the first building may be performed (S560). If it is determined that the robot does not exist indoors of the first building, it may mean that the robot is located outdoors. In this case, the control device may control the robot to move outdoors to the second building (S680).

If it is determined that the robot exists indoors of the first building, a step of analyzing whether interfloor movement is necessary may be performed (S570). If it is analyzed that the movement of the robot between floors is unnecessary, it may mean that the current floor is the target floor. Accordingly, the control device may control the robot to move to the destination for work (S700).

If it is determined that the robot needs to move between floors, a step of moving the robot in front of an elevator door capable of boarding on the first current floor may be performed (S580).

When it is confirmed that the robot has arrived in front of the elevator door (S590), a step of generating a control protocol for the elevator monitoring panel of the first building may be performed (S600). If it is determined that the robot has not arrived at the elevator door (S590), the control method may return to establishing a travel plan and work plan for the robot (S530).

When the elevator monitoring panel moves the elevator to the floor where the robot is waiting according to the control protocol and confirms that the elevator door is open (S610), a step of controlling the robot to board the elevator may be performed (S620). If boarding of the robot in the elevator is not completed, the control device may increase the waiting time of the elevator for the first current floor until boarding is completed (S630).

When the boarding of the elevator is completed (S620), a step of controlling the elevator monitoring panel of the first building so that the elevator moves to the first destination floor may be performed (S640).

When the elevator arrives at the first destination floor by the elevator monitoring panel (S640), a step of controlling the robot to get off the elevator may be performed (S650).

When getting off the elevator is completed (S650), a step of determining whether outdoor movement is necessary may be performed (S670). If getting off of the robot is incomplete (S650), the control device may increase the waiting time of the elevator for the first destination floor until the robot gets off the elevator (S660).

If it is determined that outdoor movement of the robot is necessary (S670), a step of controlling the robot to enter a second building after leaving the first building may be performed (S680). If it is determined that the outdoor movement of the robot is unnecessary (S670), it means that the robot has arrived at the final floor to perform the work. Accordingly, the control device may move the robot to a work place located on the corresponding floor (S700).

When the robot enters the inside of the second building (S690), it is determined whether or not to use the elevator in the second building, and according to the determination result, the robot is moved to the work place on the second current floor, or the elevator in the second building is used. After that, a step of moving the robot to a work place of the second destination floor may be performed.

When the robot arrives at the final work place (S 710), the robot can be controlled to perform a setting task (S 720). When the task is completed (S 730), the first mission request may be completed or terminated (S 740).

According to the present invention, the mobile robot can communicate with the control device, download a travel plan including a driving sequence and a work plan including a work sequence, and drive to and from a plurality of buildings. The control device can establish a plan suitable for robot movement and work by using robot model information (size, locomotion, mounted sensors), indoor semantic map of buildings, and outdoor semantic maps for movement between buildings. The control device enables the robot to get on and off the elevator by communicating with the elevator monitoring panel when the robot moves between floors. The elevator monitoring panel may receive and execute the elevator operation protocol from the control device.

In order for a robot to move from one building to another, an elevator suitable for the size and loco motion of the robot must be selected. In addition, when the robot gets on and off the elevator, it needs to be linked with the monitoring team. In addition, the control device can perform driving and work through replanning for failure of the robot's work (mission) due to various obstacles. First, in planning the robot's driving and work, the control device considers the robot's size and locomotion according to the semantic information of the size of the door connecting the elevator or space (room, hallway), excluding doors that cannot pass through. Planning the driving and working sequence considering the open time information. For example, in FIG. 3 , the robot can move by selecting elevator No. 2, which operates to the first floor, where it can go outdoors even if there is another elevator that is close in distance due to the loco motion that cannot rotate in place.

In a driving sequence defined as corridor 36 - 32 - 35 - door 317 - elevator 2 in FIG. 3 , the robot may move to the control device and arrive in front of the elevator to move between floors. The control device may transmit corresponding elevator identification information (ID), robot identification information (ID), departure floor and arrival floor information to the elevator monitoring panel through a protocol. The control device may send an elevator entry command to the robot after receiving a departure floor arrival message from the elevator monitoring panel. Thereafter, the robot may recognize the place on the arrival floor, check the destination floor, complete getting off, and report this to the control device. The control device may transmit a protocol for getting off the robot to the elevator monitoring panel. If the robot does not complete getting off until the waiting time for the elevator passes, the control device transmits an extension protocol to the elevator monitoring panel. Upon receiving the extension protocol, the elevator monitoring panel may control the corresponding elevator so that the corresponding elevator waits at the arrival floor with the door open for a set time.

The robot can go outdoors to move between buildings. Although the driving route targets the minimum distance, the control device may set the outdoor driving sequence by prioritizing a road having a low roughness by giving a step of the roughness of the road, which is semantic information, from 1 to 5.

Finally, when the robot arrives at the final floor for work, the control device can generate a movement sequence from the elevator to the work room (work place) and control the robot to travel according to the movement sequence. The robot arriving at the work room can perform the work obtained from the control device (ex: delivery, guidance, security, etc.).

Communication between the robot-controller-elevator monitoring panel can use wireless or wired Wi-Fi communication. In preparation for a communication failure between the robot and the control device, a LoRa (Long Range) communication module may be additionally mounted on the robot and the control device.

The control device may generate a modified sequence and retransmit it to the robot when receiving a failure message of the robot in various driving stages and elevator use stages.

According to the present invention, the control device may create a driving and working sequence using a model of the mobile robot, a current location, loco motion, a semantic map inside a building, and a semantic map between buildings outdoors, and transmit the sequence to the robot through communication.

The robot transmits its current location (indoor/outdoor possible) to the control device through precise location recognition using a place recognizer and SLAM, and can move to a location to perform a task based on the current location.

The control device may select a driving path of the robot based on indoor and outdoor semantic map information.

When inter-floor movement is included in the movement sequence, the control device may select an elevator to be boarded through the elevator monitoring panel and send a completion message to the server after the robot arrives at the floor to be boarded.

The control device may transmit elevator identification information (ID), robot identification information (ID), and protocols of departure and arrival floors to the elevator monitoring panel for inter-floor movement, and may control the elevator monitoring panel.

The control device may confirm the notification of getting on and off the robot through communication, and send a protocol to increase the waiting time to the elevator monitoring panel when getting on and off the robot fails. The elevator monitoring team that received the corresponding protocol can wait for the elevator at the current location until the robot gets on and off.

After moving to the floor of a building where work is required, the robot can move to a work place on the same floor and start working.

After completing the mission, the robot can report to the control unit and wait for the next task.

The control device may perform re-planning and transmit the message to the robot upon receipt of the robot driving and operation failure message.

6 is a diagram illustrating a computing device according to an embodiment of the present invention. The computing device TN100 of FIG. 6 may be a device (eg, a control device, etc.) described in this specification.

In the embodiment of FIG. 6 , the computing device TN100 may include at least one processor TN110, a transceiver TN120, and a memory TN130. In addition, the computing device TN100 may further include a storage device TN140, an input interface device TN150, and an output interface device TN160. Elements included in the computing device TN100 may communicate with each other by being connected by a bus TN170.

The processor TN110 may execute program commands stored in at least one of the memory TN130 and the storage device TN140. The processor TN110 may mean a central processing unit (CPU), a graphics processing unit (GPU), or a dedicated processor on which methods according to embodiments of the present invention are performed. Processor TN110 may be configured to implement procedures, functions, methods, and the like described in relation to embodiments of the present invention. The processor TN110 may control each component of the computing device TN100.

Each of the memory TN130 and the storage device TN140 may store various information related to the operation of the processor TN110. Each of the memory TN130 and the storage device TN140 may include at least one of a volatile storage medium and a non-volatile storage medium. For example, the memory TN130 may include at least one of read only memory (ROM) and random access memory (RAM).

The transmitting/receiving device TN120 may transmit or receive a wired signal or a wireless signal. The transmitting/receiving device TN120 may perform communication by being connected to a network.

Meanwhile, the embodiments of the present invention are not implemented only through the devices and/or methods described so far, and may be implemented through a program that realizes functions corresponding to the configuration of the embodiments of the present invention or a recording medium in which the program is recorded. And, such implementation can be easily implemented by those skilled in the art from the description of the above-described embodiment.

Although the embodiments of the present invention have been described in detail above, the scope of the present invention is not limited thereto, and various modifications and improvements of those skilled in the art using the basic concept of the present invention defined in the following claims are also provided. fall within the scope of the invention.

Claims

a planning unit that generates a travel plan and a work plan of the robot;

a location unit that determines the current location of the robot;

A control unit for controlling the robot located inside the first building to move to the inside of the second building using the travel plan, the work plan, and the current location,

A control device comprising a.
According to claim 1,

An acquisition unit, a planning unit, and a generation unit are provided,

The acquisition unit acquires model information of the robot, indoor/outdoor map information of the first building, indoor/outdoor map information of the second building, and job information;

The planning unit grasps the size and loco motion of the robot using the model information,

The planning unit establishes the work plan using the work information,

The planning unit establishes the driving plan using the work plan, the size of the robot, the loco motion, indoor/outdoor map information of the first building, and indoor/outdoor map information of the second building;

Wherein the generating unit generates a movement sequence of the robot according to the driving plan.
According to claim 1,

The control unit analyzes whether interfloor movement is necessary when the robot is present indoors of the first building,

When the robot needs to move between floors, the control unit moves the robot in front of an elevator door capable of boarding on the first current floor;

When it is confirmed that the robot has arrived in front of the elevator door, the control unit generates a control protocol of the elevator monitoring panel of the first building,

The control unit controls the robot to board the elevator when the elevator monitoring panel moves the elevator to the floor where the robot is waiting and confirms that the elevator door is open according to the control protocol,

The control unit controls an elevator monitoring panel of the first building so that the elevator moves to the first destination floor when the robot is boarded in the elevator,

The control unit controls the robot to get off the elevator when the elevator arrives at the first destination floor by the elevator monitoring panel;

When the robot gets off the elevator, the control unit determines that outdoor movement is not necessary;

The control unit controls the robot to enter a second building after leaving the first building when outdoor movement of the robot is required;

When the robot enters the second building, the control unit determines whether or not to use the elevator of the second building, and uses the elevator of the second current floor or the second building according to the determination result, and then gets off the second building. A control device for moving the robot from the destination floor to the work place.
In the control method performed by the control device,

acquiring robot model information, indoor/outdoor map information of a first building, indoor/outdoor map information of a second building, and job information;

determining the size and locomotion of the robot using the model information;

Establishing a work plan for the robot using the work information;

establishing a traveling plan for the robot using the work plan, the size of the robot, the loco motion, indoor/outdoor map information of the first building, and indoor/outdoor map information of the second building;

generating a movement sequence of the robot according to the traveling plan;

determining whether the robot exists inside the first building;

analyzing whether interfloor movement is necessary if it is determined that the robot exists indoors of the first building;

If it is determined that the robot needs to move between floors, moving the robot in front of an elevator door capable of boarding on the first current floor;

generating a control protocol of an elevator monitoring panel of the first building when it is confirmed that the robot has arrived in front of the elevator door;

controlling the elevator to board the elevator when the elevator monitoring panel moves the elevator to the floor where the robot is waiting and when it is confirmed that the elevator door is open;

controlling an elevator monitoring panel of the first building so that the elevator moves to a first destination floor when the robot boarding of the elevator is completed;

controlling the robot to get off the elevator when the elevator arrives at the first destination floor by the elevator monitoring panel;

determining whether outdoor movement is necessary when the robot is completely removed from the elevator;

controlling the robot to enter a second building after leaving the first building when it is determined that outdoor movement of the robot is necessary;

When the robot enters the second building, it determines whether or not to use the elevator in the second building, and depending on the determination result, moves the robot to the work place on the second current floor or uses the elevator in the second building. Moving the robot to a work place of a second destination floor after getting off the vehicle after using the robot;

A control method comprising a.