KR102285813B1 - Method for controlling a mobile robot, apparatus for supporting the same, and delivery system using a mobile robot - Google Patents

Abstract

Various delivery systems using a mobile robot and delivery methods performed in the system are provided. A cooperative delivery system using a drone and a ground robot according to some embodiments of the present disclosure includes a service system for generating mission information for delivering goods to a delivery destination, a ground robot for transporting the goods based on the mission information, and the It may include a drone that transports the goods in cooperation with a ground robot. In this case, the mission information may include collaboration information for cooperation between the ground robot and the drone.

Description

A method for controlling a mobile robot, a device supporting the method, and a delivery system using the mobile robot

The present disclosure relates to a method for controlling a mobile robot, an apparatus supporting the method, and a delivery system using the mobile robot. More specifically, a control method for a mobile robot designed to solve the problem of increasing the risk of an accident of the mobile robot due to the communication delay between the remote control system and the mobile robot and the user's sense of difference in operation, a control device supporting the method, and It relates to various delivery systems using one or more mobile robots.

In recent years, interest in autonomous mobile robots such as autonomous vehicles has been growing. However, research on autonomous mobile robots is still in its infancy, and most robot systems include a remote control system in which a manipulator controls a mobile robot in a remote location.

In a system for controlling a mobile robot in a remote location through a remote control system, the most problematic is a communication delay generated in a wireless communication process. As shown in FIG. 1 , the communication delay is a control delay ( 5 , delay _control ) generated in a process in which the mobile robot 3 receives a control signal of the remote control system 1 and the remote control system 1 moves. _{It includes a monitoring delay (7, delay monitor} ) generated in the process of receiving the surrounding image from the robot (3).

The two delays 5 and 7 are the main factors that increase the accident risk of the mobile robot. This is because, due to the monitoring delay 7 , the user may be late in recognizing the obstacles surrounding the mobile robot 3 , or the control signal of the operator may arrive to the mobile robot 3 late due to the control delay 5 .

Conventionally, in order to solve the above problem, a method of changing the driving mode of the mobile robot to the autonomous driving mode has been mainly used when the communication delay is greater than or equal to a threshold value. That is, when remote control is not smoothly performed due to a communication delay, a method of avoiding the risk of an accident by entrusting the control right of the mobile robot to the autonomous driving module has been mainly used. However, as shown in FIG. 2 , repeated mode changes while driving the mobile robot discontinuously generate the final control value 15 , resulting in unstable driving of the mobile robot (eg T ₁ , T ₂ , T _{3 ).} At that point in time, the robot's control value changes abruptly), it can distract the user's attention. Moreover, the sudden change to the autonomous driving mode is also a factor that maximizes the sense of heterogeneity of robot control felt by the operator.

Accordingly, there is a need for a mobile robot control method capable of resolving problems of safety of the mobile robot and a sense of heterogeneity felt by a user in controlling a mobile robot located at a remote location.

On the other hand, as ground robot or drone technology develops in recent years, many attempts are being made to apply these technologies to delivery services. For example, Google is planning to use driverless cars for delivery, and Amazon is trying out a drone-based delivery service called prime air. In addition, Domino, a world-class pizza delivery company, has demonstrated a service that delivers pizza with robots. The purpose of such a delivery service is to deliver goods to users more quickly to improve customer satisfaction and to increase delivery efficiency by reducing labor costs.

However, in order for the delivery service using ground robots and drones to be put to practical use in various environments, a lot of research has yet to be done. For example, in order for the delivery service as described above to be put to practical use, various topics such as a collaborative delivery method using a ground robot and a drone, a method of delivering large items using multiple drones, and a method of safely delivering fragile items such as food and beverages research should be done.

Korean Patent Registration No. 10-1293247 (published on August 10, 2007) Korean Patent Publication No. 10-2018-0070624 (published on June 26, 2018)

A technical problem to be solved by the present disclosure is to provide a method for controlling a mobile robot and an apparatus supporting the method, which can solve the problem of increasing the risk of an accident of the mobile robot caused by a communication delay.

Another technical problem to be solved by the present disclosure is to provide a method for controlling a mobile robot capable of solving the problem of heterogeneity in robot manipulation felt by a user due to communication delay and/or repeated mode change, and an apparatus supporting the method .

Another technical problem to be solved by the present disclosure is to provide a system for providing a delivery service using a mobile robot and an operating method of the system.

Another technical problem to be solved by the present disclosure is to provide a system for providing a collaborative delivery service using a ground robot and a drone together, and an operating method of the system.

Another technical problem to be solved by the present disclosure is to provide a method for safely delivering large items using a plurality of drones.

Another technical problem to be solved by the present disclosure is to provide a method for safely delivering fragile items such as food and drink.

The technical problems of the present disclosure are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

In order to solve the above technical problem, a cooperative delivery system using a drone and a ground robot according to some embodiments of the present disclosure is a service system that generates mission information for delivering goods to a delivery destination, and based on the mission information, the It may include a ground robot that transports goods and a drone that transports the goods in cooperation with the ground robot. In this case, the mission information may include collaboration information for cooperation between the ground robot and the drone.

In some embodiments, the service system may receive a delivery request for the item from a user terminal, monitor the location of a ground robot or drone that delivers the item, and provide the monitored location to the user terminal there is.

In some embodiments, the collaboration information includes a place of handover of the item, and the drone recognizes a marker located in the vicinity of the place of delivery, places the item in the recognized marker area, and the The ground robot may take over and transport the article located in the marker area.

In some embodiments, the ground robot may recognize an obstacle existing on a movement path to the destination, and in response to determining that it cannot pass the recognized obstacle, hand over the article to the drone.

In some embodiments, the drone includes a plurality of drones, and the plurality of drones transport the article using a rope connected to the article, wherein the first drone among the plurality of drones is relatively relative to other drones. estimate the posture of the article based on the position and the current length of the rope, and in response to determining that the estimated posture does not satisfy a predetermined balance condition, adjust the length of the rope or the relative position of the first drone there is.

In some embodiments, the service system further includes a remote control system for remotely controlling the ground robot, and the target control value for controlling the ground robot includes a first control value input through the remote control system and the It is determined based on a weighted sum of the second control values generated by the autonomous driving module of the ground robot, wherein the first weight applied to the first control value and the second weight applied to the second control value are determined by the remote control It is determined based on a communication delay between the system and the ground robot, and the first weight and the second weight may have an inverse relationship.

A food and beverage delivery system using a mobile robot according to some embodiments of the present disclosure for solving the above-described technical problem, receives a delivery request from a user terminal, and in response to the request, a task for delivering the food and beverage to the delivery destination A service system for generating information and a mobile robot for delivering the food and drink stored in the cooking device based on the mission information may be included. In this case, the task information includes the cooking time of the food and beverage, and the mobile robot calculates an expected required time to the delivery destination, compares the calculated estimated required time and the cooking time, and the comparison result The cooking appliance may be operated based on the

In some embodiments, the mobile robot may monitor the cooking state of the food and drink through a sensor provided in the cooking appliance, and may adjust the cooking intensity of the cooking appliance based on the cooking state.

In some embodiments, the mobile robot calculates an accident risk for a current moving path based on sensing data about the surrounding environment, and in response to determining that the calculated accident risk is greater than or equal to a threshold, moving to the delivery destination You can adjust the path.

A food and beverage delivery system using a mobile robot according to some embodiments of the present disclosure for solving the above-described technical problem, receives a delivery request from a user terminal, and in response to the reception, a task for delivering food and drink to a delivery destination It may include a mobile robot that delivers the food and drink contained in the service system and the delivery container for generating information based on the mission information. At this time, the mobile robot, when the type of food and drink is the first type, adjusts the posture of the delivery container during movement according to the first adjustment method, and when the type of food and beverage is the second type, the second It is possible to adjust the posture of the delivery container during movement according to the adjustment method.

In some embodiments, the mobile robot, when the food and drink is a liquid, to measure the surface inclination of the food and drink, and may adjust the posture of the delivery container so that the surface inclination is gentle.

In some embodiments, the mobile robot, when the food and drink is not a liquid, may measure the ground inclination, and adjust the posture of the delivery container based on the ground inclination.

1 is a diagram for explaining a communication delay between a remote control system and a mobile robot.
2 is a view for explaining the problems of the conventional mobile robot control method.
3 is an exemplary configuration diagram illustrating a robot control system according to some embodiments of the present disclosure.
4 and 5 are diagrams schematically illustrating a control device according to some embodiments of the present disclosure.
6 is a diagram for explaining an operation of a server according to some embodiments of the present disclosure.
7 is an exemplary block diagram illustrating an apparatus for controlling a mobile robot according to some embodiments of the present disclosure.
8 is an exemplary flowchart illustrating a method for controlling a mobile robot according to a first embodiment of the present disclosure.
9 and 10 are diagrams for explaining a weight for each control value that may be referred to in some embodiments of the present disclosure.
11 is an exemplary flowchart illustrating a weight determination process according to some embodiments of the present disclosure.
12 is an exemplary flowchart illustrating a weight determination process according to some other embodiments of the present disclosure.
13 is an exemplary flowchart illustrating a method for controlling a mobile robot according to a second embodiment of the present disclosure.
14 and 15 are exemplary views illustrating a driving pattern learning method according to some embodiments of the present disclosure.
16 is a view for explaining a delivery system using a mobile robot according to some embodiments of the present disclosure.
17 is an exemplary configuration diagram illustrating a service system according to some embodiments of the present disclosure.
18 is an exemplary flowchart illustrating a mobile robot-based delivery service providing method that may be performed in a delivery system according to some embodiments of the present disclosure.
19 is an exemplary diagram for explaining a method for identifying a user and a user terminal according to some embodiments of the present disclosure.
20 is an exemplary diagram for explaining a cooperative delivery method according to the first embodiment of the present disclosure.
21 is an exemplary diagram for explaining a cooperative delivery method according to a second embodiment of the present disclosure.
22 and 23 are exemplary views for explaining a collaborative delivery method according to a third embodiment of the present disclosure.
24 and 25 are exemplary views for explaining a collaborative delivery method according to a fourth embodiment of the present disclosure.
26 is an exemplary diagram for explaining a cooperative delivery method according to a fifth embodiment of the present disclosure.
27 is an exemplary diagram for explaining a cooperative delivery method according to a sixth embodiment of the present disclosure.
28 is an exemplary diagram for explaining a multi-drone-based cooperative delivery method according to the first embodiment of the present disclosure.
29 is an exemplary diagram for explaining a multi-drone-based cooperative delivery method according to a second embodiment of the present disclosure.
30 is an exemplary view for explaining a food and beverage delivery method using a mobile robot according to the first embodiment of the present disclosure.
31 and 32 are exemplary views for explaining a food and beverage delivery method using a mobile robot according to a second embodiment of the present disclosure.
33 is an exemplary hardware configuration diagram illustrating an exemplary computing device that may implement an apparatus and/or system according to various embodiments of the present disclosure.

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Advantages and features of the present disclosure, and methods for achieving them, will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the technical spirit of the present disclosure is not limited to the following embodiments, but may be implemented in various different forms, only the present embodiments make the present disclosure complete, and common knowledge in the technical field to which the present disclosure belongs It is provided to fully inform those who have the scope of the present disclosure, and the technical spirit of the present disclosure is only defined by the scope of the claims.

In adding reference numerals to the components of each drawing, it should be noted that the same components are given the same reference numerals as much as possible even though they are indicated on different drawings. In addition, in describing the present disclosure, if it is determined that a detailed description of a related known configuration or function may obscure the gist of the present disclosure, the detailed description thereof will be omitted.

Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used with the meaning commonly understood by those of ordinary skill in the art to which this disclosure belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless clearly defined in particular. The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present disclosure. As used herein, the singular also includes the plural unless specifically stated otherwise in the phrase.

In addition, in describing the components of the present disclosure, terms such as first, second, A, B, (a), (b), etc. may be used. These terms are only for distinguishing the components from other components, and the essence, order, or order of the components are not limited by the terms. When a component is described as being “connected”, “coupled” or “connected” to another component, the component may be directly connected or connected to the other component, but another component is between each component. It should be understood that elements may be “connected,” “coupled,” or “connected.”

As used herein, “comprises” and/or “comprising” refers to the presence of one or more other components, steps, operations and/or elements mentioned. or addition is not excluded.

Prior to the description of the present specification, some terms used in the present specification will be clarified.

In this specification, a mobile robot refers to all types of robots equipped with a movement (ie, traveling) function. The mobile robot may include a ground robot equipped with a ground movement function and a drone equipped with an air movement function. In addition, the mobile robot may include a remote control robot controlled by a remote control device (or system), an autonomous robot that moves according to autonomous judgment, and a semi-autonomous robot equipped with both remote control and autonomous movement functions. .

In this specification, an instruction is a series of instructions grouped based on a function, and refers to a component of a computer program and to be executed by a processor.

Hereinafter, some embodiments of the present disclosure will be described in detail with reference to the accompanying drawings.

3 is an exemplary configuration diagram illustrating a robot control system according to some embodiments of the present disclosure.

Referring to FIG. 3 , the robot control system may be configured to include a remote control system 20 , a mobile robot 30 , a control device 100 (not shown in FIG. 3 ), and a server 50 . However, this is only a preferred embodiment for achieving the purpose of the present disclosure, and it goes without saying that some components may be added or deleted as necessary. In addition, it should be noted that each component of the robot control system shown in FIG. 3 represents functional elements that are functionally separated, and that at least one component may be implemented in a form that is integrated with each other in an actual physical environment. For example, the control device 100 may be implemented as a specific logic of the remote control system 20 or the mobile robot 30 .

In addition, in the actual physical environment, each of the components may be implemented in a form separated into a plurality of detailed functional elements. For example, a first function of the server 50 may be implemented in a first computing device, and a second function may be implemented in a second computing device. Hereinafter, each component of the robot control system will be described.

In the robot control system, the remote control system 20 is a device used by the operator to remotely control the mobile robot 30 . The remote control system 20 may control the mobile robot 30 located at a remote location by transmitting the control values (eg, steering angle, speed) input by the operator to the mobile robot 30 in real time. Hereinafter, for convenience of description, a control value input by the operator through the remote control system 20 will be referred to as a “remote control value”.

For improved convenience, the remote control system 20 may be configured to include a haptic interface device such as a handle or a pedal. For example, the operator may control the steering angle of the mobile robot 30 through the handle and control the speed of the mobile robot 30 through the pedal. However, the technical scope of the present disclosure is not limited thereto, and the user interface device of the remote control system 20 may be implemented in any manner.

In addition, the remote control system 20 may be configured to include a display device for displaying surrounding image data captured by the mobile robot 30 . Accordingly, the operator may recognize the surrounding environment of the mobile robot 30 through the display device and appropriately perform remote control.

In the robot control system, the mobile robot 30 is a robot controlled by the remote control system 20 . In particular, although FIG. 3 illustrates that the mobile robot 30 is a vehicle-based ground robot as an example, as described above, the mobile robot 30 may include any type of robot such as a drone.

In the robot control system, the control device 100 is a computing device that generates target control values for controlling the mobile robot 30 based on various control values. Here, the computing device may be a notebook, a desktop, a laptop, etc., but is not limited thereto and may include any type of device equipped with a calculation means. Referring to FIG. 33 for an example of the computing device.

Since the target control value is a value indicating a target control state (eg, steering angle, speed) of the mobile robot 30 , the final control value input to the mobile robot 30 may be different from the target control value. For example, the final control value may be generated by various control algorithms based on the current control state (eg, current speed, current steering angle) of the mobile robot 30 and the target control value. However, a detailed description thereof will be omitted so as not to obscure the subject matter of the present disclosure.

As shown in FIG. 4 , the control device 100 includes a remote control value 23 input through the remote control system 20 and a control value 25 generated by the autonomous driving module 21 , hereinafter referred to as “autonomous control”. The target control value 29 may be generated based on the 'value') or the like. At this time, the control device 100 sets the remote control value 23 and the autonomous control value 25 based on various factors such as communication delay between the remote control system 20 and the mobile robot 30, the risk of collision, and the complexity of the surrounding environment. ) may be adjusted to be reflected in the target control value 29 .

That is, the control device 100 does not generate the target control value as the specific control values 23 and 25 at any one time, but appropriately combines the remote control value 23 and the autonomous control value 25 to control the target. It produces the value (29). Accordingly, as shown in FIG. 5 , the target control value 29 generated by the control device 100 appears in the form of a continuous value and does not change rapidly. Accordingly, a problem in which discontinuous control values are generated according to repeated control mode changes (refer to FIG. 2 ) can be solved, and the problem of manipulation heterogeneity felt by the operator can also be alleviated. A detailed description of the configuration and operation of the control device 100 will be described in detail with reference to the drawings below with reference to FIG. 7 .

The control device 100 is preferably implemented on the mobile robot 30 side, but the technical scope of the present disclosure is not limited thereto. However, for convenience of understanding, it is assumed that the control device 100 is implemented on the side of the mobile robot 30 and the description will be continued.

In the robot control system, the server 50 is a computing device that receives driving data from the remote control system 20 or the mobile robot 30, learns the driving data, and builds a machine learning model for a user's driving pattern. . Here, the computing device may be a notebook computer, a desktop computer, a laptop computer, etc., but is not limited thereto and may include all types of devices equipped with a calculation means and a communication means.

The driving data may include, for example, control values of the remote control systems 20a to 20n, sensing data (e.g. image data) of the mobile robot, and the like.

Specifically, as shown in FIG. 6 , the server 50 receives driving data of a specific user from a plurality of remote control systems 20a to 20n, learns the driving data, and a machine associated with each user's driving pattern. You can build a learning model. Also, the server 50 may predict and provide a control value (hereinafter, referred to as a “pattern control value”) for the current driving environment of the mobile robot 30 through the built-up machine learning model. The pattern control value may also be used by the control device 100 to generate a target control value, and a detailed description thereof will be described later with reference to FIG. 12 . In addition, an exemplary method for the server 50 to learn the driving pattern will be described later with reference to FIGS. 12 and 12 .

For reference, some of the functions of the server 50 may be implemented in the remote control system 20 or the mobile robot 30 . For example, the remote control system 20 or the mobile robot 30 may machine-lear a specific user's driving pattern based on driving data and provide the learned machine learning model to the server 50 . there is. In this case, the server 50 may only perform a function of providing a predetermined pattern control value to the control device 100 using a machine learning model from which driving patterns of a plurality of users are learned.

At least some of the respective components 20 , 30 , 50 , and 100 of the robot control system may communicate through a network. The network between the remote control system 20 and the mobile robot 30 will typically be implemented as a wireless network, but the technical scope of the present disclosure is not limited to the type of network. That is, the network may be implemented as any type of network, such as a local area network (LAN), a wide area network (WAN), a mobile radio communication network, and a Wibro (Wireless Broadband Internet). .

So far, the robot control system according to an embodiment of the present disclosure has been described with reference to FIGS. 3 to 6 . Next, the configuration and operation of the apparatus 100 for controlling a mobile robot according to an embodiment of the present disclosure will be described with reference to FIG. 7 .

7 is an exemplary block diagram illustrating the control apparatus 100 according to some embodiments of the present disclosure.

Referring to FIG. 7 , the control device 100 includes a control value obtaining module 110 , a delay determining module 120 , an autonomous driving module 130 , an information obtaining module 140 , a weight determining module 150 , and a target control. It may be configured to include a value determination module 160 . However, only components related to the embodiment of the present disclosure are illustrated in FIG. 7 . Accordingly, those skilled in the art to which the present disclosure pertains can see that other general-purpose components other than the components shown in FIG. 7 may be further included. Hereinafter, each component is demonstrated.

The control value acquisition module 110 acquires various control values that are the basis of the target control value. Specifically, the control value obtaining module 110 may obtain a remote control value input by a user from the remote control system 20 , and obtain an autonomous control value generated by the autonomous driving module 130 . . Both the remote control value and the autonomous control value may include control values for the speed and steering angle of the mobile robot 30 .

In addition, the control value acquisition module 110 may further acquire from the server 50 a pattern control value derived based on the driving pattern of the corresponding user, a pattern control value derived based on the average driving pattern of a plurality of users, and the like. there is.

Next, the delay determining module 120 determines a communication delay between the remote control system 20 and the mobile robot 30 . The communication delay may be determined based on the control delay 5 and the monitoring delay 7 as shown in FIG. 1 . For example, the communication delay may be determined in various forms, such as a sum, average, and weighted average of the two delays 5 and 7 .

Next, the autonomous driving module 130 generates an autonomous control value for the mobile robot 30 . To this end, the autonomous driving module 130 may use at least one autonomous driving algorithm widely known in the art, and any autonomous driving algorithm may be used.

Next, the information acquisition module 140 acquires various types of information, such as the degree of collision risk for the surrounding objects of the mobile robot 30 and the complexity of the surrounding environment. The collision risk and the complexity may be used by the weight determination module 150 to determine a weight for each control value.

The information acquisition module 140 may directly calculate the risk of collision and the complexity, or may receive information calculated by another device. This may vary according to the embodiment. A method in which the information acquisition module 140 determines the risk of collision and complexity of a surrounding object based on the sensing data of the mobile robot will be described with reference to FIGS. 12 and 12 .

Next, the weight determination module 150 determines a weight for each control value, and provides the determined weight to the target control value determination module 160 . In this case, the weight may be understood as a value indicating a weight in which each control value is reflected in the target control value.

In some embodiments, the weight determination module 150 may determine a weight for each control value based on the communication delay provided by the delay determination module 120 . A detailed description of the present embodiment will be described in detail with reference to FIGS. 10 and 11 .

In some embodiments, the weight determination module 150 may determine a weight for each control value by further considering the risk of collision with the surrounding objects of the mobile robot 30 provided by the information obtaining module 140 . A detailed description of the present embodiment will be described later with reference to FIG. 12 .

In some embodiments, the weight determination module 150 may determine a weight for each control value by further considering the complexity of the surrounding environment of the mobile robot 30 provided by the information obtaining module 140 . A detailed description of the present embodiment will be described later with reference to FIG. 12 .

Next, the target control value determination module 160 determines a target control value based on a plurality of control values provided by the control value obtaining module 110 and a weight for each control value provided by the weight determination module 150 . do.

For example, the target control value determination module 160 may determine the target control value based on a weighted average of the plurality of control values. However, this is only for describing some embodiments of the present disclosure, and the technical scope of the present disclosure is not limited thereto.

For reference, it should be noted that not all components shown in FIG. 7 are essential components of the control device 100 . That is, according to another embodiment of the present disclosure, the control device 100 may be implemented by some of the components shown in FIG. 7 .

Each of the components 110 to 160 shown in FIG. 7 may mean software or hardware such as a field programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). However, the above components are not meant to be limited to software or hardware, and may be configured to reside in an addressable storage medium, or may be configured to execute one or more processors. The functions provided in the components may be implemented by more subdivided components, or may be implemented as one component that performs a specific function by combining a plurality of components.

So far, the configuration and operation of the control device 100 according to some embodiments of the present disclosure have been described with reference to FIG. 7 . Hereinafter, a method for controlling a mobile robot according to some embodiments of the present disclosure will be described in detail with reference to FIGS. 8 to 12 .

Each step of a method for controlling a mobile robot according to some embodiments of the present disclosure to be described below may be performed by a computing device. That is, each step of the control method may be implemented as one or more instructions executed by a processor of a computing device. All steps included in the control method may be executed by one physical computing device, but the first steps of the method are performed by a first computing device, and the second steps of the method are performed by a second computing device may be performed by Hereinafter, it is assumed that each step of the control method is performed by the control device 100 to continue the description. However, for convenience of description, the description of the operating subject of each step included in the control method may be omitted.

8 is an exemplary flowchart illustrating a method for controlling a mobile robot according to a first embodiment of the present disclosure. However, this is only a preferred embodiment for achieving the purpose of the present disclosure, and it goes without saying that some steps may be added or deleted as needed.

Referring to FIG. 8 , the method for controlling the mobile robot according to the first embodiment starts at step S100 in which the control device 100 obtains a remote control value input through the remote control system 20 . In this case, the remote control value may include control values for the speed and steering angle of the mobile robot 30 .

In step S200 , the control device 100 acquires an autonomous control value generated by the autonomous driving module. The autonomous control value may also include control values for the speed and steering angle of the mobile robot 30 .

In step S300, the control device 100 determines a weight for each control value.

In some embodiments, the control device 100 may determine a weight for each control value based on a communication delay between the remote control system 20 and the mobile robot 30 . A specific example of this embodiment is shown in FIG. 9 .

As shown in FIG. 9 , the weight for the remote control value (41, hereinafter referred to as “remote weight”) and the weight for the autonomous control value (hereinafter referred to as “autonomous weight”) are inversely proportional to each other as shown in FIG. 9 . value can be determined. Specifically, as the communication delay increases, the remote weight 41 may be determined as a large value and the autonomous weight 43 may be determined as a small value. This is to improve the safety of the mobile robot 30 by increasing the proportion in which the autonomous control value is reflected to the target control value as the communication delay increases.

Also, in response to determining that the communication delay is equal to or greater than the predetermined threshold value 45 , the control device 100 may determine the autonomous weight 43 as the maximum value and the remote weight 41 as the minimum value. For example, as shown in FIG. 9 , when the weight for each control value has a value between 0 and 1, the control device 100 determines the autonomous weight 43 as “1”, and the remote weight (41) can be determined as “0”. In this case, the mobile robot 30 may be controlled as if operating in an autonomous driving mode.

The threshold value 45 may be a preset fixed value or a variable value that varies according to circumstances. For example, the threshold 45 may be a variable value that varies according to the current traveling speed of the mobile robot 30 . As a more detailed example, the threshold 45 is determined to be a value inversely proportional to the current running speed of the mobile robot 30 as shown in FIG. 10 , and as the current running speed of the mobile robot 30 is changed, may be subject to change. In this case, even if the communication delay is the same, if the current traveling speed of the mobile robot 30 is higher, the control device 100 determines the autonomous weight 43 as a larger value. Accordingly, the risk of an accident of the mobile robot 30 may be further reduced, and safety may be improved. However, the above examples are only for explaining some embodiments of the present disclosure, and the technical scope of the present disclosure is not limited thereto.

In some embodiments, as shown in FIG. 11 , the control device 100 may determine a weight for each control value by further considering the risk of collision with a surrounding object of the mobile robot 30 ( S310 , S330 , S310 , S330 , S350). For example, in order to improve the safety of the mobile robot 30 , as the risk of collision increases, the control device 100 may determine the autonomous weight as a larger value.

The collision risk may be determined based on sensing data of the mobile robot 30 . In this case, the sensing data may include data measured by various sensors, such as distance data to a nearby object measured by a proximity sensor, and image data captured by a camera sensor.

According to some embodiments of the present disclosure, the control device 100 controls the number of surrounding objects derived from the image data, the recognition result of the surrounding objects, the distance from the surrounding objects, and the characteristics of the surrounding environment (eg flat, uphill, downhill, lane). The collision risk may be determined based on an image analysis result such as presence or absence, size of lanes, intersection existence, traffic light existence, etc.). In this case, the recognition result may include information on whether the surrounding object is a movable or fixed object, a person or an object, and information on the size and material of the corresponding object.

A method for the control device 100 to determine the risk of collision with respect to a nearby object may vary according to an embodiment. For example, as the number of surrounding objects located within a predetermined distance increases, the risk of collision may be determined as a higher value. As another example, as the number of surrounding objects corresponding to the movable object within a predetermined distance increases, the risk of collision may be determined as a higher value. As another example, as the number of surrounding objects corresponding to a person within a certain distance increases, the risk of collision may be determined as a higher value. This is because people are more likely to show irregular movement patterns than objects. As another example, even if a plurality of surrounding objects are located within a predetermined distance, if most of the surrounding objects are located in different lanes, the risk of collision may be determined as a lower value. As another example, the higher the complexity of the surrounding environment, the higher the risk of collision may be determined. As another example, the larger the size of the surrounding object and the harder the material, the higher the risk of collision may be determined.

In some embodiments, as shown in FIG. 12 , the control device 100 may determine a weight for each control value by further considering the complexity of the surrounding environment of the mobile robot 30 ( S310 , S320 , and S340 ). ).

The complexity may also be determined based on sensing data of the mobile robot 30 . In this case, the sensing data may include data measured by various sensors, such as distance data to a nearby object measured by a proximity sensor, and image data captured by a camera sensor.

According to some embodiments of the present disclosure, the control device 100 controls the number of surrounding objects derived from the image data, the recognition result of the surrounding objects, the distance from the surrounding objects, and the characteristics of the surrounding environment (whether there is a lane, whether there is an intersection, or whether there is a lane). The complexity may be determined based on the image analysis results, such as the size of , the presence or absence of a traffic light, etc.), the location distribution of surrounding objects, and the density. For example, as the number of surrounding objects increases, the complexity may be determined as a higher value. For another example, as there are more movable objects, the complexity may be determined to be a higher value. As another example, as the number of movable objects exhibiting irregular movement patterns increases, the complexity may be determined as a higher value. As another example, as the types of surrounding objects are more diverse, the complexity may be determined as a higher value. In addition, the complexity may be determined as a higher value when there is an intersection in the surrounding environment, when the size of the lane is small, when there is no lane, when a plurality of surrounding objects are distributed within a predetermined distance, and the like.

In addition, the control device 100 may determine a weight for each control value according to various combinations of the above-described exemplary embodiments.

Referring back to FIG. 8 , in step S400 , the control device 100 generates a target control value of the mobile robot based on a remote control value, an autonomous control value, and a weight for each control value. For example, the control device 100 may generate the target control value by using a method such as weighted sum or weighted average.

Meanwhile, in some embodiments, the control device 100 may further perform the step of adjusting the target control value so that the difference between the target control value and the remote control value is equal to or less than a predetermined threshold value. For example, in response to determining that the difference between the target control value and the remote control value generated in step S400 is equal to or greater than a threshold value, the control device 100 may adjust the generated target control value to be within the threshold value. there is.

In this case, the threshold value may be a preset fixed value or a variable value that varies according to circumstances. For example, the threshold value may be a variation value determined to be larger as the current traveling speed or communication delay of the mobile robot 30 increases. This is because, when the current traveling speed or communication delay is large, the weight for the remote control value will be small. According to the present embodiment, the target control value may be appropriately adjusted so that the difference between the target control value and the remote control value does not become excessively large. Accordingly, a sense of heterogeneity in remote operation felt by the user may be further reduced.

So far, the method for controlling the mobile robot according to the first embodiment of the present disclosure has been described with reference to FIGS. 8 to 12 . According to the above-described method, since the target control value is generated based on the combination of each control value, the problem that the driving of the mobile robot becomes unstable due to repeated control mode switching and the problem of maximizing the manipulator's heterogeneity in operation can be alleviated. there is. In addition, the risk of an accident of the mobile robot can be minimized by adjusting the weight for each control value without switching the control mode.

Hereinafter, a method for controlling a mobile robot according to a second embodiment of the present disclosure will be described with reference to FIGS. 13 to 15 . In order to exclude duplicate description, the description will be continued focusing on differences from the first embodiment described above, but it goes without saying that the contents mentioned in the first embodiment may also be included in the second embodiment.

13 is an exemplary flowchart illustrating a method for controlling a mobile robot according to a second embodiment of the present disclosure. However, this is only a preferred embodiment for achieving the purpose of the present disclosure, and it goes without saying that some steps may be added or deleted as needed.

Referring to FIG. 13 , similarly to the first embodiment, the control method according to the second embodiment starts at steps S100 and S200 in which the control device 100 acquires a remote control value and an autonomous control value. .

In step S250 , the control device 100 acquires a pattern control value for the mobile robot 30 . The pattern control value means a control value generated by the machine learning model that has learned the user's driving pattern as described above. That is, the pattern control value means a control value predicted for the current surrounding environment of the mobile robot 30 based on a past driving pattern. A method of learning the driving pattern will be described later with reference to FIGS. 14 and 15 .

The pattern control value is generated based on a first pattern control value generated based on a driving pattern of a specific user who controls the mobile robot 30 through the remote control system 20 and/or an average driving pattern of a plurality of users and a second pattern control value. The pattern control value may be provided, for example, from a predetermined server (e.g. 50), but the technical scope of the present disclosure is not limited thereto.

In step S350 , the control device 100 determines a weight for each control value. For example, the weight for the pattern control value may be determined as a larger value as the communication delay increases or the current traveling speed of the mobile robot 30 increases.

In step S450 , the control device 100 generates a target control value of the mobile robot 30 based on a remote control value, an autonomous control value, a pattern control value, and a weight for each control value. For example, the control device 100 may generate the target control value by using a method such as weighted sum or weighted average.

So far, a method for controlling a mobile robot according to the second embodiment of the present disclosure has been described with reference to FIG. 13 . According to the above-described method, the target control value is generated by further considering the pattern control value generated based on the driving pattern for the corresponding pilot or a plurality of pilots. When the pattern control value according to the driving pattern for the corresponding manipulator is reflected to the target control value, the manipulation heterogeneity felt by the manipulator may be alleviated. In addition, when the pattern control value according to the driving pattern for the plurality of operators is reflected in the target control value, errors in other control values can be corrected by the pattern control value, so the safety of the mobile robot is further improved can be

Hereinafter, a driving pattern learning method according to some embodiments of the present disclosure will be described with reference to FIGS. 14 and 15 . Each step of the driving pattern learning method to be described below may be performed by a computing device. For example, the computing device may be the server 50 , the control device 100 , the remote control system 20 , the mobile robot 30 , and the like. However, for convenience of understanding, it is assumed that the operating subject of each step included in the driving pattern learning method is the server 50 and the description is continued. Of course, for convenience of description, the description of the operating subject of each step may be omitted.

First, learning data used for learning a driving pattern will be described with reference to FIG. 14 .

As shown in FIG. 14 , the learning data 69 may include image data 63 obtained by photographing the surrounding environment of the mobile robot 30 and a remote control value 61 corresponding to the surrounding environment. Therefore, when the driving pattern of the corresponding user (eg, the degree of deceleration when it appears on a surrounding object, the degree of deceleration on a curved road, etc.) is learned through machine learning of the learning data 69, the corresponding user for a specific surrounding environment A pattern control value of may be predicted.

In addition, the learning data 69 may further include information such as the accident risk 65 and the complexity 67 calculated based on the image data 63 . Since the method of calculating the accident risk 65 and the complexity 67 is the same as described above, a description thereof will be omitted.

Next, a machine learning-based driving pattern learning method will be described with reference to FIG. 15 .

The driving pattern learning method is an extraction process of extracting the surrounding environment feature 71 from the image data 63 among the training data 69 and the machine for the surrounding environment feature 71 and other learning data 61 , 65 , 67 . It may include a learning process for performing learning.

The extraction process is a process of extracting various surrounding environment features 71 through analysis of the image data 69, where the surrounding environment features 71 are the presence or absence of a lane, the object recognition result, the presence of an intersection, the presence of a curve, and a traffic light. It may include various characteristics such as presence or absence. Although FIG. 15 illustrates extracting various environmental features 71 from the image data 63 using a pre-trained convolutional neural network (CNN) as an example, the technical scope of the present disclosure is not limited thereto. For example, the surrounding environment feature 71 may be extracted using a computer vision algorithm widely known in the art.

The learning process is a process of performing machine learning on the surrounding environment feature 71 , the collision risk 65 , the complexity 67 , and the remote control value 61 . At this time, the surrounding environment features 71, collision risk 65, complexity 67, etc. are used as input data of the machine learning model, and the remote control value 61 is compared with the predictive control value 73 of the output layer. It can be used to update the weights of the machine learning model through 15 illustrates that machine learning is performed using an artificial neural network (ANN)-based deep learning model as an example, but the technical scope of the present disclosure is not limited thereto. Learning about the driving pattern may be performed using any number of different types of machine learning algorithms.

Since the learning of the driving pattern for a plurality of users is merely an extension of the learning data to the learning data for the plurality of users, a description thereof will be omitted. Of course, according to the embodiment, a plurality of machine learning models that learn the driving patterns of individual users may be built, or a single machine learning model that learns the driving patterns of a plurality of users may be built. When a plurality of machine learning models are built, a pattern control value for a plurality of users may be determined based on an average, a weighted average, etc. of the pattern control values of individual users. In this case, the weight used for the weighted average may be, for example, the accuracy and learning maturity of each machine learning model.

So far, a machine learning-based driving pattern learning method according to some embodiments of the present disclosure has been described with reference to FIGS. 14 and 15 .

So far, the robot control system, the control method of the mobile robot, and the control apparatus 100 supporting the control method have been described with reference to FIGS. 3 to 15 . Hereinafter, a delivery system using a mobile robot according to some embodiments of the present disclosure and a method of operating the system will be described with reference to the drawings below with reference to FIG. 16 . The aforementioned concept of controlling the mobile robot may also be applied to a mobile robot to be described later.

16 is a view for explaining a delivery system using a mobile robot according to some embodiments of the present disclosure.

As shown in FIG. 16 , the delivery system may include a service system 200 and a mobile robot 300 in charge of delivery (hereinafter, referred to as “delivery robot”). However, this is only a preferred embodiment for achieving the purpose of the present disclosure, and it goes without saying that some components may be added or deleted as necessary.

Briefly looking at each component, the service system 200 is a computing system for providing a mobile robot-based delivery service to a user. The service system 200 may perform various functions, such as receiving a delivery request, allocating a delivery task according to the delivery request, and monitoring delivery status.

In some embodiments, as shown in FIG. 17 , the service system 200 may include a service server 210 , a control system 230 , and a remote control system 250 .

The service server 210 is a computing device that performs overall functions related to a delivery service, such as providing a web page for a delivery request or managing a delivery history. The service server 210 may receive a delivery request from the user terminal 400 and request the control system 230 to create a task according to the delivery request.

Next, the control system 230 is a system that performs overall control functions for the delivery robot 300 such as delivery task creation, delivery task assignment, and delivery robot monitoring. The control system 230 may perform real-time communication with the delivery robot 300 and perform various monitoring and control functions. As shown in FIG. 17 , the control system 230 may be implemented to include a plurality of displays to monitor a plurality of delivery robots 300 .

Next, the remote control system 250 is a system equipped with a remote control function for the delivery robot 300 . In some embodiments, the operator may directly control the delivery robot 300 through the remote control system 250 and provide delivery services. For the description of the remote control system 250, reference is made to the description of the drawings such as FIG. 3 .

Referring back to FIG. 16 , the delivery robot 300 is a robot that delivers a delivery target article (hereinafter, “delivery article”). The delivery robot 300 may autonomously perform delivery based on mission information, or may perform delivery under the control of the remote control system 250 . Of course, as described above, the delivery robot 300 may be controlled by a combination of a remote control value of the remote control system 250 and an autonomous control device.

The mission information includes all information necessary for the delivery robot 300 to perform delivery. For example, the mission information may include delivery origin information, delivery start time information, delivery destination information, target arrival time information, collaboration information for performing collaboration between delivery robots, movement route information to the delivery destination, etc. The technical scope of the disclosure is not limited thereto.

The collaboration information includes all information necessary to perform collaboration. For example, information related to the role of each delivery robot (e.g. delivery, surrounding monitoring, etc.), the delivery point of the goods, and the handover method may be included. An example of the collaboration information will be described later along with an example in which cooperative delivery is performed.

The delivery robot 300 performing collaboration may be configured in various forms. For example, a ground robot and a drone may cooperate to perform delivery, and a plurality of ground robots or a plurality of drones may perform delivery through collaboration.

Next, the user terminal 400 is a terminal used by the user to request delivery. The user terminal 400 may be implemented in any device.

So far, a delivery system using a mobile robot has been schematically described with reference to FIGS. 16 and 17 . Hereinafter, a mobile robot-based delivery service providing method performed in the delivery system will be described with reference to FIG. 18 .

18 is an exemplary flowchart illustrating a mobile robot-based delivery service providing method performed in a delivery system according to some embodiments of the present disclosure. However, this is only a preferred embodiment for achieving the purpose of the present disclosure, and it goes without saying that some steps may be added or deleted as needed.

As shown in FIG. 18 , the delivery service providing method starts in step S510 of receiving a delivery request. As described above, the service server 210 may receive the delivery request.

In step S520, the service system 200 generates mission information for delivery in response to the delivery request. As described above, the control system 230 may designate a robot to be in charge of a delivery task from among the plurality of delivery robots 300 , and may generate task information for the robot.

In step S530 , the service system 200 transmits the generated task information to the delivery robot 300 to allocate the delivery task.

In step S540, the delivery robot 300 starts delivery based on the received mission information. For example, the delivery robot 300 may start delivery by moving to the delivery start location at the delivery start time and loading the delivered goods based on the delivery start time and delivery origin information included in the mission information.

In step S550, the delivery robot 300 reports its location information periodically or aperiodically. By doing so, the administrator can monitor the position of the delivery robot 300 in real time through the control system 230 .

In addition, the delivery robot 300 reports the surrounding environment information (eg surrounding environment image, identified obstacle information, etc.) and various event information (eg product damage, delivery delay occurrence, etc.) measured through the sensor to the service system 200 . can do.

In step S560 , the service system 200 provides information such as location information and delivery status of the delivery robot 300 to the user terminal 400 in real time. By doing so, a more satisfactory delivery service can be provided.

In steps S570 and S580, the delivery robot 300 arrives at the delivery destination according to the received mission information and provides an arrival notification to the user. The movement to the delivery destination may be performed by the movement path information included in the mission information, or may be performed by the autonomous driving function of the delivery robot 300 .

The arrival notification may be performed in any way, such as a visual message such as flashing a light, an audio message such as a sound or music, a text message, or a phone call.

In some embodiments, as shown in FIG. 19 , the delivery robot 301 may perform the terminal location identification 303 and user identification 305 processes before performing the arrival notification. Specifically, the delivery robot 301 may estimate the approximate location of the user terminal 401 by using a signal strength (e.g. RSSI) with the user terminal 401 . Also, in the vicinity of the estimated location, the face of the user 403 may be recognized by analyzing the image captured by the delivery robot 301 . Then, the delivery robot 301 may move near the recognized user 403 to provide a delivery notification. Here, the face image of the user 403 used for face recognition, identification information for identifying the user terminal 401, a delivery notification method, etc. may be predefined in the task information.

So far, a mobile robot-based delivery service providing method has been described with reference to FIGS. 18 and 19 . According to the above-described method, by providing a prompt and accurate delivery service using a mobile robot, it is possible to increase customer satisfaction and increase delivery efficiency by reducing labor costs.

Meanwhile, according to some embodiments of the present disclosure, the delivery robot 300 includes a ground robot and a drone, and delivery may be performed through collaboration between the ground robot and the drone. Through this, a door-to-door delivery service may be provided or a delivery range may be expanded. Hereinafter, various embodiments related to cooperative delivery will be described with reference to FIGS. 20 to 27 .

20 is an exemplary diagram for explaining a cooperative delivery method according to the first embodiment of the present disclosure.

As shown in FIG. 20 , in the first embodiment, the drone 311 monitors the surrounding environment of the ground robot 313 in the air, and the ground robot 313 is responsible for transporting the delivery product 315 . Such role division may be defined in collaboration information between the drone 311 and the ground robot 313 .

More specifically, the ground robot 313 transports the delivery product 315 to the delivery destination 317 , the drone 311 captures the surrounding environment of the ground robot 313 , and obstacle information and terrain information through image analysis. , and provides monitoring information such as traffic information to the ground robot 315 . Then, the ground robot 315 may correct the movement path based on the received monitoring information. For example, the ground robot 315 may avoid obstacles or re-route its movement to a path where flat terrain continues. By doing so, a safer delivery service can be provided. As another example, the ground robot 315 may reset the movement path to a path in which traffic can avoid enemy obstacles. By doing so, an expedited delivery service can be provided.

21 is an exemplary diagram for explaining a cooperative delivery method according to a second embodiment of the present disclosure.

21, in the second embodiment, the ground robot 324 and the drone 326 are in charge of transport together. For example, the ground robot 324 is responsible for transporting the delivery product 325 from the origin 321 to the intermediate destination 322, and the drone 326 is the delivery product 325 from the intermediate destination 322 to the delivery destination 323. may be responsible for the transportation of In this case, information such as location information (ie, a handover place) and handover time of the intermediate destination 322 may be predefined in the collaboration information, or may be dynamically determined according to a situation. In order to dynamically collaborate, event information that causes collaboration (hereinafter, referred to as a "collaboration event") may be defined in the collaboration information.

In some embodiments, the collaboration event may occur when the ground robot 324 or the drone 326 is no longer able to proceed with transportation. For example, when an abnormality occurs in the robot, when it is no longer possible to move to a destination due to an obstacle, etc., a collaboration event may occur. In this case, the point where the collaboration event occurs may be the takeover location 322 . A case in which a collaboration event occurs due to an obstacle will be described later with reference to FIG. 22 .

In some embodiments, the collaboration event may be generated based on an accident risk. For information on the accident risk, refer to the description of FIG. 31 .

22 and 23 are exemplary views for explaining a collaborative delivery method according to a third embodiment of the present disclosure.

22 and 23 , in the third embodiment, the ground robot 331 cooperates with the drone 333 in response to recognizing the obstacle 337 existing on the movement path. Of course, like the first embodiment described above, the drone 333 may detect or identify an obstacle and provide obstacle information to the ground robot 331 .

In this embodiment, the ground robot 331 may determine whether the obstacle can pass based on the information on the obstacle 337 . Here, the obstacle passing may include a case of passing over an obstacle, a case of avoiding an obstacle, and the like. Also, as shown in FIG. 23 , the ground robot 331 may hand over the delivery product 335 to the drone 333 in response to a determination that it cannot pass through the obstacle 337 .

Here, the method of handing over the delivery product 335 may be any method. For example, the ground robot 331 may operate the drone 333 when an obstacle 337 appears while transporting the delivery product 335 with the drone 333 mounted thereon. Of course, the ground robot 331 may report that it cannot pass the obstacle 337 to the control system 230 , and the drone 333 may be operated under the control of the control system 230 . For another example, the ground robot 331 may take over the delivery product 335 by calling a nearby drone.

In some embodiments, the ground robot 331 may determine whether or not the obstacle can pass based on the recognized size of the obstacle 337 and the risk of damage to the delivery product 335 . In this case, the risk of damage refers to the degree to which the value of the article is damaged due to damage, and the higher the original value of the article (eg, an expensive article), the more easily breakable in nature (eg, an article of a material that is easily broken) ) and/or an item whose value is greatly depreciated due to damage may be determined as a higher value. For a more specific example, when a shipment with a high risk of breakage is being transported, even if the size of the recognized obstacle is small (that is, even if it is an obstacle that can be sufficiently passed due to the performance of the robot), the ground robot 331 may pass through the obstacle. You can decide that you can't. By doing so, a safer delivery service can be provided for items with a high risk of breakage.

24 and 25 are exemplary views for explaining a collaborative delivery method according to a fourth embodiment of the present disclosure.

24 and 25 , in the fourth embodiment, a door-to-door delivery service may be provided through a drone (e.g. 341) and a ground robot (e.g. 345 to 347). Specifically, air transport or roof-to-roof transport is performed through the drone 341, and indoor delivery is performed to the front of the user's door through the ground robot 345 placed on the roof (ie, handover place) of the delivery destination 348 . can be done

25 illustrates a process in which the delivery product 353 is delivered at the delivery location (e.g. the roof of the delivery destination building 348).

As shown in FIG. 25 , a marker 355 that can be identified by the drone 351 may be pre-installed at the handover location (eg rooftop), and the location and shape of the marker 355 may be pre-installed in the mission information. may be defined.

The drone 351 may identify the installed marker 355 through image capturing, and drop the delivery product 353 on the area where the marker 355 is installed. A safety device for preventing the delivery product 353 from being damaged may be pre-installed in the marker area. Next, the drone 351 transmits the release location information (ie, location information of the marker 355 area) to the ground robot 355 . Then, the ground robot 355 may move to the received position and take over the delivery product 353 .

For reference, when the target for collaboration is the first drone instead of the ground robot 355 , the drone 351 may transmit the location information of the marker area to the first drone. Then, the first drone may take over the delivered product 353 and transport it to the delivery destination. Of course, the first drone may also hand over the delivery product 353 to another ground robot.

In addition, the above-described embodiment can be applied as it is without changing the actual technical idea even in the case of performing collaboration between ground robots. For example, if the first ground robot places a delivery product in the marker area and transmits the location information of the marker area to the second ground robot, the second ground robot can take over the delivered product using the location information of the marker area there is.

The drop location information may be GPS information of the marker 355 area, image information of the marker 355 area, etc., but the technical scope of the present disclosure is not limited thereto.

26 is an exemplary diagram for explaining a cooperative delivery method according to a fifth embodiment of the present disclosure.

As shown in FIG. 26 , the fifth embodiment relates to a cooperative delivery method for providing a door-to-door delivery service. In the present embodiment, the ground robot 361 delivers the delivered product 363 to the drone 363 at the handover location 365 , and the drone 363 delivers the delivery to the user 369 .

As shown in FIG. 26 , the drone 367 may deliver the delivered product 363 by approaching from the delivery destination building 366 to the residence of the user 369 . In the mission information, height information (e.g. number of floors) of the residence may be defined.

In some embodiments, the drone 367 may provide a delivery notification to the user 369 by continuously generating a visual or audible message in the vicinity of the delivery destination 366 . Alternatively, the drone 367 and/or the service system 200 may provide a delivery notification by transmitting a message to the user's terminal 369 .

In some embodiments, the drone 367 may identify the user 369 in the manner shown in FIG. 19 , and provide an arrival notification only when the user 369 is identified.

27 is an exemplary view for explaining a cooperative delivery method according to a sixth embodiment of the present disclosure.

27 , in the sixth embodiment, the drone 375 provides a delivery service in cooperation with a transport vehicle 371 (eg, a delivery vehicle). 27 shows as an example that the transport vehicle 371 provides a delivery service in cooperation with the drone 375 , but the transport vehicle 371 cooperates with another delivery robot (eg, a ground robot) instead of the drone 375 . You may.

As shown in FIG. 27 , when the transport vehicle 371 transports the delivered product to the designated handover location 373 , the drone 375 may transport the delivered product to the delivery destination 377 . In some embodiments, the drone 375 may provide a door-to-door delivery service as shown in FIG. 26 .

In some embodiments, the delivery destination 377 may refer to a place that the transportation vehicle 371 cannot enter for various reasons. For example, the delivery destination 377 may be a transport vehicle 371 for geographic reasons (eg, where access roads are narrow or absent, mountainous areas, island areas, etc.) or for legal reasons (eg areas where vehicle access is restricted by law). This could mean an inaccessible place. Even in this case, in the present embodiment, delivery to the destination 377 may be made by using the drone 375 . Accordingly, the effect of greatly expanding the delivery range can be achieved.

So far, a cooperative delivery method according to various embodiments of the present invention has been described with reference to FIGS. 20 to 27 . According to the above-described method, a safe delivery service can be provided quickly through collaboration between a ground robot and a drone. In particular, even if the ground robot cannot reach the destination, the delivery range can be greatly expanded by performing the delivery to the destination through the drone. Furthermore, by providing a door-to-door delivery service using a ground robot and a drone, customer satisfaction can be greatly improved.

Hereinafter, with reference to FIGS. 28 and 29, a method for safely delivering a large item in cooperation with a plurality of drones will be described.

28 is an exemplary diagram for explaining a multi-drone-based cooperative delivery method according to the first embodiment of the present disclosure.

As shown in FIG. 28 , in the first embodiment, a plurality of drones 381 to 387 relates to a method for safely delivering a large delivery product 387 using an electric rope. The large delivery product 387 may mean an item having a weight greater than or equal to a threshold, but the technical scope of the present disclosure is not limited thereto.

In order for the large shipment 387 to be delivered safely, it is important that the center of gravity of the large shipment 387 is kept low while the drones 381 to 387 are moving (that is, it is important that the posture of the large shipment is maintained in a balanced way) do). To this end, each of the drones 381 to 385 estimates the posture of the large delivery product 387 based on various factors, and determines whether the estimated posture satisfies a predetermined balance condition. It is also possible to adjust the length of the electric rope or its relative position in response to a determination that the above balance condition is not satisfied.

In some embodiments, the factor may include the length of the powered rope, the relative position of the drone, the direction of the powered rope, and the like.

In some embodiments, the balancing condition may be, for example, a condition based on the degree of inclination of the large shipment 387 (e.g. parallel to the horizontal plane in the case of 0 degrees), but the technical scope of the present disclosure is not limited thereto. In addition, the balance condition may be strictly set as the risk of damage of the delivered product 399 increases, but the technical scope of the present disclosure is not limited thereto.

29 is an exemplary diagram for explaining a multi-drone-based cooperative delivery method according to a second embodiment of the present disclosure.

29, in the second embodiment, a specific drone 391 that does not participate in transportation operates as a coordinator. More specifically, the coordinator drone 391 shoots the drones 393 to 397 and the large delivery product 399 that transport the large delivery product 399, and analyzes the image to determine the posture of the large delivery product 399 and the transport drones. Measure the relative position of (393-397).

In addition, the coordinator drone 391 responds to the determination that the posture of the large delivery item 399 does not satisfy the predetermined balance condition for the predetermined balance condition, and in response to the determination that the posture of the large delivery item 399 is corrected, the specific drone (eg 395 ) ) to send an adjustment command. In this case, the adjustment command may include commands related to adjusting the position of the drone, adjusting the length of the rope, and adjusting the direction of the rope.

In this embodiment, the coordinator drone 391 may continuously monitor the transport drones 393 to 397 and the large delivery product 399 and continuously adjust the posture balance of the large delivery product 399 to be maintained. Accordingly, a safer delivery service may be provided.

So far, a method for safely delivering large items using a plurality of drones has been described with reference to FIGS. 28 and 29 . According to the above-described method, it is possible to deliver large items by using a plurality of drones, and a safe delivery service can be provided through posture correction.

Hereinafter, various embodiments of the present disclosure for enhancing customer satisfaction in providing a food delivery service will be described with reference to FIGS. 30 to 32 .

The following embodiments relate to a case in which the delivery product is food and drink. In such an embodiment, the service system 200 may receive a delivery request from the user terminal 400 (e.g. food order), and in response to the request, may generate mission information for delivering the food and beverage to the delivery destination. In addition, the mobile robot 300 (hereinafter referred to as a "delivery robot") for delivering food and beverages may perform food and beverage delivery based on the mission information. Hereinafter, various embodiments of the present disclosure will be described in detail with reference to FIGS. 30 to 32 .

30 is an exemplary view for explaining a food and beverage delivery method using a mobile robot according to the first embodiment of the present disclosure.

30 , in the first embodiment, the delivery robot 405 stores food and drink in the cooking appliance 407 and delivers it. The delivery robot 405 performs food and beverage delivery based on the task information generated by the service system 200 , wherein the task information includes a cooking time of the food and beverage.

In this embodiment, the delivery robot 405 moves from the origin 401 to the destination 403 and calculates the estimated required time to the destination 403 . Any method of calculating the expected required time may be used. In addition, the delivery robot 405 compares the cooking time of the food and beverage including the task information with the expected required time, and operates the cooking appliance 407 based on the comparison result. For example, the delivery robot 405 may operate the cooking appliance 407 when the expected required time is equal to the cooking time, so that food and drink are properly cooked when reaching the destination 403 . By doing so, an optimal food delivery service can be provided to the consumer.

If a separate cooking time is not specified, the delivery robot 405 may deliver the food while operating the cooking appliance 407 in a warm or cooled state so that the taste or freshness of the food can be maintained.

The cooking appliance 407 refers to an appliance equipped with various cooking functions. For example, the cooking device 70 may be a device provided with various cooking functions such as heating, cooling, frying functions, keeping warm, cooking, and oven. Accordingly, the delivery robot 405 uses the cooking device 407 to cook food while moving (eg, evacuate a pizza, fry a fry, cut a stew, cook rice, bake a pizza in an oven, etc.) ), and the consumer can receive freshly cooked food and drink.

Meanwhile, as the surrounding conditions (e.g. traffic volume, etc.) change, the actual required time to the destination 403 may be different from the expected required time. For example, the estimated required time may be longer or shorter than the actual time. In this case, the delivery robot 405 may adjust the cooking intensity (e.g., the intensity of heating, etc.) of the cooking appliance 407, so that the cooking completion time of the food and drink can be matched with the arrival time.

More specifically, the delivery robot 405 may periodically update the estimated required time to the delivery destination 403 and adjust the cooking intensity of the cooking appliance 407 according to the updated estimated required time. For example, when the updated estimated required time is longer than before, the delivery robot 405 may weakly adjust the cooking intensity of the cooking appliance 407 . In the opposite case, the delivery robot 405 may strongly adjust the cooking intensity of the cooking appliance 407 .

In some embodiments, the delivery robot 405 may monitor the cooking state of the stored food through a sensor provided in the cooking appliance 407 . For example, an image sensor, a temperature sensor, etc. may monitor the cooking state of the food. Also, the delivery robot 405 may adjust the cooking intensity of the cooking appliance 407 based on the monitored cooking state. For example, the cooking intensity may be strongly or weakly adjusted according to the cooking state.

In addition, in some embodiments, the delivery robot 405 may sense the surrounding environment, and calculate an accident risk for the current moving path based on the sensed data. In addition, the delivery robot 405 may adjust the movement route to the delivery destination 403 to another route in response to the determination that the calculated accident risk is greater than or equal to the threshold. This is because food and drink can be easily damaged unlike other delivered products, and by doing so, a safe delivery service to the destination 403 can be provided.

The threshold may be a preset fixed value or a variable value that varies according to circumstances. For example, the threshold may be a variation value determined based on the risk of damage to the stored food and drink. Here, the risk of damage refers to the degree to which the value of the food and beverage is damaged according to the damage, and the higher the original value of the food and drink (eg expensive food and drink), the more easily damaged in nature (eg pizza) Foods that are easily damaged, such as toppings) and/or food and beverages whose value is greatly depreciated due to damage may be determined to have a higher value. As a more specific example, when food and drink having a high risk of damage is being transported, the threshold is determined to be a smaller value, so that delivery can be performed through a safer route.

A specific method of calculating the accident risk may vary according to embodiments.

In some embodiments, the accident risk may be calculated based on at least one of a curvature of the road ahead and a slope of the road ahead. For example, the greater the curvature of the road or the higher the slope, the higher the accident risk may be calculated.

In some embodiments, the accident risk may be calculated based on a recognition result of an object located within a predetermined distance. The object recognition result may be obtained through image analysis. Here, the object recognition result may include the number of moving objects and the degree of irregular movement of the objects. For example, as the number of moving objects increases or the movement pattern of the moving objects becomes irregular, the accident risk may be determined as a high value.

In some embodiments, the accident risk may be calculated in a manner similar to the aforementioned collision risk and complexity of the surrounding environment. For a method of calculating the collision risk and the complexity of the surrounding environment, refer to the above description.

In some embodiments, the accident risk may be calculated based on a combination of the above-described embodiments.

Also, according to some embodiments of the present disclosure, in response to determining that the accident risk is greater than or equal to a threshold, the ground robot 405 may hand over the cooking appliance 407 to the drone. By doing so, a safe delivery can be made via the drone, and a door-to-door delivery service can be provided. Of course, depending on the embodiment, the drone may also take over the cooking appliance to the ground robot. For example, when flying is difficult due to a complex surrounding environment (ie, when the risk of an accident is high), the drone may hand over the cooking appliance to the ground robot.

31 and 32 are exemplary views for explaining a food and beverage delivery method using a mobile robot according to a second embodiment of the present disclosure.

31 and 32, the second embodiment relates to a method of providing a safe delivery service by adjusting the posture of a delivery container (e.g. cooking appliance) containing food and drink.

In the second embodiment, the delivery robots 411 and 421 adjust the posture of the delivery container during movement according to the first adjustment method when the type of food and drink is the first type, and the type of food and drink is the second In the case of the type, the posture of the delivery container may be adjusted during movement according to the second adjustment method. That is, it is possible to ensure safe delivery in an optimal posture according to the type of food and drink.

Here, the type of food and drink may be divided into liquid food such as soup and stew, and non-liquid food such as pizza and rice, but the technical scope of the present disclosure is not limited thereto.

31 illustrates a method of adjusting the posture of a delivery container with respect to liquid food.

As shown in FIG. 31 , when the delivery robot 411 accelerates traveling, an inclination is generated on the surface of the food and drink due to the reaction acceleration, and the liquid food and drink may overflow the delivery container 413 . In order to prevent such a problem, the delivery robot 411 may adjust the posture of the delivery container 413 in the direction in which the reaction acceleration is offset.

More specifically, the delivery robot 411 can measure the surface inclination of the food and drink in the delivery container 413, and adjust the posture of the delivery container 413 so that the surface inclination is gentle. The measurement of the surface inclination may be performed through image analysis, but the technical scope of the present disclosure is not limited thereto. For example, the delivery robot 411 may tilt the delivery container 413 so that the surface of the food and drink is parallel to the horizontal plane, as shown in Figure 31, when the surface inclination occurs due to the reaction acceleration. By doing so, it is possible to prevent the liquid food or drink from overflowing from the delivery container 413 .

In some embodiments, the delivery robot 411 may measure the traveling acceleration, and adjust the posture of the delivery container 413 based on the magnitude of the traveling acceleration. For example, the greater the running acceleration, the greater the reaction acceleration will be, so that the delivery container 413 is tilted more (that is, the food surface inside is parallel to the horizontal plane), the posture of the delivery container 413 can be adjusted .

For reference, FIG. 31 illustrates that the ground robot 411 accelerates traveling on a flat road as an example, but the above-described technical idea is when traveling on a road with a slope or curvature, or when the drone accelerates flying in the air. It should be noted that this can also be applied to In any case, the delivery robot may adjust the posture of the delivery container 413 so that the surface inclination of the food and drink contained in the delivery container 413 becomes gentle.

Figure 32 illustrates a method of adjusting the posture of the delivery container for non-liquid food.

32, the delivery robot 421 may measure the ground inclination, and adjust the posture of the delivery container 423 based on the ground inclination. For example, if you are moving the terrain with a high slope, the delivery robot 421 may adjust the posture of the delivery container 423 so that the delivery container 423 or the food inside is parallel to the horizontal plane. By doing so, safe delivery can be achieved while maintaining the appearance of the food (e.g. the toppings of the pizza do not run off).

For reference, FIG. 32 illustrates that the ground robot 421 has an up and down inclination direction as an example, but the above-described technical idea is a case in which the inclination direction is left or right or when traveling on a road with curvature, or when the drone accelerates flight in the air. It should be noted that it can be applied even if In any case, the delivery robot may adjust the posture of the delivery container 413 to keep the posture of the delivery container 413 parallel to the horizontal plane.

So far, various embodiments of the present disclosure related to a food delivery service have been described with reference to FIGS. 30 to 32 . According to the above description, by performing cooking while moving, the food and drink in a freshly cooked state can be delivered to the consumer. Accordingly, customer satisfaction can be improved. Furthermore, by adjusting the posture of the delivery container according to the type of food and drink, a safe delivery service can be provided. Accordingly, customer satisfaction may be further improved.

Hereinafter, an exemplary computing device 500 that may implement the devices eg 100 , 210 , and 300 and/or the systems eg 230 and 250 according to various embodiments of the present disclosure will be described with reference to FIG. 33 . let it do

33 is an exemplary hardware configuration diagram illustrating the exemplary computing device 500 .

33 , the computing device 500 loads one or more processors 510 , a bus 550 , a communication interface 570 , and a computer program 591 executed by the processor 510 . It may include a memory 530 and a storage 590 for storing the computer program (591). However, only components related to the embodiment of the present disclosure are illustrated in FIG. 33 . Accordingly, those skilled in the art to which the present disclosure pertains can see that other general-purpose components other than the components shown in FIG. 33 may be further included.

The processor 510 controls the overall operation of each component of the computing device 500 . The processor 510 includes a central processing unit (CPU), a micro processor unit (MPU), a micro controller unit (MCU), a graphic processing unit (GPU), or any type of processor well known in the art of the present disclosure. can be Also, the processor 510 may perform an operation on at least one application or program for executing the method according to the embodiments of the present disclosure. Computing device 500 may include one or more processors.

The memory 530 stores various data, commands, and/or information. The memory 530 may load the computer program 591 from the storage 590 to perform methods/operations according to various embodiments of the present disclosure. The memory 530 may be implemented as a volatile memory such as RAM, but the technical scope of the present disclosure is not limited thereto.

The bus 550 provides communication between components of the computing device 500 . The bus 550 may be implemented as various types of buses, such as an address bus, a data bus, and a control bus.

The communication interface 570 supports wired/wireless Internet communication of the computing device 500 . In addition, the communication interface 570 may support various communication methods other than Internet communication. To this end, the communication interface 570 may be configured to include a communication module well known in the technical field of the present disclosure.

The storage 590 may non-temporarily store one or more programs 591 . The storage 590 is a non-volatile memory, such as a read only memory (ROM), an erasable programmable ROM (EPROM), an electrically erasable programmable ROM (EEPROM), a flash memory, a hard disk, a removable disk, or well in the art to which the present disclosure pertains. It may be configured to include any known computer-readable recording medium.

The computer program 591, when loaded into the memory 530, may include one or more instructions that cause the processor 510 to perform an operation/method according to various embodiments of the present disclosure. The processor 510 may perform the operations/methods according to various embodiments of the present disclosure by executing the one or more instructions.

For example, the computer program 591 performs an operation of obtaining a remote control value for the mobile robot 30 input through the remote control system 20 , and an autonomous driving module for the mobile robot 30 generated by the autonomous driving module. An operation of obtaining a control value, an operation of determining a weight for each control value based on a delay between the mobile robot 30 and the remote control system 20, and the determined weight, the first control value, and the second control value It may include instructions for performing an operation of generating a target control value of the mobile robot 30 based on the . In this case, the control device 100 may be implemented through the computing device 500 .

So far, an exemplary computing device 500 capable of implementing devices and/or systems according to various embodiments of the present invention has been described with reference to FIG. 33 .

So far, some embodiments of the present disclosure and effects according to the embodiments have been described with reference to FIGS. 1 to 33 . Effects of the present disclosure are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

The concepts of the present disclosure described with reference to FIGS. 1 to 33 so far may be implemented as computer-readable codes on a computer-readable medium. The computer-readable recording medium may be, for example, a removable recording medium (CD, DVD, Blu-ray disk, USB storage device, removable hard disk) or a fixed recording medium (ROM, RAM, computer-equipped hard disk). can The computer program recorded in the computer-readable recording medium may be transmitted to another computing device through a network, such as the Internet, and installed in the other computing device, thereby being used in the other computing device.

In the above, even if all the components constituting the embodiment of the present disclosure are described as being combined or operating in combination, the present disclosure is not necessarily limited to this embodiment. That is, within the scope of the object of the present disclosure, all of the components may operate by selectively combining one or more.

Although acts are shown in a specific order in the drawings, it should not be understood that the acts must be performed in the specific order or sequential order shown, or that all shown acts must be performed to obtain a desired result. In certain circumstances, multitasking and parallel processing may be advantageous. Moreover, the separation of the various components in the embodiments described above should not be construed as necessarily requiring such separation, and the program components and systems described may generally be integrated together into a single software product or packaged into multiple software products. It should be understood that there is

Although embodiments of the present disclosure have been described above with reference to the accompanying drawings, those of ordinary skill in the art to which the present disclosure pertains may realize that the present disclosure may be embodied in other specific forms without changing its technical spirit or essential features. can understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive. The protection scope of the present disclosure should be construed by the following claims, and all technical ideas within the scope equivalent thereto should be construed as being included in the scope of the present disclosure.

Claims (24)

a service system for generating mission information for delivering goods to a delivery destination;
a ground robot for transporting the goods based on the mission information; and
Including a drone that transports the goods in cooperation with the ground robot,
The mission information is
Including collaboration information for the ground robot and the drone to collaborate,
The collaboration information includes a place of delivery of the article,
The drone is
Recognizing a marker located in the vicinity of the handing over place, and positioning the article in the recognized marker area,
The ground robot is
receiving the goods located in the marker area, and transporting the goods to the delivery destination;
The drone transmits, to the ground robot, the information on the dropping position of the article, which is generated according to the image information photographed with the marker,
The ground robot moves to the drop location information and takes over the article,
The drone includes a plurality of drones,
The plurality of drones,
The article is transported using a rope connected to the article,
A first drone among the plurality of drones,
Estimate the posture of the article based on the relative position with other drones and the length of the current rope, and in response to determining that the estimated posture does not satisfy a predetermined balance condition, the length of the rope or the length of the first drone adjust the relative position,
The predetermined equilibrium condition is,
Characterized in the condition of adjusting the height of the center of gravity according to the size of the article,
Collaborative delivery system using drones and ground robots. According to claim 1,
The service system is
Receiving a delivery request for the article from a user terminal, monitoring the location of a ground robot or drone that delivers the article, and providing the monitored location to the user terminal,
Collaborative delivery system using drones and ground robots. According to claim 1,
The task information includes identification information of a user who ordered the item,
The ground robot or the drone,
identifying the user in the vicinity of the delivery destination using the identification information, and providing a notification message to the user in response to determining that the user has been identified,
Collaborative delivery system using drones and ground robots. delete delete According to claim 1,
The ground robot is
Recognizing an obstacle existing on the movement path to the destination, and in response to a determination that the recognized obstacle cannot be passed through, handing over the article to the drone,
Collaborative delivery system using drones and ground robots. 7. The method of claim 6,
The ground robot is
Characterized in that it is determined whether the recognized obstacle can pass based on the size of the recognized obstacle and the risk of damage to the article,
Collaborative delivery system using drones and ground robots. delete According to claim 1,
The drone includes a first plurality of drones for transporting the article using a rope and a second drone for photographing the article and the first plurality of drones,
The second drone,
The posture of the article is measured by analyzing the image generated through the photographing, and in response to determining that the measured posture does not satisfy a predetermined balance condition, the posture of the article is given to at least some of the first plurality of drones Characterized in sending a control command to correct the,
Collaborative delivery system using drones and ground robots. According to claim 1,
The service system further comprises a remote control system for remotely controlling the ground robot,
The target control value for controlling the ground robot is,
It is determined based on a weighted sum of the first control value input through the remote control system and the second control value generated by the autonomous driving module of the ground robot,
A first weight applied to the first control value and a second weight applied to the second control value are determined based on a communication delay between the remote control system and the ground robot,
The first weight and the second weight are characterized in that inversely proportional relationship,
Collaborative delivery system using drones and ground robots. delete delete delete delete delete delete delete delete delete delete delete delete delete delete

Images