WO2023187859A1

WO2023187859A1 - Device and method for controlling operation of autonomous mobile robot

Info

Publication number: WO2023187859A1
Application number: PCT/JP2022/014818
Authority: WO
Inventors: 克希小林; 宏治田中; 国郎成政
Original assignee: 三菱電機株式会社
Priority date: 2022-03-28
Filing date: 2022-03-28
Publication date: 2023-10-05
Also published as: JP7400998B1; JPWO2023187859A1

Abstract

Provided are a device and a method for controlling an operation of an autonomous mobile robot (AMR), which is free from reduction of convenience even when the ARM goes around in an area in which there are mixedly present a person requesting a service for the AMR and a person requesting no service.　The device for controlling an operation of an autonomous mobile robot for providing a service to a person comprises: a request determination unit that determines the degree of a service request from the person to the autonomous mobile robot by analyzing the behavior of the person using surrounding information for the autonomous mobile robot, and sets a proximate range of the autonomous mobile robot with respect to the person in accordance with the degree of the service request; and an operation determination unit that determines whether the position of the person is within the proximity range, and determines stoppage or deceleration of the autonomous mobile robot.

Description

Autonomous mobile robot motion control device and method

The present invention relates to an operation control device and method for an autonomous mobile robot that can improve the convenience of the autonomous mobile robot.

The use of autonomous mobile robots (hereinafter referred to as AMR) that autonomously move within an area and provide various services is becoming widespread. For example, it is known that an AMR is equipped with a trash can and assists people who visit the area to throw away trash.

The conventional AMR technology includes a person discrimination means for determining whether a detected obstacle is a person or not, and when it is determined that a person is present, the traveling speed is different from that for a simple obstacle other than a person (for example, An AMR that travels by reducing the traveling speed has been disclosed (for example, Patent Document 1).

Patent No. 4645108

However, the method described in Patent Document 1 has the following problems. In this prior art, the traveling speed of the AMR is changed only by determining whether there is a person or not. Therefore, since AMR performs the same operation for all people, it is not possible to limit the operation to those who request services from AMR (for example, people who have a request to throw away garbage), and AMR becomes less convenient.

The purpose of the present disclosure is to provide an AMR operation control device and method that does not reduce convenience even if there are people who request services from AMR and people who do not in an area. .

The operation control device for an autonomous mobile robot according to the present disclosure includes:
An operation control device for an autonomous mobile robot for providing a service to a person,
Determine whether a person has a request for a service for an autonomous mobile robot by analyzing the person's behavior using surrounding information of the autonomous mobile robot,
a request determination unit that sets the proximity range of the autonomous mobile robot to the person depending on whether there is a request for service;
and a motion determining unit that determines whether the position of the person is within the proximity range and determines whether the autonomous mobile robot should decelerate or stop.

The autonomous mobile robot operation control method according to the present disclosure includes:
A method for controlling the operation of an autonomous mobile robot for providing a service to a person, the method comprising:
a request determination unit determines whether a person has a request for a service for an autonomous mobile robot by analyzing the behavior of the person using surrounding information of the autonomous mobile robot;
The proximity range of the autonomous mobile robot to the person is set depending on the presence or absence of a service request.
The motion determining unit determines whether the position of the person is within the proximity range, and determines whether the autonomous mobile robot should decelerate or stop.

According to the present disclosure, even if there are people who request services from AMR and people who do not coexist in the area, AMR will not change its operation at unnecessary timing, so convenience will not decrease. This has the advantage of being able to provide an AMR operation control device and method that do not require the use of AMR.

1 is a functional block diagram showing the overall configuration of an AMR operation control device in Embodiment 1. FIG. FIG. 2 is a configuration diagram of hardware included in the AMR operation control device in the first embodiment. 5 is a flowchart showing the operation of the AMR operation control device in the first embodiment. 5 is a flowchart representing internal processing of a request determination unit in the first embodiment. 5 is a flowchart showing internal processing of the operation determining unit in the first embodiment. 1 is an example (part 1) of a specific operation of the AMR operation control device in the first embodiment; 3 is an example (part 2) of a specific operation of the AMR operation control device in the first embodiment; FIG. 2 is a functional block diagram showing the overall configuration of an AMR operation control device in a second embodiment. 7 is a flowchart showing the operation of the AMR operation control device in Embodiment 2. FIG. 7 is an example of a specific operation of the AMR operation control device in the second embodiment.

In the description of the embodiments and the drawings, the same elements and corresponding elements are denoted by the same reference numerals. Descriptions of elements labeled with the same reference numerals will be omitted or simplified as appropriate. In the following embodiments, "unit" may be read as "circuit", "process", "procedure", or "process" as appropriate.

Embodiment 1.
<Configuration>
The AMR operation control device in Embodiment 1 will be explained using FIGS. 1 to 7.

FIG. 1 is a functional block diagram showing the overall configuration of an AMR operation control device in the first embodiment. In FIG. 1, the AMR 100 includes an input section 1, a control section 5, and an operation control device 200. The motion control device 200 includes a person detection section 2, a request determination section 3, proximity range setting data 4, a motion determination section 5, and a control section 6.

In this embodiment, an example will be described in which the AMR 100 is equipped with a trash can and supports people who visit the area to throw away trash. Areas are, for example, facilities such as tourist spots, theme parks, shopping malls, stadiums, hotels, and factories. People who visit the area are, for example, tourists, employees, workers, etc., and are referred to as "persons." Hereinafter, when describing the general operation, structure, and configuration of the AMR 100, the number may be omitted and simply referred to as "AMR".

The AMR 100 autonomously moves within an area in order to provide a service (in this embodiment, assistance with trash disposal) to a person. The AMR 100 is, for example, a cart-type robot, a drone-type robot, or the like. The AMR 100 includes a trash can (not shown) and various moving devices (not shown) for running or flying within the area, and also includes a housing for housing the main components of the AMR. Examples of the various moving devices include drive devices such as motors and engines, steering mechanisms, driven devices such as wheels and rotary blades, and power sources such as batteries and fuel.

The AMR 100 includes various sensors (not shown) for acquiring self information of the AMR 100 and information about the surroundings of the AMR 100. Various sensors include, for example, image sensors such as cameras and infrared sensors, positioning sensors such as GNSS (Global Navigation Satellite System) and IMU (Inerial Measurement Unit), microphones, acoustic sensors such as vibration pickups, ultrasonic sensors, and millimeter wave radars. , a distance measurement sensor such as LiDAR (Light Detection and Ranging), a wireless LAN (Local Area Network), a wireless communication device such as Bluetooth (registered trademark), and the like.

The input unit 1 uses various sensors included in the AMR 100 to acquire self information of the AMR 100 and information about the surroundings of the AMR 100. The self-information of the AMR 100 includes, for example, its own position (latitude/longitude, altitude, relative coordinates from the starting point, etc.), moving speed, remaining amount of power source, and the like. The surrounding information of the AMR 100 includes, for example, video data captured around the AMR 100 using an image sensor, audio data collected from the surroundings of the AMR 100 using a microphone, sensing data obtained from sensing the surroundings of the AMR 100 using a millimeter wave radar, LiDAR, etc., Bluetooth ( (registered trademark) and other communication data with surrounding terminals.
Note that the various sensors for acquiring the self information of the AMR 100 and the surrounding information of the AMR 100 do not need to be included in the AMR 100. For example, regarding video data, the input unit 1 may acquire data from a device attached to the outside of the AMR 100, such as a surveillance camera. A device attached to the outside of the AMR 100 can detect a person who is outside the detection range of the various sensors provided in the AMR, and can expand the action range of the AMR.

The person detection unit 2 uses the information around the AMR 100 obtained from the input unit 1 to detect people around the AMR 100 within a predetermined detection range. The detected person is output as person information. The predetermined detection range is a detection range of a person. For example, the detection range of a person is a predetermined area surrounding the AMR 100, but is not limited thereto. For example, the detection range of a person may be a predetermined area scattered around the AMR 100, or may be determined by the detection performance of various sensors. Note that the width (or narrowness) of the detection range of a person may be set based on the distance from the AMR 100. In this case, since the distance can be defined as a relative value with the AMR 100 as the center (origin), the detection range can be set without using the self-information (for example, self-position) of the AMR 100.

For the detection of a person in the person detection unit 2, for example, a method of performing image recognition using video data from a camera can be used. Furthermore, a method of combining a plurality of pieces of information in the surrounding information of the AMR 100 may be used to detect a person. For example, if an object and its position can be detected using sensing data, and speech from that position can be obtained using audio data, the object may be detected as a person. Note that the person detection unit 2 may detect obstacles other than people (for example, objects that obstruct the movement of the AMR, such as other AMRs and facility structures).

Subsequently, the person detection unit 2 associates the detected person with surrounding information of the AMR 100. For example, a unique ID is given to each detected person, and surrounding information of the AMR 100 related to the person, such as video data, audio data, and sensing data within a range that includes the person, is linked to the person's ID. Note that the person detection unit 2 may also store obstacles other than people in association with the surrounding information of the AMR 100. A person's ID can be included in the person information.

Furthermore, the person detection unit 2 may estimate the physical characteristics of the person (for example, height, physique, gender, age, etc.). For example, a person's physical characteristics can be estimated by image recognition of video data of surrounding information of the AMR 100 that can be obtained from the input unit 1. The estimated physical characteristics of the person can be included in the person information together with the person's ID.

The request determination unit 3 analyzes the behavior of each person detected by the person detection unit 2 using surrounding information of the AMR 100 (for example, video data of the person). Then, based on the analysis result, it is determined whether there is a request for service for the AMR 100. A person's behavior is a preliminary movement or action before a person takes a certain action, and includes, for example, gestures, gestures, attitude, line of sight movement, facial expressions, words and actions, and the like. In other words, a request for a service for the AMR 100 is a request from a person to receive a service from the AMR, and in this embodiment, a request for a person to throw garbage into a trash can installed in the AMR 100. . Note that for obstacles other than people, it is determined that there is no request for service to the AMR 100.

As a method for determining the demand for the AMR 100, for example, it is possible to use pattern recognition using machine learning, rule-based processing, etc., or a classification method using the surrounding information of the AMR 100 linked to the person information ID by the person detection unit 2. be. As a judgment method using machine learning, for example, a trained model created using deep learning, reinforcement learning, or a neural network can be used. Note that the following method is possible as a method for creating a trained model. For example, using video data of the surrounding information of AMR100, an image of a person who wants to throw away trash (for example, a person taking out trash from a bag, a person looking around to throw away trash, etc.) Machine learning may be performed using learning data that is given as input data and that is given a correct answer label depending on whether there is a request for service for the AMR 100 or not. Note that the correct answer label may be assigned not only to the presence or absence of a service request but also to the strength (or weakness) of the service request. Further, in the rule-based processing, the judgment results may be manually classified heuristically, for example, using the above-mentioned labeled learning data.
Hereinafter, a service request for the AMR 100 may be simply referred to as a "service request."

Subsequently, the request determination unit 4 sets the proximity range according to the determination result of whether or not there is a request for the service. For the proximity range, a separate value is set for each detected person. Here, the proximity range is, for example, a predetermined area surrounding the AMR 100, and when a person enters the area, the AMR changes its operation.

The proximity range is not limited to a predetermined area surrounding the AMR. For example, it may be predetermined areas scattered around the AMR 100, or may be determined by the detection performance of various sensors. Note that the width (or narrowness) of the proximity range may be set based on the distance from the AMR 100. In this case, the proximity range can be defined as a relative value with the AMR 100 as the center (origin), so the proximity range can be set without using the self-information (for example, self-position) of the AMR 100. Note that separate proximity ranges may be set for each obstacle other than a person.

The proximity range may be set to individual values depending on the physical characteristics of the person. For example, since there is a physical difference between adults and children, it is desirable that the proximity ranges are different. Furthermore, since the walking speeds of young people and elderly people are different, it is desirable that the proximity ranges be different. Furthermore, it is desirable that the proximity ranges are different for pedestrians and wheelchairs. Note that a person's physical characteristics can be acquired from person information.

The proximity range setting data 4 holds proximity range data set according to the result determined by the request determination unit 3.

The operation determining unit 5 determines whether the position of each person is within the proximity range using the proximity range set according to the determination result of whether each person has a request for service. Then, the operation of the AMR 100 is determined based on the determination result. Then, according to the determined operation, a control signal for controlling the operation of the AMR 100 is output.

Whether or not each person's position is within the proximity range can be determined from the surrounding information of the AMR 100 obtained from the input unit 1. For example, it is possible to use a method of estimating the distance from the person to the AMR 100 by using image information of the person included in video data of surrounding information of the AMR 100 linked to the ID of the person. Then, using the estimated distance, it can be determined whether the distance from each person to the AMR 100 is within the proximity range. Note that it is possible to determine whether or not each person's position is within the proximity range by using both the self-information of the AMR 100 (for example, the coordinates of its own position) and the surrounding information of the AMR 100 (for example, the coordinates of the person's position obtained from sensing data). You may be judged.

The control unit 6 controls various mobile devices of the AMR 100 according to the control signals output by the operation determining unit 5. For example, according to the control signal, the steering mechanism and brakes of the AMR 100 are controlled, the rotation speed of the drive device, etc. are controlled, and the AMR 100 autonomously moves within the area.

<Hardware configuration>
Each configuration of the operation control device 200 of the AMR 100 shown in FIG. 1 can be realized by, for example, a computer that is an information processing device with a built-in processor. FIG. 2 is a configuration diagram of hardware included in the operation control device 200 of the AMR 100 in the first embodiment. In FIG. 2, the operation control device 200 includes a processor 300, a volatile storage device 301, a nonvolatile storage device 302, and a signal path 303. Further, an input device 304 and a control device 305 are connected to each component of the operation control device 200 via a signal path 303.

The computer incorporating the processor 300 is, for example, a microcomputer for device built-in use, a SoC (System on Chip), or the like. Alternatively, it may be a portable computer such as a smartphone or a tablet computer, or a stationary computer such as a personal computer or a server computer.

The processor 300 controls the entire operation control device 200. For example, the processor 300 is a CPU (Central Processing Unit), an FPGA (Field Programmable Gate Array), a DSP (Digital Signal Processor), or the like. Processor 300 may be a single processor or multiple processors. Furthermore, the operation control device 200 may include a processing circuit such as an ASIC (Application Specific Integrated Circuit) in addition to the computer. The processing circuit may be a single circuit or a composite circuit.

The volatile storage device 301 is the main storage device of the operation control device 200. For example, the volatile storage device 301 holds data necessary for the operation of the processor 300. For example, the volatile storage device 301 is a RAM (Random Access Memory).

The nonvolatile storage device 302 is an auxiliary storage device of the operation control device 200. For example, the nonvolatile storage device 302 stores initial value data, programs, error logs, etc. necessary for the operation of the operation control device 200. Further, the nonvolatile storage device 302 may store proximity range setting data 4. For example, the nonvolatile storage device 302 is a ROM (Read Only Memory), an HDD (Hard Disk Drive), or an SSD (Solid State Drive).

The signal path 303 is a transmission path (bus) for transmitting and receiving data between each component shown in FIG.

The input device 304 is an input interface of the operation control device 200, which has the function of the input section 1. For example, the input device 304 is a device that inputs information necessary for controlling the AMR 100, including self information of the AMR 100 and surrounding information of the AMR 100, and an execution command for control processing of the AMR 100 to the processor 300.

The control device 305 is a device that has the functions of the control unit 6 and controls operations such as movement, stopping, and direction change of the AMR 100. For example, the control device 305 controls the steering mechanism and brakes, which are various moving devices of the AMR 100, and also controls the rotation speed of a drive device such as a motor.

Processor 300 uses volatile storage 301 (e.g., RAM) as working memory and stores computer programs (i.e., operating control program). Note that the operation control program may be supplied from outside the operation control device 200 through the input device 304. Further, the operation control program may be distributed in a computer-readable nonvolatile storage medium (eg, a CD (Compact Disc), a DVD (Digital Versatile Disc), a flash memory, etc.).

Note that the operation control device 200 may be installed inside the AMR 100, but is not limited to this, and may be configured by a computer or the like located at a separate location from the AMR 100. In this case, the self information of the AMR 100 and the surrounding information of the AMR 100 may be input to the operation control device 200 on the computer by data transmission via a communication network connected to the input device 304, for example. Further, the control information generated by the operation control device 200 on the computer is sent to the control device 305 of the AMR 100 via a communication network, for example, as in the case of the self information of the AMR 100 and the surrounding information of the AMR 100. good. Note that the communication network is a wired or wireless network, such as a dedicated line such as a LAN (Local Area Network), the Internet, or the like.

<Processing operation>
Next, the operation of the AMR operation control device in the first embodiment will be explained. FIG. 3 is a flowchart showing the operation of the AMR operation control device in the first embodiment.

In step ST1, the input unit 1 acquires the self information of the AMR 100 and the surrounding information of the AMR 100 (step ST1).

In step ST2, the person detection unit 2 uses the surrounding information of the AMR 100 acquired in step ST1 to detect people and non-person obstacles existing within a predetermined detection range (step ST2). Note that in the following description, obstacles other than people are included in people who do not have a request for service.

In step ST3, the person detection unit 2 branches the process depending on whether one or more people have been detected in step ST2. If one or more people are detected (Yes in step ST3), the process moves to step ST4, and it is determined whether or not the person has a service request. Subsequently, the process moves to step ST5, where it is determined whether or not the position of the person is within the proximity range, and the operation of the AMR 100 is determined. On the other hand, if no person is detected (No in step ST3), since there is no person around the AMR 100, the AMR 100 determines that there is no need to stop or decelerate, and proceeds to step ST6, where appropriate Continue patrolling.

In step ST4, the request determination unit 3 analyzes the behavior of each person detected in step ST2 using the surrounding information of the AMR 100. Then, based on the analysis result, it is determined whether there is a request for service or not, and an individual proximity range is set for each person depending on whether there is a request for service (step ST4).

Here, detailed processing contents in step ST4 will be explained.
FIG. 4 is a flowchart showing the internal processing of the request determination unit 3.

First, in step ST401, one person is selected from among the detected persons for whom the process of determining whether or not there is a service request has not been completed (step ST401).

Next, in step ST402, the behavior of the person selected in step ST401 is analyzed, and based on the analysis result, it is determined whether there is a request for service (step ST402). As a method for determining the presence or absence of a service request, for example, a determination method using a learned model by machine learning, a rule-based determination method, or A determination method that combines machine learning and rule-based methods is used to determine whether there is a request for a service (for example, whether there is a request to throw garbage into an AMR trash can).

Here, as a method for determining whether there is a request for a service, for example, a method of analyzing a person's behavior using an image recognition method using video data can be used. Specifically, for example, when a person picks up trash, such as holding trash or trying to take trash out of a bag, making a gesture such as waving at the AMR100, or moving one's gaze such as staring at the AMR100. Recognize and judge that you are making a gesture with the intention of throwing it away.

Additionally, as a method for determining whether there is a request for a service, a method of analyzing a person's behavior using a voice recognition method using voice data, for example, can be used. Specifically, for example, when recognizing an utterance that prompts the AMR100 to stop, such as "stop" or "where is the trash can," or an utterance that asks the AMR100 to look for a trash can, the person who made the utterance wants to throw out the trash. It is determined that there is a demand for this.

Further, as a method for determining whether there is a request for a service, for example, a method of analyzing a person's behavior using sensing data obtained from a ranging sensor can be used. Specifically, for example, if the relative speed between the person and the AMR 100 is above a certain level, and the person is coming toward the AMR 100 (the distance is getting closer), the person wishes to throw away trash. It is determined that there is.

Furthermore, as a method for determining whether or not a person has a request for a service, for example, the person's request may be directly input using a terminal carried by the person (for example, a smartphone, a tablet terminal, etc.). In this case, the act of inputting data into a terminal carried by a person can be considered as the behavior of the person. Specifically, if a person visiting the area has previously entered a request to throw away trash into AMR100 using application software installed on a mobile device, then the person who is visiting the area will be asked to throw away trash. It is determined that there is a demand for this.

Note that the method for determining whether there is a request for a service described above is an example, and the method for determining whether there is a request for a service is not limited to this. Furthermore, the presence or absence of a service request may be comprehensively determined by combining a plurality of determination methods.

If it is determined in step ST402 that the person has a request for service (Yes in step ST402), then in the subsequent step ST403, the proximity range of the person is set as if there is a request for service (step ST403). On the other hand, if it is determined that there is no request for service (No in step ST402), the proximity range of the person is set in step ST404 as if there is no request for service (step ST404).

The proximity range is set to a preset value with reference to the proximity range setting data 4. Further, the proximity range is set to a longer distance when there is a request for service than when there is no request for service. Specifically, for example, the proximity range when there is a request for service can be set to 5 (m), and the proximity range when there is no request for service can be set to 3 (m).

Next, in step ST405, the process of determining whether or not the person selected in step ST401 has a service request is completed (step ST405).

Next, in step ST406, it is checked whether there are still people among the people detected in step ST2 whose determination of whether or not they have a service request has not yet been completed (step ST406). If there is an unfinished person (Yes in step ST406), the process returns to step ST401 again, and steps ST401 to ST405 are sequentially executed. If there is no incomplete person (No in step ST406), processing step ST4 of the request determining unit 3 in the flowchart of FIG. 4 is ended (END).

Returning to the overall flowchart in FIG.
In step ST5, the operation determining unit 5 determines whether the position of each person is within the proximity range using the proximity range set according to the determination result of whether each person has a request for service obtained in step ST4. Based on the determination result, the operation of the AMR 100 is determined (step ST5). Hereinafter, the determination as to whether or not the object is within the proximity range will be abbreviated as "proximity determination."

Here, detailed processing contents in step ST5 will be explained.
FIG. 5 is a flowchart showing the internal processing of the motion determining section 5. As shown in FIG.

First, in step ST501, one person whose proximity determination has not been completed is selected from among the detected persons (step ST501).

Next, in step ST502, with respect to the person selected in step ST501, the proximity range set in step ST4 and the surrounding information of the AMR 100 linked to the person's ID in step ST2 are referred to, and the position of the person is determined in the proximity range. Determine whether it is within the range. In other words, it is determined whether the distance from the person to the AMR 100 is within the proximity range.

Here, as a method of determining whether the distance from the person to the AMR 100 is within the proximity range, for example, a method of directly measuring the distance to the person using sensing data obtained from a distance measurement sensor can be used.

In addition, as a method for determining whether the distance from a person to the AMR 100 is within the proximity range, for example, a method may be used that uses a Bluetooth (registered trademark) communication function with a mobile terminal held by the person and measures the radio field strength. I can do it.

In addition, as a method of determining whether the distance from a person to the AMR 100 is within the proximity range, for example, the current position of the AMR 100 obtained from the self-information of the AMR 100 can be used by using GPS (Global Positioning System) information of a mobile terminal held by the person. The distance can be calculated using the relative coordinates.

Furthermore, as a method for determining whether the distance from the person to the AMR 100 is within the proximity range, for example, a method can be used that uses image information of the person included in the video data of the camera to estimate the distance to the person. .

The method for determining the distance from the person to the AMR 100 described above is an example, and the method for determining the distance from the person to the AMR 100 is not limited to this. Furthermore, the distance from the person to the AMR 100 may be comprehensively determined by combining a plurality of distance determination methods.

In step ST502, if it is determined that the position of the person is within the proximity range (Yes in step ST502), the process moves to step ST503, and control is performed to stop or decelerate the movement of the AMR 100 (step ST503). Thereafter, even if there is a person whose proximity judgment has not been completed, the process of stopping or decelerating remains unchanged, and therefore the process of the motion determining unit 5 in step ST5 ends (END).

On the other hand, if it is determined in step ST502 that the person is not within the proximity range (No in step ST502), the process moves to step ST504, and the proximity determination is completed for the person selected in step ST501 (step ST504).

Next, in step ST505, it is checked whether there are any persons whose proximity determination has not yet been completed among the persons detected in step ST2. If there is an unfinished person (Yes in step ST505), the process returns to step ST501 again, and each process from step ST501 to step ST504 is sequentially performed. If there is no unfinished person (No in step ST505), this is a case where a person is detected around the AMR 100 but there is no one within the proximity range, so the process moves to step ST506 and the AMR 100 is operated at the normal speed. resume movement. This completes the process of the motion determining unit 5 in step ST5 (END).

Returning to the overall flowchart in FIG.
In step ST6, the control unit 6 controls the AMR 100 based on the control signal generated by the operation determining unit 5 in step ST5 (step ST6). After completing the process in step ST6 (END), the process returns to the beginning (START) of the overall flowchart and continues each process from step ST1 to step ST6.

FIGS. 6 and 7 show an example of a specific operation of the AMR operation control device in the first embodiment. To simplify the explanation, it is assumed that there is one AMR and three people in the examples of FIGS. 6 and 7. In FIG. 6(a), the AMR 100a is patrolling within the area. A person M1 (hereinafter referred to as person M1) who has no desire to throw away trash is walking near the AMR 100a. A person M2a (hereinafter, person M2a) who has a desire to throw away garbage and a person M2b (hereinafter, person M2b) who has a desire to throw away garbage each hold a bottle as garbage and use the AMR100a while searching for a garbage can. walking near.

In FIG. 6(b), the proximity range E1 on the side closer to the AMR 100a is an area within a circle with a radius d ₁ (m) centered on the AMR 100a. The proximity range E2 on the side far from the AMR 100a is similarly an area within a circle with a radius d ₂ (m) centered on the AMR 100a. The proximity range E2 is a wider area than the proximity range E1 (ie, radius d ₂ >radius d ₂ ). The values of the proximity range E1 and the proximity range E2 (that is, the distance from the AMR 100a) are each stored in advance in the proximity range setting data 4. The human detection range E3 is an area within a circle with a radius d ₃ (m) centered on the AMR 100a. The detection range E3 of the person is a wider area than the proximity range E2 (that is, radius d ₃ >radius d ₂ ), and is set as a predetermined constant value.

The proximity range E1 is the proximity range when there is no request for service. Further, the proximity range E1 is also a proximity range for the safety of all persons. The proximity range E2 is a proximity range when there is a request for service. The proximity range E2 is set to a distance where the distance from the AMR to the person is not too short and allows the person to easily throw away trash at an early timing. In either case, when a target person enters the proximity area, the AMR 100a stops or decelerates.

Returning to the explanation of FIG. 6(a). A person M1, a person M2a, and a person M2b existing within the person detection range E3 are detected from the surrounding information of the AMR 100a acquired by various sensors of the AMR 100a.

Subsequently, based on the video data included in the AMR100 information, it is determined that the person M1 has no service request (no desire to throw away trash). On the other hand, person M2a and person M2b are determined to be people who have a request for service (have a request to throw away trash). Therefore, a proximity range E1 is set for the person M1, and a proximity range E2 is set for the person M2a and the person M2b.

Moving on to Figure 7. As shown in FIG. 7C, consider a case where the person M1 enters the proximity range E2 due to the movement of the AMR 100a. Since the proximity range E1 is set for the person M1, even if the person M1 enters the proximity range E2, the AMR 100a does not stop or decelerate and passes in front of the person M1. Note that when the person M1 enters the close proximity range E1, the AMR 100a stops or decelerates. This is a safety control for everyone.

Next, consider a case where, after the AMR 100a passes in front of the person M1, the persons M2a and M2b enter the proximity range E2, as shown in FIG. 7(d). Since the proximity range E2 is set for the person M2a and the person M2b, the AMR 100a stops or decelerates when the person M2a or the person M2b enters the proximity range E2.

As mentioned above, since the proximity range for a person who has a desire to throw away garbage is set wider than the proximity range for a person who does not have a desire to throw away garbage, the AMR 100a can be used for persons M2a and M2b. Therefore, it is possible to stop or decelerate at a stage when the distance from the AMR is long (that is, at an early timing). Therefore, the person M2a and the person M2b can easily throw garbage into the trash can of the AMR 100a.

Note that since the AMR 100a does not move toward the person M2a, the person M2a does not have to wait until the AMR 100a moves toward him. On the other hand, the person M2b does not need to move any further (to the place where the AMR 100a has moved toward the person M2a) to throw away trash. In other words, even if AMR detects a plurality of people who want to throw away trash, the convenience of AMR does not decrease.

On the other hand, the AMR 100a maintains the moving state of the person M1 unless the person enters a range where stopping or deceleration is required for safety. That is, the AMR 100a does not cause unnecessary stopping or deceleration of the person M1.

As described above, the AMR 100a can stop or decelerate only when a person who has a service request for the AMR 100a approaches, and does not stop or decelerate unnecessarily at an unnecessary timing. Therefore, the AMR 100a not only does not provide excessive services, but also can provide services more quickly to the other persons M2a and M2b who have a desire to throw away trash. In other words, the convenience of AMR is improved for all people visiting the area.

As described above, in the first embodiment, the proximity range for a person who has a desire to throw away trash is set wider than the proximity range for a person who does not have a desire to throw away trash.
Therefore, even if there are people in the area who request AMR services and people who do not, the AMR will not change its operation at unnecessary times, so the AMR operation will not deteriorate in convenience. A control device and method can be provided.

Embodiment 2.
A second embodiment for implementing the present invention will be described. FIG. 8 is a functional block diagram showing the overall configuration of an AMR operation control device according to Embodiment 2 of the present invention.

In FIG. 8, the different configurations from FIG. 1 are the map/facility data 7 and the people flow distribution prediction unit 8. The rest of the configuration is the same as that in FIG. 1, so the explanation will be omitted.

The map/facility data 7 holds map information and facility information of the area where the AMR 100 moves. The map/facility data 7 may include facility time information (for example, opening/closing times, event holding times, etc.).

The people flow distribution prediction unit 8 refers to the self-position of the AMR 100 acquired by the input unit 1, and acquires the current movement location and time of the AMR 100. Then, with reference to the map/facility data 7, the distribution of the flow of people at the current location and time is predicted. Here, the distribution of people flow is information regarding the flow of people, including, for example, the degree of crowding of people, the direction of movement of people, and the like.

If the AMR is moving indoors, for example, in a part of the facility that corresponds to the flow of people or near a popular facility, the flow of people tends to be larger than in other facilities. Further, when the AMR moves outdoors, there is a high possibility that people prefer to move to a place with a roof, and such a place also tends to have a larger flow of people than other places. Furthermore, there is a tendency for the number of people to be larger during certain time periods (for example, opening hours in the morning, lunch time, returning home in the evening, event times, holidays, etc.) compared to other times.

If the people flow distribution prediction unit 8 predicts that the movement location or time period of the AMR will have a large distribution of people as described above compared to the normal case, the person detection unit 2 will detect the number of people around the AMR. Notify to change detection range. It also notifies the control unit 5 to change the proximity range for determining whether the person's position is within the proximity range. Here, the normal case is a case where the above-mentioned location or time zone does not have a large distribution of people.

As a method of changing the detection range of people around the AMR, for example, when there is a large distribution of people, the detection range of people can be narrowed under the condition that the safety of all people in the area can be ensured. For example, the detection range of a person when there is a large distribution of people is 70% of the detection range of a person in a normal case. Note that the size of the detection range can be appropriately set depending on the density of people within the area.
This makes it possible to prevent over-detection of people, and to make it possible to more accurately determine the demand for AMR services and to control the movement of AMR.

In addition, as a method of changing the proximity range for determining whether a person's position is within the proximity range, for example, in a place where the distribution of people is larger than usual, it is possible to Under the conditions that can be ensured, the proximity range can be narrower than in the case where the distribution of people flows is normal. Note that the proximity range may be changed by changing both the proximity range of a person who has a service request and the proximity range of a person who does not have a service request, or only one of the proximity ranges may be changed. For example, the proximity range when there is a large distribution of people is 70% of the proximity range in the normal case. Note that the size of the proximity range can be appropriately set depending on the density of people within the area.
This makes it possible to suppress unnecessary stopping or deceleration every time a person approaches an AMR in a place where there is a large flow of people, and it becomes possible to more accurately determine the demand for AMR service and to control the movement of the AMR.

FIG. 9 is a flowchart showing the operation of the motion control device in the second embodiment. Compared to the flowchart of the first embodiment shown in FIG. 3, the process of step ST7 is added. Since the rest is the same as in Embodiment 1, description of processes not related to the operation of step ST7 will be omitted.

In step ST7, the people flow distribution prediction unit 8 refers to the self-location of the AMR 100 acquired in step ST1 and the map/facility data 7, and predicts the distribution of people flow at the time and place where the AMR 100 is currently moving ( Step ST7).
If the person flow distribution prediction unit 8 determines that the place where the AMR 100 is currently moving is a place where there is a large distribution of people, the person detection range in step ST3 and the proximity range in step ST5 are The distribution of is set narrower than in the normal case (step ST7).

As a method for predicting the distribution of people, for example, map information, information regarding the structure of facilities, information based on time zones, etc. can be used. The map information is, for example, the location where the AMR 100 is moving, which is the flow line within the facility, or the vicinity of a popular facility. Information regarding the structure of the facility includes, for example, whether it has a roof or whether the road is wide (or narrow). Information based on time zones includes, for example, opening hours in the morning, lunch hours, returning home time in the evening, and event holding times. If the location or time the AMR 100 is moving meets these conditions, it is predicted that the location has a larger distribution of people than usual.

FIG. 10 shows an example of a specific operation of the AMR operation control device in the second embodiment. In the example of FIG. 10, there is one AMR and five people. The area around AMR will be explained as a place where there is a large distribution of people. FIG. 10A shows an example where the detection range and proximity range of a person are not controlled due to the distribution of the flow of people. FIG. 10B is an example of a case where the detection range and proximity range of a person are controlled based on the distribution of the flow of people according to the second embodiment.

In each of FIGS. 10(a) and 10(b), the AMR 100b is patrolling within the area. Persons M3a to M3c (hereinafter referred to as person M3a, person M3b, and person M3c) who have no intention of throwing away garbage are walking near the AMR 100b. Persons M4a to M4b (hereinafter referred to as person M4a and person M4b) who intend to throw away garbage are walking near the AMR 100b while holding bottles as garbage and looking for a trash can.

In FIG. 10A, the proximity range E4a, the proximity range E5a, and the person detection range E6a are each set when the distribution of the flow of people is normal. The person M4a and the person M4b are within the person detection range E6a, and are each detected as a person who has a request for service. Further, the person M3a, the person M3b, and the person M3c are also within the person detection range E6a, and are each detected as a person who does not have a request for service.
The person M3a and the person M3b have entered the proximity range E3a, and the AMR 100b stops or decelerates for safety. Therefore, the AMR 100b remains in place and cannot promptly move to the persons M4a and M4b. As a result, the AMR 100b cannot smoothly tour within the area. Further, although the person M3c has not entered the proximity range E3a, unnecessary service request determination processing is performed for the person M3c, which increases the processing load on the motion control device 200.

On the other hand, in FIG. 10(b), the people flow distribution prediction unit 8 predicts that the place where the AMR 100b is moving is a place where there is a lot of people. Then, the proximity range E4b, the proximity range E5b, and the person detection range E6b are set narrowly within ranges that can ensure the safety of all people.
As a result, the AMR 100b does not stop or decelerate the person M3a and M3b who were in the same position as in FIG. 10(a). Therefore, the AMR 100b can quickly move to the person M4a and the person M4b. As a result, the AMR 100b can smoothly tour within the area. Furthermore, since the person M3c is outside the person detection range E6b, unnecessary service request determination processing is not performed, and the processing load on the motion control device 200 does not increase.

As described above, in the second embodiment, when the distribution of the flow of people around the AMR is larger than in the normal case, the detection range and the proximity range of the person are made narrower than in the normal case.
Therefore, it is possible to suppress over-detection of a person who has no service request, and it is possible to prevent AMR from remaining in the same place. As a result, the AMR can travel within the area more efficiently, and the convenience of the AMR can be further improved.

In the second embodiment, the people flow distribution prediction unit 8 uses the map/facility data 7 to predict the people flow distribution, but the present invention is not limited to this. For example, as a method for predicting the distribution of people, the degree of crowding of people may be predicted using camera video data included in the surrounding information of the AMR 100. Other configurations may be used as long as they provide similar functions and effects.

In the second embodiment, when there is a large flow of people, the detection range and the proximity range of a person are made narrower than in the normal case, but the present invention is not limited to this. For example, when the distribution of the flow of people is smaller than in the normal case, the detection range and proximity range of the person may be set wider than in the normal case. Further, the detection range and the proximity range of the person do not need to be changed at the same time. For example, only the detection range of the person may be changed, or only the proximity range may be changed.
When the distribution of people around the AMR is smaller than normal, the AMR can provide service to people further away at an earlier timing by making the detection range and proximity range wider than normal. can be provided. Therefore, the convenience of AMR can be further improved.

In each of the embodiments described above, in order to simplify the explanation, the areas representing the detection range and the proximity range of a person are circular, but the invention is not limited to this. For example, the area representing the detection range and proximity range of a person may be elliptical or polygonal such as a quadrangle. Alternatively, the area representing the detection range and proximity range of the person may be a polygon shaped like the outline of the AMR. Furthermore, the area representing the detection range and proximity range of a person does not need to be a plane, and may be a three-dimensional shape (three-dimensional shape) such as a sphere or a polyhedron.

Furthermore, in each of the above-described embodiments, in order to simplify the explanation, the areas representing the detection range and the proximity range of a person are concentric circles centered on the AMR, but the invention is not limited to this. For example, the area representing the detection range and proximity range of a person may be an asymmetric area such that the direction of movement of the AMR is wider than in the case of concentric circles, and the area behind the AMR is narrower than in the case of concentric circles.

Furthermore, in each of the embodiments described above, the request determining unit 3 makes a binary determination of whether or not there is a request for a service, but the present invention is not limited to this. For example, a service request is not a binary value, but a continuum such as the degree of service request (for example, the strength of the service request, such as wanting to throw away the trash right away, or being able to throw away the trash when I find a trash can). It may be calculated as a value. For example, the level of service demand may be determined from the magnitude of a person's behavior. For example, if the movement of a person in the camera video data is large, it is determined that the level of demand for the service is strong, and if the movement of the person is small, it is determined that the level of demand for the service is weak. Further, the degree of service request may be the certainty (probability) of the service request.

Specifically, as a result of the analysis of the person's behavior by the request determination unit 3, if the person has a strong desire for the service (for example, the person frequently looks around because they want to throw out the trash right away, etc.) When the person's behavior is large) and when the degree of service request is weak (for example, the person's behavior is small, such as the person's awareness of throwing away garbage when he finds a garbage can, and the person's line of sight is slightly focused on the AMR). A continuous value of the degree of service demand may be generated between the two, and the moving speed of the AMR may be slowed down as the degree of service demand becomes stronger.
By subdividing the AMR operation control according to the degree of a person's AMR service request, it becomes possible to perform AMR operation control that is more adapted to the person's AMR service request.

Further, in each of the above-described embodiments, the proximity range is set to two levels: presence/absence of service request, but is not limited to this. For example, the proximity range may be set in multiple stages of three or more. For example, when there is a demand for a service, it is subdivided into cases where the demand for service is strong and cases where the demand for service is weak. may also be set widely. Alternatively, the proximity range may be set as a continuous value proportional to the degree of service demand.
By subdividing the proximity range according to the degree of a person's desire for AMR service, it becomes possible to control the operation of AMR that is more adapted to the person's AMR service.

Further, in each of the embodiments described above, a trash can-type AMR in which a trash can is mounted on the AMR has been described, but the AMR operation control device according to the present disclosure is not limited to a trash can-type AMR. For example, the present invention can also be applied to an AMR equipped with a vending machine. In this case, the service request for AMR is, for example, a desire to use a vending machine (to purchase a product from a vending machine). Further, the behavior of the person is, for example, taking out a wallet. Then, the AMR operation control device determines whether the person wants to purchase the product or not, and sets a wide proximity range for the person who wants to purchase the product. Therefore, since the AMR can be stopped or decelerated at an early timing, a person who wants to purchase a product can purchase the product at an early timing.

The present invention can also be applied to AMRs equipped with signage. In this case, the service request for AMR is, for example, a desire to view content displayed on a signage (a desire to obtain information within the signage). Further, the behavior of the person is, for example, keeping pace with the AMR, taking out glasses, etc. Then, the AMR operation control device determines whether or not the person wants to acquire the information in the signage, and sets a wide proximity range for the person from whom the information in the signage is desired to be acquired. Therefore, since the AMR can be stopped or decelerated at an early timing, a person who wants to acquire the information in the signage can acquire the information in the signage at an early timing.

Furthermore, the present invention can also be applied to AMR for transporting parts in factory work. In this case, the service request for the AMR is, for example, a desire to remove parts mounted on the AMR. Further, the behavior of the person includes, for example, an operator picking a component mounted on the AMR, an operator moving his/her line of sight to the component, and the like. Then, the AMR operation control device determines whether or not the person wants to take out the part, and sets a wide proximity range for the person who wants to take out the part. Therefore, since the AMR can be stopped or decelerated at an early timing, a person who wants to take out parts can take out the parts mounted on the AMR at an early timing.

As described above, if the AMR has a function to provide services to people and there is a strong need for the AMR to stop or decelerate in order to utilize that function, the present invention AMR operation control devices can be applied. Therefore, it is possible to provide an AMR operation control device and method with improved convenience in various applications.

The AMR operation control device of the present disclosure prevents the AMR from unnecessary stopping or deceleration at unnecessary timing. Furthermore, the AMR does not move toward the person every time. Therefore, the AMR operation control device of the present disclosure can reduce the moving distance of the AMR and the frequency of starting and stopping, and can reduce the running costs (power, fuel, maintenance costs, etc.) of the AMR.

In the AMR operation control device of the present disclosure, a request for AMR service is determined by analyzing a person's behavior. In other words, the presence or absence of a request for service cannot be determined based on differences in the person's appearance (for example, differences in clothing, name tag, etc.). Therefore, the AMR operation control device of the present disclosure can control all the people in the area, such as a factory, even if the people in the area wear the same clothes, or even if the people in the area, such as an exercise facility, wear different clothes depending on the group. It is possible to correctly judge whether a person has a request for service.

In addition to the above, any component of the embodiments of the present disclosure may be modified or any component of the embodiments may be omitted within the scope of the disclosure.

1 input unit, 2 person detection unit, 3 request determination unit, 4 proximity range setting data, 5 movement determination unit, 6 control unit, 7 map/facility data, 8 people flow distribution prediction unit,
100, 100a, 100b autonomous mobile robot,
200 operation control device,
300 processor, 301 volatile storage device, 302 non-volatile storage device, 303 input/output device, 304 signal path.

Claims

An operation control device for an autonomous mobile robot for providing a service to a person,
determining the degree of the person's request for a service from the autonomous mobile robot by analyzing the behavior of the person using surrounding information of the autonomous mobile robot;
a request determination unit that sets a proximity range of the autonomous mobile robot to the person according to a degree of request for the service;
An operation control device for an autonomous mobile robot, comprising: a motion determining unit that determines whether the position of the person is within the proximity range and determines whether to stop or decelerate the autonomous mobile robot.
The autonomous mobile robot according to claim 1, wherein the request determining unit sets the proximity range for a person who has a request for the service to be wider than the proximity range for a person who does not have a request for the service. Motion control device.
The distribution of the flow of people around the autonomous mobile robot is predicted, and the detection range of the person or the proximity range, or the detection range of the person and the detection range are changed depending on the distribution of the flow of people. The operation control device for an autonomous mobile robot according to claim 1 or 2, further comprising a distribution prediction section.
Any one of claims 1 to 3, wherein the request determining unit sets the proximity range of the autonomous mobile robot to the person to three or more levels depending on the degree of the request for the service. The operation control device for an autonomous mobile robot according to item 1 above.
5. The request determination unit sets the proximity range for a person who has a strong desire for the service to be wider than the proximity range for a person who has a weak desire for the service. Motion control device for autonomous mobile robots.
A method for controlling the operation of an autonomous mobile robot for providing a service to a person, the method comprising:
a request determination unit determines whether or not the person requests a service for the autonomous mobile robot by analyzing the behavior of the person using surrounding information of the autonomous mobile robot;
setting a proximity range of the autonomous mobile robot to the person depending on whether there is a request for the service;
A motion control method for an autonomous mobile robot, wherein a motion determining unit determines whether the position of the person is within the proximity range and determines whether to stop or decelerate the autonomous mobile robot.
The autonomous mobile robot according to claim 6, wherein the request determining unit sets the proximity range for a person who has a request for the service to be wider than the proximity range for a person who does not have a request for the service. Operation control method.
A people flow distribution prediction unit predicts the distribution of people flow around the autonomous mobile robot, and depending on the distribution of the people flow, the detection range of the person or the proximity range, or the detection range of the person and the The method for controlling the operation of an autonomous mobile robot according to claim 6 or 7, characterized in that the detection range is changed.
Any one of claims 6 to 8, wherein the request determining unit sets the proximity range of the autonomous mobile robot to the person to three or more levels depending on the degree of the request for the service. The method for controlling the operation of an autonomous mobile robot according to item 1.
10. The request determination unit sets the proximity range for a person who has a strong desire for the service to be wider than the proximity range for a person who has a weak desire for the service. Operation control method for autonomous mobile robots.