JP2022185811A - Autonomous traveling system, autonomous traveling method, and autonomous traveling program - Google Patents

Abstract

To provide an autonomous traveling system, an autonomous traveling method, and an autonomous traveling program capable of improving the work efficiency of a worker who generates a traveling route for an autonomous traveling device.SOLUTION: An autonomous traveling device 10 comprises: a recognition processing unit 413 that recognizes a worker based on an image captured by a camera 43; a travel processing unit 418 that causes the autonomous traveling device 10 to travel while following the worker; a detection processing unit 415 that detects a predetermined gesture operation of the worker based on the captured image; a registration processing unit 414 that registers action information in which a specific action corresponding to a specific gesture operation and position information of the autonomous traveling device 10 are associated with each other when the detected gesture operation is a preset specific gesture operation; and a generation processing unit 416 that generates a traveling route based on locus information of a traveling locus along which the autonomous traveling device 10 travels following the worker and action information registered by the registration processing unit 414.SELECTED DRAWING: Figure 4

Description

TECHNICAL FIELD The present invention relates to an autonomous traveling system, an autonomous traveling method, and an autonomous traveling program for causing an autonomous traveling device capable of autonomous traveling to travel.

In recent years, due to labor shortages and soaring labor costs, there has been a shortage of workers to guard (monitor) and clean facilities such as shopping malls, stations, and airports. Therefore, the introduction of mobile robots (autonomous mobile devices) such as autonomous security robots and cleaning robots for industrial use, which are designed to be able to travel autonomously and have safety, are progressing.

Some autonomous mobile devices of this type have a teaching function (teaching function) that stores a travel route based on an operator's operation (see Patent Document 1, for example). For example, the operator sets the route teaching mode, drives the autonomous mobile device in the work area, drives the autonomous mobile device along a desired route, and stores the trajectory traveled by the autonomous mobile device as the travel route. When the autonomous traveling device is set to the autonomous traveling mode, the autonomous traveling device reproduces and travels the traveling route stored in the route teaching mode. This enables the autonomous mobile device to autonomously travel along the travel route generated based on the operator's operation (teaching operation).

JP 2017-182175 A

However, in the conventional technology, it is necessary for the worker to manually move the autonomous mobile device while performing operations such as setting the travel route on the operation terminal. operation) takes a long time, resulting in a problem of reduced work efficiency.

An object of the present invention is to provide an autonomous traveling system, an autonomous traveling method, and an autonomous traveling program capable of improving the work efficiency of a worker who creates a traveling route for an autonomous traveling device.

An autonomous mobile system according to one aspect of the present invention is a system that causes an autonomous mobile device to travel based on a travel route. The autonomous traveling system includes a recognition processing unit that recognizes a worker based on detection information from a detection unit mounted on the autonomous traveling device, and a traveling processing unit that causes the autonomous traveling device to follow and travel the worker. a detection processing unit that detects a predetermined registration operation of the worker based on the detection information; and if the registration operation detected by the detection processing unit is a preset specific registration operation, the a registration processing unit that registers motion information in which a specific motion corresponding to a specific registration operation and position information of the autonomous mobile device are associated with each other; and a generation processing unit that generates the travel route based on the motion information registered by the registration processing unit.

An autonomous traveling method according to another aspect of the present invention is a method of causing an autonomous traveling device to travel based on a travel route. In the autonomous traveling method, a recognition step in which one or more processors recognize a worker based on detection information from a detection unit mounted on the autonomous traveling device; a running step for running the vehicle with the vehicle, a detection step for detecting a predetermined registration operation of the worker based on the detection information, and a case where the registration operation detected in the detection step is a preset specific registration operation a registration step of registering motion information in which a specific motion corresponding to the specific registration operation and position information of the autonomous mobile device are associated with each other; and a generating step of generating the travel route based on the trajectory information and the motion information registered in the registering step.

An autonomous traveling program according to another aspect of the present invention is a program for causing an autonomous traveling device to travel based on a travel route. The autonomous traveling program includes a recognition step of recognizing a worker based on detection information of a detection unit mounted on the autonomous traveling device, a traveling step of causing the autonomous traveling device to follow the worker, and a detection step of detecting a predetermined registration operation of the worker based on the detection information; a registration step of registering motion information in which a corresponding specific motion and position information of the autonomous mobile device are associated with each other; trajectory information of a travel trajectory along which the autonomous mobile device travels following the worker; and a generation step of generating the travel route based on the operation information registered in the above.

ADVANTAGE OF THE INVENTION According to this invention, the autonomous traveling system, the autonomous traveling method, and the autonomous traveling program which can improve the work efficiency of the worker who generates the traveling route of an autonomous traveling apparatus can be provided.

FIG. 1 is a perspective view showing the appearance of the front side of an autonomous mobile device according to an embodiment of the present invention. FIG. 2 is a perspective view showing the appearance of the rear side of the autonomous mobile device according to the embodiment of the present invention. FIG. 3 is a front view of an autonomous mobile device according to an embodiment of the invention. FIG. 4 is a functional block diagram showing the configuration of the autonomous mobile device according to the embodiment of the invention. FIG. 5 is a diagram showing an example of map information registered in the storage unit of the autonomous mobile device according to the embodiment of the present invention. FIG. 6 is a diagram showing an example of teaching information registered in the storage unit of the autonomous mobile device according to the embodiment of the present invention. FIG. 7 is a diagram showing an example of gesture information registered in the storage unit of the autonomous mobile device according to the embodiment of the present invention. FIG. 8 is a diagram showing how the autonomous mobile device according to the embodiment of the present invention travels while following a worker. FIG. 9 is a diagram showing how the teaching operation is performed according to the embodiment of the present invention. FIG. 10 is a diagram showing an example of an operation screen displayed on the autonomous mobile device according to the embodiment of the present invention. FIG. 11 is a diagram showing an example of an operation screen displayed on the autonomous mobile device according to the embodiment of the present invention. FIG. 12 is a diagram showing an example of an operation screen displayed on the autonomous mobile device according to the embodiment of the present invention. FIG. 13 is a diagram showing an example of an operation screen displayed on the autonomous mobile device according to the embodiment of the present invention. FIG. 14 is a diagram showing an example of an operation screen displayed on the autonomous mobile device according to the embodiment of the present invention. FIG. 15 is a diagram showing an example of an operation screen displayed on the autonomous mobile device according to the embodiment of the present invention. FIG. 16 is a diagram showing an example of an operation screen displayed on the autonomous mobile device according to the embodiment of the present invention. FIG. 17 is a diagram showing an example of an operation screen displayed on the autonomous mobile device according to the embodiment of the present invention. FIG. 18 is a diagram showing an example of an operation screen displayed on the autonomous mobile device according to the embodiment of the present invention. FIG. 19 is a diagram showing an example of motion information registered in the storage unit of the autonomous mobile device according to the embodiment of the present invention. FIG. 20 is a flowchart showing an example of travel route generation processing executed by the autonomous mobile device according to the embodiment of the present invention. FIG. 21 is a flowchart showing an example of travel route generation processing executed by the autonomous mobile device according to the embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. In addition, the following embodiment is an example which actualized this invention, and does not limit the technical scope of this invention.

[Autonomous mobile device 10]
FIG. 1 is a perspective view showing the appearance of the front side of an autonomous mobile device 10 according to an embodiment of the present invention, and FIG. 2 is a schematic diagram showing the appearance of the rear side of the autonomous mobile device 10. As shown in FIG. In the following description, the up-down direction D1, the front-rear direction D2, and the left-right direction D3 shown in each drawing are used. The autonomous mobile device 10 is an example of the autonomous mobile device of the present invention.

The autonomous mobile device 10 is an autonomous mobile device that moves by autonomously traveling within facilities such as shopping malls, stations, and airports, and is also called a mobile robot. The autonomous mobile device 10 is used for various purposes such as security robots, cleaning robots, guidance robots, nursing care robots, and luggage transport robots. For example, the autonomous mobile device 10 guards (monitors) a guarded area in the facility while moving by autonomous driving. Further, for example, the autonomous mobile device 10 cleans the cleaning target area while moving by autonomous travel. Further, for example, the autonomous mobile device 10 guides the user to a destination and displays guidance information or advertisement information on the display panel 21 while moving by autonomous travel. The autonomous mobile device 10 autonomously travels based on various setting information such as a preset travel route, a time period, and a return position to return for charging.

A security robot having a security function will be described below as the autonomous mobile device 10 of the present embodiment. For example, in a shopping mall, the autonomous mobile device 10 executes security processing for detecting the presence or absence of an abnormality such as a suspicious person while executing travel processing according to a preset travel route during a predetermined time period (for example, at night). do.

The autonomous mobile device 10 includes a device main body 11 and functional units provided in the device main body 11 . Specifically, the device main body 11 includes a traveling portion 12, a motor (not shown), a battery (not shown), an operation portion 20 (see FIG. 2), a display panel 21 (see FIG. 1), a charging connection portion 30 ( 2), and a control unit 40 (control device).

As shown in FIG. 1, the apparatus main body 11 has an exterior cover 11A that constitutes its exterior. Further, as shown in FIG. 2, the device main body 11 has a chassis 11B at its lower portion. The chassis 11B is provided substantially parallel to the floor surface. Further, inside the device body 11, a support frame for supporting each of the functional units described above is appropriately provided.

The running portion 12 is provided in the lower portion of the device main body 11 . The traveling portion 12 is attached to the chassis 11B to transmit the conveying force in the advancing direction to the floor surface while maintaining the traveling posture of the apparatus main body 11 . The traveling part 12 has a pair of wheels 121 for traveling and four casters 122 .

The wheels 121 are rotatably supported at the center of the chassis 11B in the front-rear direction and at both ends in the left-right direction D3 (width direction). The four casters 122 are for maintaining the running posture of the apparatus main body 11, and are rotatably supported at both ends of the front end of the chassis 11B and both ends of the rear end of the chassis 11B. The outer peripheral surfaces of the wheels 121 and casters 122 are supported by the floor while the autonomous mobile device 10 is placed on the floor. As a result, the device main body 11 is maintained in the running posture shown in FIGS. 1 and 2 .

The output shaft of the motor is connected to the rotating shaft of the wheel 121 via a transmission mechanism such as a reduction gear. Therefore, when the motor is driven and its rotational driving force is output from the output shaft, the rotational driving force of the motor is transmitted to the wheels 121 . In this embodiment, the motors are individually provided for each of the pair of wheels 121 . Therefore, the rotational speed of each wheel 121 is controlled by individually driving and controlling each motor. For example, when the rotational speed of each wheel 121 is controlled to be uniform, the autonomous mobile device 10 moves straight or backward. The autonomous mobile device 10 turns.

The battery is provided in the central portion of the device main body 11 . The battery supplies driving power to the motor.

As shown in FIG. 2, a back cover 17 is provided on the back of the device body 11 to cover the back. The rear cover 17 is provided with a charging connection section 30 used when charging the battery. The charging connection unit 30 has three power receiving terminals 31 that are connected to the three power supply terminals of the charging station. When the autonomous mobile device 10 is a cleaning robot, components related to the cleaning function are arranged inside the rear cover 17 .

FIG. 3 is a front view of the autonomous mobile device 10. FIG. As shown in FIG. 3 , a front laser sensor 41 , a sonar sensor 42 and a camera 43 are provided on the front of the autonomous mobile device 10 .

The front laser sensor 41 is provided in a groove portion 175 extending in the width direction formed in the lower front portion of the apparatus body 11 . The front laser sensor 41 is arranged at the center inside the groove portion 175 . The front laser sensor 41 includes a laser transmission element, a laser driver for driving the laser transmission element, a light receiving element, a light reception processing circuit for converting the output of the light receiving element into a digital signal, and the like. The front laser sensor 41 is connected to the controller 40 and controlled by the controller 40 . The front laser sensor 41 scans a laser beam forward in the width direction (horizontal direction) within a predetermined scanning angle (for example, 120°). When the front laser sensor 41 receives the laser beam that has returned after being reflected by the irradiated object (object), the control unit 40 measures the time until the laser beam returns, and measures the measured value. , the distance to the object at each scanning position is calculated. Thereby, the control unit 40 can grasp the distance and position to an object existing on the front side (advance direction side) of the autonomous mobile device 10, and the shape and size of the object in the width direction.

The sonar sensor 42 is provided below the display panel 21 . Sonar sensors 42 are provided at both ends in the width direction on the front surface of the apparatus main body 11 . The sonar sensor 42 is connected to the controller 40 and controlled by the controller 40 . The sonar sensor 42 detects an object using sound waves, and measures the distance to the object based on the time it takes for the sound waves to reflect off the object and return.

The camera 43 is provided above the display panel 21 . The camera 43 is arranged at the center in the width direction on the front surface of the apparatus main body 11 . The camera 43 is connected to the control section 40 and controlled by the control section 40 . The camera 43 captures an image of a predetermined range in front (for example, a range of 120 degrees in the horizontal direction and 100 degrees in the vertical direction) at a predetermined frame rate, and outputs the captured image data to the control unit 40 . The control unit 40 sequentially acquires the image data from the camera 43 and executes predetermined image processing (described later). The camera 43 is an example of the detection section of the present invention, and an example of the camera of the present invention.

Further, a side laser sensor 45 is provided on each of both side surfaces of the apparatus main body 11 . The side laser sensor 45 has substantially the same configuration as the front laser sensor 41, and includes a laser transmitting element, a laser driver for driving the laser transmitting element, a light receiving element, a light receiving processing circuit for converting the output of the light receiving element into a digital signal, and the like. I have. The side laser sensor 45 is connected to the controller 40 and controlled by the controller 40 . The side laser sensor 45 scans the laser beam from the front, downward, and rearward within a predetermined scanning angle range (eg, 180°). When the side laser sensor 45 receives the laser beam that has returned after being reflected by the irradiated object (object), the control unit 40 measures the time until the laser beam returns, and measures the measured value. , the distance to the object at each scanning position is calculated. As a result, the control unit 40 can determine the distances and positions of objects existing on the front side (advance direction side) of the autonomous mobile device 10, steps and obstacles existing on the floor, and their shapes and sizes in the scanning direction. can grasp.

A monitoring camera 44 is provided on the upper surface of the device body 11 . The monitoring camera 44 is arranged in the center of the top plate of the device main body 11 . The monitoring camera 44 is connected to the control unit 40 and controlled by the control unit 40 . The monitoring camera 44 is configured to be able to change its imaging range, and images a predetermined imaging range in accordance with an instruction from the control unit 40 . For example, the surveillance camera 44 has a rotation mechanism capable of rotating the lens horizontally by 360 degrees and vertically by 180 degrees. Take an image of the range. The monitoring camera 44 captures an image of a predetermined range at a predetermined frame rate, and outputs captured image data to the control unit 40 . The control unit 40 sequentially acquires the image data from the monitoring camera 44 and executes predetermined image processing (described later).

The autonomous mobile device 10 uses map (environmental map) information corresponding to the guarded area and detection information from the front laser sensor 41, sonar sensor 42, camera 43, surveillance camera 44, and side laser sensor 45. , autonomously travels along the preset travel route. Note that the camera 43 and the monitoring camera 44 may be configured by one camera. For example, a monitoring camera 44 provided on the upper surface of the device main body 11 may serve as the camera 43 .

An operation unit 20 (see FIG. 2) is provided on the upper part of the rear surface of the device main body 11 . The operation unit 20 is attached to the exterior cover 11A. The operating unit 20 is a device operated by an operator, and is, for example, a device having a touch panel capable of touch operation. The operation unit 20 displays an operation screen for the operator to perform various operations (teaching operation, registration operation, setting operation, travel instruction operation, etc.). Operation information for the operation unit 20 is transferred to the control unit 40 and used for travel control by the control unit 40 . Note that the operation unit 20 may be provided on the upper surface of the apparatus main body 11 .

A display panel 21 (see FIG. 1) is provided on the front surface of the device main body 11 . The display panel 21 is, for example, a liquid crystal panel. Various operation screens (see FIGS. 10 to 18) are displayed on the display panel 21 by the controller 40 during the teaching operation. Also, announcement information such as security information is displayed on the display panel 21 by the control unit 40 while the autonomous mobile device 10 is autonomously traveling (during security). The security information is, for example, information for announcing a warning or notification when a suspicious person is detected. The display panel 21 is an example of the display section of the present invention.

The operating handle 22 is provided at the uppermost portion of the rear surface of the device main body 11 . The operating handle 22 is attached to the exterior cover 11A. The operating handle 22 is an operating member held by the operator when the operator manually operates the autonomous mobile device 10 .

The communication unit 25 (see FIG. 4) connects the autonomous mobile device 10 to a network by wire or wirelessly, and exchanges data according to a predetermined communication protocol with an external device such as a server (not shown) via the network. A communication interface for executing communication.

The storage unit 50 (see FIG. 4) is a non-volatile storage unit such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) that stores various types of information. Specifically, the storage unit 50 stores data such as map information 51, teaching information 52, gesture information 53, motion information 54, and the like. FIG. 5 is a diagram showing an example of the map information 51. As shown in FIG. FIG. 6 is a diagram showing an example of the teaching information 52. As shown in FIG. FIG. 7 is a diagram showing an example of the gesture information 53. As shown in FIG. The motion information 54 (see FIG. 19) will be described later.

As shown in FIG. 5, in the map information 51, environment map information corresponding to the security area (work area) in which the autonomous mobile device 10 travels is registered. One or more environmental maps are registered in the map information 51 . FIG. 5 illustrates one environmental map M1. For example, when an environment map is generated for each floor of the facility, the map information 51 registers a plurality of environment maps corresponding to each floor of the facility. Specifically, in the map information 51, an environment map M1 of the facility floor F1, an environment map M2 of the floor F2, and an environment map M3 of the floor F3 are registered. The environment map may be one generated in advance by an external device, or the distance to obstacles detected by each sensor (front laser sensor 41, sonar sensor 42, etc.) while the autonomous mobile device 10 is traveling. and location. That is, the control unit 40 may generate an environmental map of the travel area by causing the autonomous mobile device 10 to travel. The environment map M1 shown in FIG. 5 will be described below as an example.

As shown in FIG. 6, in the teaching information 52, information on the teaching operation by the operator is registered.

Here, in the teaching operation according to the present embodiment, the worker A who is in charge of the teaching operation of the travel route of the facility corresponding to the environment map M1 shown in FIG. It runs behind the worker A as shown. FIG. 9 shows a worker A who is performing a teaching operation and an autonomous mobile device 10 which follows the worker A and travels. Reference P0 in FIG. 9 represents the starting position P1 of the teaching operation. The autonomous mobile device 10 acquires various kinds of information and registers them in the teaching information 52 while following the worker A and traveling.

Specifically, in the teaching information 52, information such as "positional information", "gesture detection", and "gesture type" is registered at each predetermined time interval. “Position information” is information indicating the position (coordinates) of the autonomous mobile device 10 . The position information is calculated based on the environment map M1 and detection results of the front laser sensor 41, sonar sensor 42, camera 43, and the like. "Gesture detection" is information indicating whether or not the autonomous mobile device 10 has detected a gesture operation by the worker A. The gesture operation is a predetermined action of the operator A's hands (right and left hands), face, and whole body. For example, the gesture operation includes an action of raising the right hand, an action of raising the left hand, a pointing action, an action of shaking the hand horizontally or vertically, an action of bending the waist or knees to lower the posture (a bending action), and the like. When the control unit 40 analyzes the captured image of the camera 43 and detects the gesture operation, it registers “1” in “gesture detection” of the teaching information 52 . "Gesture type" is information indicating the operator's motion corresponding to the gesture operation. In the teaching operation, the worker A performs a predetermined gesture operation at a predetermined position while the autonomous mobile device 10 is following the robot. When detecting the gesture operation of the worker A, the control unit 40 registers the motion information of the gesture operation in the "gesture type".

As shown in FIG. 7, the gesture information 53 registers information about the gesture operation by the worker. Specifically, information such as “gesture type” and “action content” is registered in the gesture information 53 . Various types of information of the gesture information 53 are set in advance by an administrator or the like and stored in the storage unit 50 . "Gesture type" is the type of gesture operation to be recognized as the gesture operation. The control unit 40 can recognize the gesture operation registered in the gesture type. The “action content” is information indicating the action content to be executed by the autonomous mobile device 10 when the control unit 40 recognizes the gesture operation.

The gesture type "pointing" is registered in association with the operation content of "registering an action of photographing the pointing direction with a monitoring camera for the duration of the gesture operation". For example, when the controller 40 detects the operator A's "pointing" gesture operation based on the captured image of the camera 43 in the teaching operation, the control unit 40 causes the monitoring camera 44 to photograph the pointing direction for the duration of the pointing operation. The motion information is registered in association with the position information of the autonomous mobile device 10 (motion information 54 in FIG. 19).

Further, the gesture type "waving hand" is registered in association with the action content of "registration of action of photographing 360 degrees with surveillance camera for gesture operation time". For example, when the control unit 40 detects a gesture operation of “waving hand” by the worker A based on the captured image of the camera 43 in the teaching operation, the control unit 40 detects the operation time of waving the hand in 360 degrees to the monitoring camera 44 . The information of the motion of only shooting is registered in association with the position information of the autonomous mobile device 10 (motion information 54 in FIG. 19).

Further, the gesture type “bend over” is registered in association with the action content of “registering the action of displaying the specific information for the duration of the gesture operation”. For example, when the controller 40 detects the operator A's "bend over" gesture operation based on the image captured by the camera 43 in the teaching operation, the control unit 40 transmits information on the operation of displaying predetermined information on the display panel 21 to the autonomous traveling robot. It is registered in association with the position information of the device 10 (operation information 54 in FIG. 19).

Further, the gesture type "raise right hand" is registered in association with the action content of "transition to action information registration mode". For example, when the control unit 40 detects a gesture operation of “raising right hand” by the worker A based on the captured image of the camera 43 in the teaching operation, the control unit 40 detects a specific gesture operation (“pointing”, “waving”, "Bend down") is shifted to a registration mode in which the operation content associated with the position information of the autonomous mobile device 10 is associated and registered. When the control unit 40 detects the specific gesture operation after shifting to the registration mode, the control unit 40 registers the information on the operation content associated with the gesture operation in association with the position information of the autonomous mobile device 10 (the operation shown in FIG. 19). Information 54).

Further, the gesture type "raise left hand" is registered in association with the action content of "finish teaching operation". For example, when the control unit 40 detects a gesture operation of “raise left hand” by the worker A based on the captured image of the camera 43 in the teaching operation, the control unit 40 terminates the teaching operation, and performs registration processing of the operation content and location information. End the registration process.

In this way, when detecting the gesture operation of the worker A, the control unit 40 refers to the gesture information 53 and executes the operation content associated with the detected gesture operation.

As another embodiment, some or all of the information such as the map information 51, the teaching information 52, the gesture information 53, and the motion information 54 are stored in a data server or the like accessible from the autonomous mobile device 10 via a network. good too.

Further, the storage unit 50 stores a control program such as a travel route generation program for causing the control unit 40 to execute a travel route generation process (see FIGS. 20 and 21), which will be described later. For example, the travel route generation program is non-temporarily recorded on a computer-readable recording medium such as a CD or DVD, and read by a reading device (not shown) such as a CD drive or DVD drive provided in the autonomous mobile device 10. It is read and stored in the storage unit 50 .

The control unit 40 is provided on the upper part of the apparatus main body 11 (not shown). FIG. 4 shows a functional block diagram showing the configuration of the control section 40. As shown in FIG. The control unit 40 has control devices such as a CPU, ROM, and RAM. The CPU is a processor that executes various kinds of arithmetic processing. The ROM is a non-volatile storage unit in which control programs such as BIOS and OS for causing the CPU to execute various arithmetic processes are stored in advance. The RAM is a volatile or nonvolatile storage unit that stores various information, and is used as a temporary storage memory (work area) for various processes executed by the CPU. The control unit 40 controls the autonomous mobile device 10 by causing the CPU to execute various control programs pre-stored in the ROM or storage unit 50 .

Specifically, as shown in FIG. , a travel processing unit 418 and other various processing units. The control unit 40 functions as the various processing units by executing various types of processing according to the travel route generation program by the CPU. Also, part or all of the processing unit may be configured by an electronic circuit. The travel route generation program may be a program for causing a plurality of processors to function as the processing unit.

The acquisition processing unit 411 acquires the image data of the captured image captured by the camera 43 during the teaching operation. Specifically, as shown in FIG. 8, in the teaching operation, while the autonomous mobile device 10 is following the worker A, the camera 43 captures an image of the front at a predetermined frame rate. The image data are sequentially output to the control unit 40 . The acquisition processing unit 411 repeatedly acquires image data from the camera 43 during the teaching operation.

The display processing unit 412 causes the display panel 21 to display various information. Specifically, the display processing unit 412 causes the display panel 21 to display an operation screen (see FIGS. 10 to 18) regarding the teaching operation. The display processing unit 412 supports the teaching operation by the operator A by displaying announcement information on each operation screen.

The recognition processing unit 413 recognizes the worker based on the captured image corresponding to the image data acquired by the acquisition processing unit 411 . Specifically, when starting the teaching operation, the operator A photographs himself/herself with the camera 43 of the autonomous mobile device 10 . The display processing unit 412 causes the display panel 21 to display the captured image captured by the camera 43, as shown in FIG. The display processing unit 412 may display information (environmental map information) of the target area (environmental map M1) of the teaching operation, the travel route ID, the travel route name, and the like on the operation screen.

When a person is detected from the captured image, the recognition processing unit 413 displays the detected person image to confirm whether it is the operator A (see FIG. 11). When the worker A nods in front of the camera 43, the recognition processing unit 413 recognizes the detected person as the worker performing the teaching operation. When the recognition processing unit 413 recognizes the operator, the control unit 40 starts teaching operation (see FIG. 12). When the teaching operation is started, the display processing unit 412 displays the starting position P0 of the teaching operation, the current position (“★” mark) of the autonomous mobile device 10, the operation The current position of person A (“●” mark) is displayed.

When the teaching operation is started, the control unit 40 controls traveling of the autonomous mobile device 10 so as to follow the recognized worker A. Specifically, the control unit 40 controls the traveling speed and traveling direction of the autonomous mobile device 10 based on the position of the worker A included in the captured image acquired from the camera 43 . The display processing unit 412 displays the travel locus of the autonomous mobile device 10 on the operation screen while the autonomous mobile device 10 follows the worker A (see FIG. 13).

When the recognition processing unit 413 cannot recognize the worker A after the teaching operation is started (when the worker A is lost), the notification processing unit 417 outputs a message (warning information) indicating that recognition is impossible. is displayed on the operation screen (see FIG. 14). Note that the notification processing unit 417 may output the message by voice, or may output a predetermined warning sound. That is, the notification processing unit 417 notifies warning information when the autonomous mobile device 10 cannot follow the worker A.

In this way, the recognition processing unit 413 recognizes the worker based on the captured image acquired from the camera 43 . As another embodiment, the recognition processing unit 413 may recognize the worker based on the detection results of sensors mounted on the autonomous mobile device 10 (for example, the front laser sensor 41, the sonar sensor 42, etc.). The sensor is an example of the detection section of the present invention.

The registration processing unit 414 registers the position information of the current position of the autonomous mobile device 10 during the teaching operation in the teaching information 52 (see FIG. 6).

The detection processing unit 415 detects a predetermined gesture operation (registration operation of the present invention) of the worker A based on the captured image. Specifically, the detection processing unit 415 refers to a plurality of gesture operations (gesture types) registered in the gesture information 53 (see FIG. 7) to identify the gesture operations of the worker A included in the captured image. do. When the detection processing unit 415 detects a predetermined gesture operation, the control unit 40 executes a specific operation (operation details in FIG. 7) associated with the gesture operation.

Here, for example, when the worker A stops at an arbitrary position for a predetermined time during the teaching operation, the display processing unit 412 displays the operation screen shown in FIG. 15 . Specifically, the display processing unit 412 provides announcement information regarding a gesture operation (for example, “raise right hand”) for shifting to a registration mode for registering a specific action and a gesture operation for ending a teaching operation (for example, “raise left hand”). display. When the detection processing unit 415 detects a specific gesture operation of “raising right hand” by the worker A from the captured image of the camera 43, the control unit 40 associates the specific operation content with the position information of the autonomous mobile device 10 and registers it. Move to registration mode. Note that when the detection processing unit 415 detects "right hand raised", the registration processing unit 414 registers information on "right hand raised" in the teaching information 52 (time t12 in FIG. 6).

After shifting to the registration mode, if the gesture operation detected by the detection processing unit 415 is a specific gesture operation, the registration processing unit 414 registers the operation content corresponding to the specific gesture operation and the position information of the autonomous mobile device 10. and register the operation information associated with each other. Specifically, when the gesture operation detected by the detection processing unit 415 is the first gesture operation, the registration processing unit 414 stores the first action information that causes the autonomous mobile device 10 to perform the first action at the first position. When the gesture operation detected by the detection processing unit 415 is the second gesture operation, second action information is registered to cause the autonomous mobile device 10 to perform the second action at the second position.

Further, the registration processing unit 414 allows the operator to perform a registration operation (for example, Nodding operation) is received, the motion information is registered.

For example, when operator A performs a pointing operation at an arbitrary position, the display processing unit 412 displays inquiry information for confirming the gesture operation on the operation screen shown in FIG. 16 . When the worker A nods toward the camera 43, the controller 40 executes a specific action associated with the pointing operation. Here, the registration processing unit 414 performs a process of registering the “movement of photographing the pointing direction with the monitoring camera for the duration of the gesture operation (pointing operation)” in association with the current position [x10, y10] of the autonomous mobile device 10. Execute (time t13 in FIG. 6). Further, the display processing unit 412 displays the registered contents on the operation screen shown in FIG. Thereby, the worker A can recognize that the intended specific action is definitely registered.

For example, when worker A waves his or her hand at an arbitrary position, the display processing unit 412 displays inquiry information for confirming the gesture operation on the operation screen. When the worker A nods toward the camera 43, the control unit 40 executes a specific action associated with the hand waving operation. Here, the registration processing unit 414 executes a process of registering "movement of photographing 360 degrees with a surveillance camera for the duration of a gesture operation (waving a hand)" in association with the current position [x20, y20] of the autonomous mobile device 10. (time t23 in FIG. 6).

The generation processing unit 416 generates the travel route based on the locus information of the travel locus along which the autonomous mobile device 10 travels following the worker A and the motion information registered by the registration processing unit 414 . Specifically, the generation processing unit 416 generates the trajectory information shown in FIG. 18 based on the position information registered in the teaching information 52 (see FIG. 6). The trajectory information includes information such as a position P0 at which autonomous travel is started, a route R1 to be autonomously traveled, a position G at which autonomous travel is terminated, and the like. The display processing unit 412 displays the generated trajectory information on the operation screen shown in FIG. 18 to prompt the operator A to confirm. Further, the display processing unit 412 may display the positions P1 and P2 of the specific action on the operation screen shown in FIG. The position P1 corresponds to the position information [x10, y10] in FIG. 6, and the position P2 corresponds to the position information [x20, y20] in FIG.

Further, the registration processing unit 414 registers motion information 54 based on the teaching information 52 shown in FIG. FIG. 19 shows an example of the motion information 54. As shown in FIG. Information indicating operation information when the autonomous mobile device 10 is caused to travel autonomously is registered in the operation information 54 . For example, position information [x1, y1] is registered in association with motion information when the autonomous mobile device 10 starts autonomous travel. Further, the position information [x10, y10] is registered in association with motion information of a specific motion registered by the worker A by performing a specific gesture operation (pointing). Further, the position information [x20, y20] is registered in association with motion information of a specific motion registered by the worker A by performing a specific gesture operation (waving hand). Further, the position information [x50, y50] is registered in association with operation information when the autonomous mobile device 10 ends autonomous traveling. The generation processing unit 416 generates a travel route including the travel locus and the motion information 54 . The generation processing unit 416 stores the generated travel route information in the storage unit 50 in association with the travel route ID and the travel route name.

For example, the storage unit 50 stores information of a travel route (travel route ID “0001”) corresponding to the environment map M1 on the facility floor F1 and a travel route (travel route ID “0001”) corresponding to the environment map M2 on the facility floor F2. "0002") and information on the travel route (travel route ID "0003") corresponding to the environment map M3 of the facility floor F3.

Note that the generation processing unit 416 generates a map (environmental map) based on the image captured by the camera 43 while the autonomous mobile device 10 is traveling following the worker A, and displays the map on the map. A driving route may be generated.

Here, since the travel trajectory of the autonomous mobile device 10 following the worker A corresponds to the movement trajectory of the worker A, there is a possibility that it may become oblique or zigzag with respect to the passage. . Therefore, the generation processing unit 416 may generate the running route by correcting the running locus to a route corresponding to the running motion of the autonomous mobile device 10 . For example, the generation processing unit 416 corrects the traveling route to a straight traveling route, or corrects the traveling route so that a left-right turning route turns 90 degrees. In addition, the generation processing unit 416 may generate a travel route by correcting the movement locus of the worker A to a position that allows avoidance of obstacles in the security target area.

The travel processing unit 418 executes autonomous travel according to the travel route generated by the generation processing unit 416 . For example, the facility manager operates the operation unit 20 (see FIG. 2) to select a travel route (travel route ID, travel route name, etc.) for autonomous travel. When the travel route is selected, the travel processing unit 418 reads the information of the travel route (see FIGS. 18 and 19) from the storage unit 50, autonomously travels along the route R1 shown in FIG. An operation corresponding to the indicated operation information 54 is executed. As a result, the autonomous mobile device 10 starts autonomous traveling at, for example, the position P0, and when it reaches the position P1[x10, y10], it temporarily stops and causes the surveillance camera 44 to photograph the direction of 90 degrees to the left for 30 seconds. After that, the autonomous mobile device 10 travels along the route R1, and when it reaches the position P2[x20, y20], it temporarily stops and rotates the surveillance camera 44 360 degrees to capture images for 90 seconds. After that, the autonomous mobile device 10 travels along the route R1 and ends the autonomous travel when it reaches the position G[x50, y50].

As described above, when the autonomous mobile device 10 according to the present embodiment is a security robot, the autonomous mobile device 10 autonomously travels along the travel route in accordance with the registered contents of the teaching operation by the worker A, and moves to the position P1. , P2 to guard the guarded area.

[Driving route generation processing]
Hereinafter, the travel route generation processing executed in the autonomous mobile device 10 will be described with reference to FIGS. 20 and 21. FIG. Specifically, in the present embodiment, the control unit 40 of the autonomous mobile device 10 executes the travel route generation process.

The present invention can be regarded as an invention of a travel route generation method that executes one or more steps included in the travel route generation process. Also, one or more steps included in the travel route generation process described here may be omitted as appropriate. The execution order of each step in the travel route generation process may differ within the range of producing the same effect. Furthermore, here, a case where the control unit 40 executes each step in the travel route generation processing will be described as an example, but one or more processors perform the steps in the travel route generation processing in a distributed manner. A path generation method is also contemplated as another embodiment.

Upon receiving an instruction to start the teaching operation from the operator, the control unit 40 executes the travel route generation process.

First, in step S<b>11 , the control unit 40 acquires a captured image from the camera 43 . For example, when the operator A performs an operation to start a teaching operation on the autonomous mobile device 10 , the control unit 40 outputs a photographing instruction to the camera 43 and acquires a captured image captured by the camera 43 .

When the control unit 40 acquires the captured image from the camera 43 (S11: Yes), in step S12, a person is detected from the captured image. When a person is detected (S12: Yes), the control unit 40 causes the display panel 21 to display an image of the detected person in step S13 (see FIG. 10).

Next, in step S14, the control unit 40 displays an inquiry screen asking whether the person displayed on the display panel 21 is a worker (see FIG. 11). When the worker nods in front of the camera 43, the control unit 40 obtains confirmation of the worker and recognizes the detected person as the worker performing the teaching operation (S14: Yes). The control unit 40 repeats the processing of steps S1 to S4 until it recognizes the operator performing the teaching operation.

When the controller 40 recognizes (determines) the operator who performs the teaching operation (S14: Yes), the controller 40 executes the teaching process in step S15 (see FIG. 12). After starting the teaching process, the control unit 40 detects the operator from the captured image acquired from the camera 43 in step S16 (see FIG. 13). If the operator cannot be detected from the captured image acquired from the camera 43 (S16: No), the controller 40 displays warning information indicating that the operator cannot be recognized on the operation screen in step S161 (see FIG. 14). reference).

When the control unit 40 detects the operator from the captured image acquired from the camera 43 (S16: Yes), the control unit 40 acquires the position information of the autonomous mobile device 10 while the autonomous mobile device 10 does not stop for a predetermined time (S17: No). (S171). The control unit 40 registers the acquired position information in the teaching information 52 (see FIG. 6). In this way, the control unit 40 accumulates the position information of the movement trajectory of the autonomous mobile device 10 while the autonomous mobile device 10 recognizes and follows the worker (see FIG. 6).

Here, when the operator stops at an arbitrary position and the autonomous mobile device 10 stops for a predetermined period of time (S17: Yes), the control unit 40 registers a specific action on the operation screen shown in FIG. 15 in step S18. Announcement information regarding a gesture operation (for example, "raise right hand") to shift to the mode and a gesture operation (for example, "raise left hand") for ending the teaching operation is displayed.

Next, in step S<b>19 , the control unit 40 determines whether or not the operator's “raise right hand” gesture operation is detected from the captured image acquired from the camera 43 . When the control unit 40 detects the gesture operation of "raising right hand" (S19: Yes), the process proceeds to step S20. On the other hand, when the control unit 40 does not detect the gesture operation of "raising right hand" (S19: No), the process proceeds to step S191.

In step S<b>20 , the control unit 40 shifts to a registration mode for registering specific actions to be executed by the autonomous mobile device 10 .

Next, in step S21, the control unit 40 determines whether or not the gesture operation detected from the captured image acquired from the camera 43 is a specific gesture operation. If the gesture operation is the specific gesture operation (S21: Yes), the process proceeds to step S22. If the gesture operation is not the specific gesture operation (S21: No), the process returns to step S19. Note that the specific gesture operation here is a gesture operation for registering a specific action, such as "pointing", "waving", or "bending down", to be performed when the autonomous mobile device 10 autonomously travels. .

In step S22, the control unit 40 displays inquiry information for confirming the detected specific gesture operation on the operation screen shown in FIG.

Next, in step S23, the control unit 40 determines whether or not an operation for registering motion information corresponding to the specific gesture operation has been received from the operator. For example, when the worker nods toward the camera 43 (see FIG. 16), the control unit 40 accepts an operation for registering motion information corresponding to the specific gesture operation.

Next, in step S24, the control unit 40 registers motion information corresponding to the specific gesture operation. For example, when the specific gesture operation at the current position [x10, y10] of the autonomous mobile device 10 is "pointing", the control unit 40 performs "an operation of photographing the pointing direction by the monitoring camera for the pointing operation time". is associated with the position [x10, y10] (time t13 in FIG. 6). Further, for example, when the specific gesture operation at the current position [x20, y20] of the autonomous mobile device 10 is "waving", the control unit 40 performs "moving 360 degrees with the monitoring camera for the duration of the hand waving". ” is associated with the position [x20, y20] (time t23 in FIG. 6). The control unit 40 registers motion information about the specific motion in the motion information 54 (see FIG. 19). Further, the control unit 40 displays the registered contents on the operation screen shown in FIG. 17 . After the control unit 40 registers the motion information, the process returns to step S15. After returning to step S15, the control unit 40 causes the autonomous mobile device 10 to follow the operator.

Returning to step S15, when the autonomous mobile device 10 travels following the worker and detects the specific gesture operation "waving hand" at the position [x20, y20], the control unit 40 performs "360 degree is registered in association with the position [x20, y20] (time t23 in FIG. 6). The control unit 40 registers motion information about the specific motion in the motion information 54 (see FIG. 19).

Here, in step S191, when the control unit 40 detects the operator's "raise left hand" gesture operation (S191: Yes), the control unit 40 ends the teaching process (S192). After that, the control unit 40 determines the travel route based on the trajectory information (see FIG. 18) of the travel trajectory along which the autonomous mobile device 10 travels following the worker and the registered motion information 54 (see FIG. 19). Generate. The control unit 40 stores the generated travel route information in the storage unit 50 .

As described above, the control unit 40 generates the travel route of the autonomous mobile device 10 . The operator selects a desired travel route from one or a plurality of travel routes (route patterns) registered in the storage unit 50 when causing the autonomous mobile device 10 to travel autonomously. The control unit 40 causes the autonomous mobile device 10 to autonomously travel according to the travel route selected by the operator.

As described above, the autonomous mobile device 10 according to the present embodiment recognizes the worker based on the detection information (eg, captured image) of the detection unit (eg, camera 43) mounted on the autonomous mobile device 10 . In addition, the autonomous mobile device 10 travels following the recognized worker. Further, the autonomous mobile device 10 detects a predetermined registration operation (for example, a gesture operation) of the worker based on the detection information, and when the detected registration operation is a preset specific registration operation, the Motion information in which the specific motion corresponding to the specific registration operation and the position information of the autonomous mobile device 10 are associated with each other is registered. In addition, the autonomous mobile device 10 generates a travel route for autonomous travel based on trajectory information of a travel trajectory that follows the worker and the registered motion information.

According to this configuration, in the teaching operation, the worker can register the travel route by moving (walking) the route on which the autonomous mobile device 10 is desired to travel autonomously. Further, when the autonomous mobile device 10 wants to perform a specific action at a predetermined position on the travel route, the worker can register the specific action by performing a predetermined gesture operation in the teaching operation. can be done. In this way, the operator does not need to set or register the travel route using an operation terminal or the like, so that the teaching operation can be easily performed. Therefore, even an operator unfamiliar with the teaching operation can perform the teaching operation. Also, the working efficiency of the teaching operation can be improved.

Note that the method of registering the specific action in the teaching operation is not limited to a method of gesture operation using the operator's body. For example, the specific action may be registered by the voice of the operator. For example, when the worker utters a voice saying "take a picture of 90 degrees to the left for 30 seconds with a surveillance camera" at the position [x10, y10], the control unit 40 acquires the voice and identifies the content. The motion is registered in association with the position [x10, y10].

In the above-described embodiment, the case where the autonomous mobile device 10 is a security robot was exemplified, but the autonomous mobile device of the present invention may be, for example, a cleaning robot. When the autonomous mobile device is a cleaning robot, the autonomous mobile device performs cleaning work while autonomously traveling along the generated travel route. Also, the autonomous mobile device executes specific actions at the positions P1 and P2. Here, as the specific action, for example, an action of focusing on cleaning work at the position P1, an action of spraying chemicals to remove dirt at the position P2, and the like are registered.

As another embodiment, the autonomous mobile device of the present invention may be, for example, a guide robot. When the autonomous mobile device is a guide robot, the autonomous mobile device autonomously travels along the generated travel route and performs specific actions at positions P1 and P2. Here, as the specific action, for example, an action of displaying a specific advertisement according to the position, an action of announcing the advertisement information by voice, etc. are registered.

In the above-described embodiment, the autonomous mobile device 10 alone corresponds to the autonomous mobile system according to the present invention. It may contain components. For example, when a plurality of components of the autonomous mobile device 10 and the server cooperate to share and execute the travel route generation processing, the present invention includes a system including a plurality of components that perform the processing. It can be regarded as such an autonomous driving system. For example, the server alone may configure the autonomous driving system according to the present invention. Specifically, the server includes each processing unit (acquisition processing unit 411, display processing unit 412, recognition processing unit 413, registration processing unit 414, detection processing unit 415, generation processing unit 416 , a notification processing unit 417, and a traveling processing unit 418) to control the autonomous mobile device 10. Also, the server may store map information 51 , teaching information 52 , gesture information 53 , and motion information 54 . The server may be configured by a physical server, or may be configured by a virtual server (cloud server).

10 : Autonomous mobile device 21 : Display panel 40 : Control unit 51 : Map information 52 : Teaching information 53 : Gesture information 54 : Motion information 411 : Acquisition processing unit 412 : Display processing unit 413 : Recognition processing unit 414 : Registration processing unit 415 : Detection processing unit 416 : Generation processing unit 417 : Notification processing unit 418 : Travel processing unit

Claims (11)

An autonomous traveling system that causes an autonomous traveling device to travel based on a travel route,
a recognition processing unit that recognizes a worker based on detection information from a detection unit mounted on the autonomous mobile device;
a travel processing unit that causes the autonomous mobile device to travel while following the worker;
a detection processing unit that detects a predetermined registration operation of the worker based on the detection information;
When the registration operation detected by the detection processing unit is a preset specific registration operation, registration of operation information in which a specific operation corresponding to the specific registration operation and position information of the autonomous mobile device are associated with each other. a registration processing unit that
a generation processing unit that generates the travel route based on trajectory information of the travel trajectory along which the autonomous mobile device travels following the worker and the motion information registered by the registration processing unit;
autonomous driving system. The detection unit is a camera,
The recognition processing unit recognizes the worker based on an image captured by the camera,
The detection processing unit detects a predetermined registration operation of the worker based on the captured image.
The autonomous driving system according to claim 1. The detection processing unit detects a predetermined gesture operation of the worker,
When the gesture operation detected by the detection processing unit is a preset specific gesture operation, the registration processing unit mutually registers the specific action corresponding to the gesture operation and the position information of the autonomous mobile device. registering the associated operation information;
The autonomous driving system according to claim 1 or 2. The registration processing unit registers first action information for causing the autonomous mobile device to perform a first action at a first position when the gesture operation detected by the detection processing unit is a first gesture operation, registering second motion information for causing the autonomous mobile device to perform a second motion at a second position when the gesture motion detected by the detection processing unit is a second gesture motion;
The autonomous driving system according to claim 3. Further comprising a display processing unit that displays the detection information on a display unit mounted on the autonomous mobile device,
The autonomous driving system according to any one of claims 1 to 3. The registration processing unit receives a registration operation from the worker for the detected information displayed on the display unit, and the registration operation detected by the detection processing unit is the specific registration operation. registering the operation information when
The autonomous driving system according to claim 5. The generation processing unit generates a map based on the detection information detected by the detection unit while the autonomous mobile device is following the worker and generates the travel route on the map. do,
The autonomous driving system according to any one of claims 1 to 6. The generation processing unit generates the driving route by correcting the driving locus to a route according to the driving operation of the autonomous mobile device.
The autonomous driving system according to any one of claims 1 to 7. Further comprising a notification processing unit that notifies warning information when the autonomous mobile device cannot follow the worker,
The autonomous driving system according to any one of claims 1 to 8. An autonomous traveling method for traveling an autonomous mobile device based on a traveling route,
one or more processors
a recognition step of recognizing a worker based on detection information from a detection unit mounted on the autonomous mobile device;
a traveling step of causing the autonomous mobile device to travel while following the worker;
a detection step of detecting a predetermined registration operation of the worker based on the detection information;
When the registration operation detected in the detection step is a preset specific registration operation, registering operation information in which a specific operation corresponding to the specific registration operation and position information of the autonomous mobile device are associated with each other. a registration step;
a generation step of generating the travel route based on trajectory information of the travel trajectory along which the autonomous mobile device travels following the worker and the motion information registered in the registration step;
Autonomous driving method to perform. An autonomous traveling program for causing an autonomous mobile device to travel based on a travel route,
a recognition step of recognizing a worker based on detection information from a detection unit mounted on the autonomous mobile device;
a traveling step of causing the autonomous mobile device to travel while following the worker;
a detection step of detecting a predetermined registration operation of the worker based on the detection information;
When the registration operation detected in the detection step is a preset specific registration operation, registering operation information in which a specific operation corresponding to the specific registration operation and position information of the autonomous mobile device are associated with each other. a registration step;
a generation step of generating the travel route based on trajectory information of the travel trajectory along which the autonomous mobile device travels following the worker and the motion information registered in the registration step;
is executed by one or more processors.

Images