JP2022003437A - Control system and control method of mobile body - Google Patents

Abstract

To provide a mobile body control system which has a small load to an operator and a small load to a circuit and can confirm travel to a destination of an automatic mobile body.SOLUTION: A mobile body control system includes a mobile body A, environmental information acquisition means B, a terminal device C provided separately from the mobile body, and movement control means D. The environmental information acquisition means detects an obstacle existing around the mobile body to provide the terminal device with the obstacle. The terminal device includes map creation means C1 for creating map information including information of the obstacle, image display means C2 capable of displaying the information of the obstacle, input means C4 capable of changing the information of the obstacle, and route calculation means C3 for calculating a route to a destination. The map creation means creates virtual map information including the changed information of the obstacle. The route calculation means calculates a virtual route to the destination on the basis of the virtual map information. The terminal device provides the movement control means with the information of the obstacle, the virtual map information and/or the virtual route. The movement control means starts movement of the mobile body along the virtual route.SELECTED DRAWING: Figure 2

Description

The present invention relates to a control system and a control method for an autonomous mobile body, and more particularly to a control system and a control method for the mobile body including the mobile body and a terminal device for a remote operator that can be connected to the mobile body.

Conventionally, a robot that memorizes an environmental map and autonomously moves to a target point has been known. For example, Patent Document 1 discloses a mobile robot capable of autonomously moving along a predetermined route while avoiding obstacles in an environment with obstacles. The mobile robot superimposes the environmental map information and the map information indicating the position of the obstacle observed by the stereovision sensor to generate an integrated grid map by the integrated map creation unit, and weights according to the type of obstacle. Calculate the product with the reciprocal of the distance. For example, use a weight of 1.0 for a wall and 0.5 for a human to set a safe distance between the robot and each object. A threshold is set for this value to determine the free space that is open in front of the robot, and the optimum route is selected with all open free spaces as the destinations.

Various methods for determining the type of obstacle have also been proposed. In Patent Document 2, an obstacle detecting means for detecting an obstacle existing in the vicinity, a map information storing means for storing map information, and a position on a map of the apparatus main body are calculated from the amount moved by the moving means. In a self-propelled device equipped with a microcomputer means for monitoring, by counting the number of times the device body has passed and the number of times an obstacle detected by the obstacle detecting means has been detected for each map position, a temporary obstacle can always be detected. It is disclosed to distinguish existing obstacles and estimate the travelable area even in a dynamically changing environment. In Patent Document 3, an image is acquired via a camera, a point cloud is acquired via a light detector, an object in the image is associated with a point in an object cluster of the point cloud, and the object is attached to the object in the image. It is disclosed that the three-dimensional (3D) object of the point cloud is identified by the label, and the dynamic traffic participants such as automobiles, pedestrians, bicycles, or animals in the point cloud are identified and dynamically. It is disclosed to remove traffic participants from the point cloud to perform more accurate location estimation and mapping.

Furthermore, a technique in which a human intervenes in the movement of a moving body has also been proposed. The above-mentioned Patent Document 2 is provided with a map information correction means capable of correcting a part or all of the map information stored in the map information storage means, so that the user can grasp the current situation. However, the map information is rewritten according to the user's intention, for example, the obstacles that the self-propelled device mistakenly recognizes are removed, or the obstacles that the self-propelled device does not recognize, the steps, the stove, and other places that you do not want to drive are the obstacles. It is disclosed that any shape intended by the user can be specified as a boundary wall. Further, in Patent Document 4, an obstacle that has existed while the autonomous traveling robot is traveling on a set path is imaged by using an image pickup unit, and the type of the obstacle is identified by pattern matching. Select an avoidance action from the avoidance action data to avoid the obstacle, and if the obstacle cannot be avoided, output a remote control request signal to the operation terminal and avoid the obstacle by remote control of the operator of the operation terminal. Is disclosed.

Japanese Unexamined Patent Publication No. 2007-249632 Japanese Unexamined Patent Publication No. 2007-323402 Japanese Unexamined Patent Publication No. 2019-207220 Japanese Unexamined Patent Publication No. 2013-206237

In the conventional autonomous mobile body, the route is calculated by avoiding obstacles when calculating the route, but in a situation where the route cannot be calculated in an environment where obstacles are crowded, or a detour route is used to avoid obstacles. May be calculated. In particular, when setting a safe distance between the robot and each object as in Patent Document 1, a region (non-movable region) that does not serve as a path in a range wider than the actual size of the obstacle for a safety margin. Is set. Therefore, in reality, even if there is a gap through which the moving body can pass, the route may be blocked by the immovable area, and the route may not be calculated or the detour route may be calculated. As described in Patent Documents 2 and 3, even when temporary or dynamic obstacles are discriminated and ignored to perform more accurate position estimation and cartography, route calculation is performed. In some cases, if a temporary or dynamic obstacle actually exists, it cannot be ignored, and the route is calculated while avoiding the obstacle, so that the above problem cannot be solved.

Further, Patent Document 2 discloses that the user removes an obstacle mistakenly recognized by the self-propelled device, but this only corrects the misrecognition of an obstacle that does not actually exist. It does not apply when an obstacle actually exists and the route cannot be calculated. Furthermore, it is unclear to the user what the route will be after removing the obstacle of misrecognition, and it is not certain that the self-propelled device can travel to the destination.

Furthermore, although there are robots that can be remotely controlled by humans with a controller, since one person needs to operate one robot, the merits of automation, productivity improvement, cost reduction, mass production, etc. by the robot are reduced. When managing a plurality of robots over a long period of time, there is also a problem that the burden on the operator becomes large. In addition, in order to grasp the situation of the robot, it is necessary to constantly send various information (voice information, image information, etc.) from various sensors acquired by the robot from the robot to the controller, and further, the controller must send the robot to the controller. It is also necessary to send the operation details of the robot. For this reason, the amount of communication transmitted and received between the robot and the controller becomes enormous, the processing speed may decrease, and the responsiveness may deteriorate. In addition, when a human remotely controls the robot, it is necessary to check the image around the robot in real time while manipulating the robot, but there is also a problem of difficulty in maneuvering due to a time lag. Patent Document 2 discloses that an arbitrary shape intended by the user is designated as a boundary wall and travels in a space inside the boundary wall intended by the user, but a route is calculated in an environment where obstacles are crowded. It cannot solve the situation that cannot be solved. Further, Patent Document 4 discloses that an obstacle is avoided by remote control of an operator of an operation terminal. However, even in Patent Document 4, it is only an ad hoc response to the obstacles that it faces in the end, and it does not determine the completion of traveling to the final destination.

The present invention aims to solve a part of the above-mentioned problems, and in particular, the burden on the operator and the burden on the line are small, and the traveling body of the autonomous moving body can be confirmed to travel to the destination. One of the purposes is to provide a control system or control method for the above.

In order to solve the above problems, the moving body control system of the present invention is provided separately from the moving body having the moving body, the environmental information acquisition means for acquiring the environmental information around the moving body, and the moving body. A moving body control system including a remote operator terminal device and a moving control means for controlling the moving means of the moving body, wherein the environmental information acquisition means is around the moving body as environmental information. It is possible to detect obstacles existing in the remote operator and provide environmental information to the remote operator terminal device, and the remote operator terminal device creates map information including information on obstacles included in the environmental information. The creation means, the image display means capable of displaying the information of the obstacle included in the map information, the input means capable of changing the information of the obstacle displayed on the image display means, and the departure position based on the map information. The map creation means includes virtual map information including information on obstacles changed by the input means, and the route calculation means includes the route calculation means for calculating a route from the destination to the destination. Based on the virtual map information, the virtual route from the self-position or the departure position of the moving body to the destination is calculated, and the remote operator terminal device informs the movement control means of the obstacle information changed by the input means. , The virtual map information and / or the virtual route can be provided, and the movement control means can provide information on obstacles changed by the input means provided from the remote operator terminal device, the virtual map information, and the virtual map information. / Or, based on the virtual path, the moving means of the moving body is controlled to start the movement of the moving body along the virtual path.

Further, in the moving body control system, the moving body control system further includes an image information acquisition means for acquiring an image of an obstacle, and the image information acquisition means has an obstacle on the remote operator terminal device. The image information can be provided, and the remote operator terminal device may be able to display the image information of the obstacle on the image display means.

Further, in the control system of the mobile body, the environmental information acquisition means and the movement control means are provided in the mobile body, and the mobile body is the same route calculation method as the route calculation means of the remote operator terminal device. It may have a route calculation means capable of executing.

Further, in the moving body control system, the moving body may have a map making means capable of executing the same map making method as the map making means of the remote operator terminal device.

Further, in the moving body control system, the map information is divided into a plurality of cells, a cost based on the obstacle is set as information on the obstacle in each cell, and the input means is the cell. The cost may be variable, and the remote operator terminal device may be based on the selected obstacle when the obstacle included in the map information displayed on the image display means is selected by the input means. The cost of the area where the cost is set may be changed. Further, when at least one cell of the map information displayed on the image display means is selected by the input means, the remote operator terminal device passes around an obstacle including the selected cell or around the selected cell. It may be changed to a possible cost.

Further, in the mobile control system, when the remote operator terminal device is set as a movable obstacle by the input means, it creates virtual map information in a state where the set obstacle does not exist. Then, a virtual route is calculated, and information on the movable obstacle is provided to the movement control means. When the information on the movable obstacle is provided, the movement control means moves and passes through the movable obstacle. A wait process may be executed that waits until it becomes possible.

Further, in the mobile body control system, the remote operator terminal device is set in advance when the route to the destination cannot be calculated by the route calculation means or when an obstacle blocking the moving route is detected. When the information on the obstacle is acquired, the route is changed, or the moving body moves off the route, the operator may be requested to change the information on the obstacle.

Further, in the moving body control system, the remote operator terminal device displays the virtual route calculated by the route calculating means on the image display means, and after receiving the transmission permission signal from the operator to the moving body. , The obstacle information changed by the input means, the virtual map information and / or the virtual route may be provided to the movement control means.

Further, in the mobile control system, the remote operator terminal device may display a candidate for a change content or a candidate for a virtual route obtained by simulation on the image display means.

Further, in the moving body control system, the moving control means may execute a contact operation of bringing the moving body into contact with an obstacle.

Further, the method for controlling a moving body of the present invention is for a moving body having the moving means, an environmental information acquisition means for acquiring environmental information around the moving body, and a remote operator provided separately from the moving body. A method for controlling a moving body including a terminal device and a moving control means for controlling the moving means of the moving body, and an obstacle existing around the moving body as environmental information by the environmental information acquisition means. And a step of providing environmental information to the remote operator terminal device, and a step of creating map information including information on obstacles included in the environmental information by the map creating means of the remote operator terminal device. , The information of the obstacle included in the map information is displayed on the image display means of the remote operator terminal device, and the information of the obstacle displayed on the image display means by the input means of the remote operator terminal device is changed. A step of creating virtual map information including information on obstacles changed by the input means by the map creating means, and a route calculation means of the remote operator terminal device based on the virtual map information. The step of calculating the virtual route from the self-position or the departure position of the moving body to the destination, the information of the obstacle changed by the input means, the virtual map information and / or the virtual route to the movement control means. And the movement control means, the movement is based on the obstacle information, the virtual map information and / or the virtual route changed by the input means provided from the remote operator terminal device. It comprises a step of controlling the moving means of the body and initiating the movement of the moving body along the virtual path.

Further, in the moving body control method, an image information acquisition means for acquiring an image of an obstacle is further included, and the image information of the obstacle is acquired by the image information acquisition means, and the obstacle is attached to the remote operator terminal device. It may include a step of providing image information of an object.

Further, in the method for controlling the mobile body, the environmental information acquisition means and the movement control means are provided on the mobile body, and the mobile body is the same route calculation method as the route calculation means of the remote operator terminal device. It may have a route calculation means capable of executing.

Further, in the method of controlling the moving body, the moving body may have a map-creating means capable of executing the same map-creating method as the map-creating means of the remote operator terminal device.

Further, in the moving body control method, the map information is divided into a plurality of cells, and a cost based on the obstacle is set as information on the obstacle in each cell, and the cell is set by the input means. In addition, the cost of the area may be changed by selecting an obstacle included in the map information displayed on the image display means by the input means and setting the cost based on the selected obstacle. The moving body may be changed to a movable value. Further, at least one cell of the map information displayed on the image display means by the input means may be selected and changed to a cost capable of passing through an obstacle around the selected cell or including the selected cell.

Further, in the above-mentioned mobile object control method, when an obstacle is set as a movable obstacle by the input means, virtual map information in a state where the set obstacle does not exist is created, a virtual route is calculated, and a movable obstacle is obtained. Information on the object may be provided to the movement control means, and the movement control means may pass the position of the movable obstacle after the movable obstacle has moved.

Further, in the method for controlling the moving body, the remote operator terminal device is set in advance when the route to the destination cannot be calculated by the route calculating means or when an obstacle blocking the moving route is detected. When the information on the obstacle is acquired, the route is changed, or the moving body moves off the route, the operator may be requested to change the information on the obstacle.

Further, in the method for controlling the moving body, when the virtual route can be calculated by the route calculation means based on the virtual map information, the obstacle information changed by the input means, the virtual map information and / or the above. The virtual route may be provided to the movement control means.

Further, in the method for controlling a mobile body, the remote operator terminal device may display a candidate for a change content or a candidate for a virtual route obtained by simulation on the image display means.

Further, in the method of controlling the moving body, the moving control means may execute a contact operation of bringing the moving body into contact with an obstacle.

According to the mobile control system of the present invention, when a problem caused by a surrounding obstacle occurs in an autonomous mobile, the remote operator terminal device (hereinafter, also referred to as “terminal device”) is used. It is possible to seek the judgment of a remote operator for problems caused by obstacles. In the terminal device, the provided obstacle information can be displayed on the image display means, the operator can confirm the obstacle, and the input means can change the obstacle information included in the map information. Obstacles include empty cardboard, packing materials, paper bags, balloons, and other light movable obstacles that are not damaged by a moving object and can be pushed away by the moving object, soft deformable obstacles, etc. There is also. In addition, there are desks and chairs that can pass underneath, or doors that can easily move obstacles depending on the pushing direction, and the operator can contact from the obstacle information. It is possible to determine whether or not the obstacle can be pushed away, and the moving object can be made to perform a contact operation. Further, if it is a temporary obstacle, the operator can determine that the obstacle can move and pass through after waiting for a while, and the moving body can be made to execute the standby process. In addition, for unknown obstacles, the safety margin may be set high and the cost area may be set excessively wide, but the operator confirms what kind of obstacle it is from the obstacle information and sets it in the appropriate cost area. By changing it, it may be possible to calculate the route to the destination while avoiding obstacles. The mobile control system of the present invention can respond to various situations that require the judgment of such an operator by changing the information of obstacles. Moreover, since the terminal device of the present invention calculates the virtual route to the destination based on the virtual map information including the information of the obstacle changed by the input means by the route calculation means, it is possible to have the operator confirm the virtual route. It is possible to send information to the moving object after confirming that it has arrived at the destination, so highly reliable support can be provided with a small amount of communication. In particular, the operator's work is only to select an obstacle and change the cost or the like, and the burden on the operator can be reduced. In addition, the information received by the operator during operation is static map information or environmental information (obstacle position), and the virtual route is calculated on the terminal device, eliminating the real-time nature of the operation and operating due to the communication time lag. Can solve the difficulties of. Furthermore, by limiting the information sent to and received from the operator to static map information, environmental information, virtual routes, and changes in obstacle information, and calculating the virtual route on the terminal device, the burden on the communication line is increased. Can be reduced. Other effects will be described in the embodiments.

An example of a flowchart of a mobile body control method of the present invention Block diagram of components required for the mobile control system of the present invention An example of hardware that realizes the control system of the present invention A block diagram showing a schematic configuration of the control system of the present invention. (A) is a schematic plan view showing the floor plan of the moving range, and (B) is a figure showing the middle of the environmental information acquisition work for creating the basic map information. (A) is an arrangement pattern of measurement points of environmental information in the entire movement range, and (B) is a plan view of map information. (A) is a plan view schematically showing the situation at the time of creating the measurement map information, and (B) and (C) are image information of newly detected obstacles. (A) is measurement map information based on environmental information, and (B) and (C) are diagrams showing virtual map information and virtual routes. An example of a flowchart of a moving body route calculation assisting process in a moving body An example of a flowchart of a mobile route calculation assisting process in a terminal device An example of a flowchart of a process for presenting virtual route candidates in a terminal device An example of a flowchart when shooting a moving subject in a moving object

FIG. 1 is an example of a flowchart of a mobile body control method of the present invention. In the control system and control method for a moving body of the present invention, when the moving body cannot calculate the route to the destination due to an obstacle, or when an obstacle that blocks the moving route is detected, the route of the moving body is changed. When the operator's judgment is required due to surrounding obstacles, such as when the vehicle is changed due to an obstacle or when the moving object moves off the route, the remote operator terminal device (terminal device) is used. Through this, the judgment of a remote operator can be sought for problems caused by obstacles. In S1 of FIG. 1, the terminal device receives information on obstacles such as environmental information, image information, and map information around the moving body, and in S2, the map information including the received information on obstacles is used as an image display means. indicate. Further, the terminal device makes the input means operable, requests the operator to change the obstacle information as needed, and in S3, the operator changes the obstacle information. In S4, the terminal device acquires the information of the obstacle changed by the operator, and in S5, creates the virtual map information including the information of the obstacle changed by the input means by the map creating means. Then, in S6, the terminal device calculates the virtual route from the self-position or the departure position of the moving body to the destination based on the virtual map information by the route calculation means. Next, in S7, it is determined whether or not the virtual route can reach the destination. When the virtual route cannot be calculated (No in S7), the operator is notified that the route to reach the destination cannot be calculated, and a new obstacle information change is requested. When the destination is reached (Yes in S7), the changed contents and the like are transmitted to the moving body or the moving control means in S8. The changed contents include the changed obstacle information, virtual map information, and virtual route. Before S8, the virtual route may be displayed to the operator, the necessity of transmission to the mobile body may be confirmed, and the information may be transmitted after receiving the transmission permission signal from the operator.

In this way, in the terminal device, the obstacle information included in the map information is displayed on the image display means, and the operator can change the obstacle information by the input means while checking the obstacle information. .. Further, the terminal device can create virtual map information including information on obstacles changed by the input means by the map information creating means, and calculate a virtual route to the destination based on the virtual map information by the route calculation means. Therefore, it is possible to confirm whether the virtual route to the destination can be calculated before actually starting the movement of the moving object. The mobile body solves the problem caused by the surrounding obstacles by moving along a virtual route that reflects the judgment of the operator at a remote place. As described above, in the moving body control system and the control method of the present invention, by calculating the virtual path of the moving body in advance, it can be confirmed whether the changed contents by the operator are sufficient or insufficient, and the virtual path can be calculated. It is possible to start the movement of the moving body after making sure. In addition, obstacles include light movable obstacles, such as empty cardboard, packing materials, paper bags, balloons, and cushions, which are not very damaged by the collision of moving objects and can be pushed away by the moving objects, and are softly deformable. There are also obstacles. In addition, there are desks and chairs that can pass underneath, or doors that can easily move obstacles depending on the pushing direction, and the operator can contact from the obstacle information. You can judge whether it is an obstacle that can be pushed away. As described above, in the control system and control method for the moving body of the present application, it is possible to execute the operation of bringing the moving body into contact with an obstacle (contact operation) at the discretion of the operator. Further, if it is a temporary obstacle, the operator can determine that the obstacle can move and pass through after waiting for a while, and the standby process may be executed accordingly. Also, when creating map information for an unknown obstacle, the safety margin may be set high and the cost area may be set excessively wide, but the operator confirms what kind of obstacle it is from the obstacle information. By changing to an appropriate cost area, it may be possible to avoid obstacles and calculate the route to the destination. The mobile control system of the present invention can respond to various situations that require the judgment of such an operator by changing the information of obstacles, and the terminal device executes up to route planning. After the operator confirms the virtual route, the changed information can be provided to the moving object.

Here, "obstacles" are objects and structures that hinder movement, such as people, luggage, walls, pillars, stairs, doors, windows, equipment (including furniture and furniture), and other moving objects. Etc. are included. A "fixed obstacle" is basically an obstacle whose arrangement does not change, and includes, for example, walls, pillars, stairs, fixed equipment, etc., and is the basic configuration of basic map information created in advance. Is preferable. A "movable obstacle" is a moving obstacle, including people, luggage, doors, windows, movable equipment, other moving objects, etc., and may have an irregular configuration in map information. Some movable obstacles do not move for a long period of time, some move in a short time, and those that move in a short time are called "temporary obstacles". Temporary obstacles include, for example, people, animals, vehicles, other moving objects, automatic doors, and the like. The "measurement point" is a point measured by the environmental information acquisition means, and the "measurement point arrangement pattern" is the mutual positional relationship of the point cloud (point cloud) of the measurement points by the environmental information acquisition means. For example, a figure obtained by connecting a group of points is included.

"Map information" indicates the arrangement of surrounding obstacles, etc., and is created from the basic map information of the basic obstacle arrangement prepared in advance and the environmental information acquired by the environmental information acquisition means. Includes measurement map information showing the placement of obstacles around the moving object at the time of measurement. Basic map information may be created by incorporating existing maps such as road maps, topographic maps, floor plans of buildings, and architectural drawings in advance, or the range of movement of moving objects may be determined in advance by using environmental information acquisition means. By SLAM (Simultaneous Localization and Mapping), self-position estimation and environmental map creation may be performed and acquired at the same time, or basic map information created by another moving object or another computer may be used. The map information may be stored, for example, in the storage means of the moving body, may be stored in the storage means of the terminal device, or may be stored in the storage server. The map information includes at least basic map information to the destination in order to calculate the route by the route calculation means, and sequentially measured map information may be accumulated by the environmental information acquisition means while traveling the route to the destination. .. As map information, a two-dimensional or three-dimensional map is divided into a plurality of cells (typically, a grid having a uniform width, a grid of voxels, etc., but is not limited to the grid), and for each cell, a moving object is used. Costs related to mobility may be set. The cost is the energy required for the movement of the moving body, and the higher the cost, the larger the energy consumed. For example, set the cost as a binary value of 0 that can be moved and 1 that cannot be moved, set the cell at the position of the obstacle to 1 that cannot be moved, and set the other cells to 0 that can be moved. Alternatively, the cost may be set to 1 which cannot be moved even for a cell having a certain distance around the obstacle. A cell in which a cost is set based on an obstacle is called a "cost area" of the obstacle, and a distance from the obstacle to the cost area is called a cost distance. By making the obstacle larger than the actual obstacle in the cost area, the moving body can be replaced with a point in the calculation of the route calculation, and the route can be calculated as a line. Further, an intermediate value between a movable value and a non-movable value may be set as the cost, and the cost may be set according to the cost distance. In this case, for example, cells near the obstacle have a high cost, cells at a distant position have a low cost, and if the route calculation means preferentially passes through the cell with a low cost, the cell is separated from the obstacle. The cell at the specified position can be set as the route. Further, the cost may be set based on the terrain and the orientation of the moving object. For example, the cost is set to be high when the moving body moves uphill on a sloping terrain, and the cost is low when the moving body moves downhill. Further, even when traveling the same distance, the cost of the latter is high depending on whether the moving body travels in the traveling direction or changes the direction. The map information can be created by the map creation means of the terminal device. Each moving body may receive map information from the terminal device, but each moving body may be provided with a map making means of the same map making program as the map making means of the terminal device.

"Obstacle information" includes the cost area of the obstacle, the arrangement of the obstacle, the coordinates of the obstacle measurement point, the arrangement pattern of the measurement point and / or a part of the image information (including the image of the obstacle). Is done. The "subject database" is a database that stores pattern images of subjects. The "obstacle database" is a database that stores information on obstacles, and includes obstacles and correspondence in virtual routes to them. For a part of the image information included in the obstacle information, the subject database can be used as the obstacle database by associating each pattern image stored in the subject database with whether or not it is an obstacle.

FIG. 2 is a block diagram of components required for a mobile control system for realizing the present invention. In FIG. 2, the control system is a mobile body A having a mobile body A1, an environmental information acquisition means B for acquiring environmental information around the mobile body A, and a remote operator terminal provided separately from the mobile body A. The device C, the movement control means D for controlling the movement means of the moving body A, and the image information acquisition means E and the storage means F may be included. In FIG. 2, the dotted line connecting each element means that information can be provided to each other or to one side, and even if both are mounted inside the same device and connected by an internal bus, they are separately connected. This includes cases where the device is mounted on the device and connected by a wired or wireless network.

The mobile body A has at least a moving means A1 for moving the moving body and a communication means A2 capable of communicating with an external device. Even if the moving body A further has an environmental information acquisition means, an image information acquisition means, a route calculation means, a movement control means, a position estimation means, a map creation means, an image recognition means, a storage means, and the like, if necessary. good. The mobile body A normally patrols a preset route, and is calculated by the terminal device C according to a control signal from the mobile control means D only when the operator's judgment is required due to surrounding obstacles. It may be configured to move to the destination along the virtual route. In this case, for example, by allowing the moving body to grasp its own position on the map, a preset route may be patrolled, or the surrounding wall, ceiling, floor, etc. may not have map information. By guiding the course by means for guiding the moving body A, for example, a magnetic tape, a transmitter, or the like, a preset route may be patrolled. In addition, the moving body A is configured to automatically move autonomously to the destination in normal times, and to move to the destination along a virtual route when the operator's judgment is required due to surrounding obstacles. You may. The mobile body A includes, for example, an automatic guided vehicle, a manned vehicle, a mobile industrial robot, a mobile robot, a mobile service robot, and an unmanned aerial vehicle (including a drone).

The moving means A1 moves the moving body A, and includes, for example, wheels, tracks, a multi-legged walking device, a propeller, and the like, and further includes driving means (motor, engine, etc.) for driving them. By controlling the output of each moving means A1 by the moving control means D, the moving body A can be moved in a predetermined direction.

The communication means A2 is a means that can be connected to an external device via a network, and is connected to, for example, a movement control means D, a terminal device C, an environmental information acquisition means B, an image information acquisition means E, a storage means F, and the like. You may. When the movement control means D is installed outside, the movement control means D may be connected to receive a control signal of the movement means A1 from the movement control means D, or the moving body A may be an environmental information acquisition means, an image. If it has information acquisition means or the like, it may be connected to the terminal device C to transmit environmental information and image information to the terminal device C. Further, if the mobile body A has a route calculation means, a movement control means, a map creation means, etc., the obstacle information, the virtual map information, or the virtual route changed from the terminal device C by connecting to the terminal device C can be obtained. You may receive it. As the communication means A2, it is preferable to adopt a wireless communication means so as not to restrict the movement of the mobile body A. For example, as a wireless LAN, Wi-Fi (Wireless Fidelity), Bluetooth (registered trademark), ZigBee (registered trademark) Trademark) or the like may be used. A router, a modem, a terminating device, a repeater, or the like may be separately installed between the network and the router.

The environmental information acquisition means B is a sensor that acquires environmental information around the moving body A, and can at least detect an obstacle. The environmental information acquisition means B may be any means that can detect the existence and position (relative position or absolute position) of the obstacle, and for example, measure the distance and direction to the obstacle existing in the surroundings. By doing so, the surrounding environment information is acquired. Examples of the environmental information acquisition means B include LiDAR (Light Detection and Ranging or Laser Imaging Detection and Ranging) using laser light, a range sensor, a stereo camera using a plurality of cameras, a pattern projection camera, and a radio wave (millimeter wave). ) Includes radars, magnetic sensors, etc. The environmental information acquisition means B is preferably provided on the moving body A, but may be provided on the outside of the moving body A, for example, on the surrounding wall, ceiling, floor, or the like. When the environmental information acquisition means B is provided on the moving body A, at least a part of the environmental information around the moving body A can be acquired, for example, the environment within a predetermined angle range around the moving body A. Information can be obtained. It is preferable to be able to acquire environmental information of the entire circumference (360 °), but at least to acquire environmental information in front of the moving body in the moving direction. When it is provided outside the moving body A, it is preferable to provide a plurality of environmental information acquisition means B on a wall, a ceiling, a floor, or the like so as to cover the moving range of the moving body A. The externally provided environmental information acquisition means B can specify the distance and direction from the environmental information acquisition means B to the obstacle, and can also detect the position of the moving body A. Although at least a part of the acquired environmental information can be provided to the terminal device C, it may be possible to provide it to the mobile body A. Further, it is preferable that the environmental information acquisition means B can acquire not only two-dimensional (2D) environmental information but also three-dimensional (3D) environmental information. The environmental information acquired by the environmental information acquisition means B varies depending on the type of the sensor, but includes the distance to the measurement point of the sensor. For example, in the case of LiDAR, micropulse laser light sequentially scans within a predetermined angle range, receives the light reflected by an obstacle and returns, measures the time until it returns, and reaches the obstacle. To get the distance and direction of. Here, the measurement point is the position of an obstacle hit by the irradiated laser beam. In the case of a stereo camera, the difference in position (misparity) in each image and between multiple cameras for a part of a specific subject (area of one pixel or several pixels) included in the images taken by multiple cameras. It is possible to recognize the distance and shape of a specific subject from the distance between the two. Here, a part of the subject for which the parallax is obtained in each pixel is a measurement point. In a stereo camera, a range in which images from a plurality of cameras overlap is a predetermined angle range, and 3D environmental information can be acquired. A plurality of cameras may be provided so as to be movable (rotatable). In addition, even with a normal camera, by attaching a marker (graphic, character, shape, etc.) of a known size to an obstacle, the presence of the obstacle can be seen from the number of dots (size) and angle of the marker in the image information. And the position can be detected. Further, a camera such as an infrared camera that can detect the distance may be used. When a camera (including a stereo camera) is adopted as the environmental information acquisition means B, it can also be used as the image information acquisition means E.

The remote operator terminal device C is provided separately from the moving body A, and displays map creating means C1 for creating map information, map information created by map creating means C1, image information of obstacles, and the like. Possible image display means C2, route calculation means C3 for calculating a route from a departure position to a destination using information on obstacles included in map information, and input means C4 for changing obstacle information. , Communication means C5 and control means C6 for controlling these are included, and storage means is included as needed. The terminal device C is provided with at least a part of the environmental information from the environmental information acquisition means B. Further, at least a part of the image information may be provided from the image information acquisition means E. The terminal device C is a device operated by a human (operator), can create map information from the environmental information provided by the environmental information acquisition means B by the map creation means C1, and can be converted into map information by the route calculation means C3. The route to the destination can be calculated based on this, and the map information or environmental information acquisition means B or the obstacle information provided by the image information acquisition means E is displayed on the image display means C2 to display the obstacle as a human. The input means C4 can cause a human to change the information of obstacles included in the map information, and the virtual means based on the information of obstacles changed by the map creating means C1 and the route calculation means C3. By creating map information and calculating a virtual route to the destination, it is possible to make a human confirm that the destination can be reached with the changed contents. Further, the terminal device C can provide the changed obstacle information, the virtual map information and / and the virtual route to the moving body A or the moving control means B. Further, the terminal device C may have a controller that allows a human to operate the mobile body A. The terminal device C may include, for example, a computer, a laptop computer, a personal digital assistant (including a smartphone, a tablet terminal), etc., and the terminal device C may be realized by one hardware, or a plurality of hardware (for example, a plurality of hardware). , The portable information terminal may be used as an image display means, an input means, a server as a map creation means, a route calculation means, etc.), and the terminal device C may be realized. Further, a plurality of mobile bodies A may be connected to one terminal device C. In the present invention, it is sufficient for a human to support only when the moving body A recognizes an obstacle that affects the route calculation, and it is possible for one person to support a plurality of moving bodies.

The map creating means C1 is realized by executing a map making program, and is realized as a part of the function of the control means C6 in FIG. The map creating means C1 creates measurement map information based on the environmental information provided by the environmental information acquisition means B, and creates virtual map information including information on obstacles changed by the input means C4. The map creation means C1 creates map information that integrates basic map information prepared in advance, past measurement map information, measurement map information based on environmental information from another moving object A or a plurality of environmental information acquisition means B, and the like. You may. The cartographic means C1 may set a cost or cost area for use in the route calculation means C3. Since the map information based on the environmental information from the plurality of environmental information acquisition means created by the map creation means C1 of the terminal device C includes the measurement map information beyond the measurement range of one environmental information acquisition means, a plurality of movements are performed. It may be shared between bodies.

The image display means C2 is a means for displaying various information, and includes, for example, a liquid crystal display (LCD), an organic EL display, a projector, and the like. If a touch panel type display is used as the image display means C2, it can also be used as the input means C4. Further, a plurality of image display means C2 may be provided corresponding to the plurality of moving bodies A, or information of the plurality of moving bodies A may be simultaneously or sequentially displayed on one image display means C2. .. The image display means C2 may display at least image information of an obstacle acquired from the moving body A, and may further display environmental information and / or map information. Even if the image of the image display means C2 is divided into two, the image information is displayed on one side, and the environmental information and / or the map information is displayed on the other side, or the environmental information and the image information can be displayed in a switchable manner. However, if the cost of the map information is superimposed on the image of the image information, the information of the obstacle in the image information can be easily grasped, and the information of the obstacle can be easily changed. In particular, the remote operator terminal device C is a process in which a human judgment is involved because a human operates the input means C4 to specify obstacle information. However, the operator's judgment is based solely on experience and intuition. It is difficult to operate a stable system due to individual differences. For this reason, it is preferable to prepare a UI (user interface) for changing obstacle information and build a system that is less likely to cause misjudgment, and it is preferable to display the cost of map information on top of the image of the image information. preferable.

The route calculation means C3 is realized by executing a route calculation program, and is realized as a part of the function of the control means C6 in FIG. The route calculation means C3 calculates the route from the departure position to the destination by using the information of the obstacle included in the map information, and in particular, based on the virtual map information including the information of the changed obstacle. Calculate the virtual route from the self-position or departure position of the moving object to the destination. When the mobile body A also has the route calculation means, the route calculation means C3 of the terminal device C needs to be able to execute the same route calculation method as the route calculation means of the mobile body A, and when calculating the virtual route, it is necessary. Execute the same route calculation method. When the mobile body A does not have the route calculation means, even if the mobile body control means D moves the mobile body A to the destination based on the route calculated by the route calculation means C3 of the terminal device C at normal times. good. The route calculation means C3 may calculate a plurality of virtual routes and display them to the operator in a selectable manner. When the route to the destination cannot be calculated due to an obstacle or the like, the route calculation means C3 may indicate to the image display means C2 that the route cannot be calculated, and may request the operator to change again, or it may be obtained by simulation. Candidates for changes or virtual routes may be presented.

The input means C4 is a means capable of changing at least a part of the obstacle information included in the map information, and includes, for example, a keyboard, a pointing device (mouse, touch pad, etc.), a touch panel (including a touch screen), and the like. .. The change of the obstacle information by the input means C4 is such that the operator can change the setting such as the cost related to the selected obstacle by selecting the obstacle included in the map information displayed on the image display means C2, for example. Alternatively, the setting of the cell may be changed by selecting at least one cell of the map information displayed on the image display means C2, which is the basis of the setting of the cell. It may be possible to change settings such as costs related to obstacles. It is preferable that the cost of the map information is superimposed on the image of the image information because the map information can be selected while looking at the image of the obstacle. Further, as a change of the obstacle information, the selected obstacle may be set as a movable obstacle (preferably a temporary obstacle). As an input method, an obstacle or cell within the range surrounded by the pointing device may be selected so that the cost within the range can be changed, or the cost of the cell in the orbit moved by moving with the pointing device, etc. May be able to be changed. Further, the cost based on the obstacle may be changed by moving or erasing the obstacle by the input means C4. Further, the terminal device C may present one or a plurality of changes or virtual route candidates before input by the input means C4 or after the operator selects an obstacle or a cell in the input means C4. The means C4 may be able to select a change content or a candidate for a virtual route. For example, when environmental information and image information are acquired, the terminal device determines a movable obstacle, and after temporarily changing the cost of the movable obstacle, it is virtual whether the destination can be reached by the cartographic means and the route calculation means. The route may be simulated and one or more changes or virtual routes that can be reached may be presented as candidates. Further, when the operator selects an obstacle, the cost of the selected obstacle may be temporarily changed and simulated. Further, when the operator selects a cell, the cost around the cell may be temporarily changed and simulated so that a virtual route that passes through the cell and reaches the destination can be calculated.

The communication means C5 is a means that can be connected to an external device via a network, and is connected to, for example, a mobile body A, a movement control means D, an environmental information acquisition means B, an image information acquisition means E, a storage means F, and the like. To. The communication means C5 may be wired or wireless, for example, a wireless mobile communication system (including 3G, 4G, 5G lines, etc.), Ethernet as a wired LAN, Wi-Fi (Wireless Fidelity), and Bluetooth (as a wireless LAN). Registered trademark), ZigBee (registered trademark) and the like may be used. A router, a modem, a terminating device, a repeater, or the like may be separately installed between the network and the router. The communication means C5 is controlled by the control means C6, receives environmental information and image information from the environmental information acquisition means B and the image information acquisition means E, and receives obstacle information and virtual map information changed by the input means C4. Alternatively, the virtual route is transmitted to the moving body A or the moving control means D. Since the virtual map information includes the information of the changed obstacle, the transmission of the virtual map information is also included in the transmission of the information of the changed obstacle.

The control means C6 is a processing device that controls various operations and functions of the terminal device C, and is realized by, for example, a CPU (Central Processing Unit) and a working memory, or by an integrated circuit. The CPU can perform the function of the control means C6 by reading the program stored in the storage means or the like into the working memory and executing the program. The control means C6 controls the image display means C2 to display various information (at least environmental information and / or image information), and controls the map creation means C1 and the route calculation means C3 to change the contents or the virtual route. Candidates may be simulated, or the communication means C5 may be controlled to send and receive predetermined information.

The movement control means D is realized by executing a movement control program, and controls the output of each movement means A1 along the route calculated by the route calculation means to move the moving body A to a predetermined position (destination). Let me. When a plurality of motors are provided in the moving body A as the moving means A1, the movement control means D may control the rotation angle and the rotation speed of each motor, and may control the current value supplied to each motor. You may control it. The movement control means D is preferably provided on the mobile body A, but may be provided outside the mobile body A, for example, on a management server, a terminal device C, or the like. Further, when the movement control means D recognizes a movable obstacle (preferably a temporary obstacle), the movement control means D may execute the standby process. The standby process is a process of waiting until a movable obstacle moves and becomes passable. For example, the departure timing may be delayed until the movable obstacle moves, or the movable obstacle may be delayed. After moving to the vicinity of, you may wait until a temporary obstacle moves. If the timing of movement of the movable obstacle is known, the moving speed to the movable obstacle may be adjusted so that the movable obstacle passes after the movement. The standby process may be executed when the obstacle detected by the environmental information acquisition means B or the image information acquisition means E is recognized as a movable obstacle by the image recognition means, or as information on the obstacle changed from the terminal device C. It may be when the setting of a movable obstacle is received.

The image information acquisition means E is a camera that acquires at least image information around the moving body A, and for example, a CCD camera, a CMOS sensor, or the like can be used. It is preferable that the image information acquisition means E can acquire at least an image (still image or moving image) in front of the moving body A in the moving direction, and can acquire an image of the entire measurement range of the environmental information acquisition means B, and is an image information acquisition means. It is preferable to include a movable portion that can change the imaging direction of E. The image information acquisition means E can acquire image information of an obstacle, determine the obstacle by the image recognition means, or display it on the image display means C2 of the terminal device C so that the operator can confirm the obstacle. However, if high-definition 3D LiDAR or the like is used as the environmental information acquisition means B, it is possible to determine or confirm the obstacle by the pattern of the measurement points, and the environmental information acquisition means B determines or confirms the obstacle. If possible, it is not necessary to provide the image information acquisition means E. Further, if the camera is used as the environmental information acquisition means B, it can also be used as the image information acquisition means E. The image information acquisition means E is preferably provided on the moving body A, but may be provided on the outside of the moving body A, for example, on the surrounding wall, ceiling, floor, or the like. When the moving body A is provided with the environmental information acquisition means B and the image information acquisition means E, the image information acquisition means E may be configured such that the camera is installed in a movable portion that can rotate in the circumferential direction, for example. It is preferable that the image pickup direction of the image information acquisition means E acquires an image of the measurement point in cooperation with the scanning of the measurement point of the environment information acquisition means B. The imaging timing may be always imaging, or imaging may be performed at a predetermined timing. When always taking an image, the amount of image data becomes enormous, which puts pressure on the data area of the relatively small-capacity storage means mounted on the moving body A. Further, the acquired image information can be provided to the terminal device C via the network, but if the acquired image information is always provided, a huge amount of image information is always transmitted and received, which causes a problem that the communication line is congested. Therefore, it is preferable to provide at least the image information to the terminal device C only at a predetermined timing. That is, the image information is not transmitted during normal operation, but is stored in the storage means of the moving body A, and when the operator's judgment is required due to an obstacle in the surroundings, or the image information from the terminal device C. It is preferable to transmit the image information when the signal requesting the above is received. If the storage capacity of the moving object is small, the oldest image may be automatically deleted. Further, in the case of taking an image at a predetermined timing without always taking an image, for example, the image information may be acquired at the timing when the obstacle blocking the moving path is detected by the environmental information acquisition means B. When the image information acquisition means E is provided outside the moving body A, for example, a surveillance camera may be used as the image information acquisition means E.

The storage means F is a means for storing various information and databases, and includes, for example, a USB memory, a semiconductor memory, a CD, a DVD, an HDD, and the like. The storage means F may be provided individually for each of the mobile body A, the environmental information acquisition means B, the terminal device C, the movement control means D, and the image information acquisition means E, or may be provided in common. The storage means F may be integrally provided with each configuration, or may be connected via a network. Various programs, acquired environmental information and image information, map information, subject database, obstacle database, and the like may be stored in the storage means F.

FIG. 3 is an example of hardware that realizes the mobile control system 1 of the present invention. The control system 1 in the present embodiment is a network 3 to which an autonomous mobile body 2 capable of calculating a route to a destination, a remote operator terminal device 4, a mobile body 2 and a terminal device 4 are connected. And, if necessary, the storage server 5 and the management server 6 that can be connected via the network 3 may be included. The mobile control system 1 of the present embodiment moves via the network 3 when a human judgment is required in the autonomous mobile 2 capable of calculating the route to the destination. By connecting the body 2 and the remote operator terminal device 4, it is possible to obtain the judgment of the remote operator by appropriately communicating with the terminal device 4 through the network 3. Further, the information acquired by the mobile body 2 is sent to the terminal device 4, the status of the mobile body 2 is confirmed by the terminal device 4, the information created by the terminal device 4 is sent to the mobile body 2, and such information is used. The moving body 2 can be controlled.

FIG. 4 is a block diagram showing a schematic configuration of the mobile body 2 and the terminal device 4 in the present embodiment. The moving body 2 in the present embodiment includes a moving means 21 for moving the moving body, an environment information acquisition means 22 for acquiring surrounding environment information, an image information acquisition means 23 for acquiring surrounding image information, and a communication means 24. The route calculation means 25 that calculates the route from the self-position or the departure position to the destination based on the map information, and the movement control means 26 that operates the movement means along the calculated route to move the moving body to the destination. And, if necessary, a position estimation means 27, a map creation means 28, an image recognition means 29, a storage means 30, and the like for calculating the self-position of the moving body are included. In the present embodiment, the route calculation means 25, the movement control means 26, the position estimation means 27, the map creation means 28, and the image recognition means 29 are realized by using a part of the functions of the control means 31. Details of each configuration will be described later. When the destination is set, the moving body 2 calculates the route from its own position or the departure position to the destination by the route calculating means 25, and operates the moving means 21 by the moving control means 26 to operate the moving means 21 along the route. It is an autonomous mobile body that can move up to. The mobile body 2 includes, for example, an automatic guided vehicle, a manned vehicle, a mobile industrial robot, a mobile robot, a mobile service robot, and an unmanned aerial vehicle (including a drone).

The network 3 is not particularly limited, and includes the Internet, for example, a public telephone network, ISDN (abbreviation of Integrated Service Digital Network, also referred to as a digital integrated service network), ADSL (Asymmetric Digital Subscriber Line), CATV ( Community Antenna TeleVision) networks, optical fiber networks, wireless LAN (Local Area Network), CS (Communication Satellite) broadcasting, mobile communication system (including 3G, 4G, 5G lines, etc.) networks can be used. A plurality of mobile bodies 2 may be connected to the network 3, and information is transmitted / received in each mobile body 2. However, when the capacity of the information to be transmitted / received becomes large, the communication line becomes congested and the communication speed decreases. Problems such as interruption of work and disconnection of communication connection occur due to the inability to acquire necessary information. In particular, the moving body 2 includes an image information acquisition means 23, and transmission / reception of image information is planned. However, when image information is constantly transmitted from a plurality of moving bodies 2, the amount of communication becomes enormous. Therefore, in some embodiments of the present invention, the information using the network 3 is devised in terms of time or content, the amount of communication is reduced, and even if the existing network 3 is used, a plurality of mobile units are used. Is made to work smoothly.

The remote operator terminal device 4 in the present embodiment moves with the map creating means 41 for creating map information, the image displaying means 42 capable of displaying the map information and the image information of obstacles created by the map creating means 41, and the moving. A route calculation means 43 capable of executing the same route calculation method as the route calculation means 25 of the body 2, an input means 44 capable of changing the cost of map information, a communication means 45, and a control means 46 for controlling these. It includes a storage means 47. The communication means 24 of the mobile body 2 and the communication means 45 of the terminal device 4 are connected via the network 3. The details of each configuration are as described above. .. The terminal device 4 may include, for example, a computer, a laptop computer, a personal digital assistant (including a smartphone, a tablet terminal), etc., and the terminal device 4 may be realized by one hardware, or a plurality of hardware (for example, a plurality of hardware). , The portable information terminal may be used as an image display means, an input means, a server as a map creation means, a route calculation means, etc.), and the terminal device 4 may be realized. Further, a plurality of mobile bodies 2 may be connected to one terminal device 4. In the present invention, it is sufficient for a human to support only when the moving body 2 recognizes an obstacle that affects the route calculation, and it is possible for one person to support a plurality of moving bodies.

The storage server 5 stores various information and databases, and can share data with the mobile body 2 and the terminal device 4 via the network 3. A storage server 5 may be provided separately from the storage means 47 of the mobile body 2 or the storage means 47 of the terminal device 4, or either storage means may be used in combination. The storage server 5 may store map information, environmental information and image information acquired from each moving object, obstacle information, subject database, obstacle database, and the like. The storage server 5 may be provided with a calculation function, or a part of the functions of the mobile body 2 or the terminal device 4 (for example, creation of virtual map information) may be performed. In this case, the storage server 5 is also a terminal device 4.

The management server 6 manages the movement of each moving body 2, and may acquire the self-position of each moving body 2 and manage the position of the moving body together with the map information, or each moving body 2 may be managed. The location information of the destination, the work content, and the like may be transmitted to the user. Further, the management server 6 may be made to carry out a part of the functions of the mobile body 2 or the terminal device 4 (for example, movement control means, creation of virtual map information, etc.). The storage server 5 and the management server 6 may be realized by one server or may be realized by separate servers. Further, the terminal device 4 may be used as the management server 6.

In the present embodiment, the moving means 21, the environmental information acquiring means 22, the image information acquiring means 23, the communication means 24, the moving control means 26, and the storage means 30 of the moving body 2 are the moving means A1 and the environmental information acquiring means of FIG. 2, respectively. B corresponds to image information acquisition means C, communication means A2, movement control means D, and storage means F, and the movement control means 26 is realized as a part of the function of the control means 31 in FIG.

The route calculation means 25 is realized by executing a route calculation program, and is realized as a part of the function of the control means 31 in FIG. The route calculation means 25 calculates the route from the departure position to the destination by using the information of the obstacle included in the map information, and the departure position may be the current position of the moving body 2 or is separately designated. It may be in any position. The route calculation means 25 can adopt various route calculation methods. For example, an algorithm called Global Dynamic Window Approach (Global DWA) may be used to obtain a global path, which is an overall route plan, and a local path (running speed), which is a local motion plan, may be obtained based on the global path. The route calculation means 25 sets a route so that the moving body 2 does not pass at least the position of the obstacle in order to avoid the obstacle. When the map information is composed of a plurality of cells in which a movable area, an immovable area based on an obstacle, an unknown area, etc. are set, the route calculation means 25 avoids the immovable cell and can move. Select a cell and calculate the route. Further, if the cost of the intermediate value is set for the cell in the movable area as the map information, the route calculation means 25 selects a cell having a low cost and calculates the route, so that the cost is constant from the obstacle. It is possible to calculate a route that is more than a certain distance away. When the route calculated by the route calculation means 43 of the terminal device 4 is received and used, the route calculation means 25 of the mobile body 2 is unnecessary, but due to the alleviation of congestion of the communication line, the time difference between the measurement and the route reception, etc. It is preferable to provide the route calculation means 25 in the mobile body 2 itself, and in that case, the same route calculation program as the route calculation means 43 of the terminal device 4 is used.

The position estimation means 27 is realized by executing the position estimation program, and is realized as a part of the function of the control means 31 in FIG. As the position estimation means 27, it is sufficient if the self-position can be specified, and the means is not particularly asked. However, the environmental information acquired by the environmental information acquisition means is collated with the map information prepared in advance, and based on the collation result. The self-position of the moving body may be calculated. As another position estimation means 27, it is specified by GPS, by a communication antenna or a base station, or by displaying a symbol indicating a position on a surrounding wall, ceiling, or floor and recognizing it as an image. You may do it. Further, if the position of the moving body 2 is transmitted from the outside (for example, the terminal device 4 or the like), the moving body 2 does not need to be provided with the position estimation means 27. In this case, a plurality of sensors for specifying the position of the moving body 2 may be arranged around the moving body 2.

The map creating means 28 is realized by executing a map creating program, and is realized as a part of the function of the control means 31 in FIG. The map creating means 28 creates measurement map information of the surrounding area from at least the environmental information acquired by the environmental information acquiring means 22. Further, the map information may be created by updating the basic map information and the past measurement map information with the latest measurement map information. The cartographic means 28 may set a cost for use in the route calculation means 25. When receiving and using the map information created by the map creating means 41 of the terminal device 4, the map creating means 28 of the moving body 2 is unnecessary, but due to the alleviation of congestion on the communication line, the time difference between measurement and map reception, etc. It is preferable to provide the map creating means 28 on the moving body 2 itself, and in that case, the same map creating program as the map creating means 41 of the terminal device 4 is used.

The image recognition means 29 is realized by executing an image recognition program, and is realized as a part of the function of the control means 31 in FIG. The image recognition means 29 extracts, identifies, and detects a subject included in the image from the image information. As the image recognition means 29, for example, the contour of each subject is specified from the image information, extracted separately from the background, collated using a subject database in which pattern images of various subjects are accumulated, and each subject is recognized. You may. If the subject database includes an obstacle pattern image and can be used as an obstacle database, the obstacle in the image can be identified. Further, the image recognition means 29 may identify an obstacle in the environmental information by image-recognizing the arrangement pattern of the measurement points in the environmental information and collating it with the arrangement pattern of the measurement points stored in the obstacle database. can. When an image authentication means is provided outside the mobile body 2 (for example, the terminal device 4) and the authentication result from the outside is used, the image recognition means 29 of the mobile body 2 is unnecessary, but the congestion of the communication line is reduced and the image is displayed. It is preferable to provide the image recognition means 29 on the moving body 2 itself because of the time difference between acquisition and authentication. If image authentication is not used, or if an image acquisition means is provided outside the moving body 2, the moving body may not be provided with the image recognition means 29.

The storage means 47 is a means for storing information used for controlling the mobile body 2, information acquired by the mobile body 2, and the like as needed, and is, for example, a USB memory, a semiconductor memory, a CD, a DVD, an HDD, or the like. including. The storage means 47 may store various programs, acquired environmental information and image information, a subject database, an obstacle database, and the like. Further, even if the storage means 47 is not provided, the communication means 24 can acquire necessary information from the external storage server 5 via the network 3, but the storage means is congested because the communication line is congested. It is preferable to store the necessary information such that the 47 is provided and used all the time in the storage means 47.

The control means 31 is a processing device that controls various operations and functions of the mobile body 2, and is realized by, for example, a CPU (Central Processing Unit) and a working memory, or by an integrated circuit. The CPU can fulfill the function of the control means 31 by reading the program stored in the storage means 47 or the like into the working memory and executing the program. In FIG. 4, as a part of the functions of the control means 31, the route calculation means 25, the movement control means 26, the position estimation means 27, the map creation means 28, and the image recognition means 29 are realized, but the configuration is limited to such a configuration. Instead, the route calculation means 25, the movement control means 26, the position estimation means 27, the map creation means 28, or the image recognition means 29 may be realized by a dedicated electronic circuit, or may be realized by separate hardware. May be good. Further, the control means 31 controls the environmental information acquisition means 22 to acquire the surrounding environment information, controls the image information acquisition means 23 to acquire the surrounding image information at a predetermined imaging timing, and the communication means. 24 may be controlled to send and receive predetermined information at a predetermined timing as described above. In this way, each part of the moving body 2 is controlled by the control means 31, and the moving body 2 can move to the destination along a predetermined route while performing self-position detection.

The map creation means 41, the image display means 42, the route calculation means 43, the input means 44, the communication means 45, the control means 46, and the storage means 47 of the terminal device 4 in the present embodiment are the figure creation means C1 and the image in FIG. 2, respectively. Corresponding to the display means C2, the route calculation means C3, the input means C4, the communication means C5, the control means C6, and the storage means F, the map creation means 41 and the route calculation means 43 create a map of the map creation means 28 of the moving body 2. It is possible to execute the same program and the route calculation program of the route calculation means 25.

[Acquisition of basic map information]
FIG. 5A is a schematic plan view showing the floor plan of the moving range of the moving body, and FIG. 5B is a diagram showing the middle of the environment information acquisition work for creating the basic map information. The moving body 2 has LiDAR as the environmental information acquisition means 22, a camera capable of rotating the image pickup direction in the circumferential direction as the image information acquisition means 23, and four wheels as the moving means 21. As shown in FIG. 5A, in the floor plan within the moving range, a rectangular large room 51 is arranged on the right side, and a square small room 52 connected to the wall on the left side via an opening 52a and an opening 53a. A square small room 53 connected via is arranged. In order to acquire the basic map information, the entire environmental information is acquired by SLAM using the environmental information acquisition means 22 of the moving body 2 in a state where unnecessary obstacles are cleared. As shown in FIG. 5B, the control means 31 of the moving body 2 (hereinafter, simply referred to as the moving body 2) sequentially irradiates the LiDAR 22 with the micropulse laser light 22a at predetermined step angles to affect the surrounding environment. Obtain environmental information by scanning with laser light. In FIG. 5B, only the laser beam directed to the left wall is shown, but scanning is performed in all directions, and only the measurement points are shown for the walls in other directions. It should be noted that the illustrated step angle and the angle range of the laser beam are schematically shown, and are not particularly limited. There are obstacles such as walls in the surroundings, and each laser beam 22a irradiates the obstacles, generates reflected light or scattered light (not shown) at each measurement point 22b, and returns the reflected light or The scattered light is detected by the detector of LiDAR22, and the distance and direction to each measurement point 22b are measured by measuring the time from irradiation to return. In this way, polar coordinates (distance, angle) centered on the environmental information acquisition means 22 can be obtained for each measurement point. The polar coordinates of each measurement point may be converted into Cartesian coordinates. In this way, the position of the wall of the large room 51, the position of a part of the wall of the small rooms 52 and 53 by the laser beam passing through the openings 52a and 53a, and the relative self-position can be grasped. Then, by repeating this work while moving within the moving range, the position of the obstacle in the entire moving range can be grasped.

FIG. 6A is an arrangement pattern of measurement points of environmental information in the entire movement range, and FIG. 6B is a plan view of map information. As shown in FIG. 6A, obstacles within the moving range can be grasped by the pattern of the measurement point 22b of the environmental information acquisition means 22. The image information of the obstacles grasped at the stage of creating the basic map information may be acquired by the image information acquisition means 23 of the moving body 2. FIG. 6B is map information shown in a plan view, and the position and orientation of the moving body are indicated by triangles 61. The map is divided vertically and horizontally by a plurality of cells 62 composed of a square grid, and a cost may be set for each cell. The shaded cell 63 in the figure is a cost area of the wall, and in order to prevent contact between the wall and the moving body, a high cost based on the wall is set, and it is regarded as an immovable area that cannot be set in the path of the moving body 2. ing. By setting the cost area of the obstacle to the range obtained by adding the safety margin to the width of the moving body 2, the moving body 2 can be replaced with a point and the route can be calculated as a line in the calculation of the route calculation.

[Acquisition of measurement map information]
FIG. 7A is a plan view schematically showing the situation at the time of creating the measurement map information, and FIGS. 7B and 7C are image information of newly detected obstacles which are not included in the basic map information. be. The moving body 2 is arranged at the same position as in FIG. 5, but a circular obstacle 54 and a square obstacle 55 are arranged in the vicinity of the opening 52a of the large room 51. As shown in FIG. 7A, the mobile body 2 scans the laser beam 22a of the LiDAR 22 around in order to acquire environmental information. Since a part of the wall and the opening 52a is a shadow of the obstacles 54 and 55, the laser beam 21a does not reach the shadowed area, and the measurement is performed on the surface of the obstacles 54 and 55 facing the moving body 2. Points 54a and 55a are detected. As shown in FIG. 7A, the environmental information shows that the ends of the measurement points 54a and 55a on the walls of the obstacles 54 and 55 are discontinuous and spatially separated from the wall. can. Further, in order to acquire the image information of the newly detected obstacle, the moving body 2 has a camera in the direction in which the obstacles 54 and 55 exist (the irradiation direction (measurement direction) of the laser beam 22a of the environmental information acquisition means 22). The image is taken by pointing at 23. FIG. 7B is image information of the obstacle 54, and the columnar obstacle 54 can be confirmed. FIG. 7C is image information of the obstacle 55, and the rectangular parallelepiped obstacle 55 can be confirmed. It is preferable to acquire the image information in a state where the moving body 2 is stopped, but the image information may be acquired during the movement. When acquiring image information, if the image information is acquired while moving the moving body 2, the image may be blurred. Therefore, it is preferable to use the algorithm shown in FIG. 12 below.

FIG. 8A is measurement map information based on the environmental information acquired in FIG. 7A. From the pattern of the measurement point 54a of the obstacle 54, the shape (cylindrical shape) of the surface of the obstacle 54 facing the moving body 2 can be grasped, and the cost area 54b is set for the cells within a certain range according to the surface shape. ing. Further, the shape of the surface (rectangular parallelepiped) of the obstacle 54 facing the moving body 2 can be grasped from the pattern of the measurement point 55a of the obstacle 55, and the cost area 55b is set in the cell within a certain range according to the surface shape. Has been done. Compared to the wall cost area 63 confirmed by the basic map information, the cost areas 54b and 55b of the unconfirmed obstacles 54 and 55 have a higher safety margin and are in a wider range. As shown in FIG. 8A, when the star mark of the small room 52 is the destination 64, the movement range is divided by the cost area 63 of the wall, the cost area 54b of the obstacle 54, and the cost area 55b of the obstacle 55. Therefore, the moving body 2 cannot calculate the route to the destination 64.

[Route calculation auxiliary processing]
According to the moving body control system 1 of the present invention, when the autonomous moving body 2 cannot calculate the route to the destination due to an obstacle, or when it detects an obstacle blocking the moving route, it moves. When the operator's judgment is required due to surrounding obstacles, such as when the route of the body is changed by an obstacle or when the moving object moves off the route, the operator's judgment is required via the communication means 24. The operator of the terminal device 4 can be requested to assist in route calculation. FIG. 9 is an example of a flowchart of the route calculation assist processing of the mobile body in the mobile body 2, and FIG. 10 is an example of a flowchart of the route calculation assist processing of the mobile body in the terminal device 4. In S71 of FIG. 9, the moving body 2 acquires surrounding environment information by the environment information acquisition means 22, and acquires surrounding image information by the image information acquisition means 23 in S72. The order of S71 and S72 is arbitrary, and the image information acquisition of S72 may be performed after the route calculation of S73 and before the information transmission of S75. Next, in S73, the mobile body 2 calculates the route based on the map information including the acquired environmental information by the route calculation means 25, and in S74, determines whether or not the route can be calculated. The map information may be created by the map creating means 28 of the moving body 2 or may be acquired via the communication means 24. When the route can be calculated by the route calculation means 25 (No in S74), in S77, the moving body 2 moves to the destination along the calculated route by the movement control means 26, and ends the process. When the route cannot be calculated by the route calculation means 25 (Yes in S74), necessary information is transmitted to the terminal device 4 via the communication means 24. As the information to be transmitted to the terminal device 4, at least the environmental information including the obstacle detected by the environmental information acquisition means 22 and the image information of the obstacle acquired by the image information acquisition means 23 are transmitted. For example, the environmental information is transmitted. If the information is transmitted to the terminal device 4 at the time of acquisition, the image information of the obstacle may be simply transmitted here. Further, by transmitting the map information to the terminal device, the environmental information included in the map information may be transmitted to the terminal device. Further, other information (for example, the self-position of the moving body, ID, the position of the destination, etc.) may be transmitted. After that, in S76, the change content and the like transmitted from the terminal device 2 by the process of the terminal device 2 in FIG. 10 are received. The received change contents include the changed obstacle information, virtual map information, virtual route, and the like. When the changed obstacle information is received, the map creating means 38 creates the map information based on the information, and the route calculating means 25 calculates the route. When the virtual map information is received, the route calculation means 25 calculates the route based on the virtual map information. Then, in S77, the mobile body 2 moves to the destination along the calculated route or the received virtual route by the movement control means 26.

The terminal device 4 receives the information transmitted from the mobile body 2 in S75 in S81 of FIG. In S82, the received image information of the obstacle is displayed on the image display means 42. Further, the map information created by the map creating means 41 from the received map information or the received environmental information may be displayed. The screen of the image display means 42 may be divided to display map information and image information, or the map information may be displayed in a plan view, and when an obstacle is selected, the image information of the obstacle is displayed. Alternatively, the map information may be displayed on top of the three-dimensional image information. Further, the terminal device 4 makes the input means 44 operable so that the operator can change the information of the obstacle. In S83, the terminal device 4 acquires the information of the obstacle changed by the operator, and in S84, the map creating means 41 creates the virtual map information. Then, in S85, the terminal device 4 uses the route calculation means 43 to calculate the route from the self-position or the departure position of the mobile body 2 to the destination by the same route calculation method as the route calculation means 25 of the mobile body 2 based on the virtual map information. Calculate the virtual route. Next, in S86, it is determined whether or not the virtual route can reach the destination. When the virtual route cannot be calculated (No in S86), the operator is notified that the route to reach the destination cannot be calculated, and a new obstacle information change is requested. When the destination is reached (Yes in S86), the changed contents and the like are transmitted to the moving body 2 in S87. The changed contents include the changed obstacle information, virtual map information, and virtual route. Before S87, the virtual route may be displayed to the operator to confirm the necessity of transmission to the mobile body.

[Obstacle information change processing]
In the terminal device 4, the operator can operate the input means 44 to change the obstacle information in the map information while confirming the obstacle by the image information of the obstacle displayed on the image display means 42. As a method of changing the obstacle information, for example, by selecting an obstacle included in the map information displayed on the image display means 42, the operator can change the setting such as the cost related to the selected obstacle. Alternatively, the setting of the cell may be changed by selecting at least one cell of the map information displayed on the image display means 42, or the obstacle underlying the setting of the cell may be changed. It may be possible to change settings such as costs related to things. It is preferable that the cost of the map information is superimposed on the image of the image information because the map information can be selected while looking at the image of the obstacle. Also, changing the obstacle information includes setting the selected obstacle as a movable obstacle (preferably a temporary obstacle). Obstacles include light movable obstacles such as empty cardboard, packing materials, paper bags, and balloons that are not damaged much when the moving body 2 collides and can be pushed away by the moving body 2, and soft deformable obstacles. There are things. In addition, there are desks and chairs that can pass underneath, or doors that can easily move obstacles depending on the pushing direction, and the operator can contact from the image information of the obstacle. It is possible to judge whether it is an obstacle that can be pushed away. Furthermore, if it is a temporary obstacle, the operator can determine that the obstacle can move and pass through after waiting for a while. In addition, for unknown obstacles, the safety margin may be high and the cost area may be set excessively wide, but the operator confirms from the image information and changes to an appropriate cost area to change the obstacle. It may be possible to avoid and calculate the route to the destination. Then, the operator can send the data after confirming that the destination can be reached by the route calculation means of the terminal device 4.

FIG. 8B is virtual map information in which one of the measurement points 54a of the obstacle 54 is selected by the pointer 65 and the cost area of the obstacle 54 is changed from the dotted line 54b to the shaded portion 54c. The cost area reduction may be changed manually or automatically. For example, selecting one of the measurement points 54a may allow the operator to change the cost or range of the cost area of each cell within the cost area for the obstacle 54, or the cost area of the obstacle 54. The range may be automatically reduced by the terminal device 4. In FIG. 8B, the cost area 55b of the obstacle 55 does not change, but the cost area 54c of the obstacle 54 is narrowed, so that the route calculation means 43 calculates the virtual route 66 to the destination 64 based on the virtual map information. can do. The terminal device 4 may display the virtual path 66 on the image display means 42 and confirm with the operator whether or not transmission to the mobile body 2 is necessary. When the operator confirms the virtual route and permits transmission, the terminal device 4 transmits the changed content or the like to the mobile body 2. In addition, in FIGS. 8 (B) and 8 (C), the grid lines are erased.

In FIG. 8C, at least one cell 67 of the map information is selected by the pointer 65, the cost area of the obstacle 54 is changed from the dotted line 54b to the shaded portion 54d, and the cost area of the obstacle 55 is changed from the dotted line 55b. It is the virtual map information changed to the shaded part 55d. When the operator selects the cell 67 displayed on the image display means 42 with the pointer 65, the terminal device 4 changes the selected cell 67 so that it can pass through the vicinity of the selected cell 67 if it is in the cost area, and the selected cell 67 is an obstacle. If so, change to the state where the obstacle is removed, create the virtual map information after the change, and calculate the virtual route based on the virtual map information. If the virtual route cannot be calculated, or if the virtual route is not appropriate, additional cells may be selected or new cells may be selected.

[Virtual route candidate presentation processing]
FIG. 11 is an example of a flowchart of a process of presenting a simulated virtual route candidate in order for the terminal device 4 to assist the operator's judgment. Candidates for virtual routes are presented, for example, before the operator changes the obstacle information, when the virtual route cannot be calculated even with the changed virtual map information, or when there is a request from the operator to present candidates. You may. In S91, the terminal device 4 creates virtual map information ignoring obstacles that hinder the route calculation, and calculates a preliminary virtual route. Next, in S92, the terminal device 4 superimposes the preliminary virtual path on the map information including the obstacle information, and extracts the obstacle or the cost area on the preliminary virtual path. In S93, the terminal device 4 changes the extracted obstacle information or the extracted cost area, in S94, calculates a virtual route based on the changed information, and in S95, the virtual route can reach the destination. Judge whether or not. If the virtual route cannot be calculated (No in S95), the obstacle information is further changed in S93. When the destination is reached (Yes in S95), a virtual route candidate is displayed to the operator in S96. If the terminal device 4 can calculate the candidate of one virtual route, the candidate of the virtual route may be displayed, or the candidate of a plurality of virtual routes can be selected after calculating the candidate of a plurality of virtual routes. It may be displayed in. The operator may adopt the presented candidate, change the obstacle information on his own, or add the change to the presented candidate. Further, the terminal device 4 may display the changed contents in order to further obtain the candidate of the virtual route.

[Image acquisition process on the move]
FIG. 12 is an example of a flowchart of processing for preventing or reducing image blurring when an image of a subject such as an obstacle is taken by the moving body image information acquisition means 23. In S101, the moving body 2 points the image information acquisition means 23 (here, the camera) toward the subject. If the orientation of the camera can be changed, the orientation of the camera is directed toward the subject, and if the orientation of the camera cannot be changed, the orientation of each moving object is changed and the camera is directed toward the subject. In S102, it is confirmed whether or not the subject exists within the angle of view. The camera is pointed at the subject in S101, but the subject may deviate from the angle of view because the moving body is moving. If there is no subject within the angle of view (No in S102), the process returns to S101 and the camera is pointed at the subject. When the subject exists within the angle of view (Yes in S102), it is confirmed in S103 whether or not there is a subject whose moving speed on the image is equal to or less than a certain level. The moving speed on the image is the displacement of the coordinates on the image per time with respect to a certain point of the subject, and when the moving body 2 is moving toward the subject or when the moving body 2 moves away from the subject. Is suitable for shooting because the moving speed of the camera toward the subject on the image is small. On the other hand, if the moving speed of the subject on the image is high, the image tends to be blurred. Therefore, when there is no subject whose movement speed is below a certain level on the image (No in S103), the movement conditions (movement speed, movement direction, etc.) of the moving body are changed in S104, and the process returns to S101 again. Point the camera at the subject. When there is a subject whose moving speed on the image is below a certain level (Yes in S103), the subject is photographed in S105.

1 Mobile control system 2 Mobile 3 Network 4 Remote operator terminal device 21 Mobile means 22 Environmental information acquisition means 23 Image information acquisition means
24 Communication means 25 Route calculation means 26 Movement control means 27 Position estimation means 28 Map creation means 29 Image recognition means 30 Storage means 31 Control means 41 Map creation means 42 Image display means 43 Route calculation means 44 Input means 45 Communication means 46 Control means 47 Means of storage

Claims (24)

A moving body having a moving means, an environmental information acquisition means for acquiring environmental information around the moving body, a terminal device for a remote operator provided separately from the moving body, and the moving means of the moving body. A mobile control system that includes a mobile control means to control.
The environmental information acquisition means can detect an obstacle existing around the moving body as environmental information and provide the environmental information to the remote operator terminal device.
The remote operator terminal device includes a map creating means for creating map information including information on obstacles included in the environmental information, an image display means capable of displaying information on obstacles included in the map information, and the above-mentioned. It includes an input means capable of changing the information of the obstacle displayed on the image display means and a route calculation means for calculating a route from a departure position to a destination based on map information.
The map creating means creates virtual map information including information on obstacles changed by the input means.
The route calculation means calculates a virtual route from the self-position or departure position of the moving body to the destination based on the virtual map information.
The remote operator terminal device can provide the movement control means with information on obstacles changed by the input means, the virtual map information and / or the virtual route.
The movement control means provides the movement means of the mobile body based on the obstacle information, the virtual map information and / or the virtual route changed by the input means provided from the remote operator terminal device. A moving body control system that controls and initiates the movement of the moving body along the virtual path. The moving body control system further includes an image information acquisition means for acquiring an image of an obstacle, and the image information acquisition means can provide image information of an obstacle to the remote operator terminal device.
The mobile control system according to claim 1, wherein the remote operator terminal device can display image information of the obstacle on the image display means. The environmental information acquisition means and the movement control means are provided on the moving body.
The mobile control system according to claim 1 or 2, wherein the mobile has a route calculation means capable of executing the same route calculation method as the route calculation means of the remote operator terminal device. The moving body control system according to any one of claims 1 to 3, wherein the moving body has a map making means capable of executing the same map making method as the map making means of the remote operator terminal device. The map information is divided into a plurality of cells, and a cost based on the obstacle is set as information on the obstacle in each cell.
The mobile control system according to any one of claims 1 to 4, wherein the input means can change the cost of the cell. When an obstacle included in the map information displayed on the image display means is selected by the input means, the remote operator terminal device changes the cost of the area in which the cost is set based on the selected obstacle. The moving body control system according to claim 5, which makes it possible. When at least one cell of the map information displayed on the image display means is selected by the input means, the remote operator terminal device can pass around an obstacle including the selected cell or around the selected cell. The mobile control system according to claim 5 or 6, which is converted into a cost. When the obstacle is set as a movable obstacle by the input means, the remote operator terminal device creates virtual map information in a state where the set obstacle does not exist, calculates a virtual route, and is movable. Information on obstacles is provided to the movement control means,
Any one of claims 1 to 7, wherein the movement control means executes a standby process of waiting until the movable obstacle moves and becomes passable when the information of the movable obstacle is provided. The mobile control system described in the section. When the remote operator terminal device cannot calculate the route to the destination by the route calculation means, detects an obstacle blocking the moving route, or acquires information on a preset obstacle. The moving body according to any one of claims 1 to 8, which requests the operator to change the information of the obstacle when the route is changed or the moving body moves off the route. Control system. The remote operator terminal device displays the virtual route calculated by the route calculation means on the image display means, and after receiving a transmission permission signal from the operator to the moving body, the obstacle changed by the input means. The moving body control system according to any one of claims 1 to 9, wherein the information, the virtual map information and / or the virtual route is provided to the movement control means. The mobile control system according to any one of claims 1 to 10, wherein the remote operator terminal device displays a candidate for a change content or a candidate for a virtual route obtained by simulation on the image display means. .. The mobile body control system according to any one of claims 1 to 11, wherein the movement control means executes a contact operation of bringing the mobile body into contact with an obstacle. A moving body having a moving means, an environmental information acquisition means for acquiring environmental information around the moving body, a terminal device for a remote operator provided separately from the moving body, and the moving means of the moving body. A method of controlling a moving body including a moving control means for controlling.
A step of detecting an obstacle existing around the moving body as environmental information by the environmental information acquisition means and providing the environmental information to the remote operator terminal device.
A step of creating map information including information on obstacles included in the environmental information by the map creating means of the terminal device for a remote operator, and
Information on obstacles included in the map information is displayed on the image display means of the remote operator terminal device, and the information on the obstacles displayed on the image display means is changed by the input means of the remote operator terminal device. Steps and
A step of creating virtual map information including information on obstacles changed by the input means by the map creation means, and
A step of calculating a virtual route from the self-position or departure position of the moving body to the destination based on the virtual map information by the route calculation means of the remote operator terminal device.
A step of providing the movement control means with information on obstacles changed by the input means, the virtual map information, and / or the virtual route.
The movement control means provides the movement means of the mobile body based on the obstacle information, the virtual map information and / or the virtual route changed by the input means provided from the remote operator terminal device. A method of controlling a moving body, comprising a step of controlling and initiating the movement of the moving body along the virtual path. Further including an image information acquisition means for acquiring an image of an obstacle,
13. The method for controlling a moving body according to claim 13, comprising a step of acquiring image information of an obstacle by the image information acquisition means and providing the image information of the obstacle to the remote operator terminal device. The environmental information acquisition means and the movement control means are provided on the moving body.
The mobile body control method according to claim 13, wherein the mobile body has a route calculation means capable of executing the same route calculation method as the route calculation means of the remote operator terminal device. The method for controlling a moving body according to any one of claims 13 to 15, wherein the moving body has a map making means capable of performing the same map making method as the map making means of the remote operator terminal device. The map information is divided into a plurality of cells, and a cost based on the obstacle is set as information on the obstacle in each cell.
The method for controlling a moving body according to any one of claims 13 to 16, wherein the cost of the cell is changed by the input means. An obstacle included in the map information displayed on the image display means is selected by the input means, and the cost of the area where the cost is set based on the selected obstacle is changed to a value movable by the moving body. The method for controlling a moving body according to claim 17. Claim 18 or 19 that selects at least one cell of map information displayed on the image display means by the input means and changes it to a cost that can pass through an obstacle around the selected cell or including the selected cell. The method for controlling a moving body according to the above. When an obstacle is set as a movable obstacle by the input means, virtual map information in a state where the set obstacle does not exist is created, a virtual route is calculated, and information on the movable obstacle is provided to the movement control means. death,
The method for controlling a moving body according to any one of claims 13 to 19, wherein the movement control means passes the position of the movable obstacle after the movable obstacle has moved. When the remote operator terminal device cannot calculate the route to the destination by the route calculation means, detects an obstacle blocking the moving route, or acquires information on a preset obstacle. The moving body according to any one of claims 13 to 20, which requests the operator to change the information of the obstacle when the route is changed or the moving body moves off the route. Control method. When the virtual route can be calculated based on the virtual map information by the route calculation means, the obstacle information changed by the input means, the virtual map information and / or the virtual route is provided to the movement control means. , The method for controlling a moving body according to any one of claims 13 to 21. The method for controlling a mobile body according to any one of claims 13 to 22, wherein the remote operator terminal device displays a candidate for a change content or a candidate for a virtual route obtained by simulation on the image display means. .. The method for controlling a moving body according to any one of claims 13 to 23, wherein the movement controlling means executes a contact operation in which the moving body is brought into contact with an obstacle.

Images